hurui200320
bde74c82b5
This MR fixed the pipeline so it can run on forgejo runner. Currently we have two pipelines set up: + One pipeline will run pytest to test the code and collect coverage data, this pipeline will run on all branches and PRs. + One pipeline will build the wheel file and publish it to the company's pypl repo, this pipeline will only run on the master branch and develop branch. This RP also made the following changes to make the pipeline working: + Remove `setup.py` and `requirements.txt`, now the project is fully embraced `pyproject.toml`, which according to ChatGPT, is the most modern way of managing a python project. + Move all test files to `tests` folder, so it won't be packed into the final wheel file and affect the coverage rate. + Change version to `X.Y.Z-dev`, before building the wheel file, the pipeline will automatically add a UTC timestamp after it, so it becomes `X.Y.Z-devYYYYMMDDHHmmSS`. This is added due to the limitation of the pypl repo, unlike maven snapshot versions, you can't push the same version twice with our pypl repository. Reviewed-on: #10 Reviewed-by: Stanislav Hejny <stanislav.hejny@cleverthis.com> Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
119 lines
4.9 KiB
Markdown
119 lines
4.9 KiB
Markdown
|
|
|
|
[](https://git.cleverthis.com/cleverthis/clevermicro/amq-adapter-python/-/commits/master)
|
|
[](https://git.cleverthis.com/cleverthis/clevermicro/amq-adapter-python/-/commits/master)
|
|
[](https://semver.org/spec/v2.0.0.html)
|
|
[](https://matrix.to/#/#CleverThis:qoto.org)
|
|
|
|
# amq-adapter-python
|
|
|
|
This project contains the Python implementation of the AMQ adapter
|
|
for routing the traffic over the RabbitMQ to intended service and automated service discovery.
|
|
|
|
## Usage
|
|
|
|
### Install dependencies
|
|
|
|
```bash
|
|
pip install -e .[dev]
|
|
```
|
|
|
|
This will install dependencies for developing, but won't install this project as an library.
|
|
|
|
To build the wheel files:
|
|
|
|
```bash
|
|
python -m build
|
|
```
|
|
|
|
The wheel file should be located under `dist` folder.
|
|
You can use `pip install /path/to/wheel.whl` to install this library.
|
|
|
|
### Use as a library
|
|
|
|
The library has been published to pulp registry at `https://pkg.cleverthis.io/pypi/clevermicro/amq-adapter-python/`.
|
|
Configure the dependency `amq_adapter` with the above registry using
|
|
your build tools, then:
|
|
|
|
```python
|
|
from amqp.service import amq_service
|
|
|
|
# start using the lib
|
|
```
|
|
|
|
#### How to build distributable Wheel archive
|
|
|
|
install project requirements and dev dependencies
|
|
`pip install -e .[dev]`
|
|
|
|
update version in:
|
|
`pyproject.toml`
|
|
|
|
> For dev versions, keep the `-dev` suffix. The pipeline will replace it with a unique timestamp,
|
|
> act like maven SNAPSHOT versions.
|
|
|
|
build the distributable archive (will be located in `./dist` directory)
|
|
`python -m build --wheel`
|
|
|
|
# Integrating with actual CleverThis Service
|
|
|
|
The following steps are required to integrate with the CleverThis service. This example uses ClverSwarm as specific use case,
|
|
other python-based services would follow identical pattern.
|
|
|
|
## Prerequisites
|
|
IMPORTANT: AMQ Adapter relies on the service to use FastAPI to implement its REST endpoints
|
|
and that those endpoints are properly annotated with FastAPI annotations.
|
|
AMQ Adapter will look for these annotations to recognize the service API entry points and
|
|
will automatically use the annotation to map the actual function to the incoming request the very same way FastAPI does.
|
|
|
|
|
|
1. **Install the amq-adapter library**, as indicated above
|
|
2. **Ensure the needed dependencies were installed** together with the amq-adapter. This is the list of dependencies:
|
|
```
|
|
aiohttp~=3.11.11
|
|
fastapi==0.115.6
|
|
aio_pika~=9.5.5
|
|
opentelemetry-api
|
|
opentelemetry-sdk
|
|
opentelemetry-exporter-otlp
|
|
opentelemetry-exporter-prometheus
|
|
opentelemetry-instrumentation-pika
|
|
python_multipart
|
|
```
|
|
|
|
Go to amq-dapter-python repo source code, and copy the service specific directory, e.g. `<git-root>/cleverswarm`
|
|
directory to your service's sources. For the CleverSwarm service, the file is `cleverswarm/clever_swarm.py`,
|
|
and contains the AMQ Adapter to CleverSwarm service custom facade, implemented as
|
|
```python
|
|
class CleverSwarmAmqpAdapter(CleverThisServiceAdapter):
|
|
```
|
|
|
|
to instatiate and initialize the AMQ adapter, add following line to the init routine of the CleverThis service:
|
|
```python
|
|
adapter = CleverSwarmAmqpAdapter(AMQConfiguration("./application.properties.local"))
|
|
```
|
|
and ensure that the `adapter` stays in the context of the service, so it is not garbage collected.
|
|
|
|
At version 0.2.21, AMQ Adapter uses local configuration file, relative path to which is passed
|
|
to the AMQConfiguration constructor. This may change in the future, when the central configuration service
|
|
will become available in CleverMicro, but for now please ensure correct relative path wrt the current working directory.
|
|
|
|
For the config entries, please refer to the wiki page: [AMQ Adapter Config](https://docs.cleverthis.com/en/architecture/microservices/amq-adapter-configuration)
|
|
|
|
### Environment variables
|
|
|
|
The AMQ Adapter uses the following environment variables to configure the service:
|
|
|
|
PARALLEL_WORKERS - number of parallel workers to use for processing incoming requests. This is currently mandatory value,
|
|
shall be greater than 0.
|
|
|
|
AMQ_VERBOSE_LOGGING - if set to 1, the adapter will indicate in the log next to the message itself also module name, line and function name where the message was generated
|
|
|
|
```
|
|
## Donating
|
|
|
|
[](https://opencollective.com/cleverthis)
|
|
|
|
As an open-source project we run entierly off donations. Buy one of our hardworking developers a beer by [donating at OpenCollective](https://opencollective.com/cleverthis). All donations go to our bounty fund and allow us to place bounties on important bugs and enhancements. You may also use Beerpay linked via the above badges.
|
|
|