amq-adapter-python

This project contains the Python implementation of the AMQ adapter for routing the traffic over the RabbitMQ to intended service and automated service discovery.

Usage

Install dependencies

pip install -e .[dev]

This will install dependencies for developing, but won't install this project as an library.

To build the wheel files:

python -m build

The wheel file should be located under dist folder. You can use pip install /path/to/wheel.whl to install this library.

Use as a library

The library has been published to pulp registry at https://pkg.cleverthis.io/pypi/clevermicro/amq-adapter-python/. Configure the dependency amq_adapter with the above registry using your build tools, then:

from src.amqp.service import amq_service

# start using the lib

How to build distributable Wheel archive

install project requirements and dev dependencies pip install -e .[dev]

update version in: pyproject.toml

For dev versions, keep the -dev suffix. The pipeline will replace it with a unique timestamp, act like maven SNAPSHOT versions.

build the distributable archive (will be located in ./dist directory) python -m build --wheel

Development Setup

Pre-commit Hooks

This project uses pre-commit hooks to ensure code quality and consistency. See PRE_COMMIT.md for detailed information.

Quick setup:

# Install pre-commit
pip install pre-commit

# Install hooks
pre-commit install

# Run all hooks manually
./run_precommit.sh

Integrating with actual CleverThis Service

The following steps are required to integrate with the CleverThis service. This example uses ClverSwarm as specific use case, other python-based services would follow identical pattern.

Prerequisites

IMPORTANT: AMQ Adapter relies on the service to use FastAPI to implement its REST endpoints and that those endpoints are properly annotated with FastAPI annotations. AMQ Adapter will look for these annotations to recognize the service API entry points and will automatically use the annotation to map the actual function to the incoming request the very same way FastAPI does.

1. Install the amq-adapter library, as indicated above
2. Ensure the needed dependencies were installed together with the amq-adapter. This is the list of dependencies:

aiohttp~=3.11.11
fastapi==0.115.6
aio_pika~=9.5.5
opentelemetry-api
opentelemetry-sdk
opentelemetry-exporter-otlp
opentelemetry-exporter-prometheus
opentelemetry-instrumentation-pika
python_multipart

Go to amq-dapter-python repo source code, and copy the service specific directory, e.g. <git-root>/cleverswarm directory to your service's sources. For the CleverSwarm service, the file is cleverswarm/clever_swarm.py, and contains the AMQ Adapter to CleverSwarm service custom facade, implemented as

class CleverSwarmAmqpAdapter(CleverThisServiceAdapter):

to instatiate and initialize the AMQ adapter, add following line to the init routine of the CleverThis service:

    adapter = CleverSwarmAmqpAdapter(AMQConfiguration("./application.properties.local"))

and ensure that the adapter stays in the context of the service, so it is not garbage collected.

At version 0.2.21, AMQ Adapter uses local configuration file, relative path to which is passed to the AMQConfiguration constructor. This may change in the future, when the central configuration service will become available in CleverMicro, but for now please ensure correct relative path wrt the current working directory.

For the config entries, please refer to the wiki page: AMQ Adapter Config

Environment variables

The AMQ Adapter uses the following environment variables to configure the service:

PARALLEL_WORKERS - number of parallel workers to use for processing incoming requests. This is currently mandatory value, shall be greater than 0.

AMQ_VERBOSE_LOGGING - if set to 1, the adapter will indicate in the log next to the message itself also module name, line and function name where the message was generated

## Donating

[![Open Collective backers](https://img.shields.io/opencollective/all/CleverThis)](https://opencollective.com/cleverthis)

As an open-source project we run entierly off donations. Buy one of our hardworking developers a beer by [donating at OpenCollective](https://opencollective.com/cleverthis). All donations go to our bounty fund and allow us to place bounties on important bugs and enhancements. You may also use Beerpay linked via the above badges.

## Running with Docker Compose

This project includes a Docker Compose configuration for easy local development and testing.

### Prerequisites

- Docker and Docker Compose installed on your system

### Starting the services

```bash
docker-compose up -d

This will start:

• The demo application (accessible at http://localhost:8000)
• Jaeger for distributed tracing (UI at http://localhost:16686)
• Prometheus for metrics collection (UI at http://localhost:9090)
• Alertmanager for alert handling (UI at http://localhost:9093)
• RabbitMQ for messaging (Management UI at http://localhost:15672)

Stopping the services

docker-compose down

To remove volumes as well:

docker-compose down -v