clevermicro/amq-adapter-python

Fork 0

Verify backpressure feature #20

New issue

Closed

opened 2025-04-25 08:25:41 +00:00 by hurui200320 · 2 comments

hurui200320 commented

2025-04-25 08:25:41 +00:00

Member

Ref epic: CleverSwarm Onboarding #58

This ticket will verify if the backpressure feature works in python adapter, and fix it if there is any issue.

Ref epic: [CleverSwarm Onboarding #58](https://git.cleverthis.com/clevermicro/clevermicro/issues/58) This ticket will verify if the backpressure feature works in python adapter, and fix it if there is any issue.

hurui200320 added the

labels

2025-04-25 08:25:41 +00:00

hurui200320 self-assigned this

2025-04-25 08:25:41 +00:00

hurui200320 referenced this issue from a commit

2025-04-25 08:27:53 +00:00

fix(General): fix the backpressure event

hurui200320 added a new dependency

2025-04-25 08:28:27 +00:00

clevermicro/management-service#10 - Change backpressure even enum

hurui200320 added a new dependency

2025-04-25 08:28:33 +00:00

clevermicro/amq-adapter-java#29 - Change backpressure even enum

hurui200320 referenced this issue

2025-04-25 08:32:25 +00:00

Fix the backpressure event #21

hurui200320 added a new dependency

2025-04-25 08:32:54 +00:00

#21 Fix the backpressure event

hurui200320 added the

Points

label

2025-04-25 08:34:03 +00:00

hurui200320 commented

2025-04-29 05:38:33 +00:00

Author

Member

Update at the end of the sprint: currently due to the python repo is not available, I can't pack cleverswarm-endpoint into docker image. Thus, I can only test it outside the docker swarm. However, based on the results, I can confirm that python adapter can send out the correct message, and the management service can receive and decode it. According to previous test and demo, once the management service can got the message, it can perform scale up and down based on the swarm service id from the message.

This ticket will be paused and resume when we can do full-scale docker swarm deployment test.

hurui200320 added

and removed

labels

2025-04-29 05:38:45 +00:00

aleenaumair added a new dependency

2025-05-01 10:39:43 +00:00

cleverthis/clevermicro#8 - Onboarding CleverSwarm with core setup for CleverMicro

hurui200320 added

and removed

labels

2025-05-01 13:43:14 +00:00

hurui200320 commented

2025-05-01 13:44:18 +00:00

Author

Member

(quote from matrix message)

Hi Stanislav Hejny (he/him, UTC+1) , I finished the testing with cleverswarm. There are several changes for the python adapter, I commited the change to branch fix-version. For the cleverswarm, the following lines is needed to install the python adapter in the requirement.txt file:

--extra-index-url https://pkg.cleverthis.io/pypi/clevermicro/amq-adapter-python/simple/
amq_adapter==0.2.23-dev20250430124934

The suffix will pin a specific dev build to prevent unexpected upgrades.

Other than the versioning issue, I also noticed that, when deploy a new version of the stack, different components will restarted. If the rabbitmq is restarted after the cleverswarm, then the adapter will no longer send out overload event. This may require some design change in the adapter to crash the program when connection is lost, so the docker swarm will bring up a new instance, or let it recover if we can.

FYI, this is the service def for cleverswarm:

  cleverswarm-endpoint:
    image: cleverswarm-endpoint # change to actual image name
    networks:
      - Mercury
      - Venus
    environment:
      - SWARM_SERVICE_ID={{.Service.ID}}
      - SWARM_TASK_ID={{.Task.ID}}
    deploy:
      labels:
        - com.cleverthis.clevermicro=swarm

(quote from matrix message) Hi Stanislav Hejny (he/him, UTC+1) , I finished the testing with cleverswarm. There are several changes for the python adapter, I commited the change to branch `fix-version`. For the cleverswarm, the following lines is needed to install the python adapter in the requirement.txt file: ``` --extra-index-url https://pkg.cleverthis.io/pypi/clevermicro/amq-adapter-python/simple/ amq_adapter==0.2.23-dev20250430124934 ``` The suffix will pin a specific dev build to prevent unexpected upgrades. Other than the versioning issue, I also noticed that, when deploy a new version of the stack, different components will restarted. If the rabbitmq is restarted after the cleverswarm, then the adapter will no longer send out overload event. This may require some design change in the adapter to crash the program when connection is lost, so the docker swarm will bring up a new instance, or let it recover if we can. FYI, this is the service def for cleverswarm: ``` cleverswarm-endpoint: image: cleverswarm-endpoint # change to actual image name networks: - Mercury - Venus environment: - SWARM_SERVICE_ID={{.Service.ID}} - SWARM_TASK_ID={{.Task.ID}} deploy: labels: - com.cleverthis.clevermicro=swarm ```

hurui200320 closed this issue

2025-05-01 13:44:19 +00:00