clevermicro/amq-adapter-python
clevermicro/amq-adapter-python
13
0
Fork 0
Code Issues 27 Pull requests 1 Projects Releases Packages 2 Wiki Activity Actions

Fix for cleverswarm backend #125

Merged
hurui200320 merged 10 commits from fix/channel-invalid-state into develop 2026-02-02 10:50:41 +00:00
Conversation 1 Commits 10 Files changed 14 +110 -93
hurui200320 commented 2026-02-02 08:37:50 +00:00
Member
Copy link

This PR contains a temporary fix for the cleverswarm:

  • (frontend) Test if the method result is coroutine (return value of async def) or not, only await if it's coroutine
  • (backend) Raise ConsumerChannelBrokenError when backend queue iter exits, HOWEVER, when channel fails, other aio-pika exception might get raised, not strictly this
  • (backend) Removed the timeout parameter for amq_service.data_message_generator
  • Remove connection in AMQ Message
  • RoutingService uses its own channel. TODO: how to detect and report this channel failure?
  • data_message_generator use its own channel.
  • Remove reinitialize_if_disconnected in amq service, the reinitializing process is not complete, it can't fix the problem at all.
  • release 0.3.3
This PR contains a temporary fix for the cleverswarm: + (frontend) Test if the method result is coroutine (return value of `async def`) or not, only await if it's coroutine + (backend) Raise `ConsumerChannelBrokenError` when backend queue iter exits, **HOWEVER, when channel fails, other aio-pika exception might get raised, not strictly this** + (backend) Removed the timeout parameter for `amq_service.data_message_generator` + Remove connection in AMQ Message + RoutingService uses its own channel. TODO: how to detect and report this channel failure? + `data_message_generator` use its own channel. + Remove `reinitialize_if_disconnected` in amq service, the reinitializing process is not complete, it can't fix the problem at all. + release 0.3.3
since nobody use it and it doesn't make any sense
Always close connection so we don't leak other channels
This method cause resource leak, doesn't call close/cleanup on the discarded object, and new object will not be picked up by some part of the program. This will only cause more issue than it's trying to fix
fix: allocate dedicated channel for data message generator
All checks were successful
/ run-unit-tests (push) Successful in 1m44s
Details
/ build-and-push-otdemo (push) Successful in 3m38s
Details
/ run-integration-tests-cleverswarm (push) Successful in 10m14s
Details
/ run-internal-integration-tests (push) Successful in 17m6s
Details
/ publish-python-library (push) Successful in 39s
Details
51d037feb8
 
This is specifically for cleverswarm backend (and otdemo backend) who need a data message generator.

This change doesn't affect the normal HTTP request handling.
fix: allocate dedicated channel for routing service
All checks were successful
/ run-unit-tests (push) Successful in 47s
Details
/ build-and-push-otdemo (push) Successful in 50s
Details
/ run-integration-tests-cleverswarm (push) Successful in 10m32s
Details
/ run-internal-integration-tests (push) Successful in 17m41s
Details
/ publish-python-library (push) Successful in 37s
Details
6726b516cb
feat: raise error when data message generator failed
All checks were successful
/ run-unit-tests (push) Successful in 46s
Details
/ build-and-push-otdemo (push) Successful in 44s
Details
/ run-integration-tests-cleverswarm (push) Successful in 10m32s
Details
/ run-internal-integration-tests (push) Successful in 18m56s
Details
/ publish-python-library (push) Successful in 45s
Details
b8e8b549ac
fix: add fallback error raise statement for backend usage
All checks were successful
/ run-unit-tests (push) Successful in 47s
Details
/ build-and-push-otdemo (push) Successful in 44s
Details
/ run-integration-tests-cleverswarm (push) Successful in 10m18s
Details
/ run-internal-integration-tests (push) Successful in 18m28s
Details
/ publish-python-library (push) Successful in 39s
Details
7bac57f590
ci: use latest cleverswarm lite as test
All checks were successful
/ run-unit-tests (push) Successful in 49s
Details
/ build-and-push-otdemo (push) Successful in 14s
Details
/ run-integration-tests-cleverswarm (push) Successful in 9m58s
Details
/ run-internal-integration-tests (push) Successful in 17m49s
Details
/ publish-python-library (push) Successful in 38s
Details
90bcdbbef9
CoreRasurae approved these changes 2026-02-02 10:14:18 +00:00
CoreRasurae left a comment
Member
Copy link

I left one comment for the future, but the PR is approved.

I left one comment for the future, but the PR is approved.
src/amqp/service/amq_service.py
@ -244,0 +223,4 @@
# So error on another channel won't break the request processing loop
async with self.rabbit_mq_client.connection.channel() as channel:
# Eventloop is single thread, process only 1 message at a time
await channel.set_qos(prefetch_count=1)
CoreRasurae commented 2026-02-02 10:12:15 +00:00
Member
Copy link

For CleverSwarm we only process one message at a time, but there may be other clients later on that might be able to process more than one request.

For CleverSwarm we only process one message at a time, but there may be other clients later on that might be able to process more than one request.
hurui200320 deleted branch fix/channel-invalid-state 2026-02-02 10:50:42 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
CoreRasurae
Labels
Clear labels
  
Blocked

A ticket in a blocked state and unable to complete until some other task is completed first.

  
Bounty
$100
A bounty of $100 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$1000
A bounty of $1000 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$10000
A bounty of $10000 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$20
A bounty of $20 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$2000
A bounty of $2000 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$250
A bounty of $250 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$50
A bounty of $50 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$500
A bounty of $500 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$5000
A bounty of $5000 for any open-source contributor who provides a MR that solves this issue

  
Bounty
$750
A bounty of $750 for any open-source contributor who provides a MR that solves this issue

  
MoSCoW
Could have
Could have feature in order to satisfy the epic/legendary.

  
MoSCoW
Must have
Must have feature in order to satisfy the epic/legendary.

  
MoSCoW
Should have
Should have feature in order to satisfy the epic/legendary.

  
Needs feedback

There are questions in the ticket that can not be completed until the project owner provides clarity.

  
Points
1
1 man-hours worth of work for an expert with no learning curve.

  
Points
13
13 man-hours worth of work for an expert with no learning curve.

  
Points
2
2 man-hours worth of work for an expert with no learning curve.

  
Points
21
21 man-hours worth of work for an expert with no learning curve.

  
Points
3
3 man-hours worth of work for an expert with no learning curve.

  
Points
34
34 man-hours worth of work for an expert with no learning curve.

  
Points
5
5 man-hours worth of work for an expert with no learning curve.

  
Points
55
55 man-hours worth of work for an expert with no learning curve.

  
Points
8
8 man-hours worth of work for an expert with no learning curve.

  
Points
88
88 man-hours worth of work for an expert with no learning curve.

  
Priority
Backlog
This ticket has backlogged priority and is not to be worked on yet

  
Priority
Critical
The priority is critical

  
Priority
High
The priority is high

  
Priority
Low
The priority is low

  
Priority
Medium
The priority is medium

  
Signed-off: Owner

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

  
Signed-off: Scrum Master

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

  
Signed-off: Tech Lead

When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.

  
Spike

A ticket for learning a tool or technology that is needed to be able to do future planning and design.

  
State
Completed
The ticket has been fully implemented, completed, and merged with the source code. This label should only be applied once a ticket is closed.

  
State
Duplicate
A ticket that represents the same content as an existing ticket.

  
State
In Progress
A ticket that is actively being developed.

  
State
In Review
A ticket that has had some code completed to implement but is waiting to pass peer review and is not yet merged in.

  
State
Paused
This ticket's work started but wasn't finished. It's on hold (likely in a feature branch) and will be resumed later, either due to a blocker or a delay.

  
State
Unverified
All new tickets start in this state. A developer may set it to show the ticket is unverified. This means we haven't agreed to work on it. It will either move to a verified state or be closed as wontdo.

  
State
Verified
The issue has been verified by a developer as legitimate. It will be worked on and verified tickets are now considered part of the backlog.

  
State
Wont Do
This ticket has been decided it wont be done. This may mean the bug has been determined to not be real (cant verify) or the feature is one we have decided we dont want to adopt.

  
Type
Bug
Something that doesnt work as intended.

  
Type
Discussion
Anytime a ticket represents a discussion about a subject and doesnt fall into one of the other categories.

  
Type
Documentation
An error or improvement needed in the documentation.

  
Type
Epic
Any first tier epic. That is, an epic which contains only issues as children and will not have sub-epics.

  
Type
Feature
Some new functionality not present.

  
Type
Legendary
A type of Epic which will contain other Epics.

  
Type
Support
Someone needs help using the project.

  
Type
Task
A generic task that doesnt fit into the other type categories.

  
Type
Testing
Work exclusively focusing on fixing or expanding testing.

No labels
Blocked
Bounty
$100
Bounty
$1000
Bounty
$10000
Bounty
$20
Bounty
$2000
Bounty
$250
Bounty
$50
Bounty
$500
Bounty
$5000
Bounty
$750
MoSCoW
Could have
MoSCoW
Must have
MoSCoW
Should have
Needs feedback
Points
1
Points
13
Points
2
Points
21
Points
3
Points
34
Points
5
Points
55
Points
8
Points
88
Priority
Backlog
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Signed-off: Owner
Signed-off: Scrum Master
Signed-off: Tech Lead
Spike
State
Completed
State
Duplicate
State
In Progress
State
In Review
State
Paused
State
Unverified
State
Verified
State
Wont Do
Type
Bug
Type
Discussion
Type
Documentation
Type
Epic
Type
Feature
Type
Legendary
Type
Support
Type
Task
Type
Testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
hurui200320
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
#124 Fix RabbitMQ channel dies silently
clevermicro/amq-adapter-python
You do not have permission to read 1 dependency
Reference
clevermicro/amq-adapter-python!125
No description provided.