clevermicro/amq-adapter-python
14
0
Fork 0
Code Issues 16 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Fix backpressure report #60

Closed
hurui200320 wants to merge 1 commits from fix/bp-report into feat-58-backpressure-reactive-streams
pull from: fix/bp-report
merge into: clevermicro:feat-58-backpressure-reactive-streams
Conversation 2 Commits 1 Files Changed 9
+94 -556
hurui200320 commented 2025-07-15 07:47:33 +00:00
Member
Copy Link
Copy Source

During testing the cleverswarm image, I noticed that python adapter has an internal conflict when reporting backpressure:

2025-07-15 07:37:38,367 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 100,\n "currentAvailability": 1.72,\n "requestType": "UPDATE"\n}'
2025-07-15 07:37:28,299 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 100,\n "currentAvailability": 1.6666666666666667,\n "requestType": "UPDATE"\n}'
2025-07-15 07:37:18,229 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 1,\n "currentAvailability": 1,\n "requestType": "IDLE"\n}'
2025-07-15 07:37:08,160 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 100,\n "currentAvailability": 1.65,\n "requestType": "UPDATE"\n}'
2025-07-15 07:36:58,092 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 100,\n "currentAvailability": 1.6533333333333333,\n "requestType": "UPDATE"\n}'

Sometimes it reports the correct bp message with integer value, which is corresponding to the remaining available from the counter. Sometimes it reports the CPU usage as availability in a float form.

This PR aims to fix the issue and also update the reporting message (changes in ticket clevermicro/management-service#14). This PR contains the following changes:

  • Remove the loop for monitoring the CPU usage, now the loop does periodically report of the availability counter with no condition
  • Report availability change when the counter changes, this aligns the latest java adapter implementation (ref: clevermicro/amq-adapter-java#37)
  • Update the reporting message to remove max availability and request type fields
  • Update unit tests, fix or disable the broken ones

This PR is part of the efforts for ticket clevermicro/amq-adapter-python#58 to integrate clevermicro with cleverswarm.

During testing the cleverswarm image, I noticed that python adapter has an internal conflict when reporting backpressure: ``` 2025-07-15 07:37:38,367 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 100,\n "currentAvailability": 1.72,\n "requestType": "UPDATE"\n}' 2025-07-15 07:37:28,299 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 100,\n "currentAvailability": 1.6666666666666667,\n "requestType": "UPDATE"\n}' 2025-07-15 07:37:18,229 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 1,\n "currentAvailability": 1,\n "requestType": "IDLE"\n}' 2025-07-15 07:37:08,160 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 100,\n "currentAvailability": 1.65,\n "requestType": "UPDATE"\n}' 2025-07-15 07:36:58,092 - root - INFO - 139968513025728:0.2.33 [*] Service Message Published to cleverthis.clevermicro.management, msg: b'{\n "version": 1,\n "serviceId": "x21vwlc7vcd5rjbxlex5u82j8",\n "taskId": "9nnss5kkj5wvpxng53ugfviq0",\n "maxAvailability": 100,\n "currentAvailability": 1.6533333333333333,\n "requestType": "UPDATE"\n}' ``` Sometimes it reports the correct bp message with integer value, which is corresponding to the remaining available from the counter. Sometimes it reports the CPU usage as availability in a float form. This PR aims to fix the issue and also update the reporting message (changes in ticket clevermicro/management-service#14). This PR contains the following changes: + Remove the loop for monitoring the CPU usage, now the loop does periodically report of the availability counter with no condition + Report availability change when the counter changes, this aligns the latest java adapter implementation (ref: clevermicro/amq-adapter-java#37) + Update the reporting message to remove max availability and request type fields + Update unit tests, fix or disable the broken ones This PR is part of the efforts for ticket clevermicro/amq-adapter-python#58 to integrate clevermicro with cleverswarm.
hurui200320 self-assigned this 2025-07-15 07:47:33 +00:00
hurui200320 added 1 commit 2025-07-15 07:47:35 +00:00
fix(General): fix availability report
/ build-and-push (push) Successful in 1m21s
Details
Unit test coverage / pytest (push) Failing after 1m27s
Details
Unit test coverage / pytest (pull_request) Failing after 1m17s
Details
45941de6b1 
Remove the loop for monitoring the CPU usage, adopt changes in the management service for the

reporting message format, update the timing for sending out report, similar to java adapter.

ISSUES RELATED: clevermicro/amq-adapter-python#58
hurui200320 requested review from stanislav.hejny 2025-07-15 07:47:58 +00:00
hurui200320 added a new dependency 2025-07-15 07:48:29 +00:00
#58 Modify backpressure to comply with Reactive Streams specs
gitea-actions bot commented 2025-07-15 07:48:50 +00:00
Copy Link
Copy Source

Coverage is 79%

Coverage is 79%
hurui200320 added a new dependency 2025-07-15 07:48:53 +00:00
#59 feat-58-backpressure-reactive-streams
stanislav.hejny commented 2025-07-23 17:46:49 +00:00
Member
Copy Link
Copy Source

I have adopted and applied the mentioned changes individually and selectively to ensure that the intended functionality allowing application to set the backpressure/currently available value and that it will still trigger correct even reporting periodicwally

I have adopted and applied the mentioned changes individually and selectively to ensure that the intended functionality allowing application to set the backpressure/currently available value and that it will still trigger correct even reporting periodicwally
stanislav.hejny closed this pull request 2025-07-23 17:46:50 +00:00
Some checks are pending
/ build-and-push (push) Successful in 1m21s
Details
Unit test coverage / pytest (push) Failing after 1m27s
Details
Unit test coverage / pytest (pull_request) Failing after 1m17s
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
stanislav.hejny
Labels
Clear labels
Blocked
A ticket in a blocked state and unable to complete until some other task is completed first.
Bounty
$100
A bounty of $100 for any open-source contributor who provides a MR that solves this issue
Bounty
$1000
A bounty of $1000 for any open-source contributor who provides a MR that solves this issue
Bounty
$10000
A bounty of $10000 for any open-source contributor who provides a MR that solves this issue
Bounty
$20
A bounty of $20 for any open-source contributor who provides a MR that solves this issue
Bounty
$2000
A bounty of $2000 for any open-source contributor who provides a MR that solves this issue
Bounty
$250
A bounty of $250 for any open-source contributor who provides a MR that solves this issue
Bounty
$50
A bounty of $50 for any open-source contributor who provides a MR that solves this issue
Bounty
$500
A bounty of $500 for any open-source contributor who provides a MR that solves this issue
Bounty
$5000
A bounty of $5000 for any open-source contributor who provides a MR that solves this issue
Bounty
$750
A bounty of $750 for any open-source contributor who provides a MR that solves this issue
MoSCoW
Could have
Could have feature in order to satisfy the epic/legendary.
MoSCoW
Must have
Must have feature in order to satisfy the epic/legendary.
MoSCoW
Should have
Should have feature in order to satisfy the epic/legendary.
Needs feedback
There are questions in the ticket that can not be completed until the project owner provides clarity.
Points
1
1 man-hours worth of work for an expert with no learning curve.
Points
13
13 man-hours worth of work for an expert with no learning curve.
Points
2
2 man-hours worth of work for an expert with no learning curve.
Points
21
21 man-hours worth of work for an expert with no learning curve.
Points
3
3 man-hours worth of work for an expert with no learning curve.
Points
34
34 man-hours worth of work for an expert with no learning curve.
Points
5
5 man-hours worth of work for an expert with no learning curve.
Points
55
55 man-hours worth of work for an expert with no learning curve.
Points
8
8 man-hours worth of work for an expert with no learning curve.
Points
88
88 man-hours worth of work for an expert with no learning curve.
Priority
Backlog
This ticket has backlogged priority and is not to be worked on yet
Priority
Critical
The priority is critical
Priority
High
The priority is high
Priority
Low
The priority is low
Priority
Medium
The priority is medium
Signed-off: Owner
When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.
 Signed-off: Scrum Master
When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.
 Signed-off: Tech Lead
When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.
 Spike
A ticket for learning a tool or technology that is needed to be able to do future planning and design.
State
Completed
The ticket has been fully implemented, completed, and merged with the source code. This label should only be applied once a ticket is closed.
State
Duplicate
A ticket that represents the same content as an existing ticket.
State
In Progress
A ticket that is actively being developed.
State
In Review
A ticket that has had some code completed to implement but is waiting to pass peer review and is not yet merged in.
State
Paused
This ticket's work started but wasn't finished. It's on hold (likely in a feature branch) and will be resumed later, either due to a blocker or a delay.
State
Unverified
All new tickets start in this state. A developer may set it to show the ticket is unverified. This means we haven't agreed to work on it. It will either move to a verified state or be closed as wontdo.
State
Verified
The issue has been verified by a developer as legitimate. It will be worked on and verified tickets are now considered part of the backlog.
State
Wont Do
This ticket has been decided it wont be done. This may mean the bug has been determined to not be real (cant verify) or the feature is one we have decided we dont want to adopt.
Type
Bug
Something that doesnt work as intended.
Type
Discussion
Anytime a ticket represents a discussion about a subject and doesnt fall into one of the other categories.
Type
Documentation
An error or improvement needed in the documentation.
Type
Epic
Any first tier epic. That is, an epic which contains only issues as children and will not have sub-epics.
Type
Feature
Some new functionality not present.
Type
Legendary
A type of Epic which will contain other Epics.
Type
Support
Someone needs help using the project.
Type
Task
A generic task that doesnt fit into the other type categories.
Type
Testing
Work exclusively focusing on fixing or expanding testing.
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
aleenaumair (Aleena Umair) argocd CoreRasurae (Luis Mendes) drew (Drew Morris) eugen.thaci (Eugen Thaci) freemo (Jeffrey Phillips Freeman) hurui200320 (Rui Hu) justin.morris khird (Kyle Hird) mngrif (Michael Griffith) org.clevermicro (System User) org.cleverswarm (System User) org.inf-cd salcedo (Brian Salcedo) stanislav.hejny (Stanislav Hejny)
No Assignees hurui200320
3 Participants
Notifications
Due Date
No due date set.
Blocks
#58 Modify backpressure to comply with Reactive Streams specs
clevermicro/amq-adapter-python
#59 feat-58-backpressure-reactive-streams
clevermicro/amq-adapter-python
Reference: clevermicro/amq-adapter-python#60
Delete Branch "fix/bp-report"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?