test(actor): add regression test for issue 4321 nested config extraction #11126

Merged
CoreRasurae merged 1 commit from fix/actor-registry-provider-extraction into master 2026-05-12 13:03:03 +00:00
Member

Summary

  • Add BDD regression test for Forgejo issue #4321: ActorRegistry.add() fails to extract provider and model from nested actor config
  • The test verifies that when type is only at the nested actors.<name> level (not at top level), provider and model are correctly extracted from the nested config block
  • This scenario was already fixed in #4300; the test ensures the fix remains in place and prevents future regressions
  • CHANGELOG entry added under ### Fixed

Changes

  • features/actor_registry_spec_yaml.feature: Added scenario with @tdd_issue @tdd_issue_4321 tags
  • features/steps/actor_registry_spec_yaml_steps.py: Added step definition for new scenario
  • CHANGELOG.md: Added entry documenting the regression test

Testing

  • All 22 scenarios in actor_registry_spec_yaml.feature pass
  • nox -s lint: All checks passed
  • nox -s typecheck: 0 errors, 3 warnings (optional langchain deps)

Closes #4321

## Summary - Add BDD regression test for Forgejo issue #4321: ActorRegistry.add() fails to extract provider and model from nested actor config - The test verifies that when `type` is only at the nested `actors.<name>` level (not at top level), provider and model are correctly extracted from the nested `config` block - This scenario was already fixed in #4300; the test ensures the fix remains in place and prevents future regressions - CHANGELOG entry added under ### Fixed ## Changes - `features/actor_registry_spec_yaml.feature`: Added scenario with `@tdd_issue @tdd_issue_4321` tags - `features/steps/actor_registry_spec_yaml_steps.py`: Added step definition for new scenario - `CHANGELOG.md`: Added entry documenting the regression test ## Testing - All 22 scenarios in `actor_registry_spec_yaml.feature` pass - `nox -s lint`: All checks passed - `nox -s typecheck`: 0 errors, 3 warnings (optional langchain deps) Closes #4321
CoreRasurae added this to the v3.5.0 milestone 2026-05-11 11:29:10 +00:00
CoreRasurae force-pushed fix/actor-registry-provider-extraction from 219f49d741
Some checks failed
CI / benchmark-publish (pull_request) Has been skipped
CI / push-validation (pull_request) Successful in 50s
CI / helm (pull_request) Successful in 52s
CI / build (pull_request) Successful in 1m13s
CI / lint (pull_request) Failing after 1m40s
CI / quality (pull_request) Successful in 1m40s
CI / typecheck (pull_request) Successful in 1m44s
CI / benchmark-regression (pull_request) Failing after 1m14s
CI / security (pull_request) Successful in 2m14s
CI / e2e_tests (pull_request) Failing after 4m22s
CI / integration_tests (pull_request) Successful in 4m36s
CI / unit_tests (pull_request) Successful in 6m16s
CI / coverage (pull_request) Has been skipped
CI / docker (pull_request) Has been skipped
CI / status-check (pull_request) Waiting to run
to 78b6b49974
Some checks failed
CI / benchmark-publish (pull_request) Has been skipped
CI / push-validation (pull_request) Successful in 37s
CI / helm (pull_request) Successful in 47s
CI / build (pull_request) Successful in 55s
CI / lint (pull_request) Successful in 1m2s
CI / benchmark-regression (pull_request) Failing after 1m23s
CI / quality (pull_request) Successful in 1m34s
CI / security (pull_request) Successful in 1m37s
CI / typecheck (pull_request) Successful in 1m38s
CI / integration_tests (pull_request) Successful in 3m28s
CI / e2e_tests (pull_request) Successful in 3m44s
CI / unit_tests (pull_request) Successful in 4m36s
CI / docker (pull_request) Successful in 1m29s
CI / coverage (pull_request) Successful in 10m52s
CI / status-check (pull_request) Successful in 3s
2026-05-11 11:36:52 +00:00
Compare
HAL9001 left a comment

First Review — PR #11126: test(actor): add regression test for issue 4321 nested config extraction

Thank you for the clear PR description and well-structured test. The regression test scenario itself is correctly written and follows the existing patterns in the codebase. All required CI gates pass (lint, typecheck, security, unit_tests, coverage, integration_tests, status-check). However, I have found four blocking issues that must be resolved before this PR can be approved.


Blockers (must be resolved)

1. Commit message does not match the issue Metadata prescription
Issue #4321's ## Metadata section prescribes the exact commit message:

fix(actor-registry): extract provider and model from nested actor LLM config

The actual commit uses:

test(actor): add regression test for issue 4321 nested config extraction

Per CONTRIBUTING.md, the commit first line must be used verbatim from the issue Metadata. The current message also has the wrong type prefix (test vs fix) and a different scope.

2. No companion TDD issue created for bug #4321
Issue #4321 is Type/Bug. CONTRIBUTING.md requires that every Type/Bug issue have a companion Type/Testing issue (with a TDD: title prefix), and the bug issue must depend on the TDD issue (TDD blocks the bug fix). No such companion issue was created for #4321. The current PR adds a test, which is good, but the process requires the TDD issue to exist separately so the dependency chain is explicit and traceable.

3. Wrong branch prefix for the work being done
Branch fix/actor-registry-provider-extraction uses the fix/ prefix. Per CONTRIBUTING.md, fix/mN- branches are for bug fix implementation and tdd/mN- branches are for TDD test-only work. This PR only adds a test with no production code changes. The correct prefix for a test-only regression guard is tdd/m6-actor-registry-provider-extraction.

Note: I acknowledge the issue Metadata says Branch: fix/actor-registry-provider-extraction. If the original intention was to fix the bug AND add the test in a single commit on this branch, but the fix ended up in #4300 instead, the issue Metadata needs to be updated to reflect the current state of the work.

4. Type/Bug issue closed by a test-only PR without verifying all acceptance criteria
Issue #4321 has six acceptance criteria — all of them describe behavioral requirements that the fix should satisfy (provider/model extraction working, actors registerable, plan execution not failing, etc.). This PR only adds a regression test. It relies on the fix from #4300 already being in master. Closing a Type/Bug issue with Closes #4321 on a Type/Testing PR bypasses the acceptance criteria verification process.

To properly close #4321, either:

  • (a) This PR should include the fix (production code changes) AND the test, OR
  • (b) The fix PR (which was #4300 or the original fix/actor-registry-provider-extraction work) should carry Closes #4321, and this test PR should reference a separate TDD issue with a Refs #4321 note only

⚠️ CI Failure — benchmark-regression

The CI / benchmark-regression job is failing (run #20256, job #1). While this check is NOT in the required-for-merge status-check gate, a failing CI job should always be explained. Please confirm whether this failure pre-dates this PR (i.e., was failing on master before this branch was cut) or was introduced by these changes. Given that this PR adds only test/changelog/feature files with no production code changes, this failure is most likely a pre-existing flake, but it should be explicitly acknowledged.


What is correct

  • The test scenario itself is well-written and follows existing patterns (see @tdd_issue_4300 scenario for comparison)
  • YAML construction and step definition are clear and correctly structured
  • @tdd_issue @tdd_issue_4321 tags are used consistently with the codebase convention
  • All required CI gates pass (status-check: )
  • CHANGELOG entry is present and clear
  • Forgejo dependency direction is correct: PR blocks issue
  • Milestone v3.5.0 matches the linked issue
  • Type/Testing label is present
  • Commit footer ISSUES CLOSED: #4321 is present

Summary

This PR is close to correct but has four process violations that must be addressed. The most critical are the missing companion TDD issue (per the required workflow for bug tickets) and the commit message mismatch with the issue Metadata. Please address all four blockers and push a corrected commit.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

## First Review — PR #11126: `test(actor): add regression test for issue 4321 nested config extraction` Thank you for the clear PR description and well-structured test. The regression test scenario itself is correctly written and follows the existing patterns in the codebase. All required CI gates pass (lint, typecheck, security, unit_tests, coverage, integration_tests, status-check). However, I have found **four blocking issues** that must be resolved before this PR can be approved. --- ### ❌ Blockers (must be resolved) **1. Commit message does not match the issue Metadata prescription** Issue #4321's `## Metadata` section prescribes the exact commit message: ``` fix(actor-registry): extract provider and model from nested actor LLM config ``` The actual commit uses: ``` test(actor): add regression test for issue 4321 nested config extraction ``` Per CONTRIBUTING.md, the commit first line **must be used verbatim** from the issue Metadata. The current message also has the wrong `type` prefix (`test` vs `fix`) and a different scope. **2. No companion TDD issue created for bug #4321** Issue #4321 is `Type/Bug`. CONTRIBUTING.md requires that every `Type/Bug` issue have a **companion `Type/Testing` issue** (with a `TDD:` title prefix), and the bug issue must *depend on* the TDD issue (TDD blocks the bug fix). No such companion issue was created for #4321. The current PR adds a test, which is good, but the process requires the TDD issue to exist separately so the dependency chain is explicit and traceable. **3. Wrong branch prefix for the work being done** Branch `fix/actor-registry-provider-extraction` uses the `fix/` prefix. Per CONTRIBUTING.md, `fix/mN-` branches are for **bug fix implementation** and `tdd/mN-` branches are for **TDD test-only work**. This PR only adds a test with no production code changes. The correct prefix for a test-only regression guard is `tdd/m6-actor-registry-provider-extraction`. Note: I acknowledge the issue Metadata says `Branch: fix/actor-registry-provider-extraction`. If the original intention was to fix the bug AND add the test in a single commit on this branch, but the fix ended up in #4300 instead, the issue Metadata needs to be updated to reflect the current state of the work. **4. `Type/Bug` issue closed by a test-only PR without verifying all acceptance criteria** Issue #4321 has six acceptance criteria — all of them describe behavioral requirements that the fix should satisfy (provider/model extraction working, actors registerable, plan execution not failing, etc.). This PR only adds a regression test. It relies on the fix from #4300 already being in master. Closing a `Type/Bug` issue with `Closes #4321` on a `Type/Testing` PR bypasses the acceptance criteria verification process. To properly close #4321, either: - (a) This PR should include the fix (production code changes) AND the test, OR - (b) The fix PR (which was #4300 or the original `fix/actor-registry-provider-extraction` work) should carry `Closes #4321`, and this test PR should reference a separate TDD issue with a `Refs #4321` note only --- ### ⚠️ CI Failure — benchmark-regression The `CI / benchmark-regression` job is failing (run #20256, job #1). While this check is NOT in the required-for-merge `status-check` gate, a failing CI job should always be explained. Please confirm whether this failure pre-dates this PR (i.e., was failing on master before this branch was cut) or was introduced by these changes. Given that this PR adds only test/changelog/feature files with no production code changes, this failure is most likely a pre-existing flake, but it should be explicitly acknowledged. --- ### ✅ What is correct - The test scenario itself is well-written and follows existing patterns (see `@tdd_issue_4300` scenario for comparison) - YAML construction and step definition are clear and correctly structured - `@tdd_issue @tdd_issue_4321` tags are used consistently with the codebase convention - All required CI gates pass (status-check: ✅) - CHANGELOG entry is present and clear - Forgejo dependency direction is correct: PR blocks issue ✅ - Milestone v3.5.0 matches the linked issue ✅ - `Type/Testing` label is present ✅ - Commit footer `ISSUES CLOSED: #4321` is present ✅ --- ### Summary This PR is close to correct but has four process violations that must be addressed. The most critical are the missing companion TDD issue (per the required workflow for bug tickets) and the commit message mismatch with the issue Metadata. Please address all four blockers and push a corrected commit. --- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
Owner

BLOCKER — Wrong branch prefix and missing companion TDD issue

Two related process violations:

  1. Branch name: fix/actor-registry-provider-extraction uses the fix/ prefix which is reserved for bug fix implementation PRs (production code). This PR only adds a test. Per CONTRIBUTING.md, test-only work for a bug regression guard should use a tdd/mN- branch. The correct name would be tdd/m6-actor-registry-provider-extraction.

  2. Missing companion TDD issue: Issue #4321 is Type/Bug. CONTRIBUTING.md requires a companion Type/Testing issue (titled TDD: <same title>) to be created for every bug, and the bug issue must depend on (be blocked by) the TDD issue. This companion issue was never created. Please create the TDD companion issue and link it correctly before this test work can be merged.

Note: The @tdd_issue_4321 tag itself is correct — it correctly identifies this scenario as the regression guard for issue #4321.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

**BLOCKER — Wrong branch prefix and missing companion TDD issue** Two related process violations: 1. **Branch name**: `fix/actor-registry-provider-extraction` uses the `fix/` prefix which is reserved for bug fix implementation PRs (production code). This PR only adds a test. Per CONTRIBUTING.md, test-only work for a bug regression guard should use a `tdd/mN-` branch. The correct name would be `tdd/m6-actor-registry-provider-extraction`. 2. **Missing companion TDD issue**: Issue #4321 is `Type/Bug`. CONTRIBUTING.md requires a companion `Type/Testing` issue (titled `TDD: <same title>`) to be created for every bug, and the bug issue must depend on (be blocked by) the TDD issue. This companion issue was never created. Please create the TDD companion issue and link it correctly before this test work can be merged. Note: The `@tdd_issue_4321` tag itself is correct — it correctly identifies this scenario as the regression guard for issue #4321. --- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
Owner

BLOCKER — Type/Bug issue should not be closed by a test-only PR

Closes #4321 in the PR body will close issue #4321 (Type/Bug) when this PR is merged. However:

  • Issue #4321 has six acceptance criteria that describe the fix (provider/model extraction working correctly, custom actors registerable, plan execution not failing)
  • This PR adds only a regression test, not the fix itself
  • The fix was supposedly already made in #4300, but that PR did not carry Closes #4321

Why this is a problem: Closing a Type/Bug issue via a test-only PR means there is no traceable record that the acceptance criteria were validated as part of the fix work. The proper workflow is:

  • The fix PR (which landed in #4300 or its branch) should carry Closes #4321 with evidence the acceptance criteria were met
  • This test PR should use Refs #4321 only (or reference the separate TDD companion issue)

Please update the PR body to replace Closes #4321 with Refs #4321, and ensure the bug issue is closed by the PR that implements the actual fix.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

**BLOCKER — `Type/Bug` issue should not be closed by a test-only PR** `Closes #4321` in the PR body will close issue #4321 (`Type/Bug`) when this PR is merged. However: - Issue #4321 has six acceptance criteria that describe the *fix* (provider/model extraction working correctly, custom actors registerable, plan execution not failing) - This PR adds only a regression test, not the fix itself - The fix was supposedly already made in #4300, but that PR did not carry `Closes #4321` **Why this is a problem**: Closing a `Type/Bug` issue via a test-only PR means there is no traceable record that the acceptance criteria were validated as part of the fix work. The proper workflow is: - The fix PR (which landed in #4300 or its branch) should carry `Closes #4321` with evidence the acceptance criteria were met - This test PR should use `Refs #4321` only (or reference the separate TDD companion issue) Please update the PR body to replace `Closes #4321` with `Refs #4321`, and ensure the bug issue is closed by the PR that implements the actual fix. --- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
Author
Member

That request cannot be satisfied. the PR #4300 has already been merged to master and cannot be changed anymore. Consider opening an exception for this case, where the current master fails to have a regression test for part of what it implemented. It is better to have a test even if later, than nothing, as it is in master right now.

That request cannot be satisfied. the PR #4300 has already been merged to master and cannot be changed anymore. Consider opening an exception for this case, where the current master fails to have a regression test for part of what it implemented. It is better to have a test even if later, than nothing, as it is in master right now.
Owner

BLOCKER — Commit message mismatch with issue Metadata

Issue #4321 ## Metadata section prescribes this exact commit message:

fix(actor-registry): extract provider and model from nested actor LLM config

The actual commit on this PR uses:

test(actor): add regression test for issue 4321 nested config extraction

Per CONTRIBUTING.md, the commit first line must match verbatim the Commit Message field in the issue Metadata section. Please amend the commit to use the prescribed message exactly.

If the prescribed message no longer accurately describes this PR (since the fix was done in #4300 and this PR only adds the test), then the issue Metadata should first be updated to reflect the actual work, and then the commit message should match that updated Metadata.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

**BLOCKER — Commit message mismatch with issue Metadata** Issue #4321 `## Metadata` section prescribes this exact commit message: ``` fix(actor-registry): extract provider and model from nested actor LLM config ``` The actual commit on this PR uses: ``` test(actor): add regression test for issue 4321 nested config extraction ``` Per CONTRIBUTING.md, the commit first line **must match verbatim** the `Commit Message` field in the issue Metadata section. Please amend the commit to use the prescribed message exactly. If the prescribed message no longer accurately describes this PR (since the fix was done in #4300 and this PR only adds the test), then the issue Metadata should first be updated to reflect the actual work, and then the commit message should match that updated Metadata. --- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
Author
Member

While that is true, this commit does not include code changes, only tests, so it is better to use the more accurate description, maybe issue #4321 metadata should be updated instead, to math what we are actually doing.

While that is true, this commit does not include code changes, only tests, so it is better to use the more accurate description, maybe issue #4321 metadata should be updated instead, to math what we are actually doing.
Owner

Formal review submitted (review ID: 8630) — REQUEST_CHANGES.

Four blocking issues were identified. See the review for full details and inline comments on the specific violations. Once all blockers are addressed, please re-request review.


Automated by CleverAgents Bot
Supervisor: PR Review | Agent: pr-review-worker

Formal review submitted (review ID: 8630) — **REQUEST_CHANGES**. Four blocking issues were identified. See the review for full details and inline comments on the specific violations. Once all blockers are addressed, please re-request review. --- Automated by CleverAgents Bot Supervisor: PR Review | Agent: pr-review-worker
hamza.khyari left a comment

@tdd_issue_4321 the issues is not related to the test

`@tdd_issue_4321` the issues is not related to the test
CoreRasurae force-pushed fix/actor-registry-provider-extraction from 78b6b49974
Some checks failed
CI / benchmark-publish (pull_request) Has been skipped
CI / push-validation (pull_request) Successful in 37s
CI / helm (pull_request) Successful in 47s
CI / build (pull_request) Successful in 55s
CI / lint (pull_request) Successful in 1m2s
CI / benchmark-regression (pull_request) Failing after 1m23s
CI / quality (pull_request) Successful in 1m34s
CI / security (pull_request) Successful in 1m37s
CI / typecheck (pull_request) Successful in 1m38s
CI / integration_tests (pull_request) Successful in 3m28s
CI / e2e_tests (pull_request) Successful in 3m44s
CI / unit_tests (pull_request) Successful in 4m36s
CI / docker (pull_request) Successful in 1m29s
CI / coverage (pull_request) Successful in 10m52s
CI / status-check (pull_request) Successful in 3s
to ad55c3a8e9
Some checks failed
CI / status-check (pull_request) Blocked by required conditions
CI / push-validation (pull_request) Successful in 41s
CI / helm (pull_request) Successful in 51s
CI / lint (pull_request) Successful in 1m6s
CI / build (pull_request) Successful in 1m8s
CI / quality (pull_request) Successful in 1m21s
CI / benchmark-publish (pull_request) Has been skipped
CI / typecheck (pull_request) Successful in 1m45s
CI / security (pull_request) Successful in 1m48s
CI / e2e_tests (pull_request) Successful in 3m45s
CI / integration_tests (pull_request) Successful in 4m17s
CI / unit_tests (pull_request) Successful in 4m59s
CI / benchmark-regression (pull_request) Failing after 1m47s
CI / coverage (pull_request) Has started running
CI / docker (pull_request) Successful in 1m37s
2026-05-12 12:26:13 +00:00
Compare
CoreRasurae force-pushed fix/actor-registry-provider-extraction from ad55c3a8e9
Some checks failed
CI / status-check (pull_request) Blocked by required conditions
CI / push-validation (pull_request) Successful in 41s
CI / helm (pull_request) Successful in 51s
CI / lint (pull_request) Successful in 1m6s
CI / build (pull_request) Successful in 1m8s
CI / quality (pull_request) Successful in 1m21s
CI / benchmark-publish (pull_request) Has been skipped
CI / typecheck (pull_request) Successful in 1m45s
CI / security (pull_request) Successful in 1m48s
CI / e2e_tests (pull_request) Successful in 3m45s
CI / integration_tests (pull_request) Successful in 4m17s
CI / unit_tests (pull_request) Successful in 4m59s
CI / benchmark-regression (pull_request) Failing after 1m47s
CI / coverage (pull_request) Has started running
CI / docker (pull_request) Successful in 1m37s
to f6ca9b7291
All checks were successful
CI / helm (pull_request) Successful in 40s
CI / build (pull_request) Successful in 53s
CI / lint (pull_request) Successful in 1m1s
CI / quality (pull_request) Successful in 1m12s
CI / security (pull_request) Successful in 1m28s
CI / typecheck (pull_request) Successful in 1m40s
CI / integration_tests (pull_request) Successful in 4m23s
CI / unit_tests (pull_request) Successful in 4m54s
CI / push-validation (pull_request) Successful in 22s
CI / docker (pull_request) Successful in 1m34s
CI / coverage (pull_request) Successful in 13m34s
CI / status-check (pull_request) Successful in 4s
2026-05-12 12:43:04 +00:00
Compare
CoreRasurae scheduled this pull request to auto merge when all checks succeed 2026-05-12 12:43:40 +00:00
CoreRasurae deleted branch fix/actor-registry-provider-extraction 2026-05-12 13:03:05 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core!11126
No description provided.