test(runtime): add failing regression test for executor skipping agent-configuration validation (#122) #127

Merged
CoreRasurae merged 1 commit from tdd/m1-executor-validate-configuration into master 2026-08-20 17:48:15 +00:00
Member

Summary

Companion TDD issue-capture PR for #122 ("Executor never calls AgentFactory.validate_configuration(), so malformed agents.<name> entries fail silently"). Adds a failing-first Behave regression scenario in features/credential_executor_validation.feature, proving that Executor.__init__ accepts a malformed agents.<name> entry (missing the required "type" key) without raising a ConfigurationError, in violation of docs/index.md §11 (agent validation MUST be enforced before any agent is instantiated).

The scenario constructs an Executor with a single-node graph route whose only agent entry has no "type" key, executes it, and asserts a ConfigurationError naming the offending agent is raised. Verified locally that removing @tdd_expected_fail makes the assertion fail against current master with AssertionError: Expected error message to contain "Agent 'worker' must specify a type", got 'missing credentials for provider: openai' — confirming the bug is genuinely reproduced (the malformed entry silently defaults to an empty type: llm agent and fails several layers downstream with an unrelated credentials error).

Tagged @tdd_issue @tdd_issue_122 @tdd_expected_fail per this project's TDD bug-fix workflow, so CI's tag-inversion policy treats the expected failure as a pass until the fix lands on a bugfix/ branch for #122.

Verification

  • nox -s unit_tests: green (3001 scenarios passed, including this one via inversion)
  • nox -s integration_tests (Robot): green
  • nox -s e2e_tests: green
  • nox -s coverage_report: 96.7% (>= 96.5% threshold)
  • nox -s lint / nox -s format -- --check: green
  • nox -s typecheck (Pyright strict): green
  • nox -s security_scan (bandit + semgrep) / nox -s dead_code (vulture): green
  • nox -s complexity: green
  • nox -s benchmark_regression (ASV vs origin/master): no regression
  • nox -s docs / nox -s build: green

Closes #124
Refs #122

## Summary Companion TDD issue-capture PR for #122 ("Executor never calls `AgentFactory.validate_configuration()`, so malformed `agents.<name>` entries fail silently"). Adds a failing-first Behave regression scenario in `features/credential_executor_validation.feature`, proving that `Executor.__init__` accepts a malformed `agents.<name>` entry (missing the required `"type"` key) without raising a `ConfigurationError`, in violation of `docs/index.md` §11 (agent validation MUST be enforced before any agent is instantiated). The scenario constructs an `Executor` with a single-node graph route whose only agent entry has no `"type"` key, executes it, and asserts a `ConfigurationError` naming the offending agent is raised. Verified locally that removing `@tdd_expected_fail` makes the assertion fail against current `master` with `AssertionError: Expected error message to contain "Agent 'worker' must specify a type", got 'missing credentials for provider: openai'` — confirming the bug is genuinely reproduced (the malformed entry silently defaults to an empty `type: llm` agent and fails several layers downstream with an unrelated credentials error). Tagged `@tdd_issue @tdd_issue_122 @tdd_expected_fail` per this project's TDD bug-fix workflow, so CI's tag-inversion policy treats the expected failure as a pass until the fix lands on a `bugfix/` branch for #122. ## Verification - `nox -s unit_tests`: green (3001 scenarios passed, including this one via inversion) - `nox -s integration_tests` (Robot): green - `nox -s e2e_tests`: green - `nox -s coverage_report`: 96.7% (>= 96.5% threshold) - `nox -s lint` / `nox -s format -- --check`: green - `nox -s typecheck` (Pyright strict): green - `nox -s security_scan` (bandit + semgrep) / `nox -s dead_code` (vulture): green - `nox -s complexity`: green - `nox -s benchmark_regression` (ASV vs origin/master): no regression - `nox -s docs` / `nox -s build`: green Closes #124 Refs #122
hurui200320 requested changes 2026-08-10 06:54:38 +00:00
Dismissed
hurui200320 left a comment

PR Review: !127 (Ticket #124)

Verdict: Request Changes

The PR adds the required failing Behave regression scenario for #122 and is correctly tagged for the TDD workflow. However, the step that builds the malformed Executor does not capture construction-time exceptions. Because #122's acceptance criteria explicitly state that Executor.__init__ should invoke the validation, a fix that validates in __init__ will make the Given step raise ConfigurationError directly. The @tdd_expected_fail hook only inverts AssertionError failures, so the scenario would fail on the bugfix branch even after the tag is removed. Restructuring the step to capture construction exceptions (or deferring executor creation to the When step) is needed before this TDD capture can safely land.

Critical Issues

None

Major Issues

  1. Given step does not catch construction-time exceptions, making the test fragile for the fix location specified in #122
    • File: features/steps/credential_executor_validation_steps.py, lines 97-131
    • The new step_executor_malformed_agent_missing_type step creates Executor(...) directly and stores it in context.executor. If the eventual fix for #122 validates agent configuration inside Executor.__init__ (as the issue's acceptance criteria state), construction itself raises ConfigurationError. This exception is not an AssertionError, so features/tdd_expected_fail.py will not invert it, and the scenario will fail both with and without the @tdd_expected_fail tag. The established pattern in features/steps/pure_graph_edge_target_validation_steps.py builds the config in Given, then creates the executor and executes in When inside a try/except that records the exception. Adopting that pattern here would make the test robust regardless of whether validation is placed in __init__, create_executor, or the dispatch path.

Minor Issues

  1. Only the missing-type case is covered; non-mapping agent entry is not
    • File: features/credential_executor_validation.feature, lines 32-36
    • Ticket #124's expected behavior requires coverage for an agents.<name> entry that is "not a mapping, or is missing the required 'type' key". The added scenario only exercises the missing-type case. Consider adding a companion scenario for a non-mapping value (e.g., worker: "not a dict") to ensure the fix covers both structural violations required by docs/index.md §11.1.3.

Nits

None

Summary

This PR is narrowly focused and follows the TDD workflow correctly: the scenario is tagged @tdd_issue @tdd_issue_122 @tdd_expected_fail, the assertion fails on current master with an AssertionError (so inversion works), and the steps extend an existing related step file. The major concern is that the test assumes validation will happen during execution rather than construction, which conflicts with #122's stated acceptance criteria. Fixing the step to capture construction exceptions will make the TDD capture usable on any bugfix branch regardless of where the validation is inserted.

## PR Review: !127 (Ticket #124) ### Verdict: Request Changes The PR adds the required failing Behave regression scenario for #122 and is correctly tagged for the TDD workflow. However, the step that builds the malformed `Executor` does not capture construction-time exceptions. Because #122's acceptance criteria explicitly state that `Executor.__init__` should invoke the validation, a fix that validates in `__init__` will make the `Given` step raise `ConfigurationError` directly. The `@tdd_expected_fail` hook only inverts `AssertionError` failures, so the scenario would fail on the bugfix branch even after the tag is removed. Restructuring the step to capture construction exceptions (or deferring executor creation to the `When` step) is needed before this TDD capture can safely land. ### Critical Issues None ### Major Issues 1. **Given step does not catch construction-time exceptions, making the test fragile for the fix location specified in #122** - File: `features/steps/credential_executor_validation_steps.py`, lines 97-131 - The new `step_executor_malformed_agent_missing_type` step creates `Executor(...)` directly and stores it in `context.executor`. If the eventual fix for #122 validates agent configuration inside `Executor.__init__` (as the issue's acceptance criteria state), construction itself raises `ConfigurationError`. This exception is not an `AssertionError`, so `features/tdd_expected_fail.py` will not invert it, and the scenario will fail both with and without the `@tdd_expected_fail` tag. The established pattern in `features/steps/pure_graph_edge_target_validation_steps.py` builds the config in `Given`, then creates the executor and executes in `When` inside a try/except that records the exception. Adopting that pattern here would make the test robust regardless of whether validation is placed in `__init__`, `create_executor`, or the dispatch path. ### Minor Issues 1. **Only the missing-type case is covered; non-mapping agent entry is not** - File: `features/credential_executor_validation.feature`, lines 32-36 - Ticket #124's expected behavior requires coverage for an `agents.<name>` entry that is "not a mapping, or is missing the required 'type' key". The added scenario only exercises the missing-type case. Consider adding a companion scenario for a non-mapping value (e.g., `worker: "not a dict"`) to ensure the fix covers both structural violations required by `docs/index.md` §11.1.3. ### Nits None ### Summary This PR is narrowly focused and follows the TDD workflow correctly: the scenario is tagged `@tdd_issue @tdd_issue_122 @tdd_expected_fail`, the assertion fails on current master with an `AssertionError` (so inversion works), and the steps extend an existing related step file. The major concern is that the test assumes validation will happen during execution rather than construction, which conflicts with #122's stated acceptance criteria. Fixing the step to capture construction exceptions will make the TDD capture usable on any bugfix branch regardless of where the validation is inserted.
CoreRasurae force-pushed tdd/m1-executor-validate-configuration from cce1575f97
Some checks failed
CI / quality (pull_request) Successful in 51s
CI / build (pull_request) Successful in 1m6s
CI / security (pull_request) Successful in 2m2s
CI / typecheck (pull_request) Successful in 2m6s
CI / lint (pull_request) Successful in 2m14s
CI / integration_tests (pull_request) Successful in 3m37s
CI / unit_tests (pull_request) Successful in 4m46s
CI / coverage (pull_request) Successful in 4m57s
CI / status-check (pull_request) Successful in 8s
CI / benchmark (pull_request) Failing after 24m19s
to 2679df70ed
Some checks failed
CI / build (pull_request) Successful in 1m45s
CI / typecheck (pull_request) Successful in 2m6s
CI / security (pull_request) Successful in 2m30s
CI / lint (pull_request) Successful in 2m40s
CI / integration_tests (pull_request) Successful in 3m27s
CI / coverage (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
CI / unit_tests (pull_request) Has been cancelled
CI / benchmark (pull_request) Has been cancelled
CI / quality (pull_request) Has been cancelled
2026-08-10 15:12:22 +00:00
Compare
Author
Member

Thanks for the review — both findings check out against docs/index.md §11.1 and #122/#124, and both are fixed in the amended commit (force-pushed, same TDD-capture commit, not squashed onto anything else):

Major (construction-time exception handling): Confirmed this was a real risk — Executor.__init__ currently never validates, but #122's acceptance criteria require the fix to validate there, so a ConfigurationError would previously be raised straight out of the unguarded Given step and never reach the Then assertion (which is the only place @tdd_expected_fail can invert an AssertionError). Restructured to build the config in Given and construct+execute inside a single try/except in When, mirroring pure_graph_edge_target_validation_steps.py. Verified locally by temporarily adding a validate_configuration() call to Executor.__init__ (simulating the eventual #122 fix, then reverted): the scenario now correctly runs through to the Then assertion and passes, with the TDD tag-policy hook correctly flagging that @tdd_expected_fail should be removed — exactly the intended signal for the future bugfix PR.

Minor (non-mapping agent entry): Added a second scenario for the non-mapping case (docs/index.md §11.1 item 3 requires both violations to be rejected), asserting the AgentFactory.validate_configuration() message "Configuration for agent 'worker' must be a dictionary".

Both new scenarios were confirmed to fail with a genuine AssertionError against current master when run without @tdd_expected_fail, and nox -s unit_tests is green (150 features / 3002 scenarios / 13946 steps).

Thanks for the review — both findings check out against docs/index.md §11.1 and #122/#124, and both are fixed in the amended commit (force-pushed, same TDD-capture commit, not squashed onto anything else): **Major (construction-time exception handling)**: Confirmed this was a real risk — `Executor.__init__` currently never validates, but #122's acceptance criteria require the fix to validate there, so a `ConfigurationError` would previously be raised straight out of the unguarded `Given` step and never reach the `Then` assertion (which is the only place `@tdd_expected_fail` can invert an `AssertionError`). Restructured to build the config in `Given` and construct+execute inside a single try/except in `When`, mirroring `pure_graph_edge_target_validation_steps.py`. Verified locally by temporarily adding a `validate_configuration()` call to `Executor.__init__` (simulating the eventual #122 fix, then reverted): the scenario now correctly runs through to the `Then` assertion and passes, with the TDD tag-policy hook correctly flagging that `@tdd_expected_fail` should be removed — exactly the intended signal for the future bugfix PR. **Minor (non-mapping agent entry)**: Added a second scenario for the non-mapping case (docs/index.md §11.1 item 3 requires both violations to be rejected), asserting the `AgentFactory.validate_configuration()` message `"Configuration for agent 'worker' must be a dictionary"`. Both new scenarios were confirmed to fail with a genuine `AssertionError` against current master when run without `@tdd_expected_fail`, and `nox -s unit_tests` is green (150 features / 3002 scenarios / 13946 steps).
test(runtime): add failing regression test for executor skipping agent-configuration validation (#122)
Some checks failed
CI / lint (pull_request) Successful in 1m2s
CI / quality (pull_request) Successful in 1m2s
CI / build (pull_request) Successful in 1m3s
CI / integration_tests (pull_request) Successful in 2m24s
CI / security (pull_request) Successful in 2m36s
CI / typecheck (pull_request) Successful in 2m42s
CI / unit_tests (pull_request) Successful in 5m22s
CI / benchmark (pull_request) Has been cancelled
CI / coverage (pull_request) Successful in 4m52s
CI / status-check (pull_request) Successful in 9s
b6eac8edeb
Adds failing-first Behave regression scenarios proving issue #122:
Executor.__init__ never calls AgentFactory.validate_configuration(),
so a malformed agents.<name> entry (missing the required "type" key,
or a non-mapping value) is accepted silently instead of being
rejected per docs/index.md §11.1, which requires agent validation to
be enforced before any agent is instantiated or message processed.

Each scenario builds a config for a single-node graph route whose
only agent entry is malformed, then constructs and executes the
Executor in one step, capturing any exception raised during either
construction or execution. This mirrors the established pattern in
pure_graph_edge_target_validation_steps.py and keeps the capture
robust regardless of whether the eventual fix for #122 validates
inside Executor.__init__ (as #122's acceptance criteria require) or
elsewhere in the dispatch path -- avoiding a scenario that would
break outright (rather than correctly flip to passing) once the fix
lands, since @tdd_expected_fail only inverts AssertionError and a
construction-time ConfigurationError would not be one.

Confirmed locally that both assertions fail against current master
when @tdd_expected_fail is removed: the malformed entries instead
default to an empty type: llm agent and fail several layers
downstream with an unrelated "missing credentials for provider" or
attribute error, exactly matching the bug's reported symptom. Tagged
@tdd_issue @tdd_issue_122 @tdd_expected_fail so CI inverts the
expected failure until the fix lands on a bugfix/ branch.

Refs: #122
ISSUES CLOSED: #124
CoreRasurae force-pushed tdd/m1-executor-validate-configuration from 2679df70ed
Some checks failed
CI / build (pull_request) Successful in 1m45s
CI / typecheck (pull_request) Successful in 2m6s
CI / security (pull_request) Successful in 2m30s
CI / lint (pull_request) Successful in 2m40s
CI / integration_tests (pull_request) Successful in 3m27s
CI / coverage (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
CI / unit_tests (pull_request) Has been cancelled
CI / benchmark (pull_request) Has been cancelled
CI / quality (pull_request) Has been cancelled
to b6eac8edeb
Some checks failed
CI / lint (pull_request) Successful in 1m2s
CI / quality (pull_request) Successful in 1m2s
CI / build (pull_request) Successful in 1m3s
CI / integration_tests (pull_request) Successful in 2m24s
CI / security (pull_request) Successful in 2m36s
CI / typecheck (pull_request) Successful in 2m42s
CI / unit_tests (pull_request) Successful in 5m22s
CI / benchmark (pull_request) Has been cancelled
CI / coverage (pull_request) Successful in 4m52s
CI / status-check (pull_request) Successful in 9s
2026-08-10 15:16:00 +00:00
Compare
hurui200320 requested changes 2026-08-10 15:25:25 +00:00
Dismissed
hurui200320 left a comment

PR Review: !127 (Ticket #124)

Verdict: Request Changes

The previous round of feedback has been addressed: construction and execution are now wrapped in the When step, and a non-mapping scenario was added. However, the non-mapping scenario uses a string value, which the codebase now treats as a valid bare package reference. This means it does not actually exercise the structural violation from docs/index.md §11.1.3 and will not produce the asserted error message once Executor is wired to call AgentFactory.validate_configuration(). Fixing the value to a non-dict, non-string type is needed before this TDD capture can land.

Critical Issues

None

Major Issues

  1. Non-mapping scenario uses a string value, which is treated as a valid package reference
    • File: features/steps/credential_executor_validation_steps.py, line 137
    • The value "not-a-mapping" is a non-empty string. Per ADR-2037 and existing tests (features/agent_package_references.feature:111 and features/steps/agent_modules_steps.py:820-831), AgentFactory.validate_configuration() accepts non-empty strings as bare package references. Consequently, this scenario does not exercise the agents.<name> value that is "not a mapping" structural violation from docs/index.md §11.1.3. On current unfixed code it fails with a package-resolution error wrapped as ConfigurationError; after the #122 fix simply calls validate_configuration(), the string would still pass validation and the expected "Configuration for agent 'worker' must be a dictionary" message would not be produced.
    • Recommendation: Change the value to a non-dict, non-string type such as a list (e.g., ["not-a-mapping"]) or integer (e.g., 123), matching the existing pattern in features/steps/agent_modules_steps.py:824.

Minor Issues

None

Nits

None

Summary

This is a narrowly focused TDD capture PR that follows the project's workflow correctly. The missing-type scenario is solid and the construction-time exception handling is now robust. Once the non-mapping scenario's value is changed to a type that is unambiguously rejected by validate_configuration(), both regression captures will accurately document the bug for #122.

## PR Review: !127 (Ticket #124) ### Verdict: Request Changes The previous round of feedback has been addressed: construction and execution are now wrapped in the `When` step, and a non-mapping scenario was added. However, the non-mapping scenario uses a string value, which the codebase now treats as a valid bare package reference. This means it does not actually exercise the structural violation from `docs/index.md` §11.1.3 and will not produce the asserted error message once Executor is wired to call `AgentFactory.validate_configuration()`. Fixing the value to a non-dict, non-string type is needed before this TDD capture can land. ### Critical Issues None ### Major Issues 1. **Non-mapping scenario uses a string value, which is treated as a valid package reference** - File: `features/steps/credential_executor_validation_steps.py`, line 137 - The value `"not-a-mapping"` is a non-empty string. Per ADR-2037 and existing tests (`features/agent_package_references.feature:111` and `features/steps/agent_modules_steps.py:820-831`), `AgentFactory.validate_configuration()` accepts non-empty strings as bare package references. Consequently, this scenario does not exercise the `agents.<name>` value that is "not a mapping" structural violation from `docs/index.md` §11.1.3. On current unfixed code it fails with a package-resolution error wrapped as `ConfigurationError`; after the #122 fix simply calls `validate_configuration()`, the string would still pass validation and the expected `"Configuration for agent 'worker' must be a dictionary"` message would not be produced. - Recommendation: Change the value to a non-dict, non-string type such as a list (e.g., `["not-a-mapping"]`) or integer (e.g., `123`), matching the existing pattern in `features/steps/agent_modules_steps.py:824`. ### Minor Issues None ### Nits None ### Summary This is a narrowly focused TDD capture PR that follows the project's workflow correctly. The missing-type scenario is solid and the construction-time exception handling is now robust. Once the non-mapping scenario's value is changed to a type that is unambiguously rejected by `validate_configuration()`, both regression captures will accurately document the bug for #122.
@ -97,0 +134,4 @@
}
},
"agents": {
agent_name: "not-a-mapping",
Member

This string is treated as a valid bare package reference by AgentFactory.validate_configuration() (ADR-2037). To exercise the agents.<name> "not a mapping" structural violation, use a non-dict, non-string value such as ["not-a-mapping"] or 123, matching the pattern in features/steps/agent_modules_steps.py:824.

This string is treated as a valid bare package reference by `AgentFactory.validate_configuration()` (ADR-2037). To exercise the `agents.<name>` "not a mapping" structural violation, use a non-dict, non-string value such as `["not-a-mapping"]` or `123`, matching the pattern in `features/steps/agent_modules_steps.py:824`.
CoreRasurae force-pushed tdd/m1-executor-validate-configuration from b6eac8edeb
Some checks failed
CI / lint (pull_request) Successful in 1m2s
CI / quality (pull_request) Successful in 1m2s
CI / build (pull_request) Successful in 1m3s
CI / integration_tests (pull_request) Successful in 2m24s
CI / security (pull_request) Successful in 2m36s
CI / typecheck (pull_request) Successful in 2m42s
CI / unit_tests (pull_request) Successful in 5m22s
CI / benchmark (pull_request) Has been cancelled
CI / coverage (pull_request) Successful in 4m52s
CI / status-check (pull_request) Successful in 9s
to dcca897e73
Some checks failed
CI / lint (pull_request) Failing after 7m49s
CI / typecheck (pull_request) Failing after 7m58s
CI / quality (pull_request) Failing after 8m20s
CI / security (pull_request) Failing after 8m24s
CI / integration_tests (pull_request) Failing after 9m17s
CI / benchmark (pull_request) Failing after 15m17s
CI / build (pull_request) Failing after 15m30s
CI / unit_tests (pull_request) Failing after 23m40s
CI / coverage (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
2026-08-15 11:33:25 +00:00
Compare
Author
Member

Thanks for catching this — confirmed against ADR-2037 (D-1) and AgentFactory.validate_configuration()'s current implementation: a non-empty string is accepted as a valid bare package reference, so "not-a-mapping" never exercised the agents.<name> "not a mapping" structural violation from docs/index.md §11.1.3.

Fixed in the amended commit (force-pushed, same TDD-capture commit, not squashed onto anything else): changed the non-mapping scenario's value from the string "not-a-mapping" to 123, matching the existing precedent in features/steps/agent_modules_steps.py:820-824 for exercising this exact "neither dict nor package reference" branch.

Confirmed locally: both scenarios still fail via a genuine AssertionError against current master when run without @tdd_expected_fail (non-mapping case now fails with "Failed to create agent 'worker': 'int' object has no attribute 'get'" instead of the previous unrelated string-resolution error), and nox -s unit_tests scoped to features/credential_executor_validation.feature is green (6 scenarios / 14 steps passed). nox -s lint and nox -s typecheck are also green.

Thanks for catching this — confirmed against ADR-2037 (D-1) and `AgentFactory.validate_configuration()`'s current implementation: a non-empty string is accepted as a valid bare package reference, so `"not-a-mapping"` never exercised the `agents.<name>` "not a mapping" structural violation from `docs/index.md` §11.1.3. Fixed in the amended commit (force-pushed, same TDD-capture commit, not squashed onto anything else): changed the non-mapping scenario's value from the string `"not-a-mapping"` to `123`, matching the existing precedent in `features/steps/agent_modules_steps.py:820-824` for exercising this exact "neither dict nor package reference" branch. Confirmed locally: both scenarios still fail via a genuine `AssertionError` against current master when run without `@tdd_expected_fail` (non-mapping case now fails with `"Failed to create agent 'worker': 'int' object has no attribute 'get'"` instead of the previous unrelated string-resolution error), and `nox -s unit_tests` scoped to `features/credential_executor_validation.feature` is green (6 scenarios / 14 steps passed). `nox -s lint` and `nox -s typecheck` are also green.
hurui200320 left a comment

PR Review: !127 (Ticket #124)

Verdict: Approve

The previous review feedback has been fully addressed. Construction and execution are now wrapped together in the When step, so the regression scenarios capture a ConfigurationError regardless of whether the eventual #122 fix validates agent configuration inside Executor.__init__ or later in the dispatch path. The non-mapping scenario now uses an integer (123) instead of a string, correctly avoiding the bare package-reference form accepted by AgentFactory.validate_configuration(). Both scenarios are appropriately tagged @tdd_issue @tdd_issue_122 @tdd_expected_fail, and their Then steps assert via assert (raising AssertionError only), so the TDD pass/fail inversion policy will work correctly.

Critical Issues

None

Major Issues

None

Minor Issues

None

Nits

None

Summary

This is a focused, well-structured TDD capture PR for #122. It adds two Behave regression scenarios covering both structural violations required by docs/index.md §11.1: an agents.<name> entry missing the required type key, and an entry that is not a mapping at all. The step definitions extend the existing executor-validation step file rather than creating a new one, matching the project’s conventions. Once the fix for #122 lands on its bugfix/ branch, these scenarios should be removed from the @tdd_expected_fail tag set and will then serve as passing regression guards. Approved.

## PR Review: !127 (Ticket #124) ### Verdict: Approve The previous review feedback has been fully addressed. Construction and execution are now wrapped together in the `When` step, so the regression scenarios capture a `ConfigurationError` regardless of whether the eventual #122 fix validates agent configuration inside `Executor.__init__` or later in the dispatch path. The non-mapping scenario now uses an integer (`123`) instead of a string, correctly avoiding the bare package-reference form accepted by `AgentFactory.validate_configuration()`. Both scenarios are appropriately tagged `@tdd_issue @tdd_issue_122 @tdd_expected_fail`, and their `Then` steps assert via `assert` (raising `AssertionError` only), so the TDD pass/fail inversion policy will work correctly. ### Critical Issues None ### Major Issues None ### Minor Issues None ### Nits None ### Summary This is a focused, well-structured TDD capture PR for #122. It adds two Behave regression scenarios covering both structural violations required by `docs/index.md` §11.1: an `agents.<name>` entry missing the required `type` key, and an entry that is not a mapping at all. The step definitions extend the existing executor-validation step file rather than creating a new one, matching the project’s conventions. Once the fix for #122 lands on its `bugfix/` branch, these scenarios should be removed from the `@tdd_expected_fail` tag set and will then serve as passing regression guards. Approved.
CoreRasurae force-pushed tdd/m1-executor-validate-configuration from dcca897e73
Some checks failed
CI / lint (pull_request) Failing after 7m49s
CI / typecheck (pull_request) Failing after 7m58s
CI / quality (pull_request) Failing after 8m20s
CI / security (pull_request) Failing after 8m24s
CI / integration_tests (pull_request) Failing after 9m17s
CI / benchmark (pull_request) Failing after 15m17s
CI / build (pull_request) Failing after 15m30s
CI / unit_tests (pull_request) Failing after 23m40s
CI / coverage (pull_request) Has been cancelled
CI / status-check (pull_request) Has been cancelled
to 052add3dbc
Some checks failed
CI / lint (pull_request) Successful in 2m4s
CI / typecheck (pull_request) Successful in 2m30s
CI / security (pull_request) Successful in 2m6s
CI / benchmark (pull_request) Has started running
CI / quality (pull_request) Successful in 1m5s
CI / build (pull_request) Successful in 1m4s
CI / integration_tests (pull_request) Successful in 2m51s
CI / unit_tests (pull_request) Successful in 4m41s
CI / coverage (pull_request) Successful in 4m55s
CI / status-check (pull_request) Successful in 5s
CI / quality (push) Successful in 51s
CI / typecheck (push) Successful in 1m8s
CI / lint (push) Successful in 1m26s
CI / security (push) Successful in 1m39s
CI / build (push) Successful in 1m49s
CI / integration_tests (push) Successful in 3m23s
CI / unit_tests (push) Successful in 6m15s
CI / coverage (push) Successful in 4m23s
CI / status-check (push) Successful in 8s
CI / benchmark (push) Failing after 22m39s
2026-08-20 17:34:51 +00:00
Compare
CoreRasurae deleted branch tdd/m1-executor-validate-configuration 2026-08-20 17:48:24 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveractors-core!127
No description provided.