TDD: Executor never calls AgentFactory.validate_configuration(), so malformed agents.<name> entries fail silently #124

New issue

Open

opened 2026-08-08 22:18:48 +00:00 by CoreRasurae · 0 comments

CoreRasurae commented

2026-08-08 22:18:48 +00:00

Member

Copy link

Metadata

Commit Message: test(runtime): add failing regression test for executor skipping agent-configuration validation (#122)
Branch: tdd/m1-executor-validate-configuration

Background and context

Companion TDD issue for #122 ("Executor never calls AgentFactory.validate_configuration(), so malformed agents.<name> entries fail silently"). Per this project's mandatory Bug Fix TDD workflow, a failing Behave scenario proving the bug exists must be written and merged to master before any fix is implemented on a bugfix/ branch for #122.

Current behavior

No Behave scenario exists that proves Executor.__init__ (or create_executor) accepts a malformed agents.<name> entry (missing the "type" key, or a non-mapping value) without raising a ConfigurationError before agent creation.

Expected behavior

A Behave scenario exists, tagged @tdd_issue @tdd_issue_122 @tdd_expected_fail, that:

Calls create_executor (or constructs an Executor directly) with a config containing an agents.<name> entry that is not a mapping, or is missing the required "type" key.
Asserts that construction raises a ConfigurationError naming the offending agent, before any credential or provider lookup is attempted.
The assertion currently fails, because the bug is present (no such validation is invoked) — the scenario passes in CI only because the @tdd_expected_fail tag inverts the result, per this project's TDD tag rules.
The failing step raises/asserts via AssertionError only — never ValueError, RuntimeError, or any other exception type — so the TDD CI hook inverts it correctly.

Acceptance criteria

The scenario carries all three required tags: @tdd_issue, @tdd_issue_122, @tdd_expected_fail.
The scenario's failing step raises/asserts via AssertionError only.
Running the scenario's assertion with @tdd_expected_fail removed locally fails against current master, confirming the bug is genuinely reproduced.
nox -s unit_tests is green (CI passes via the @tdd_expected_fail inversion).
Coverage stays >= 97% (nox -s coverage_report).
The scenario is merged to master via its own PR before work begins on #122's fix.

Supporting information

Bug issue: #122.
Relevant code: cleveractors.runtime.Executor.__init__ / create_executor, cleveractors.agents.factory.AgentFactory.validate_configuration, cleveractors.core.application.ReactiveCleverAgentsApp.load_configuration.
Forgejo dependency: #122 depends on this issue (this issue blocks #122), per this project's TDD-before-fix workflow.

Subtasks

Write a Behave scenario (in an existing features/*.feature file covering executor/runtime construction, or a new one only if none exists) that constructs an Executor/create_executor with a malformed agents.<name> entry.
Implement any missing step definitions, extending an existing shared features/steps/*_steps.py file where one already covers executor construction rather than creating a new file.
Tag the scenario @tdd_issue @tdd_issue_122 @tdd_expected_fail.
Confirm the scenario's assertion fails when @tdd_expected_fail is removed locally (proving the bug), and passes CI with the tag in place.
Verify coverage >= 97% via nox -s coverage_report.
Run nox (all default sessions), fix any errors.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details.
The commit is pushed to the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged.
This TDD issue is closed, with #122 depending on it (Forgejo dependency set at creation).

## Metadata - **Commit Message:** `test(runtime): add failing regression test for executor skipping agent-configuration validation (#122)` - **Branch:** `tdd/m1-executor-validate-configuration` ## Background and context Companion TDD issue for #122 ("Executor never calls `AgentFactory.validate_configuration()`, so malformed `agents.<name>` entries fail silently"). Per this project's mandatory Bug Fix TDD workflow, a failing Behave scenario proving the bug exists must be written and merged to `master` before any fix is implemented on a `bugfix/` branch for #122. ## Current behavior No Behave scenario exists that proves `Executor.__init__` (or `create_executor`) accepts a malformed `agents.<name>` entry (missing the `"type"` key, or a non-mapping value) without raising a `ConfigurationError` before agent creation. ## Expected behavior A Behave scenario exists, tagged `@tdd_issue @tdd_issue_122 @tdd_expected_fail`, that: - Calls `create_executor` (or constructs an `Executor` directly) with a config containing an `agents.<name>` entry that is not a mapping, or is missing the required `"type"` key. - Asserts that construction raises a `ConfigurationError` naming the offending agent, before any credential or provider lookup is attempted. - The assertion currently fails, because the bug is present (no such validation is invoked) — the scenario passes in CI only because the `@tdd_expected_fail` tag inverts the result, per this project's TDD tag rules. - The failing step raises/asserts via `AssertionError` only — never `ValueError`, `RuntimeError`, or any other exception type — so the TDD CI hook inverts it correctly. ## Acceptance criteria - [ ] The scenario carries all three required tags: `@tdd_issue`, `@tdd_issue_122`, `@tdd_expected_fail`. - [ ] The scenario's failing step raises/asserts via `AssertionError` only. - [ ] Running the scenario's assertion with `@tdd_expected_fail` removed locally fails against current `master`, confirming the bug is genuinely reproduced. - [ ] `nox -s unit_tests` is green (CI passes via the `@tdd_expected_fail` inversion). - [ ] Coverage stays >= 97% (`nox -s coverage_report`). - [ ] The scenario is merged to `master` via its own PR before work begins on #122's fix. ## Supporting information - Bug issue: #122. - Relevant code: `cleveractors.runtime.Executor.__init__` / `create_executor`, `cleveractors.agents.factory.AgentFactory.validate_configuration`, `cleveractors.core.application.ReactiveCleverAgentsApp.load_configuration`. - Forgejo dependency: #122 depends on this issue (this issue blocks #122), per this project's TDD-before-fix workflow. ## Subtasks - [ ] Write a Behave scenario (in an existing `features/*.feature` file covering executor/runtime construction, or a new one only if none exists) that constructs an `Executor`/`create_executor` with a malformed `agents.<name>` entry. - [ ] Implement any missing step definitions, extending an existing shared `features/steps/*_steps.py` file where one already covers executor construction rather than creating a new file. - [ ] Tag the scenario `@tdd_issue @tdd_issue_122 @tdd_expected_fail`. - [ ] Confirm the scenario's assertion fails when `@tdd_expected_fail` is removed locally (proving the bug), and passes CI with the tag in place. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions), fix any errors. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the first line matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details. - The commit is pushed to the branch matching the Branch in Metadata exactly. - The commit is submitted as a pull request to `master`, reviewed, and merged. - This TDD issue is closed, with #122 depending on it (Forgejo dependency set at creation).

CoreRasurae added the

labels

2026-08-08 22:18:48 +00:00

CoreRasurae added a new dependency