Commit Graph

4 Commits

Author SHA1 Message Date
freemo 8ea00f5185 fix: restore CI quality tests to passing state (#4175)
Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
2026-04-08 11:02:14 +00:00
hurui200320 1878998b7a refactor(testing): rename tdd_bug/tdd_bug_N tags to tdd_issue/tdd_issue_N
Rename the TDD tag system from tdd_bug/tdd_bug_<N> to tdd_issue/tdd_issue_<N>
across the entire codebase. The tdd_expected_fail tag is unchanged.

The TDD expected-failure workflow is not limited to bug fixes — it applies
equally to any issue type (features, tasks, refactors). The _bug suffix was
misleading and narrowed the perceived scope. The new _issue suffix accurately
reflects that the TDD tagging system applies to any Forgejo issue.

Changes span 92 files:
- features/environment.py: validate_tdd_tags(), should_invert_result(), and
  apply_tdd_inversion() updated — regex, variables, error messages
- robot/tdd_expected_fail_listener.py: _validate_tdd_tags(), _should_invert_result(),
  start_test(), end_test() updated consistently
- 33 Behave .feature files: all @tdd_bug/@tdd_bug_<N> tags renamed
- 29 Robot .robot files: all tdd_bug/tdd_bug_<N> tags renamed
- 3 Robot fixture files renamed (tdd_bug_alone, tdd_missing_tdd_bug,
  tdd_expected_fail_missing_bug_n) with content and references updated
- Tag validation tests and helpers updated (function names, command dispatch
  keys, output strings, fixture references)
- CONTRIBUTING.md: section renamed from 'TDD Bug Test Tags' to
  'TDD Issue Test Tags', all tag references and examples updated
- noxfile.py: comment references updated
- Step definition files, mock helpers, and benchmark files: docstring
  references updated

ISSUES CLOSED: #965
2026-03-27 05:58:35 +00:00
freemo 051ee7c290 test(coverage): add Behave BDD tests to improve coverage across 52 source files
Added 52 new .feature files and corresponding _steps.py files targeting
previously uncovered code paths in the following areas:

- TUI layer: app, commands, persona (state/schema/registry), widgets,
  input (shell_exec, reference_parser)
- Application services: plan lifecycle/service/executor, session,
  project, repo indexing, correction, checkpoint, actor, llm_actors,
  strategy coordinator, resource file watcher, service retry wiring
- CLI commands: session, resource, repl, plan, db, automation_profile
- Domain models: retry_policy, resource_type, cost_budget,
  docker_compose_analyzer, detail_level, _sql_string_aware,
  _postgresql_helpers
- Core: circuit_breaker, retry_service_patterns
- Infrastructure: repositories, transaction_sandbox, strategy_registry,
  plugins/loader, container
- Config: settings
- Agents: plan_generation, context_analysis, auto_debug
- A2A: facade

All new tests follow the Behave/Gherkin BDD standard. Resolved step
definition collisions with unique prefixes. Fixed Alembic fileConfig
logger disabling issue (disable_existing_loggers=False).

ISSUES CLOSED: #1068
2026-03-20 21:22:10 +00:00
hurui200320 747d8d3c9a test(cli): TDD failing tests for init --yes non-interactive (bug #783) (#1049)
## Summary

Adds TDD bug-capture tests proving that `agents init --yes` fails to bypass the
migration approval prompt in a TTY environment (bug #783). These tests follow the
mandatory TDD bug-fix workflow defined in CONTRIBUTING.md §Bug Fix Workflow.

### Changes

- **Behave feature** (`features/tdd_init_yes_no_input.feature`): Two scenarios
  tagged `@tdd_expected_fail @tdd_bug @tdd_bug_783` that invoke `agents init --yes`
  in a fresh environment with the migration prompt simulated as declining (TTY with
  no input). The tests verify the command exits successfully and does not display the
  "Apply migrations now?" prompt.

- **Behave steps** (`features/steps/tdd_init_yes_no_input_steps.py`): Step definitions
  that create an isolated temp directory, clear auto-apply and database-URL environment
  variables, patch `sys.stdin` on the real `sys` module (correct mock target per project
  convention), and replace `MigrationRunner._default_prompt_for_migration` with a
  function that returns `False` (simulating a TTY prompt where the user declines).
  Temp directory and env var cleanup is registered via `context.add_cleanup()`.

- **Robot Framework test** (`robot/tdd_init_yes_no_input.robot`): Two integration test
  cases tagged `tdd_bug`, `tdd_bug_783`, `tdd_expected_fail` matching the Behave scenarios.

- **Robot helper** (`robot/helper_tdd_init_yes_no_input.py`): Helper script that exercises
  the same code path using the same mock strategy as the Behave steps.

### Bug Reproduction Mechanism

The core challenge is that `CliRunner` replaces `sys.stdin` during `invoke()`, making
direct `isatty()` patching ineffective for controlling the prompt path. The mock strategy
addresses this with a two-pronged approach:

1. **`patch.object(sys, "stdin", mock_stdin)`** — Patches `sys.stdin` directly on the
   real `sys` module (correct mock target per the project's established pattern in
   `features/steps/migration_runner_steps.py`). Documents the intent of simulating a TTY.

2. **`patch.object(MigrationRunner, "_default_prompt_for_migration", ...)`** — Replaces
   the prompt function with one that returns `False` (simulating a TTY user declining
   migration). This is the mechanism that actually exercises the bug path, since
   CliRunner's stdin replacement bypasses the `isatty()` check.

3. **`CLEVERAGENTS_DATABASE_URL` with non-template filename** — Sets the database URL
   to a path whose filename does not match the `before_scenario` template-DB prefixes,
   forcing the real Alembic migration path instead of the test fast-path.

With bug #783 present, `require_confirmation=True` is hardcoded in
`UnitOfWork._ensure_database_initialized()`, so the prompt fires, returns `False`,
causing `MigrationNotApprovedError` and a non-zero exit code. After the fix, `--yes`
should bypass the prompt entirely.

### Root Cause (for the bug fix developer)

`init_command()` in `cleveragents.cli.commands.project` receives the `--yes` flag but
only uses it to control output format. It does not forward `yes` to the migration runner.
The migration runner's `init_or_upgrade()` is called via
`unit_of_work._ensure_database_initialized()` with `require_confirmation=True` hardcoded.

### Scenario 2 Limitation

While the bug is present, Scenario 2's first assertion (exit code 0) fails and Behave
skips subsequent steps. The "contains Initialized" assertion is only evaluated after the
bug is fixed, providing distinct post-fix regression value.

### Quality Gate Results

- `nox -s lint` —  passed
- `nox -s typecheck` —  passed (0 errors)
- `nox -s unit_tests` —  passed (387 features, 11121 scenarios, 0 failed)
- `nox -s integration_tests` —  passed (1561 tests, 0 failed)
- `nox -s e2e_tests` —  passed (16 tests, 0 failed)
- `nox -s coverage_report` —  passed (97% coverage)

Closes #842

Reviewed-on: cleveragents/cleveragents-core#1049
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
2026-03-19 06:31:20 +00:00