Commit Graph

4 Commits

Author SHA1 Message Date
freemo 8ea00f5185 fix: restore CI quality tests to passing state (#4175)
CI / unit_tests (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / security (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / build (push) Has been cancelled
CI / push-validation (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / helm (push) Has been cancelled
Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
2026-04-08 11:02:14 +00:00
hurui200320 1878998b7a refactor(testing): rename tdd_bug/tdd_bug_N tags to tdd_issue/tdd_issue_N
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 25s
CI / lint (pull_request) Successful in 3m19s
CI / typecheck (pull_request) Successful in 3m56s
CI / security (pull_request) Successful in 4m6s
CI / quality (pull_request) Successful in 4m10s
CI / integration_tests (pull_request) Successful in 7m20s
CI / unit_tests (pull_request) Successful in 7m44s
CI / docker (pull_request) Successful in 1m8s
CI / e2e_tests (pull_request) Successful in 13m1s
CI / coverage (pull_request) Successful in 12m25s
CI / status-check (pull_request) Successful in 1s
CI / build (push) Successful in 26s
CI / lint (push) Successful in 3m31s
CI / quality (push) Successful in 3m40s
CI / typecheck (push) Successful in 3m57s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m1s
CI / integration_tests (push) Successful in 8m55s
CI / unit_tests (push) Successful in 9m14s
CI / docker (push) Successful in 9s
CI / e2e_tests (push) Successful in 11m42s
CI / coverage (push) Successful in 11m37s
CI / status-check (push) Successful in 4s
CI / benchmark-publish (push) Successful in 29m25s
CI / benchmark-regression (pull_request) Successful in 54m8s
Rename the TDD tag system from tdd_bug/tdd_bug_<N> to tdd_issue/tdd_issue_<N>
across the entire codebase. The tdd_expected_fail tag is unchanged.

The TDD expected-failure workflow is not limited to bug fixes — it applies
equally to any issue type (features, tasks, refactors). The _bug suffix was
misleading and narrowed the perceived scope. The new _issue suffix accurately
reflects that the TDD tagging system applies to any Forgejo issue.

Changes span 92 files:
- features/environment.py: validate_tdd_tags(), should_invert_result(), and
  apply_tdd_inversion() updated — regex, variables, error messages
- robot/tdd_expected_fail_listener.py: _validate_tdd_tags(), _should_invert_result(),
  start_test(), end_test() updated consistently
- 33 Behave .feature files: all @tdd_bug/@tdd_bug_<N> tags renamed
- 29 Robot .robot files: all tdd_bug/tdd_bug_<N> tags renamed
- 3 Robot fixture files renamed (tdd_bug_alone, tdd_missing_tdd_bug,
  tdd_expected_fail_missing_bug_n) with content and references updated
- Tag validation tests and helpers updated (function names, command dispatch
  keys, output strings, fixture references)
- CONTRIBUTING.md: section renamed from 'TDD Bug Test Tags' to
  'TDD Issue Test Tags', all tag references and examples updated
- noxfile.py: comment references updated
- Step definition files, mock helpers, and benchmark files: docstring
  references updated

ISSUES CLOSED: #965
2026-03-27 05:58:35 +00:00
freemo 051ee7c290 test(coverage): add Behave BDD tests to improve coverage across 52 source files
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 31s
CI / typecheck (pull_request) Successful in 47s
CI / security (pull_request) Successful in 52s
CI / build (pull_request) Successful in 56s
CI / e2e_tests (pull_request) Successful in 5m1s
CI / integration_tests (pull_request) Successful in 5m30s
CI / unit_tests (pull_request) Successful in 5m42s
CI / docker (pull_request) Successful in 58s
CI / coverage (pull_request) Successful in 7m35s
CI / build (push) Successful in 21s
CI / docker (push) Has been skipped
CI / benchmark-regression (pull_request) Failing after 49m24s
CI / lint (push) Successful in 22s
CI / quality (push) Successful in 39s
CI / security (push) Successful in 48s
CI / typecheck (push) Successful in 1m26s
CI / benchmark-regression (push) Has been skipped
CI / e2e_tests (push) Successful in 5m53s
CI / coverage (push) Successful in 9m4s
CI / benchmark-publish (push) Successful in 19m10s
CI / integration_tests (push) Failing after 19m18s
CI / unit_tests (push) Failing after 19m20s
Added 52 new .feature files and corresponding _steps.py files targeting
previously uncovered code paths in the following areas:

- TUI layer: app, commands, persona (state/schema/registry), widgets,
  input (shell_exec, reference_parser)
- Application services: plan lifecycle/service/executor, session,
  project, repo indexing, correction, checkpoint, actor, llm_actors,
  strategy coordinator, resource file watcher, service retry wiring
- CLI commands: session, resource, repl, plan, db, automation_profile
- Domain models: retry_policy, resource_type, cost_budget,
  docker_compose_analyzer, detail_level, _sql_string_aware,
  _postgresql_helpers
- Core: circuit_breaker, retry_service_patterns
- Infrastructure: repositories, transaction_sandbox, strategy_registry,
  plugins/loader, container
- Config: settings
- Agents: plan_generation, context_analysis, auto_debug
- A2A: facade

All new tests follow the Behave/Gherkin BDD standard. Resolved step
definition collisions with unique prefixes. Fixed Alembic fileConfig
logger disabling issue (disable_existing_loggers=False).

ISSUES CLOSED: #1068
2026-03-20 21:22:10 +00:00
hurui200320 747d8d3c9a test(cli): TDD failing tests for init --yes non-interactive (bug #783) (#1049)
CI / lint (push) Successful in 25s
CI / build (push) Successful in 23s
CI / typecheck (push) Successful in 44s
CI / benchmark-regression (push) Has been skipped
CI / quality (push) Successful in 48s
CI / security (push) Successful in 1m11s
CI / unit_tests (push) Successful in 3m24s
CI / integration_tests (push) Successful in 3m54s
CI / e2e_tests (push) Successful in 4m7s
CI / docker (push) Successful in 1m0s
CI / coverage (push) Successful in 7m8s
CI / benchmark-publish (push) Has been cancelled
## Summary

Adds TDD bug-capture tests proving that `agents init --yes` fails to bypass the
migration approval prompt in a TTY environment (bug #783). These tests follow the
mandatory TDD bug-fix workflow defined in CONTRIBUTING.md §Bug Fix Workflow.

### Changes

- **Behave feature** (`features/tdd_init_yes_no_input.feature`): Two scenarios
  tagged `@tdd_expected_fail @tdd_bug @tdd_bug_783` that invoke `agents init --yes`
  in a fresh environment with the migration prompt simulated as declining (TTY with
  no input). The tests verify the command exits successfully and does not display the
  "Apply migrations now?" prompt.

- **Behave steps** (`features/steps/tdd_init_yes_no_input_steps.py`): Step definitions
  that create an isolated temp directory, clear auto-apply and database-URL environment
  variables, patch `sys.stdin` on the real `sys` module (correct mock target per project
  convention), and replace `MigrationRunner._default_prompt_for_migration` with a
  function that returns `False` (simulating a TTY prompt where the user declines).
  Temp directory and env var cleanup is registered via `context.add_cleanup()`.

- **Robot Framework test** (`robot/tdd_init_yes_no_input.robot`): Two integration test
  cases tagged `tdd_bug`, `tdd_bug_783`, `tdd_expected_fail` matching the Behave scenarios.

- **Robot helper** (`robot/helper_tdd_init_yes_no_input.py`): Helper script that exercises
  the same code path using the same mock strategy as the Behave steps.

### Bug Reproduction Mechanism

The core challenge is that `CliRunner` replaces `sys.stdin` during `invoke()`, making
direct `isatty()` patching ineffective for controlling the prompt path. The mock strategy
addresses this with a two-pronged approach:

1. **`patch.object(sys, "stdin", mock_stdin)`** — Patches `sys.stdin` directly on the
   real `sys` module (correct mock target per the project's established pattern in
   `features/steps/migration_runner_steps.py`). Documents the intent of simulating a TTY.

2. **`patch.object(MigrationRunner, "_default_prompt_for_migration", ...)`** — Replaces
   the prompt function with one that returns `False` (simulating a TTY user declining
   migration). This is the mechanism that actually exercises the bug path, since
   CliRunner's stdin replacement bypasses the `isatty()` check.

3. **`CLEVERAGENTS_DATABASE_URL` with non-template filename** — Sets the database URL
   to a path whose filename does not match the `before_scenario` template-DB prefixes,
   forcing the real Alembic migration path instead of the test fast-path.

With bug #783 present, `require_confirmation=True` is hardcoded in
`UnitOfWork._ensure_database_initialized()`, so the prompt fires, returns `False`,
causing `MigrationNotApprovedError` and a non-zero exit code. After the fix, `--yes`
should bypass the prompt entirely.

### Root Cause (for the bug fix developer)

`init_command()` in `cleveragents.cli.commands.project` receives the `--yes` flag but
only uses it to control output format. It does not forward `yes` to the migration runner.
The migration runner's `init_or_upgrade()` is called via
`unit_of_work._ensure_database_initialized()` with `require_confirmation=True` hardcoded.

### Scenario 2 Limitation

While the bug is present, Scenario 2's first assertion (exit code 0) fails and Behave
skips subsequent steps. The "contains Initialized" assertion is only evaluated after the
bug is fixed, providing distinct post-fix regression value.

### Quality Gate Results

- `nox -s lint` —  passed
- `nox -s typecheck` —  passed (0 errors)
- `nox -s unit_tests` —  passed (387 features, 11121 scenarios, 0 failed)
- `nox -s integration_tests` —  passed (1561 tests, 0 failed)
- `nox -s e2e_tests` —  passed (16 tests, 0 failed)
- `nox -s coverage_report` —  passed (97% coverage)

Closes #842

Reviewed-on: #1049
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
2026-03-19 06:31:20 +00:00