3 Commits

Author	SHA1	Message	Date
hurui200320	2005b8ef82	feat(tests): replace all @skip tags with proper @tdd_expected_fail tags or remove them across the entire codebase (#7221) ... ## Summary Replaces all 234 bare `@skip` occurrences across 82 Behave feature files with the correct TDD issue-capture tagging system described in CONTRIBUTING.md § Bug Fix Workflow. Previously, the noxfile ran Behave with `--tags=not @skip`, silently excluding all `@skip`-tagged scenarios from every CI run. These tests never ran, never inverted results via the `@tdd_expected_fail` mechanism, and never contributed to coverage — defeating the purpose of TDD issue-capture testing. Every `@skip` occurrence had a commented-out hint line immediately above it showing the intended proper tags (e.g., `# @tdd_issue @tdd_issue_4272 @tdd_expected_fail @skip`), confirming they were all intended for conversion. ## Changes ### Mechanical conversion (234 replacements across 82 files) - Extracted the proper TDD tags from the comment hint above each `@skip` line, removed `@skip` from the tag set, and replaced the `@skip` line with those tags. - Removed the now-redundant comment hint lines alongside each replacement. ### Bug-fixed scenarios — `@tdd_expected_fail` removed (84 scenarios) - After conversion, ran `nox -s unit_tests` to identify which newly-enabled `@tdd_expected_fail` scenarios now **pass** (their referenced bugs have already been fixed). Removed `@tdd_expected_fail` from those 84 scenarios and their corresponding feature-level tags, leaving only the permanent `@tdd_issue @tdd_issue_<N>` regression-guard tags. - Affected features include: `tdd_tool_runner_env_precedence`, `tdd_automation_profile_session_leak`, `tls_certificate_check`, `project_create_persist`, `resource_type_bootstrap_*`, and 18 others. ### Noxfile cleanup - Removed all four `--tags=not @skip` arguments from `noxfile.py` (unit_tests and coverage sessions). With zero `@skip` tags remaining in the codebase, this filter was dead code and its presence would mislead future maintainers into thinking `@skip` is still a supported escape mechanism. ### Regression guard files - Split the regression guards into two focused files: - `tdd_regression_guards_exec_env.feature` for bug #4281 (exec-env precedence) - `tdd_regression_guards_session_list.feature` for bug #4271 (session list summary) - Each file carries only its own `@tdd_issue` tags at the feature level, avoiding cross-contamination via Behave tag inheritance. The `Background` step (`session-list-summary mock`) only appears in the session-list file where it is actually needed. ### Duplicate tag cleanup - Removed duplicate `@tdd_issue @tdd_issue_4287` tag lines in `tdd_skill_add_regression.feature` (lines 20 and 29). ### Inline comment for retained `@tdd_expected_fail` - Added inline comment in `ci_workflow_validation.feature:134` explaining why this specific #4227 scenario retains `@tdd_expected_fail` despite #4227 being closed (CI YAML does not encode threshold as a machine-readable value). ### Known edge cases — `@tdd_expected_fail` retained (closed issues, fix on master, scenarios still fail) The following issues are **closed** and their fixes **are on master**, but the specific test assertions still fail because the fixes address other aspects of the bugs. The `@tdd_expected_fail` tags are functionally correct and must remain until the specific scenario assertions pass: - `tdd_exec_env_resolution_precedence.feature` — bug #1080 (closed 2026-03-31). The precedence-level-2-vs-4 scenario still fails. - `session_list_summary_dedup.feature` — bug #3046 (closed 2026-04-05). The dedup-consistency scenarios still fail. - `actor_add_update_enforcement.feature` — bug #2609 (closed 2026-04-05). The enforcement scenarios still fail. - `ci_workflow_validation.feature:134` — #4227 (closed 2026-04-08). The CI YAML threshold assertion still fails. ## Verification - `grep -r "@skip" features/ --include="*.feature"` → **zero results** ✓ - `grep -n "tags=not @skip" noxfile.py` → **zero results** ✓ - `nox -s unit_tests` → **629 features passed, 0 failed** ✓ (up from ~545 before this PR) - CI all green (coverage ≥ 97%) ✓ - Integration tests (Robot Framework) do not use `@skip` — confirmed no action needed ✓ - E2E tests (Robot Framework) do not use `@skip` — confirmed no action needed ✓ - `CHANGELOG.md` updated with entry for this change ✓ - `CONTRIBUTORS.md` — Rui Hu already listed ✓ ## Issues Addressed Closes #7025 Co-authored-by: CleverThis <hal9000@cleverthis.com> Reviewed-on: #7221 Reviewed-by: HAL 9000 <HAL9000@cleverthis.com> Reviewed-by: HAL9001 <hal9001@cleverthis.com> Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-04-13 04:56:01 +00:00
freemo	8ea00f5185	fix: restore CI quality tests to passing state (#4175) ... Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me> Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>	2026-04-08 11:02:14 +00:00
brent.edwards	efacfd61c3	test: add TDD bug-capture test for #1023 — implicit init requirement (#1113) ... ## Summary Adds TDD bug-capture tests proving that CLI commands (`resource add`, `project create`) fail in a fresh environment without explicit `agents init`, even when `CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true` is set. Bug #1023 reports that `CLEVERAGENTS_AUTO_APPLY_MIGRATIONS=true` triggers schema migrations on an existing database but does NOT create the database file or its parent directory structure, causing `sqlite3.OperationalError: unable to open database file`. ## Changes ### Behave Tests - `features/tdd_e2e_implicit_init.feature` — Two scenarios tagged `@tdd_expected_fail @tdd_bug @tdd_bug_1023`: - `resource add` in a fresh environment without prior init - `project create` in a fresh environment without prior init - Both scenarios include output content assertions - `features/steps/tdd_e2e_implicit_init_steps.py` — Step definitions using Typer's CliRunner with real CLI invocation in a pristine temp directory. - Env isolation: saves/removes `CLEVERAGENTS_AUTO_APPLY_MIGRATIONS`, `CLEVERAGENTS_DATABASE_URL`, `CLEVERAGENTS_TEST_DATABASE_URL`, `CLEVERAGENTS_HOME`, `BEHAVE_TESTING`, `CLEVERAGENTS_TEMPLATE_DB`, `CLEVERAGENTS_TESTING_USE_MOCK_AI` - CliRunner invoked without `catch_exceptions=False` to ensure proper `@tdd_expected_fail` inversion ### Robot Tests - `robot/tdd_e2e_implicit_init.robot` — Two test cases tagged `tdd_bug tdd_bug_1023 tdd_expected_fail` (listener inverts results while bug is unfixed). - `robot/helper_tdd_e2e_implicit_init.py` — Helper script exercising CLI in a fresh environment. - Env isolation includes `ROBOT_TESTING`, `CLEVERAGENTS_TEMPLATE_DB`, `CLEVERAGENTS_TESTING_USE_MOCK_AI` - Env restoration in `finally` block ensures cleanup even on exceptions - CliRunner invoked without `catch_exceptions=False` for correct inversion ### Changelog - Updated `CHANGELOG.md` with entry for this TDD test addition. ## Quality Gates | Gate | Result | |------|--------| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (462 features, 12232 scenarios, 0 failed) | | `nox -s integration_tests` | PASS (pre-existing failures only; our suite correct) | | `nox -s e2e_tests` | N/A (TDD test only, no E2E suite changes) | | `nox -s coverage_report` | PASS (98% coverage, threshold 97%) | Closes #1033 Reviewed-on: #1113 Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>	2026-03-28 00:30:26 +00:00