5 Commits

Author SHA1 Message Date
hurui200320 2005b8ef82 feat(tests): replace all @skip tags with proper @tdd_expected_fail tags or remove them across the entire codebase (#7221)
## Summary

Replaces all 234 bare `@skip` occurrences across 82 Behave feature files with the correct TDD issue-capture tagging system described in CONTRIBUTING.md § Bug Fix Workflow.

Previously, the noxfile ran Behave with `--tags=not @skip`, silently excluding all `@skip`-tagged scenarios from every CI run. These tests never ran, never inverted results via the `@tdd_expected_fail` mechanism, and never contributed to coverage — defeating the purpose of TDD issue-capture testing. Every `@skip` occurrence had a commented-out hint line immediately above it showing the intended proper tags (e.g., `# @tdd_issue @tdd_issue_4272 @tdd_expected_fail @skip`), confirming they were all intended for conversion.

## Changes

### Mechanical conversion (234 replacements across 82 files)
- Extracted the proper TDD tags from the comment hint above each `@skip` line, removed `@skip` from the tag set, and replaced the `@skip` line with those tags.
- Removed the now-redundant comment hint lines alongside each replacement.

### Bug-fixed scenarios — `@tdd_expected_fail` removed (84 scenarios)
- After conversion, ran `nox -s unit_tests` to identify which newly-enabled `@tdd_expected_fail` scenarios now **pass** (their referenced bugs have already been fixed). Removed `@tdd_expected_fail` from those 84 scenarios and their corresponding feature-level tags, leaving only the permanent `@tdd_issue @tdd_issue_<N>` regression-guard tags.
- Affected features include: `tdd_tool_runner_env_precedence`, `tdd_automation_profile_session_leak`, `tls_certificate_check`, `project_create_persist`, `resource_type_bootstrap_*`, and 18 others.

### Noxfile cleanup
- Removed all four `--tags=not @skip` arguments from `noxfile.py` (unit_tests and coverage sessions). With zero `@skip` tags remaining in the codebase, this filter was dead code and its presence would mislead future maintainers into thinking `@skip` is still a supported escape mechanism.

### Regression guard files
- Split the regression guards into two focused files:
  - `tdd_regression_guards_exec_env.feature` for bug #4281 (exec-env precedence)
  - `tdd_regression_guards_session_list.feature` for bug #4271 (session list summary)
- Each file carries only its own `@tdd_issue` tags at the feature level, avoiding cross-contamination via Behave tag inheritance. The `Background` step (`session-list-summary mock`) only appears in the session-list file where it is actually needed.

### Duplicate tag cleanup
- Removed duplicate `@tdd_issue @tdd_issue_4287` tag lines in `tdd_skill_add_regression.feature` (lines 20 and 29).

### Inline comment for retained `@tdd_expected_fail`
- Added inline comment in `ci_workflow_validation.feature:134` explaining why this specific #4227 scenario retains `@tdd_expected_fail` despite #4227 being closed (CI YAML does not encode threshold as a machine-readable value).

### Known edge cases — `@tdd_expected_fail` retained (closed issues, fix on master, scenarios still fail)
The following issues are **closed** and their fixes **are on master**, but the specific test assertions still fail because the fixes address other aspects of the bugs. The `@tdd_expected_fail` tags are functionally correct and must remain until the specific scenario assertions pass:
- `tdd_exec_env_resolution_precedence.feature` — bug #1080 (closed 2026-03-31). The precedence-level-2-vs-4 scenario still fails.
- `session_list_summary_dedup.feature` — bug #3046 (closed 2026-04-05). The dedup-consistency scenarios still fail.
- `actor_add_update_enforcement.feature` — bug #2609 (closed 2026-04-05). The enforcement scenarios still fail.
- `ci_workflow_validation.feature:134` — #4227 (closed 2026-04-08). The CI YAML threshold assertion still fails.

## Verification

- `grep -r "@skip" features/ --include="*.feature"` → **zero results** ✓
- `grep -n "tags=not @skip" noxfile.py` → **zero results** ✓
- `nox -s unit_tests` → **629 features passed, 0 failed** ✓ (up from ~545 before this PR)
- CI all green (coverage ≥ 97%) ✓
- Integration tests (Robot Framework) do not use `@skip` — confirmed no action needed ✓
- E2E tests (Robot Framework) do not use `@skip` — confirmed no action needed ✓
- `CHANGELOG.md` updated with entry for this change ✓
- `CONTRIBUTORS.md` — Rui Hu already listed ✓

## Issues Addressed

Closes #7025

Co-authored-by: CleverThis <hal9000@cleverthis.com>
Reviewed-on: cleveragents/cleveragents-core#7221
Reviewed-by: HAL 9000 <HAL9000@cleverthis.com>
Reviewed-by: HAL9001 <hal9001@cleverthis.com>
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
2026-04-13 04:56:01 +00:00
freemo 8ea00f5185 fix: restore CI quality tests to passing state (#4175)
Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
2026-04-08 11:02:14 +00:00
freemo 4e3b4180fc feat(ci): add Helm chart lint and template validation to CI
Add kubeconform manifest validation to the existing helm CI job.
The helm job already had helm lint and helm template steps (added in
#1085). This commit completes the optional acceptance criterion from
#1089 by adding kubeconform v0.7.0 to validate rendered manifests
against the Kubernetes 1.29.0 schema in strict mode.

Changes:
- Install kubeconform v0.7.0 in the helm CI job
- Add 'Validate rendered manifests with kubeconform' step after
  helm template smoke render (strict mode, ignore-missing-schemas,
  kubernetes-version 1.29.0)
- Add 5 BDD scenarios to ci_workflow_validation.feature covering:
  - helm job existence
  - helm lint step
  - helm template step
  - kubeconform validation step
  - status-check dependency on helm job

ISSUES CLOSED: #1089
2026-04-02 07:47:09 +00:00
brent.edwards 00897be24a feat(ci): CI/CD pipeline definitions (#983)
## Summary

Complete CI/CD pipeline definitions: release workflow, caching, status-check consolidation, and documentation.

### Changes

**New: Release pipeline** (`.forgejo/workflows/release.yml`):
- Triggered on `v*` tags
- 3 jobs: `build-wheel` → `build-docker` → `create-release`
- Builds wheel via `nox -s build`, Docker image via multi-stage Dockerfile
- Creates Forgejo release via API with wheel artifact attached
- Configurable registry push via `REGISTRY_*` secrets

**Updated: CI pipeline** (`.forgejo/workflows/ci.yml`):
- Added `actions/cache@v3` for `~/.cache/uv` on all 8 primary jobs (keyed on `pyproject.toml` hash)
- Added `status-check` consolidation job depending on all required checks — single gate for branch protection

**Updated: CONTRIBUTING.md**:
- New CI/CD section: pipeline overview, job table, required merge checks, release process, secrets documentation

**Tests**:
- 13 new Behave scenarios validating workflow YAML structure, tag triggers, job definitions, dependencies
- 9 new Robot tests validating file existence, content, nox session references

### Quality Gates

| Session | Result |
|---|---|
| `nox -s lint` | PASS |
| `nox -s typecheck` | PASS (0 errors) |
| `nox -s unit_tests` | PASS (10,819 scenarios) |
| `nox -s coverage_report` | 97.9% (>= 97%) |

Closes #858

Reviewed-on: cleveragents/cleveragents-core#983
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
2026-03-21 00:29:49 +00:00
brent.edwards e801eb1ee8 feat(ci): add nox-based PR validation workflow
- Rewrite .forgejo/workflows/ci.yml to route all jobs through nox sessions
- Fix coverage_report nox session: serial behave mode replaces broken parallel
  mode (22% -> 97% accuracy), raise fail-under from 85% to 97%
- Pass posargs through format nox session for CI --check support
- Add 11 CI workflow validation scenarios (Behave) + Robot smoke test + ASV bench
- Add 108 new Behave scenarios covering 6 largest coverage gaps to reach 97%:
  yaml_template_engine, actor/config, actor/registry, message_router,
  context_analysis, context_service
- Update docs/development/ci-cd.md with nox-based CI docs and 97% threshold
- Restore implementation_plan.md verbose style, check off completed CI tasks

Verified: 1673 scenarios pass, 97% coverage, lint clean, typecheck clean
2026-02-12 22:01:51 +00:00