Commit Graph

67 Commits

Author SHA1 Message Date
clever-agent dd68f47b0e docs(spec): sixth pass documenting universal label deny rule and permission audit
- Expanded Section 6.19 to document universal forgejo_add_issue_labels deny rule with 2 exceptions
- Added v1.21.0 revision entry documenting 6 agent fixes
- Verified 41 edit:deny and 3 webfetch:allow match spec claims exactly
2026-04-10 16:15:43 -04:00
clever-agent d4bf789989 docs(spec): fifth pass adding permission audit results to revision history
- Added v1.20.0 documenting 3 agent permission bugs fixed
- All 79 agents task permissions verified against spec subagent claims
2026-04-10 16:15:43 -04:00
clever-agent dbd90975c0 docs(spec): fourth pass fixing broken cross-references and stale work types
- Fixed broken Section 20.4 reference (section was removed in v1.16.0)
- Fixed wrong Section 13.2 reference (should be Section 11.2 for escalation)
- Replaced stale stale-check work type with awaiting-review in Section 10.1
- Added v1.19.0 revision entry
2026-04-10 16:15:43 -04:00
clever-agent 0902a4508d docs(spec): third pass fixing Appendix A model and adding revision history
- Fixed build-opencode model in Appendix A from (not specified) to claude-sonnet-4-6
- Added v1.18.0 revision entry documenting 6 functional bugs fixed in agent definitions
2026-04-10 16:15:43 -04:00
clever-agent 2488a34bc8 docs(spec): second pass adding pagination, PR ownership, and reviewer lockdown
- Added Section 6.20: Forgejo API Pagination Requirement
- Added Section 6.21: PR Ownership Detection rules
- Expanded Section 8.1.2: three mandatory PR dispatch rules, updated scoring table
- Added Section 21.22: PR Reviewer total bash lockdown permissions
- Fixed Appendix A note: ASV Benchmarker is subagent not primary
- Added revision history entry v1.17.0
2026-04-10 16:15:43 -04:00
clever-agent d8a31527f3 feat(docs): create agent system specification
CI / push-validation (push) Successful in 17s
CI / helm (push) Successful in 23s
CI / build (push) Successful in 28s
CI / lint (push) Successful in 32s
CI / quality (push) Successful in 33s
CI / typecheck (push) Successful in 54s
CI / security (push) Successful in 1m11s
CI / benchmark-regression (push) Has been skipped
CI / e2e_tests (push) Successful in 3m20s
CI / integration_tests (push) Successful in 4m1s
CI / unit_tests (push) Successful in 5m16s
CI / docker (push) Successful in 1m55s
CI / coverage (push) Successful in 10m19s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 1h13m44s
- Adds a comprehensive specification for the autonomous agent system.
- Details the intended behavior, interactions, and design of all agents.
- Includes Kroki-based diagrams for workflows and interactions.
- Serves as the source of truth for agent implementation.
2026-04-10 14:28:41 -04:00
HAL9000 79359baff3 docs: update CHANGELOG and automation-tracking for recent agent changes
CI / push-validation (pull_request) Successful in 17s
CI / build (pull_request) Successful in 21s
CI / helm (pull_request) Successful in 22s
CI / lint (pull_request) Successful in 33s
CI / quality (pull_request) Successful in 41s
CI / typecheck (pull_request) Successful in 50s
CI / security (pull_request) Successful in 1m12s
CI / e2e_tests (pull_request) Successful in 3m2s
CI / benchmark-publish (pull_request) Has been skipped
CI / integration_tests (pull_request) Successful in 4m2s
CI / unit_tests (pull_request) Successful in 5m10s
CI / docker (pull_request) Successful in 21s
CI / coverage (pull_request) Successful in 10m26s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-regression (pull_request) Successful in 58m15s
ISSUES CLOSED: #7116
2026-04-10 10:32:00 +00:00
clever-agent 97c45ab780 chore(policy): reduce PR review requirement from 2 approvals to 1
CI / helm (push) Successful in 23s
CI / build (push) Successful in 24s
CI / push-validation (push) Successful in 30s
CI / lint (push) Successful in 35s
CI / quality (push) Successful in 42s
CI / typecheck (push) Successful in 53s
CI / security (push) Successful in 59s
CI / e2e_tests (push) Successful in 3m11s
CI / integration_tests (push) Successful in 4m1s
CI / unit_tests (push) Successful in 5m12s
CI / docker (push) Successful in 1m18s
CI / coverage (push) Successful in 11m2s
CI / status-check (push) Successful in 9s
CI / benchmark-publish (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
- Update CONTRIBUTING.md to require only 1 approving review instead of 2
- Allow self-approval including for automated bot PRs (HAL9000)
- Approval can be formal review OR approval comment (LGTM, Approved, )
- Remove distinction between human and bot PRs in review requirements
- Update agent definitions to reflect new policy
- Update system watchdog and documentation to match new requirements

This change unblocks PR merges while maintaining quality through CI checks
and still requiring at least one approval before merge.
2026-04-10 03:28:26 +00:00
clever-agent 8109091bc3 docs: update remaining references to renamed agents
CI / lint (push) Successful in 28s
CI / quality (push) Successful in 33s
CI / push-validation (push) Successful in 21s
CI / build (push) Successful in 23s
CI / helm (push) Successful in 23s
CI / typecheck (push) Successful in 53s
CI / security (push) Successful in 1m12s
CI / benchmark-regression (push) Has been skipped
CI / e2e_tests (push) Successful in 3m0s
CI / integration_tests (push) Successful in 3m58s
CI / unit_tests (push) Successful in 5m3s
CI / docker (push) Successful in 1m20s
CI / coverage (push) Successful in 10m18s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 1h13m36s
- Fix automation-tracking.md to use new pool supervisor names
- Update session_state.md to reference implementation-pool-supervisor
- Fix pr-status-analyzer.md to reference pr-ci-test-fixer
2026-04-09 21:39:32 -04:00
HAL9000 a3762a4cc2 docs: add invariant-reconciliation module guide, extend automation tracking docs, update mkdocs nav
CI / lint (push) Successful in 25s
CI / security (push) Successful in 59s
CI / quality (push) Successful in 45s
CI / typecheck (push) Successful in 1m9s
CI / push-validation (push) Successful in 26s
CI / build (push) Successful in 49s
CI / helm (push) Successful in 42s
CI / e2e_tests (push) Successful in 4m18s
CI / integration_tests (push) Successful in 4m28s
CI / unit_tests (push) Successful in 5m47s
CI / docker (push) Successful in 11s
CI / coverage (push) Successful in 11m48s
CI / status-check (push) Successful in 1s
CI / benchmark-regression (push) Has been skipped
CI / benchmark-publish (push) Has been cancelled
docs: add invariant-reconciliation module guide, extend automation tracking docs, update mkdocs nav

- Add docs/modules/invariant-reconciliation.md: module guide for InvariantReconciliationActor (v3.8.0)
- Extend docs/development/automation-tracking.md: all 16 supervisors, centralized tracking manager
- Update mkdocs.yml: add Modules nav section, Custom Sandbox Strategy to Development

ISSUES CLOSED: #5700

Co-authored-by: CleverThis <hal9000@cleverthis.com>
Co-committed-by: CleverThis <hal9000@cleverthis.com>
2026-04-09 09:22:43 +00:00
freemo 014033eed9 feat: enhance automation tracking with health monitoring and recovery
CI / build (push) Successful in 19s
CI / helm (push) Successful in 32s
CI / push-validation (push) Successful in 17s
CI / lint (push) Failing after 40s
CI / quality (push) Successful in 40s
CI / typecheck (push) Successful in 50s
CI / security (push) Successful in 1m1s
CI / coverage (push) Has been skipped
CI / benchmark-regression (push) Has been skipped
CI / e2e_tests (push) Successful in 3m8s
CI / integration_tests (push) Failing after 4m2s
CI / unit_tests (push) Successful in 5m13s
CI / docker (push) Has been skipped
CI / status-check (push) Failing after 1s
CI / benchmark-publish (push) Has been cancelled
Add comprehensive automated health monitoring and recovery capabilities
to the automation tracking system for proactive agent management.

**Major Enhancements:**

1. **Standardized Interval Reporting**
   - Mandatory interval declaration in all tracking issues
   - Format: 'Reporting Interval: <interval> (Next report expected: <timestamp>)'
   - Enables precise staleness detection and recovery triggering

2. **Automated Health Monitoring (system-watchdog)**
   - New audit_automation_tracking_health() function runs every 5 minutes
   - Monitors all issues with 'Automation Tracking' label
   - Detects stalled agents when >20% overdue from expected interval
   - Calculates staleness ratios and time overdue metrics

3. **Automated Recovery System**
   - Kills stalled agent sessions via OpenCode Server API (port 4096)
   - Performs root cause analysis of session messages and agent definitions
   - Creates high-priority diagnostic issues with detailed findings
   - Automatically closes stale tracking issues with recovery notes
   - Provides human-readable remediation recommendations

**Agent Updates with Standardized Format:**

- **implementation-orchestrator**: Status updates (5 cycles) + health reports (10 cycles)
- **backlog-groomer**: Grooming reports (5 min) + health reports (50 min)
- **human-liaison**: Status updates (20 min monitoring cycles)
- **session-persister**: Event-driven checkpoints with standardized format
- **system-watchdog**: Enhanced with comprehensive recovery capabilities

**Template Standardization:**
- Unified header format across all tracking issues
- Health indicators and next actions sections
- Consistent metadata and automation signatures
- Support for active/warning/error status indicators

**Documentation Updates:**
- Comprehensive automated recovery process documentation
- Agent interval reference table with all timing details
- Recovery issue format and diagnostic workflow
- Health check algorithm and staleness threshold explanation

**Benefits:**
- Proactive detection of crashed or stuck agents (20% staleness threshold)
- Automated recovery reduces manual intervention requirements
- Root cause analysis provides actionable diagnostic information
- Standardized format improves searchability and monitoring
- Comprehensive health metrics enable system-wide visibility

This enhancement transforms the automation tracking system from passive
logging to active health monitoring with automated recovery capabilities.
2026-04-08 22:34:23 +00:00
freemo a323f07783 feat: implement new automation tracking system for agent supervision
CI / push-validation (push) Successful in 16s
CI / build (push) Successful in 19s
CI / helm (push) Successful in 23s
CI / lint (push) Failing after 35s
CI / quality (push) Successful in 43s
CI / security (push) Successful in 52s
CI / typecheck (push) Successful in 1m3s
CI / coverage (push) Has been skipped
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Failing after 4m4s
CI / unit_tests (push) Successful in 4m58s
CI / docker (push) Has been skipped
CI / e2e_tests (push) Successful in 6m24s
CI / status-check (push) Failing after 1s
CI / benchmark-publish (push) Has been cancelled
Replace shared session state issue tracking with individual tracking issues
per agent to reduce noise and improve searchability.

**Agent Updates:**
- session-persister: [AUTO-SESSION] prefix with cycle management
- implementation-orchestrator: [AUTO-IMP-POOL] prefix for health reports
- system-watchdog: [AUTO-WATCHDOG] prefix for system health
- backlog-groomer: [AUTO-GROOMER] prefix + backup cleanup functionality
- human-liaison: [AUTO-LIAISON] prefix for status updates

**New Features:**
- Standardized issue title format: [AUTO-<PREFIX>] <TYPE> (Cycle <N>)
- Announcement format: [AUTO-<PREFIX>] Announce: <message>
- Automatic cleanup to prevent issue accumulation
- Required 'Automation Tracking' label for filtering
- Validation script for format compliance

**Documentation:**
- Complete system documentation at docs/development/automation-tracking.md
- Added to mkdocs.yml navigation
- Validation script at scripts/validate_automation_tracking.py

**Benefits:**
- Reduced noise from shared tracking issue
- Better searchability with agent-specific prefixes
- Cleaner history per agent type
- Easier debugging with focused issue threads
- Automatic cleanup prevents accumulation

Closes automation tracking system implementation requirements.
2026-04-08 21:28:46 +00:00
freemo 194c830f98 fix(ci): resolve repository push failure in CI pipeline
ci.yml / fix(ci): resolve repository push failure in CI pipeline (push) Failing after 0s
Reviewed and APPROVED. Closes CI push failure issue.
2026-04-05 21:25:11 +00:00
freemo c14f2d0a54 docs(development): add system watchdog architecture and audit reference
CI / typecheck (pull_request) Successful in 47s
CI / quality (pull_request) Successful in 34s
CI / security (pull_request) Successful in 58s
CI / lint (pull_request) Successful in 3m24s
CI / build (pull_request) Successful in 19s
CI / helm (pull_request) Successful in 23s
CI / unit_tests (pull_request) Successful in 6m34s
CI / docker (pull_request) Successful in 11s
CI / e2e_tests (pull_request) Successful in 17m49s
CI / coverage (pull_request) Successful in 11m9s
CI / integration_tests (pull_request) Successful in 22m29s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 56m7s
Document the ca-system-watchdog continuous monitoring agent:
- All 12 audit descriptions with severity levels and detection patterns
- Session introspection via OpenCode Server API (Audits 6, 11, 12)
- Finding severity table and action dispatch table
- Health reporting format and configuration parameters
- Constraints and related documentation links

Add System Watchdog entry to mkdocs.yml Development nav section.

Covers commits 8c13e63c and 77427bd7 (system watchdog introduction
and deep session introspection enhancement).
2026-04-05 02:41:16 +00:00
freemo 8c81f13758 fix(infra): resolve TLS handshake failure on git.dev.cleveragents.com
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 18s
CI / helm (pull_request) Successful in 24s
CI / lint (pull_request) Failing after 28s
CI / quality (pull_request) Successful in 35s
CI / security (pull_request) Failing after 48s
CI / typecheck (pull_request) Failing after 50s
CI / coverage (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Has been skipped
CI / unit_tests (pull_request) Failing after 1m46s
CI / docker (pull_request) Has been skipped
CI / e2e_tests (pull_request) Failing after 13m19s
CI / integration_tests (pull_request) Failing after 21m9s
CI / status-check (pull_request) Failing after 1s
The TLS handshake failure on git.dev.cleveragents.com was caused by the
hostname being absent from the certificate's Subject Alternative Names
(SANs), or by SNI virtual-host misconfiguration on the server side.

This commit delivers the repository-side remediation:

- scripts/check-tls-cert.py: New TLS certificate health-check script.
  Connects to a hostname, verifies the certificate's SANs include the
  target hostname, checks expiry, and reports errors/warnings.  Accepts
  an injectable SSLContext for unit testing without real network access.
  Supports wildcard SAN matching and configurable expiry warning threshold.

- docs/development/ops-runbook.md: New ops runbook documenting the full
  certificate renewal procedure (Let's Encrypt/certbot and manual CA),
  SNI misconfiguration diagnosis steps, expiry monitoring with cron, and
  recommended alert thresholds (30/14/7/0 days).

- features/tls_certificate_check.feature: 14 Behave scenarios tagged
  @tdd_issue @tdd_issue_1543 covering: missing SAN detection, valid SAN
  acceptance, expired certificate detection, expiry warning threshold,
  TLS handshake errors, connection timeouts, connection refused, wildcard
  SAN matching, and _hostname_matches_san unit tests.

- features/steps/tls_certificate_check_steps.py: Step definitions for
  the above feature, using unittest.mock to inject SSL contexts and
  socket connections so no real network calls are made.

- mkdocs.yml: Added Ops Runbook to the Development section navigation.

The actual server-side certificate renewal (adding git.dev.cleveragents.com
as a SAN and reloading the web server) must be performed by the server
administrator following the procedure in docs/development/ops-runbook.md.

Closes #1543

ISSUES CLOSED: #1543
2026-04-02 23:59:37 +00:00
freemo 33f1978bd0 fix(ci): resolve repository push failure in CI pipeline
ci.yml / fix(ci): resolve repository push failure in CI pipeline (push) Failing after 0s
ci.yml / fix(ci): resolve repository push failure in CI pipeline (pull_request) Failing after 0s
Root cause: actions/checkout@v4 was not configured with explicit write
credentials (token + persist-credentials), and no git user identity
(user.name/user.email) was set. Both are required for any git push
operation in Forgejo Actions.

Changes:
- release.yml create-release job: add token: secrets.FORGEJO_TOKEN and
  fetch-depth: 0 to checkout; add 'Configure git identity for push
  operations' step using HTTPS credential store; add 'Smoke-test push
  access' step that validates write permission via Forgejo API before
  any push attempt
- ci.yml: add push-validation job that validates push credentials on
  every CI run using FORGEJO_TOKEN, including credential helper
  verification and API-based write permission check; add push-validation
  to status-check needs and result reporting
- docs/development/ci-cd.md: add FORGEJO_TOKEN, FORGEJO_URL, and
  CONTAINER_REGISTRY* secrets to the secrets table; add 'Repository
  Push Authentication' section documenting root cause, fix pattern,
  smoke-test step, setup instructions, and security notes; add
  push-validation to CI job dependency graph and quality gates table

Design decisions:
- HTTPS token authentication (not SSH deploy keys) -- simpler to manage
- ~/.git-credentials with chmod 600 for ephemeral, secure storage
- Smoke-test validates write permission via API before push attempts
- push-validation job is independent (no needs) -- runs in parallel
- No hardcoded credentials -- all secrets via Forgejo Secrets

ISSUES CLOSED: #1541
2026-04-02 23:54:52 +00:00
freemo 48cff5cfe0 refactor(cli): rename plan lifecycle-list and lifecycle-apply to match specification
CI / build (push) Successful in 18s
CI / lint (push) Failing after 31s
CI / helm (push) Successful in 33s
CI / typecheck (push) Successful in 50s
CI / security (push) Failing after 51s
CI / coverage (push) Has been skipped
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Failing after 1m50s
CI / docker (push) Has been skipped
CI / quality (push) Successful in 3m43s
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / status-check (push) Has been cancelled
Renames `plan lifecycle-list` to `plan list` and `plan lifecycle-apply` to `plan apply` to align with the specification's canonical command names. Removes legacy V2 plan commands that occupied those names.

- Renamed CLI command registrations from lifecycle-list/lifecycle-apply to list/apply
- Removed legacy V2 apply and list commands (~200 lines)
- Updated apply shortcut in main.py to delegate to v3 lifecycle
- Added defensive null check for plan existence in apply command
- Updated 63+ test, doc, and benchmark files for consistency

Closes #881

Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
2026-04-02 19:09:04 +00:00
freemo 0c301ac581 fix(test): update facade operation count from 11 to 42
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 44s
CI / lint (pull_request) Successful in 3m19s
CI / unit_tests (pull_request) Failing after 3m42s
CI / quality (pull_request) Successful in 3m48s
CI / typecheck (pull_request) Successful in 4m55s
CI / security (pull_request) Successful in 5m6s
CI / docker (pull_request) Has been skipped
CI / e2e_tests (pull_request) Successful in 5m9s
CI / integration_tests (pull_request) Successful in 6m52s
CI / coverage (pull_request) Failing after 19m27s
CI / benchmark-regression (pull_request) Successful in 48m13s
CI / status-check (pull_request) Failing after 1s
The A2A facade now exposes 42 operations (31 extension + 11 legacy)
after the spec-aligned _cleveragents/ extension methods were added.
Update the BDD assertion and docs to match the actual count.
2026-03-22 03:23:42 +00:00
freemo 2688c85769 feat(extensibility): implement Custom Sandbox Strategy Registration via SandboxStrategy Protocol
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 19s
CI / lint (pull_request) Successful in 23s
CI / quality (pull_request) Successful in 28s
CI / security (pull_request) Successful in 54s
CI / typecheck (pull_request) Successful in 1m25s
CI / e2e_tests (pull_request) Successful in 2m33s
CI / unit_tests (pull_request) Successful in 3m4s
CI / integration_tests (pull_request) Successful in 3m33s
CI / docker (pull_request) Successful in 1m7s
CI / coverage (pull_request) Successful in 5m50s
CI / lint (push) Successful in 18s
CI / build (push) Successful in 33s
CI / quality (push) Successful in 47s
CI / typecheck (push) Successful in 49s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 52s
CI / e2e_tests (push) Successful in 2m9s
CI / unit_tests (push) Successful in 3m32s
CI / integration_tests (push) Successful in 3m41s
CI / docker (push) Successful in 56s
CI / coverage (push) Successful in 6m10s
CI / benchmark-publish (push) Has been cancelled
CI / benchmark-regression (pull_request) Successful in 40m21s
Implement SandboxStrategyProtocol, a 9-method @runtime_checkable Protocol enabling
third-party sandbox strategy registration. Includes:

- SandboxStrategyProtocol with create/read/write/diff/commit/rollback/checkpoint/
  restore_checkpoint/cleanup methods
- SandboxRef (frozen dataclass) and DiffView/DiffEntry (Pydantic models)
- SandboxStrategyRegistry with config-driven registration, Protocol validation,
  thread safety, and clear/list/has operations
- BuiltInSandboxStrategyAdapter wrapping existing Sandbox implementations to conform
  to the new Protocol
- CustomStrategyConfig for YAML/dict-based strategy registration
- SandboxFactory integration with custom_registry parameter,
  has_custom_strategy() and get_custom_strategy_class()
- 25 Behave BDD scenarios (85 steps) covering protocol, registry, adapter, config,
  and factory integration
- 8 Robot Framework integration tests with real filesystem operations
- ASV benchmarks for registry and adapter operations
- Developer documentation

ISSUES CLOSED: #586
2026-03-16 22:50:27 +00:00
freemo ce722ed0ea ops(ci): configure LLM API keys in Forgejo CI for integration test execution
CI / lint (pull_request) Successful in 20s
CI / benchmark-publish (pull_request) Has been skipped
CI / quality (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 40s
CI / e2e_tests (pull_request) Successful in 36s
CI / build (pull_request) Successful in 36s
CI / security (pull_request) Successful in 1m3s
CI / unit_tests (pull_request) Successful in 3m14s
CI / docker (pull_request) Successful in 9s
CI / integration_tests (pull_request) Successful in 3m49s
CI / coverage (pull_request) Successful in 5m54s
CI / lint (push) Successful in 32s
CI / quality (push) Successful in 33s
CI / typecheck (push) Successful in 45s
CI / benchmark-regression (push) Has been skipped
CI / build (push) Successful in 23s
CI / e2e_tests (push) Successful in 1m0s
CI / security (push) Successful in 1m12s
CI / unit_tests (push) Successful in 3m23s
CI / integration_tests (push) Successful in 3m41s
CI / docker (push) Successful in 1m6s
CI / coverage (push) Successful in 6m15s
CI / benchmark-publish (push) Successful in 20m11s
CI / benchmark-regression (pull_request) Successful in 38m40s
Updated CI pipeline to inject ANTHROPIC_API_KEY and OPENAI_API_KEY
secrets as environment variables during Robot Framework integration
test execution. Added CI setup documentation.

ISSUES CLOSED: #701
2026-03-14 22:34:18 +00:00
freemo d5b122d4a3 Docs: Updated to A2A and integrating rest standard
CI / lint (push) Successful in 14s
CI / quality (push) Successful in 21s
CI / security (push) Successful in 36s
CI / build (push) Successful in 36s
CI / typecheck (push) Successful in 39s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m57s
CI / integration_tests (push) Successful in 3m23s
CI / docker (push) Successful in 53s
CI / coverage (push) Successful in 5m58s
CI / benchmark-publish (push) Successful in 19m27s
2026-03-11 13:19:55 -04:00
Brent E. Edwards d88dad94f6 fix(test): resolve race condition in M4 validation integration test
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 25s
CI / quality (pull_request) Successful in 25s
CI / build (pull_request) Successful in 26s
CI / typecheck (pull_request) Successful in 1m9s
CI / security (pull_request) Successful in 1m18s
CI / integration_tests (pull_request) Successful in 3m43s
CI / unit_tests (pull_request) Successful in 4m8s
CI / docker (pull_request) Successful in 43s
CI / coverage (pull_request) Successful in 5m35s
CI / benchmark-regression (pull_request) Successful in 32m44s
Three-pronged fix for intermittent pabot-parallel race condition in
M4 validation integration tests:

1. Composable Setup Database Isolation keyword in common.resource
   gives each suite a unique CLEVERAGENTS_DATABASE_URL so concurrent
   pabot workers never contend on the same SQLite file.

2. Per-suite CLEVERAGENTS_HOME directories prevent shared temp
   directory cleanup from racing between workers.

3. Centralised reset_global_state() in robot/helpers_common.py clears
   Settings singleton, DI container, provider registry, and engine
   cache between chained CLI invocations in helper processes.

Also:
- Setup Test Environment now accepts optional mock_ai and
  auto_apply_migrations arguments (default TRUE) for backward
  compatibility while allowing suites to opt out.
- Added Suite Teardown to cli_plan_context_commands.robot.
- Fixed _COMMANDS typing in two helpers to eliminate type: ignore.
- Updated docs/development/testing.md to reflect helpers_common
  delegation pattern.
- Added timeout=30s to all Run Process calls in
  m4_e2e_verification.robot.

Fixes: #563
2026-03-10 19:04:57 +00:00
freemo 74772280e6 perf(tests): optimize coverage instrumentation and reporting pipeline
Replace coverage.py (sys.settrace-based) with slipcover (bytecode-based
instrumentation) for significantly faster coverage collection:

- Each behave-parallel worker runs under slipcover, producing per-feature
  JSON coverage files with unique UUIDs to avoid write contention.
- After all workers finish, slipcover --merge combines per-worker data
  into a single build/coverage.json report.
- XML report generated via slipcover --merge --xml for CI tooling.
- Terminal report with --fail-under=97 threshold enforcement.
- Robust JSON key-fallback logic handles both slipcover and coverage.py
  output formats.
- CI workflow (ci.yml, nightly-quality.yml) updated with defensive key
  lookup instead of hardcoded coverage.py format.
- Documentation updated to reflect slipcover as the coverage tool.
- CHANGELOG.md updated.

ISSUES CLOSED: #482
2026-03-02 02:01:27 +00:00
brent.edwards 38e75342a9 refactor: merge from master
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / quality (pull_request) Successful in 16s
CI / build (pull_request) Successful in 23s
CI / typecheck (pull_request) Successful in 31s
CI / security (pull_request) Successful in 59s
CI / integration_tests (pull_request) Successful in 3m30s
CI / benchmark-regression (pull_request) Successful in 21m22s
CI / unit_tests (pull_request) Successful in 23m4s
CI / docker (pull_request) Successful in 16s
CI / coverage (pull_request) Successful in 43m38s
This code is a merge from master.
2026-02-28 05:49:26 +00:00
brent.edwards 348c230bc5 test(e2e): add M6 autonomy acceptance suite
CI / lint (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 28s
CI / security (pull_request) Successful in 31s
CI / benchmark-publish (pull_request) Has been skipped
CI / typecheck (pull_request) Successful in 53s
CI / build (pull_request) Successful in 24s
CI / integration_tests (pull_request) Successful in 4m25s
CI / unit_tests (pull_request) Successful in 23m33s
CI / docker (pull_request) Successful in 11s
CI / benchmark-regression (pull_request) Successful in 28m0s
CI / coverage (pull_request) Successful in 48m3s
Add comprehensive M6 autonomy acceptance test suites covering the ACP
local-mode facade, autonomy guardrails, automation profile resolution,
event queue pub/sub, HTTP transport stub, and version negotiation.

Behave suite (52 scenarios):
- ACP facade dispatch for all 11 operations
- Guard enforcement (denylist, allowlist, budget, call limit, write/apply)
- Automation profile built-in validation and custom creation
- Profile resolution precedence (plan > action > project > global)
- Event queue lifecycle (publish, subscribe, unsubscribe, close)
- HTTP transport stub rejection in local mode
- ACP version negotiation (accept/reject)
- Model validation (AcpRequest, AcpResponse, AcpEvent, AcpErrorDetail)

Robot integration suite (11 tests):
- Facade session/plan lifecycle, unknown operation error
- Event queue publish/subscribe, transport stub, version negotiation
- Guard denylist/budget enforcement, profile resolution
- Fixture loading, full end-to-end flow

ASV benchmarks (5 suites):
- Facade dispatch, guard evaluation, profile resolution
- Event queue operations, fixture loading

Fixtures: acp_facade_flows.json, autonomy_guardrails.json,
automation_profiles.json

Closes #211
2026-02-27 20:31:15 +00:00
brent.edwards ece5e61725 test(e2e): add M5 ACMS + context suites
CI / lint (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 21s
CI / security (pull_request) Successful in 40s
CI / typecheck (pull_request) Successful in 41s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 16s
CI / integration_tests (pull_request) Successful in 2m47s
CI / benchmark-regression (pull_request) Successful in 28m30s
CI / unit_tests (pull_request) Failing after 33m23s
CI / docker (pull_request) Has been skipped
CI / coverage (pull_request) Failing after 53m42s
2026-02-27 20:20:13 +00:00
brent.edwards 32b81793a2 refactor(test): rename decision persistence files to avoid conflict with master
Rename our serialization-focused decision persistence suites to
*_serialization to avoid add/add conflicts with the repository-based
decision persistence suites that landed on master independently:

- decision_persistence.feature -> decision_persistence_serialization.feature
- decision_persistence_steps.py -> decision_persistence_serialization_steps.py
- decision_persistence_bench.py -> decision_persistence_serialization_bench.py
- decision_persistence.robot -> decision_persistence_serialization.robot
- helper_decision_persistence.py -> helper_decision_persistence_serialization.py

Updated robot helper path, step docstring, and testing.md references.
2026-02-26 16:13:44 +00:00
brent.edwards 3d3c8fda33 Merge feature/m4-decision-tests into develop-brent-5 2026-02-26 02:15:01 +00:00
brent.edwards b3f126b147 Merge feature/m3-decision-validation-smoke into develop-brent-5 2026-02-26 02:14:14 +00:00
brent.edwards b99510d016 fix(test): derive decision type count from enum instead of hard-coding 11
CI / lint (pull_request) Successful in 21s
CI / typecheck (pull_request) Successful in 55s
CI / security (pull_request) Successful in 51s
CI / quality (pull_request) Successful in 32s
CI / integration_tests (pull_request) Successful in 5m12s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 24s
CI / unit_tests (pull_request) Successful in 33m37s
CI / benchmark-regression (pull_request) Successful in 26m3s
CI / docker (pull_request) Successful in 1m1s
CI / coverage (pull_request) Has been cancelled
Replace hard-coded '11 decision types' with dynamic len(DecisionType) in
the assertion and remove the literal from scenario names, step patterns,
docstrings, and docs so the tests stay correct when new DecisionType
values are added.
2026-02-26 01:19:28 +00:00
brent.edwards 93730067a2 fix(test): address review findings in M3 smoke tests
CI / lint (pull_request) Successful in 24s
CI / typecheck (pull_request) Successful in 55s
CI / security (pull_request) Successful in 47s
CI / quality (pull_request) Successful in 28s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 23s
CI / integration_tests (pull_request) Successful in 5m7s
CI / unit_tests (pull_request) Successful in 20m55s
CI / benchmark-regression (pull_request) Successful in 25m40s
CI / docker (pull_request) Successful in 14s
CI / coverage (pull_request) Successful in 1h48m41s
- Replace dead after_scenario() hook with context.add_cleanup() calls
  so patchers are properly stopped (Behave only runs hooks from
  environment.py)
- Fix invalid ULID constants: pad DECISION/CORRECTION to 26 chars,
  replace invalid 'I' with 'J' in INVARIANT_ULID
- Add temp file cleanup for NamedTemporaryFile(delete=False) via
  context.add_cleanup() in behave steps and try/finally in robot helper
- Remove decision-persistence doc sections from testing.md that belong
  in PR #438, not this PR
2026-02-26 01:13:16 +00:00
brent.edwards c80cda2590 Merge feature/m4-correction-subplan-smoke into develop-brent-5
CI / lint (pull_request) Successful in 23s
CI / typecheck (pull_request) Successful in 1m7s
CI / security (pull_request) Successful in 1m0s
CI / quality (pull_request) Successful in 28s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 25s
CI / integration_tests (pull_request) Successful in 5m50s
CI / unit_tests (pull_request) Failing after 32m24s
CI / docker (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 25m44s
CI / coverage (pull_request) Failing after 1h28m6s
2026-02-25 22:11:33 +00:00
brent.edwards 917c2bc546 test(e2e): add M4 correction + subplan suites 2026-02-25 21:51:26 +00:00
brent.edwards ace7311de4 test(e2e): add M3 decision + validation suites
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 19s
CI / build (pull_request) Successful in 20s
CI / security (pull_request) Successful in 55s
CI / typecheck (pull_request) Successful in 1m5s
CI / integration_tests (pull_request) Successful in 4m4s
CI / benchmark-regression (pull_request) Successful in 22m21s
CI / unit_tests (pull_request) Failing after 34m7s
CI / docker (pull_request) Has been skipped
CI / coverage (pull_request) Has been cancelled
2026-02-25 21:26:19 +00:00
brent.edwards f4c4884d03 test(persistence): add decision persistence suites
CI / lint (pull_request) Successful in 21s
CI / typecheck (pull_request) Successful in 40s
CI / security (pull_request) Successful in 41s
CI / benchmark-publish (pull_request) Has been skipped
CI / quality (pull_request) Successful in 37s
CI / build (pull_request) Successful in 32s
CI / integration_tests (pull_request) Successful in 4m46s
CI / unit_tests (pull_request) Successful in 10m36s
CI / docker (pull_request) Successful in 1m2s
CI / benchmark-regression (pull_request) Successful in 22m26s
CI / coverage (pull_request) Successful in 1h55m50s
Add comprehensive persistence test suites for the Decision domain model
covering serialization round-trips (model_dump/model_validate, JSON),
context-snapshot persistence, correction-chain reconstruction, and
decision-tree reconstruction from serialized data.

Deliverables:
- Behave: 21 scenarios in features/decision_persistence.feature
- Robot Framework: 7 integration tests in robot/decision_persistence.robot
- ASV benchmarks: 4 suites (17 benchmarks) in benchmarks/decision_persistence_bench.py
- Documentation: Updated docs/development/testing.md

ISSUES CLOSED: #180
2026-02-25 21:19:57 +00:00
brent.edwards 3b58432ecc docs(merge): merge new docs into branch
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 19s
CI / security (pull_request) Successful in 31s
CI / typecheck (pull_request) Successful in 36s
CI / integration_tests (pull_request) Failing after 3m10s
CI / benchmark-regression (pull_request) Has been cancelled
CI / unit_tests (pull_request) Has been cancelled
CI / coverage (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
2026-02-25 19:41:56 +00:00
CoreRasurae 57fcc880c3 feat(apply): run validation attachments
ISSUES CLOSED: #176
2026-02-25 10:03:21 +00:00
brent.edwards e230700857 Merge feature/m1-e2e-sourcecode into develop-brent-4 2026-02-25 06:36:45 +00:00
brent.edwards dbc9be9fc0 Merge feature/m2-actor-tool-smoke into develop-brent-4 2026-02-25 06:35:44 +00:00
brent.edwards 0397a00eb6 test(e2e): add M1 source-code plan lifecycle suite
CI / lint (pull_request) Successful in 24s
CI / security (pull_request) Successful in 48s
CI / quality (pull_request) Successful in 30s
CI / benchmark-publish (pull_request) Has been skipped
CI / typecheck (pull_request) Successful in 1m8s
CI / build (pull_request) Successful in 26s
CI / integration_tests (pull_request) Successful in 5m6s
CI / unit_tests (pull_request) Successful in 22m40s
CI / docker (pull_request) Successful in 15s
CI / benchmark-regression (pull_request) Successful in 23m12s
CI / coverage (pull_request) Successful in 31m33s
2026-02-25 04:50:43 +00:00
brent.edwards 9bf87a20d9 chore(pr): address code review feedback for PR #421
CI / lint (pull_request) Successful in 23s
CI / benchmark-publish (pull_request) Has been skipped
CI / quality (pull_request) Successful in 30s
CI / build (pull_request) Successful in 23s
CI / security (pull_request) Successful in 52s
CI / typecheck (pull_request) Successful in 58s
CI / integration_tests (pull_request) Successful in 4m58s
CI / unit_tests (pull_request) Successful in 12m52s
CI / docker (pull_request) Successful in 1m0s
CI / benchmark-regression (pull_request) Successful in 21m33s
CI / coverage (pull_request) Successful in 1h12m45s
- Added CHANGELOG.md entry for M2 E2E test suite (CONTRIBUTING.md §6)
- Added Brent E. Edwards to CONTRIBUTORS.md (CONTRIBUTING.md §8)
- Fixed docs/development/testing.md: corrected M2 Behave scenario count
  from 11 to 10 to match actual feature file
- Added standard explanatory comments to benchmark sys.path setup
  for consistency with project convention (advisory)
- Replaced untyped lambda handler with typed _m2_noop_handler function
  in step definitions (advisory)

All nox stages pass: lint, typecheck, format, unit_tests (280 features /
5846 scenarios / 0 failures), integration_tests (682 passed),
coverage_report (97.2%).

ISSUES CLOSED: #169
2026-02-25 04:13:55 +00:00
brent.edwards 093a74953f test(e2e): add M2 actor + tool source smoke suite
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 16s
CI / lint (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 27s
CI / security (pull_request) Successful in 56s
CI / typecheck (pull_request) Successful in 1m2s
CI / coverage (pull_request) Has been cancelled
CI / unit_tests (pull_request) Has been cancelled
CI / integration_tests (pull_request) Has been cancelled
CI / benchmark-regression (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
Add comprehensive E2E test suite for M2 (Actor Graphs + Tool Sources) epic:

- Behave BDD: 10 scenarios covering actor YAML loading, skill registry,
  tool lifecycle (discover/activate/execute/deactivate), and MCP stub
- Robot Framework: 6 integration tests via CLI helper script
- ASV benchmarks: 12 benchmarks for actor loading, skill registry,
  tool lifecycle, and MCP stub performance baselines
- MCP stub server mock: in-process fake with 3 tools (search/fetch/transform)
- Fixtures: hierarchical graph actor YAML + skill pack with tool refs and
  inline tools
- Docs: updated testing.md with M2 smoke suite section

Closes #169
2026-02-24 21:55:24 +00:00
brent.edwards 2c21611ad5 test(cli): cover action and plan extensions
CI / lint (pull_request) Successful in 25s
CI / quality (pull_request) Successful in 37s
CI / benchmark-publish (pull_request) Has been skipped
CI / security (pull_request) Successful in 55s
CI / typecheck (pull_request) Successful in 1m1s
CI / build (pull_request) Successful in 33s
CI / integration_tests (pull_request) Successful in 5m13s
CI / unit_tests (pull_request) Successful in 12m53s
CI / docker (pull_request) Successful in 9s
CI / benchmark-regression (pull_request) Successful in 21m31s
CI / coverage (pull_request) Successful in 51m7s
Added comprehensive test coverage for CLI extension features from #325:
- Behave scenarios for automation_profile resolution, invariant ordering,
  and actor override error cases
- Output snapshot assertions for JSON/YAML/table formats
- Robot integration test for action show with optional actors/invariants
- ASV benchmark for extended CLI scenario runtime baseline
- Updated testing.md with CLI extension test fixture documentation

ISSUES CLOSED: #326
2026-02-24 20:11:42 +00:00
brent.edwards a181b774c9 Merge branch 'feature/m6-perf-scale' into develop-brent-2
# Conflicts:
#	docs/development/testing.md
2026-02-20 16:31:01 +00:00
brent.edwards 5c323470e2 Merge branch 'feature/m6-validation-edge' into develop-brent-2 2026-02-20 16:30:19 +00:00
brent.edwards 4d41459800 Merge branch 'feature/m6-review-playbook' into develop-brent-2 2026-02-20 16:30:15 +00:00
brent.edwards 2cfa9eafbc Merge branch 'feature/m5-subplan-tests' into develop-brent-2 2026-02-20 16:30:11 +00:00
brent.edwards 06c01a1d7b Merge branch 'feature/m1-persistence-tests-robot' into develop-brent-2 2026-02-20 16:29:11 +00:00
brent.edwards 0dfdc02564 test(perf): add scale test fixtures 2026-02-20 03:31:43 +00:00
brent.edwards 558bcaf586 test(validation): add edge case suites 2026-02-20 02:54:08 +00:00