test/cli-docstring-example-validation
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
78be08870c
|
fix(cli): validate actor provider field at correct config nesting level
CI / lint (push) Successful in 1m10s
CI / quality (push) Successful in 1m16s
CI / build (push) Successful in 1m1s
CI / benchmark-regression (push) Has been skipped
CI / typecheck (push) Successful in 1m46s
CI / security (push) Successful in 1m58s
CI / push-validation (push) Successful in 43s
CI / helm (push) Successful in 47s
CI / e2e_tests (push) Successful in 4m9s
CI / integration_tests (push) Successful in 5m9s
CI / unit_tests (push) Successful in 7m45s
CI / docker (push) Successful in 1m49s
CI / coverage (push) Successful in 14m18s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (push) Successful in 1h31m39s
CI / lint (pull_request) Successful in 1m17s
CI / typecheck (pull_request) Successful in 1m30s
CI / benchmark-publish (pull_request) Has been skipped
CI / security (pull_request) Successful in 1m33s
CI / push-validation (pull_request) Successful in 48s
CI / helm (pull_request) Successful in 49s
CI / build (pull_request) Successful in 1m32s
CI / benchmark-regression (pull_request) Failing after 2m1s
CI / quality (pull_request) Successful in 2m9s
CI / e2e_tests (pull_request) Successful in 4m58s
CI / integration_tests (pull_request) Successful in 5m23s
CI / unit_tests (pull_request) Successful in 6m43s
CI / docker (pull_request) Successful in 1m36s
CI / coverage (pull_request) Successful in 10m58s
CI / status-check (pull_request) Successful in 5s
- Actor configuration in V3 is now obtained from the nested configuration parameter, according to the specification. - Removed the legacy V2 fallback support and the tests affected by that removal. Mocked existing steps to allow remaining V2 features to be covered/tested. - Fix add() to pass unsafe flag to add_v3() for proper unsafe actor handling - Add _extract_nested_v3_config() to extract provider/model from nested actors.<name>.config block before v3 schema validation - Remove v2 extraction test scenarios from consolidated_actor.feature that call removed _extract_v2_actor() and _extract_v2_options() methods - Update test YAML in actor_registry_spec_yaml_steps.py to exercise nested type detection (no top-level type field) - Add @tdd_issue_4300 regression test tag to existing spec-compliant actors map scenario ISSUES CLOSED: #4300 |
||
|
|
8dc55655e9 |
feat(actor): make built-in actors virtual, resolved on-demand from provider registry
CI / push-validation (push) Successful in 41s
CI / helm (push) Successful in 42s
CI / benchmark-publish (push) Failing after 56s
CI / build (push) Successful in 1m6s
CI / lint (push) Successful in 1m13s
CI / quality (push) Successful in 1m34s
CI / typecheck (push) Successful in 2m12s
CI / security (push) Successful in 2m13s
CI / e2e_tests (push) Successful in 3m45s
CI / integration_tests (push) Successful in 3m57s
CI / unit_tests (push) Successful in 4m53s
CI / docker (push) Successful in 1m34s
CI / coverage (push) Successful in 11m27s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (pull_request) Has been skipped
CI / helm (pull_request) Successful in 32s
CI / push-validation (pull_request) Successful in 33s
CI / build (pull_request) Successful in 53s
CI / lint (pull_request) Successful in 59s
CI / quality (pull_request) Successful in 1m28s
CI / security (pull_request) Successful in 1m29s
CI / typecheck (pull_request) Successful in 1m32s
CI / e2e_tests (pull_request) Successful in 4m1s
CI / integration_tests (pull_request) Successful in 5m4s
CI / unit_tests (pull_request) Successful in 5m51s
CI / docker (pull_request) Successful in 1m27s
CI / coverage (pull_request) Successful in 11m51s
CI / status-check (pull_request) Successful in 3s
Replace the DB-persistence approach for built-in actors with in-memory virtual resolution. Built-in actors (e.g. openai/gpt-4o, anthropic/claude-sonnet) are now resolved on-demand from ProviderRegistry at query time and merged with persisted custom actors — no database writes occur for built-in actors. Key changes: - Add ActorRegistry._resolve_virtual_builtin_actors(): generates virtual Actor objects in-memory from configured providers (is_built_in=True, id=None) - ActorRegistry.list()/list_actors(): merges virtual built-ins with custom DB actors; custom actors win on name collision; result sorted alphabetically - ActorRegistry.get()/get_actor(): DB-first, virtual built-in fallback, NotFoundError if neither - ActorRegistry.remove()/remove_actor(): rejects virtual built-in names with ValidationError - ActorRegistry.set_default_actor(): stores only the actor name string via new actor_preferences singleton table; no actor row created for virtual built-ins - ActorRegistry.get_default_actor(): reads preference name, resolves via DB→virtual chain, returns actor with is_default=True - Remove ensure_built_in_actors() entirely — 20+ call sites cleaned up including plan.py - Remove ActorRepository.upsert_built_in() — no longer needed - Remove is_built_in from ActorModel DB column (kept on Actor domain model for virtual actors) - New Alembic migration m10_001_virtual_builtin_actors: drops is_built_in column, adds actor_preferences singleton table - Add ActorService.set_default_actor_name() and get_default_actor_name() for preference storage without requiring a DB actor row - Update 15+ Behave step files and 5 feature files; add new features/virtual_builtin_actors.feature with 8 scenarios covering list, show, remove, set-default, get-default, no-DB-writes guarantees - Rewrite tests/actor/test_registry_builtin_yaml.py: TestEnsureBuiltInActorsWithYaml → TestResolveVirtualBuiltinActors plus new TestListActors, TestGetActor, TestRemoveActor, TestDefaultActor test classes Quality gates: lint ✓, typecheck ✓, unit_tests ✓ (15674 scenarios), coverage ✓ (97.10%), integration_tests ✓ (1997 tests) ISSUES CLOSED: #10923 |
||
|
|
bc6a41deb6 |
tdd(cli): prevent actor list from triggering database updates (#1151)
CI / build (push) Successful in 25s
CI / helm (push) Successful in 29s
CI / lint (push) Successful in 3m20s
CI / quality (push) Successful in 3m45s
CI / typecheck (push) Successful in 4m7s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m8s
CI / integration_tests (push) Successful in 9m17s
CI / unit_tests (push) Successful in 9m34s
CI / docker (push) Successful in 1m31s
CI / e2e_tests (push) Successful in 13m53s
CI / coverage (push) Successful in 11m30s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 26m47s
## Summary - Fix bug #797: `agents actor list` no longer triggers database writes (`upsert_actor`, `set_default_actor`) by removing `ensure_built_in_actors()` from `ActorRegistry.list()` and `ActorRegistry.list_actors()` - Include TDD regression tests from #841 with `@tdd_expected_fail` removed per Bug Fix Workflow - Update three existing test suites that relied on the old behavior to call `ensure_built_in_actors()` explicitly ## Root Cause `ActorRegistry.list_actors()` and `ActorRegistry.list()` both unconditionally called `self.ensure_built_in_actors()` before delegating to the actor service. `ensure_built_in_actors()` iterates all configured providers and calls `_actor_service.upsert_actor()` for each — a database WRITE operation. It may also call `_actor_service.set_default_actor()` if no default exists — another WRITE. This means every read-only `agents actor list` command triggered database writes and could prompt for pending migrations on fresh checkouts. ## Changes ### Bug Fix - **`src/cleveragents/actor/registry.py`** — Removed `self.ensure_built_in_actors()` from `list()` and `list_actors()`. Both methods now delegate directly to the service layer without triggering writes. All write-heavy methods (`add`, `upsert_actor`, `get`, `get_actor`, `remove`, `remove_actor`, `set_default_actor`, `get_default_actor`) still call `ensure_built_in_actors()`. ### TDD Tests (from #841, `@tdd_expected_fail` removed) - **`features/tdd_actor_list_no_db_update.feature`** — 2 Behave scenarios verifying `upsert_actor` and `set_default_actor` are not called during `actor list` - **`features/steps/tdd_actor_list_no_db_update_steps.py`** — Step definitions - **`robot/tdd_actor_list_no_db_update.robot`** — 2 Robot Framework integration tests - **`robot/helper_tdd_actor_list_no_db_update.py`** — Robot helper script ### Test Adjustments Three existing test suites relied on the old (buggy) behavior where `list_actors()` called `ensure_built_in_actors()`: 1. **`features/consolidated_actor.feature`** — Scenario updated to explicitly call `ensure_built_in_actors()` before `list_actors()` 2. **`features/steps/tdd_actor_list_validation_steps.py`** + **`robot/helper_tdd_actor_list_validation.py`** (bug #592) — Updated to call `ensure_built_in_actors()` explicitly before CLI invocation 3. **`features/steps/actor_list_empty_steps.py`** (bug #592) — Updated with explicit `ensure_built_in_actors()` call and capturing upsert pattern ## Quality Gates | Gate | Result | |------|--------| | `nox -e lint` | ✅ Pass | | `nox -e typecheck` | ✅ Pass (0 errors) | | `nox -e unit_tests` | ✅ Pass (468 features, 12367 scenarios, 0 failures) | | `nox -e integration_tests` | ⚠️ 6 pre-existing failures (timeouts/OOM) | | `nox -e coverage_report` | ✅ 98% (>= 97% threshold) | The 6 integration test failures are pre-existing infrastructure issues (SIGTERM/SIGKILL timeouts) unrelated to this change: Container Resolve Crash (3), M3 E2E Verification (2), Resource CLI (1). Closes #797 Reviewed-on: #1151 Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com> |
||
|
|
73d5552467 |
fix(actor): handle empty actor list without validation error
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 13s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 18s
CI / security (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 37s
CI / unit_tests (pull_request) Successful in 2m37s
CI / docker (pull_request) Successful in 39s
CI / integration_tests (pull_request) Successful in 4m42s
CI / coverage (pull_request) Successful in 7m0s
CI / benchmark-regression (pull_request) Successful in 32m19s
ActorRegistry._actor_name() built names via f"{provider}/{model}", which
produced names with multiple slashes when providers included models
containing "/" (e.g. OpenRouter's "anthropic/claude-sonnet-4-20250514").
The resulting name violated the spec pattern ^[a-z0-9_-]+/[a-z0-9_-]+$
and triggered a ValidationError during actor upsert.
Now sanitises both provider and model components by replacing "/" with "-"
and lowercasing, so multi-slash provider models no longer break actor
listing.
Includes 6 Behave BDD regression scenarios (covering zero-provider,
multi-slash, consecutive-slash, leading-slash, and name-validation
cases), Robot Framework integration smoke tests, and ASV benchmarks.
ISSUES CLOSED: #592
|
||
|
|
b2e923173f |
perf(tests): consolidate 141 trivially small feature files into 34 domain groups
CI / lint (pull_request) Successful in 18s
CI / typecheck (pull_request) Successful in 32s
CI / quality (pull_request) Successful in 19s
CI / security (pull_request) Successful in 33s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 16s
CI / unit_tests (pull_request) Successful in 2m5s
CI / integration_tests (pull_request) Successful in 2m50s
CI / docker (pull_request) Successful in 15s
CI / coverage (pull_request) Successful in 4m19s
CI / lint (push) Successful in 12s
CI / quality (push) Successful in 18s
CI / security (push) Successful in 30s
CI / build (push) Successful in 22s
CI / typecheck (push) Successful in 58s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m6s
CI / docker (push) Successful in 50s
CI / integration_tests (push) Successful in 3m21s
CI / coverage (push) Successful in 4m12s
CI / benchmark-publish (push) Has been cancelled
CI / benchmark-regression (pull_request) Successful in 24m46s
Consolidate 141 BDD feature files that each complete in under 0.1 seconds into 25 domain-grouped feature files, reducing subprocess count from 339 to ~223. Each consolidated file groups scenarios from the same domain/module that share step definitions and fixtures. All scenarios are preserved with clear comment headers indicating their original source file. This reduces subprocess overhead by ~116 invocations (141 original files replaced by 25 consolidated files), targeting the 42% of subprocess count that contributed only 0.2% of actual test runtime. ISSUES CLOSED: #485 |