Commit Graph

288 Commits

Author SHA1 Message Date
brent.edwards da3d0dfd06 chore(ci): re-enable pabot, add discovery resource stub
- Restore pabot-based integration_tests with conservative parallelism
  (<=2 processes by default) and support PABOT_PROCESSES/--processes
- Remove the temporary CI debug dump from noxfile.py
- Add robot/discovery_common.resource to silence non-fatal warning
- Document the change in implementation_plan.md
2026-02-13 04:32:45 +00:00
brent.edwards 6ffba3e2ba fix(ci): guard scientific_paper_basic cleanup when OpenAI key missing
- Avoid deleting the workspace when OPENAI_API_KEY is unset
  by guarding cleanup and initializing CONTEXT_DIR
- Prevents downstream Robot suites from losing .nox venv and
  resource files after the basic scientific paper suite is skipped
2026-02-13 04:15:14 +00:00
brent.edwards 51a6e76e9e fix(ci): merge duplicate Settings blocks, inject venv Python, add CI debug
Three fixes targeting CI integration_tests failures (18 failures on e8aa5ac):

1. Merge duplicate *** Settings *** blocks in 14 robot files into single
   blocks. Multiple Settings sections are non-standard RF practice and
   may cause resource import failures in certain Robot Framework versions
   or CI environments.

2. Replace bare 'python' with ${PYTHON} variable in all Run Process
   calls (14 files). Noxfile now passes --variable PYTHON:<venv-path>
   to robot so tests use the venv interpreter regardless of PATH. This
   fixes '/usr/local/bin/python: No module named cleveragents' on CI.

3. Add comprehensive CI debug output in noxfile.py: file existence
   checks for .resource files, PATH/Python resolution, fixture dir
   checks, and RF version. This will diagnose any remaining resource
   import issues.

Also: remove hardcoded '/app/src' sys.path.insert in
system_prompt_template_rendering.robot (not portable to CI), and
add trailing newline to common.resource.

All 204 tests pass locally (4 excluded: 2 slow, 2 discovery).
2026-02-13 03:43:44 +00:00
brent.edwards e8aa5ac268 fix(ci): restore venv PATH, use absolute resource paths, and add timeouts in robot tests
- Restore session.env["PATH"] in integration_tests nox session to ensure
  Run Process uses venv Python instead of system Python
- Convert bare Resource references to ${CURDIR}/ absolute paths across
  30 robot files to fix CI resolution failures
- Add timeout=30s to all Run Process calls in rxpy_route_validation.robot
  to prevent hanging tests
- Tag 2 rxpy tests as slow (require running actors unavailable on CI)
- Fix LangGraph test to use correct config file (LANGGRAPH_CONFIG)
- All 204 tests pass (4 excluded: 2 slow + 2 discovery)
2026-02-13 02:42:57 +00:00
brent.edwards 0059050836 fix(ci): replace pabot with robot for reliable CI execution
- integration_tests: replace pabot with sequential robot execution to
  eliminate FileNotFoundError caused by subprocess/FD exhaustion in
  constrained CI containers (pabot spawns a robot subprocess per suite;
  after ~24 suites the container cannot execve new processes)

- integration_tests: add resource debug output (open FD count, ulimit
  values) to help diagnose future CI container issues

- integration_tests: remove _pabot_parallel_args (no longer needed);
  slow_integration_tests session still available for parallel runs

- load_context_test: add env:TERM=dumb alongside NO_COLOR=1 to disable
  all Rich terminal styling (NO_COLOR only disables color, not bold/
  reset ANSI codes that may split substrings)

- load_context_test: add repr() debug logging around the --load-context
  match to reveal any invisible characters on CI
2026-02-13 01:37:32 +00:00
brent.edwards 25e6f95eb9 fix(ci): resolve security_scan and integration_tests CI failures
- security_scan: create build/ directory before bandit writes its JSON
  report (fails on fresh CI checkout where directory does not exist)

- integration_tests: cap pabot parallelism to 2 processes and explicitly
  propagate venv bin/ to PATH, preventing FileNotFoundError for the
  robot binary under CI resource constraints

- integration_tests: add env:NO_COLOR=1 to load_context_test.robot help
  text assertions so Rich ANSI escape codes do not break substring
  matching on CI

- integration_tests: tag initial_next_command_test as slow (requires
  OPENAI_API_KEY for LLM agent invocation, unavailable on CI)
2026-02-13 01:04:22 +00:00
brent.edwards 321ab18a37 fix(ci): resolve unit_tests and integration_tests CI failures
- Fix Rich Console line-wrapping breaking assertions in
  context_unit_tests_steps.py: collapse newlines before checking for
  filenames and overflow summaries (CI temp paths exceed 80 columns)
- Fix features.mocks import failure in database_integration.robot:
  replace hardcoded sys.path '/app' with portable ${CURDIR}/..
- Fix --load-context help text assertions in load_context_test.robot:
  merge stderr into stdout via stderr=STDOUT and remove duplicate test
- Add standalone dead_code nox session running vulture directly
- Rename security nox session to security_scan to match CI references
- Restore --exclude discovery to integration_tests nox session (lost
  during merge conflict resolution)
2026-02-13 00:15:58 +00:00
brent.edwards 3e3530de48 ci(git-merge): Merging from master 2026-02-12 23:01:35 +00:00
brent.edwards e5b1b04ec6 test(integration_tests-3.13): fix nox tests
`integration_tests-3.13` was failing; it is now passing.
2026-02-12 22:48:19 +00:00
CoreRasurae 3adc9c1f47 chore: Fix conflicts in the implementation plan 2026-02-12 22:17:32 +00:00
freemo 89184689dc build: Coverage was broken, now fixed. 2026-02-12 17:02:21 -05:00
brent.edwards e801eb1ee8 feat(ci): add nox-based PR validation workflow
- Rewrite .forgejo/workflows/ci.yml to route all jobs through nox sessions
- Fix coverage_report nox session: serial behave mode replaces broken parallel
  mode (22% -> 97% accuracy), raise fail-under from 85% to 97%
- Pass posargs through format nox session for CI --check support
- Add 11 CI workflow validation scenarios (Behave) + Robot smoke test + ASV bench
- Add 108 new Behave scenarios covering 6 largest coverage gaps to reach 97%:
  yaml_template_engine, actor/config, actor/registry, message_router,
  context_analysis, context_service
- Update docs/development/ci-cd.md with nox-based CI docs and 97% threshold
- Restore implementation_plan.md verbose style, check off completed CI tasks

Verified: 1673 scenarios pass, 97% coverage, lint clean, typecheck clean
2026-02-12 22:01:51 +00:00
freemo 3d96c3d9ff Docs: made the glossary a little less verbose 2026-02-12 16:29:40 -05:00
CoreRasurae 5d1825fb60 ci: CI jobs need to have git command
Some tests require it
2026-02-12 20:20:00 +00:00
CoreRasurae e7ad541b71 build(env): Update security Vulture check exceptions 2026-02-12 20:19:57 +00:00
CoreRasurae aa299966ad build(env): Update noxfile config to include a secuity check with vulture 2026-02-12 20:19:55 +00:00
CoreRasurae 091c93838c build(env): Update noxfile config to also lint the behave features and robot tests 2026-02-12 20:19:52 +00:00
CoreRasurae c8f6bbaf9a build(env): Update noxfile config to be backward compatible with older behave 2026-02-12 20:19:49 +00:00
CoreRasurae f2f7aa5dc9 feat(core): add v3 lifecycle models, automation levels, subplan support, and security hardening
Implement multiple Stage A/B/E/SEC milestones for the v3 lifecycle system:
- Stage A5.3+A5.4: Add LifecycleActionModel and LifecyclePlanModel SQLAlchemy
  models with to_domain()/from_domain() conversion methods
- Stage A5.6: Implement ActionRepository with full CRUD, namespace/state
  queries, referential integrity checks, and retry decorator
- Stage E1: Add subplan domain models (ExecutionMode, SubplanMergeStrategy,
  SubplanConfig, SubplanStatus, SubplanAttempt, SubplanFailureHandler) with
  computed properties on Plan (is_subplan, is_root_plan, depth, has_subplans)
- Stage A6: Add AutomationLevel enum (MANUAL, REVIEW_BEFORE_APPLY,
  FULL_AUTOMATION), settings integration, PlanLifecycleService auto-progression,
  pause/resume, and CLI commands (--automation-level, set-automation-level)
- Stage SEC1: Remove eval()/exec() from stream_router.py, replace with named
  operation and transform registries; code blocks and unregistered transforms
  now raise StreamRoutingError
- Add langchain-anthropic dependency
- Update BDD tests for security changes and relax ADR directory requirement
2026-02-12 20:19:42 +00:00
freemo a1c1540a00 Docs: updated plan 2026-02-12 15:01:51 -05:00
freemo 453c1479de Docs: Fixed several descrepencies within the specification document 2026-02-12 14:48:30 -05:00
freemo ea81a5856b Docs: updated the glossary to make it more succinct 2026-02-12 14:20:29 -05:00
freemo 6c7a48e7dc Docs: split out some sections from the specification 2026-02-12 14:04:10 -05:00
freemo 687bf6b6ec Docs: using standard kroki server since ours appears to be down (or broken) 2026-02-12 13:41:26 -05:00
freemo 578bfbf102 Docs: updated implementation plan to remove discrepencies between it and the specification 2026-02-12 13:24:44 -05:00
freemo 31fd332c7b Docs: Updated implementation plan to eliminate indescrepencies with specification 2026-02-12 13:11:38 -05:00
freemo dc2fa53942 Docs: Updated the implementation plan, included more schedule adherance data as well 2026-02-12 13:08:01 -05:00
freemo a4b56fa308 Docs: Cleaned up some minor issues like docs not being in nav 2026-02-12 11:08:54 -05:00
freemo f9d3aa6f3a Docs: Made the ACMS more extensible 2026-02-12 11:02:10 -05:00
freemo e4febab61e Docs: Updated the depth / detail level concept to be more extensible with additional levels as well 2026-02-12 10:07:37 -05:00
freemo bce6109e2f Docs: First iteration of ACMS added 2026-02-12 08:54:01 -05:00
freemo 013408961a Docs: Removed Context section as we are about to rewrite it in the next commit 2026-02-12 08:53:57 -05:00
brent.edwards 13a0aaeb9d Merge pull request 'feat: Add Q0: Pre-commit hooks setup.' (#41) from Q0-quality-automation into master
Reviewed-on: cleveragents/cleveragents-core#41
Reviewed-by: Rui Hu <rui.hu@cleverthis.com>
2026-02-12 07:21:52 +00:00
brent.edwards 63b54ac8ab fix(nox): run asv machine --yes before benchmarks
asv requires machine info to be configured before running benchmarks.
Without this, asv run fails on first use or in CI/containers.
2026-02-12 07:03:31 +00:00
brent.edwards 64f58f6406 fix(deps): pin behave==1.2.6 in tests extra to match dev extra
behave-parallel pulls behave>=1.2.6 which resolves to 1.3.3. Behave
1.3.3 no longer strips trailing colons from step text before tables,
breaking the architecture.feature step matcher.
2026-02-12 07:01:08 +00:00
brent.edwards 58d79558ce fix(docker): copy README.md into build stage for hatchling 2026-02-12 06:35:02 +00:00
brent.edwards d0f1c8bae3 fix(ci): use docker:dind as main container instead of services
Forgejo act_runner service container networking fails with 'network not
found'. Use docker:dind directly with --privileged and start dockerd
inline to avoid the services networking issue.
2026-02-12 06:26:56 +00:00
brent.edwards 945cced381 fix(ci): fix docker build job and Dockerfile
- Add DinD service container and docker:cli image so docker CLI is available
- Fix Dockerfile: remove missing uv.lock, fix build order (copy source before build)
- Remove helm job (k8s/ directory does not exist yet)
2026-02-12 06:17:36 +00:00
brent.edwards 04070ca3ff Merge branch 'master' into Q0-quality-automation
# Conflicts:
#	implementation_plan.md
2026-02-12 06:05:43 +00:00
brent.edwards c9605c9381 test(features/steps): remove ansi codes
Remove rich formatting from output
2026-02-12 05:44:57 +00:00
brent.edwards d4a48caeec test(features/steps): fix features that depend on running as user not superuser 2026-02-12 05:23:29 +00:00
freemo 4363fbe7c4 Docs: moved implementation plan here 2026-02-12 00:23:19 -05:00
brent.edwards 17fe46d925 test(behave): fix behave tests
There had been over 100 behave tests failing. There should be none failing now.
2026-02-12 04:47:02 +00:00
freemo 2bceddeed3 Docs: revampted implementation plan again 2026-02-11 23:24:18 -05:00
brent.edwards 3ed11f7988 build(behave.ini): remove ", src" from "paths"
The code was seeing the whole text as one path.
2026-02-12 03:36:51 +00:00
brent.edwards 4ae8a9ee3a ci(.forgejo/workflows/ci.yml): remove unused behave-parallel
No longer installing behave-parallel. Also removed other uploads that won't work.
2026-02-12 03:30:13 +00:00
brent.edwards ea783c57e9 ci(.forgejo/workflows/ci.yml): remove behave 3.11 and 3.12 2026-02-12 03:17:26 +00:00
brent.edwards 3cc662f5cc ci(.forgejo/workflows/ci.yml): remove the attempt to upload complexity report
Forgejo shows the following error: ::error::@actions/artifact v2.0.0+, upload-artifact@v4+ and download-artifact@v4+ are not currently supported on GHES.
2026-02-12 03:14:37 +00:00
brent.edwards a40d48cbea style(src/cleveragents/reactive/application.py): simplify the code that gets an F on complexity metric 2026-02-12 03:04:18 +00:00
brent.edwards 51bfee28a9 style(robot/-and-typings/numpy): fix style problems found by ruff check 2026-02-12 02:47:08 +00:00