TEST-INFRA: [missing-test-levels] Add missing tests for the templates module #2084

Open
opened 2026-04-03 03:56:22 +00:00 by freemo · 1 comment
Owner

The following files in the src/cleveragents/templates module are missing dedicated tests or have incomplete test coverage at one or more required levels:

  • src/cleveragents/templates/__init__.py
  • src/cleveragents/templates/renderer.py
  • src/cleveragents/templates/secure_renderer.py

Metadata

  • Branch: test/missing-test-levels-templates-module
  • Commit Message: test(templates): add Behave, Robot, and ASV tests for templates module
  • Milestone: v3.5.0
  • Parent Epic: #933

Background and Context

The templates module in src/cleveragents/templates is missing complete test coverage at all three required levels mandated by CONTRIBUTING.md's Multi-Level Testing Mandate:

  • Behave unit tests (in features/) — BDD/Gherkin scenarios covering unit-level behaviour
  • Robot integration tests (in robot/) — integration tests using real services, no mocks
  • ASV performance benchmarks (in benchmarks/) — performance regression tracking

While some scenarios for secure_renderer.py exist in features/consolidated_security.feature and a handful of TemplateRenderer scenarios are present, the following gaps remain:

  • templates/__init__.py — the public re-export surface has no dedicated Behave scenarios verifying that all exported symbols (SecureTemplateRenderer, TemplateConfig, TemplateError, TemplateSecurityError, TemplateSizeError, TemplateValidationError, render_template_secure, validate_template) are importable and functional via the package-level API.
  • renderer.py — the TemplateEngine enum is untested; edge cases for TemplateRenderer (e.g. non-default engine values, config passthrough, warning-path logging) lack dedicated scenarios.
  • secure_renderer.py — Robot Framework integration tests and ASV benchmarks exist but do not cover the full public API surface (e.g. SecureTemplateRenderer.validate(), _extract_placeholder_names, Unicode key acceptance, open-allowlist mode, log_rejected=False path).

This violates the testing standards outlined in CONTRIBUTING.md: "Every coding task must include or update tests at multiple levels: unit tests, integration tests, and performance benchmarks. Testing is non-optional and is part of the definition of done for any task."

Expected Behaviour

The templates module has complete test coverage at all three required levels:

  1. Behave feature files with step implementations covering all module behaviours
  2. Robot integration test suites exercising real template module dependencies
  3. ASV benchmarks tracking performance of key template module operations

Acceptance Criteria

  • Behave feature files exist in features/ covering templates/__init__.py public re-export API
  • Behave feature files exist in features/ covering templates/renderer.py (TemplateEngine enum, TemplateRenderer all branches)
  • Behave feature files exist in features/ covering templates/secure_renderer.py remaining untested paths
  • All Behave step definitions are fully implemented (no placeholder steps)
  • Robot integration test suites exist in robot/ for the templates module using real services (no mocks)
  • ASV performance benchmarks exist in benchmarks/ for key templates module operations (where applicable)
  • All new tests pass via the project's nox task runner
  • Test coverage for the templates module is >= 97%
  • All nox sessions pass (unit, integration, typecheck, lint, coverage)

Subtasks

  • Audit all 3 files in src/cleveragents/templates/ for untested code paths and branches
  • Write Behave (Gherkin) unit test scenarios in features/ covering templates/__init__.py (all 8 re-exported symbols importable and functional)
  • Write Behave (Gherkin) unit test scenarios in features/ covering templates/renderer.py (TemplateEngine enum values, TemplateRenderer.__init__ with and without config, render success path, render fallback-on-error path, logging warning branch)
  • Write Behave (Gherkin) unit test scenarios in features/ covering templates/secure_renderer.py remaining gaps (validate() method, _extract_placeholder_names, Unicode key acceptance, open-allowlist mode, log_rejected=False, reject_unknown_keys=False)
  • Write Robot Framework integration tests in robot/ for the templates module
  • Write ASV benchmark tests in benchmarks/ for the templates module (if applicable)
  • Add/update step definitions in features/steps/ as needed
  • Add/update mock implementations in features/mocks/ as needed (no mocks in production code)
  • Run nox -e unit_tests and confirm all scenarios pass
  • Run nox -e integration_tests and confirm all tests pass
  • Run nox -e coverage_report and confirm coverage >= 97%
  • Run nox -e typecheck and confirm zero Pyright errors
  • Run nox -e lint and confirm zero linting errors
  • Run full nox suite and confirm all sessions pass

Definition of Done

  • All subtasks above are checked off
  • Commit created with exact message: test(templates): add Behave, Robot, and ASV tests for templates module
  • Commit footer includes ISSUES CLOSED: #<this issue number>
  • Commit pushed to branch test/missing-test-levels-templates-module
  • Pull request submitted, reviewed, and merged
  • All nox stages pass
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

The following files in the `src/cleveragents/templates` module are missing dedicated tests or have incomplete test coverage at one or more required levels: - `src/cleveragents/templates/__init__.py` - `src/cleveragents/templates/renderer.py` - `src/cleveragents/templates/secure_renderer.py` ## Metadata - **Branch**: `test/missing-test-levels-templates-module` - **Commit Message**: `test(templates): add Behave, Robot, and ASV tests for templates module` - **Milestone**: v3.5.0 - **Parent Epic**: #933 ## Background and Context The `templates` module in `src/cleveragents/templates` is missing complete test coverage at all three required levels mandated by CONTRIBUTING.md's Multi-Level Testing Mandate: - **Behave** unit tests (in `features/`) — BDD/Gherkin scenarios covering unit-level behaviour - **Robot** integration tests (in `robot/`) — integration tests using real services, no mocks - **ASV** performance benchmarks (in `benchmarks/`) — performance regression tracking While some scenarios for `secure_renderer.py` exist in `features/consolidated_security.feature` and a handful of `TemplateRenderer` scenarios are present, the following gaps remain: - `templates/__init__.py` — the public re-export surface has no dedicated Behave scenarios verifying that all exported symbols (`SecureTemplateRenderer`, `TemplateConfig`, `TemplateError`, `TemplateSecurityError`, `TemplateSizeError`, `TemplateValidationError`, `render_template_secure`, `validate_template`) are importable and functional via the package-level API. - `renderer.py` — the `TemplateEngine` enum is untested; edge cases for `TemplateRenderer` (e.g. non-default `engine` values, config passthrough, warning-path logging) lack dedicated scenarios. - `secure_renderer.py` — Robot Framework integration tests and ASV benchmarks exist but do not cover the full public API surface (e.g. `SecureTemplateRenderer.validate()`, `_extract_placeholder_names`, Unicode key acceptance, open-allowlist mode, `log_rejected=False` path). This violates the testing standards outlined in CONTRIBUTING.md: *"Every coding task must include or update tests at multiple levels: unit tests, integration tests, and performance benchmarks. Testing is non-optional and is part of the definition of done for any task."* ## Expected Behaviour The `templates` module has complete test coverage at all three required levels: 1. Behave feature files with step implementations covering all module behaviours 2. Robot integration test suites exercising real template module dependencies 3. ASV benchmarks tracking performance of key template module operations ## Acceptance Criteria - [ ] Behave feature files exist in `features/` covering `templates/__init__.py` public re-export API - [ ] Behave feature files exist in `features/` covering `templates/renderer.py` (`TemplateEngine` enum, `TemplateRenderer` all branches) - [ ] Behave feature files exist in `features/` covering `templates/secure_renderer.py` remaining untested paths - [ ] All Behave step definitions are fully implemented (no placeholder steps) - [ ] Robot integration test suites exist in `robot/` for the `templates` module using real services (no mocks) - [ ] ASV performance benchmarks exist in `benchmarks/` for key `templates` module operations (where applicable) - [ ] All new tests pass via the project's nox task runner - [ ] Test coverage for the `templates` module is >= 97% - [ ] All nox sessions pass (unit, integration, typecheck, lint, coverage) ## Subtasks - [ ] Audit all 3 files in `src/cleveragents/templates/` for untested code paths and branches - [ ] Write Behave (Gherkin) unit test scenarios in `features/` covering `templates/__init__.py` (all 8 re-exported symbols importable and functional) - [ ] Write Behave (Gherkin) unit test scenarios in `features/` covering `templates/renderer.py` (`TemplateEngine` enum values, `TemplateRenderer.__init__` with and without config, `render` success path, `render` fallback-on-error path, logging warning branch) - [ ] Write Behave (Gherkin) unit test scenarios in `features/` covering `templates/secure_renderer.py` remaining gaps (`validate()` method, `_extract_placeholder_names`, Unicode key acceptance, open-allowlist mode, `log_rejected=False`, `reject_unknown_keys=False`) - [ ] Write Robot Framework integration tests in `robot/` for the `templates` module - [ ] Write ASV benchmark tests in `benchmarks/` for the `templates` module (if applicable) - [ ] Add/update step definitions in `features/steps/` as needed - [ ] Add/update mock implementations in `features/mocks/` as needed (no mocks in production code) - [ ] Run `nox -e unit_tests` and confirm all scenarios pass - [ ] Run `nox -e integration_tests` and confirm all tests pass - [ ] Run `nox -e coverage_report` and confirm coverage >= 97% - [ ] Run `nox -e typecheck` and confirm zero Pyright errors - [ ] Run `nox -e lint` and confirm zero linting errors - [ ] Run full `nox` suite and confirm all sessions pass ## Definition of Done - [ ] All subtasks above are checked off - [ ] Commit created with exact message: `test(templates): add Behave, Robot, and ASV tests for templates module` - [ ] Commit footer includes `ISSUES CLOSED: #<this issue number>` - [ ] Commit pushed to branch `test/missing-test-levels-templates-module` - [ ] Pull request submitted, reviewed, and merged - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.5.0 milestone 2026-04-03 03:56:34 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: Medium — test coverage gaps in the templates module
  • Milestone: v3.5.0
  • MoSCoW: Should Have — maintaining ≥97% coverage is a project-wide requirement per CONTRIBUTING.md; missing tests for the templates module is a gap that should be addressed

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — test coverage gaps in the templates module - **Milestone**: v3.5.0 - **MoSCoW**: Should Have — maintaining ≥97% coverage is a project-wide requirement per CONTRIBUTING.md; missing tests for the templates module is a gap that should be addressed --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2084
No description provided.