Improve test data quality in Behave unit test suite #2022

Open
opened 2026-04-03 02:41:35 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/improve-test-data-quality
  • Commit Message: test(features): improve test data quality in Behave unit test suite
  • Milestone: v3.8.0
  • Parent Epic: #1678

Overview

The current Behave unit test suite (features/) relies on test data and mock
implementations (features/mocks/) that lack consistency, realism, and
coverage breadth. Poor test data quality leads to false-positive test passes,
gaps in edge-case coverage, and brittle scenarios that break under minor
refactors. This issue tracks a systematic improvement of all test data used
across the unit test suite.

Key areas of concern identified:

  1. Mock data realism — mock objects in features/mocks/ use placeholder
    values (e.g., empty strings, None defaults) that do not reflect realistic
    production inputs, masking bugs that only surface with real data shapes.
  2. Edge-case coverage — boundary conditions (empty collections, maximum
    field lengths, unicode characters, null-equivalent values) are under-represented
    in scenario outlines and step definitions.
  3. Data consistency — the same logical entity (e.g., a Resource, Actor,
    or Plan) is defined differently across feature files, making cross-feature
    assertions unreliable.
  4. Shared fixtures — there is no centralised fixture catalogue; each feature
    file re-declares its own data, leading to duplication and drift.

Subtasks

  • Audit all existing mock data in features/mocks/ and document gaps
  • Define a shared test-data catalogue (realistic, typed fixtures) covering
    core domain objects: Resource, Actor, Plan, Skill, Tool
  • Replace placeholder/empty mock values with realistic, production-representative data
  • Add scenario outlines for boundary and edge-case inputs (empty, max-length,
    unicode, null-equivalent) across all affected feature files
  • Ensure all mock implementations remain strictly within features/mocks/
    per CONTRIBUTING.md (no mocks in robot/ or production code)
  • Update or add Behave step definitions to consume the new shared fixtures
  • Run nox -e unit_tests and confirm all scenarios pass
  • Run nox -e coverage_report and confirm coverage ≥ 97%
  • Run full nox suite and confirm all stages pass

Definition of Done

  • All mock data in features/mocks/ uses realistic, typed, production-representative values
  • A shared fixture catalogue exists and is consumed consistently across feature files
  • Boundary and edge-case scenarios are present for all core domain objects
  • No mock data or test doubles exist outside features/mocks/
  • All nox stages pass (nox -e lint, nox -e typecheck, nox -e unit_tests,
    nox -e integration_tests, nox -e coverage_report)
  • Coverage ≥ 97%
  • PR is reviewed and approved by ≥ 2 non-author contributors
  • PR is merged and linked issue is closed

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/improve-test-data-quality` - **Commit Message**: `test(features): improve test data quality in Behave unit test suite` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Overview The current Behave unit test suite (`features/`) relies on test data and mock implementations (`features/mocks/`) that lack consistency, realism, and coverage breadth. Poor test data quality leads to false-positive test passes, gaps in edge-case coverage, and brittle scenarios that break under minor refactors. This issue tracks a systematic improvement of all test data used across the unit test suite. Key areas of concern identified: 1. **Mock data realism** — mock objects in `features/mocks/` use placeholder values (e.g., empty strings, `None` defaults) that do not reflect realistic production inputs, masking bugs that only surface with real data shapes. 2. **Edge-case coverage** — boundary conditions (empty collections, maximum field lengths, unicode characters, null-equivalent values) are under-represented in scenario outlines and step definitions. 3. **Data consistency** — the same logical entity (e.g., a `Resource`, `Actor`, or `Plan`) is defined differently across feature files, making cross-feature assertions unreliable. 4. **Shared fixtures** — there is no centralised fixture catalogue; each feature file re-declares its own data, leading to duplication and drift. ## Subtasks - [ ] Audit all existing mock data in `features/mocks/` and document gaps - [ ] Define a shared test-data catalogue (realistic, typed fixtures) covering core domain objects: `Resource`, `Actor`, `Plan`, `Skill`, `Tool` - [ ] Replace placeholder/empty mock values with realistic, production-representative data - [ ] Add scenario outlines for boundary and edge-case inputs (empty, max-length, unicode, null-equivalent) across all affected feature files - [ ] Ensure all mock implementations remain strictly within `features/mocks/` per CONTRIBUTING.md (no mocks in `robot/` or production code) - [ ] Update or add Behave step definitions to consume the new shared fixtures - [ ] Run `nox -e unit_tests` and confirm all scenarios pass - [ ] Run `nox -e coverage_report` and confirm coverage ≥ 97% - [ ] Run full `nox` suite and confirm all stages pass ## Definition of Done - [ ] All mock data in `features/mocks/` uses realistic, typed, production-representative values - [ ] A shared fixture catalogue exists and is consumed consistently across feature files - [ ] Boundary and edge-case scenarios are present for all core domain objects - [ ] No mock data or test doubles exist outside `features/mocks/` - [ ] All nox stages pass (`nox -e lint`, `nox -e typecheck`, `nox -e unit_tests`, `nox -e integration_tests`, `nox -e coverage_report`) - [ ] Coverage ≥ 97% - [ ] PR is reviewed and approved by ≥ 2 non-author contributors - [ ] PR is merged and linked issue is closed --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 02:41:39 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: Medium (confirmed)
  • Milestone: v3.8.0 (confirmed — test infrastructure)
  • MoSCoW: Could Have — Improving test data quality is a long-term quality investment. The current tests pass and provide 97%+ coverage; this improves the reliability and realism of the test suite but does not block any deliverables.
  • Parent Epic: #1678 (confirmed correct)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium (confirmed) - **Milestone**: v3.8.0 (confirmed — test infrastructure) - **MoSCoW**: Could Have — Improving test data quality is a long-term quality investment. The current tests pass and provide 97%+ coverage; this improves the reliability and realism of the test suite but does not block any deliverables. - **Parent Epic**: #1678 (confirmed correct) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2022
No description provided.