UAT: `unittest.mock` used in 68 Robot Framework integration test helper files — violates integration test mock prohibition #3875

New issue

Open

opened 2026-04-06 07:05:28 +00:00 by freemo · 0 comments

freemo commented

2026-04-06 07:05:28 +00:00

Owner

Metadata

Branch: fix/robot-mock-prohibition/remove-unittest-mock-from-integration-tests
Commit Message: fix(tests): remove unittest.mock from 68 robot/ integration test helpers — enforce mock prohibition
Milestone: Backlog
Parent Epic: #3374

Background

CONTRIBUTING.md explicitly states:

"Unit Tests Only: Mocks, fakes, stubs, and test doubles are permitted only in unit tests. Integration tests must exercise real services, real endpoints, and real dependencies — mocking of any kind is strictly prohibited in integration tests."

68 Robot Framework integration test helper files in the robot/ directory use from unittest.mock import MagicMock, patch (and related mock utilities). These files are called by Robot Framework .robot test suites as integration tests, but they use mocking to replace real service dependencies — a direct violation of the project's testing standards.

Current Behavior

68 Robot Framework helper files under robot/ import and use unittest.mock constructs (MagicMock, patch, etc.) to stub out real service dependencies during integration test execution.

Affected Files (sample — 68 total):

robot/helper_validation_attach_type_guard.py — uses MagicMock, patch
robot/helper_tool_cli.py — uses MagicMock, patch
robot/helper_session_cli.py — uses MagicMock
robot/helper_plan_executor_subplan_spawning.py — uses MagicMock
robot/helper_plan_cli_spec.py — uses MagicMock, patch
robot/helper_actor_add_yaml_first_path.py — uses MagicMock, patch
robot/helper_actor_add_rich_output.py — uses MagicMock, patch
robot/helper_cli_formats.py — uses MagicMock, patch
robot/helper_automation_profile_cli.py — uses patch
robot/helper_a2a_facade_wiring.py — uses MagicMock
robot/helper_actor_cli_show.py — uses MagicMock, patch
robot/helper_config_cli.py — uses patch
robot/helper_cli_lifecycle_e2e.py — uses MagicMock, patch
robot/helper_cli_lifecycle.py — uses MagicMock, patch
robot/helper_m1_sourcecode_smoke.py — uses MagicMock, patch
… and 53 more files

Steps to Reproduce:

grep -r "from unittest.mock import" robot/ --include="*.py" | wc -l
# Returns: 68

Expected Behavior

Integration tests in robot/ must exercise real services, real endpoints, and real dependencies without any mocking. If a test requires mocking to function, it must be converted to a Behave unit test in features/ instead. Mocking code belongs only in features/mocks/.

Subtasks

Audit all 68 affected robot/ helper files and catalogue each mock usage
For each file: determine if the test can be rewritten to use real services, or must be moved to features/
Move mock-dependent tests to features/ as Behave scenarios (with mocks in features/mocks/)
Convert remaining tests to exercise real service dependencies without mocking
Remove all unittest.mock imports from robot/ directory
Run nox -s integration_tests and verify all pass with no mock usage in robot/ helpers
Run nox -s unit_tests and verify all pass
Verify coverage >= 97% via nox -s coverage_report

Definition of Done

Zero unittest.mock imports remain in any file under robot/
All integration tests in robot/ exercise real services without mocking
All mock-dependent tests migrated to features/ as Behave scenarios
nox -s integration_tests passes with no mock usage in robot/ helpers
nox -s unit_tests passes
All nox stages pass
Coverage >= 97%

Backlog note: This issue was discovered during autonomous operation
on milestone v3.6.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/robot-mock-prohibition/remove-unittest-mock-from-integration-tests` - **Commit Message**: `fix(tests): remove unittest.mock from 68 robot/ integration test helpers — enforce mock prohibition` - **Milestone**: Backlog - **Parent Epic**: #3374 ## Background `CONTRIBUTING.md` explicitly states: > "Unit Tests Only: Mocks, fakes, stubs, and test doubles are permitted only in unit tests. Integration tests must exercise real services, real endpoints, and real dependencies — mocking of any kind is strictly prohibited in integration tests." 68 Robot Framework integration test helper files in the `robot/` directory use `from unittest.mock import MagicMock, patch` (and related mock utilities). These files are called by Robot Framework `.robot` test suites as integration tests, but they use mocking to replace real service dependencies — a direct violation of the project's testing standards. ## Current Behavior 68 Robot Framework helper files under `robot/` import and use `unittest.mock` constructs (`MagicMock`, `patch`, etc.) to stub out real service dependencies during integration test execution. **Affected Files** (sample — 68 total): - `robot/helper_validation_attach_type_guard.py` — uses `MagicMock, patch` - `robot/helper_tool_cli.py` — uses `MagicMock, patch` - `robot/helper_session_cli.py` — uses `MagicMock` - `robot/helper_plan_executor_subplan_spawning.py` — uses `MagicMock` - `robot/helper_plan_cli_spec.py` — uses `MagicMock, patch` - `robot/helper_actor_add_yaml_first_path.py` — uses `MagicMock, patch` - `robot/helper_actor_add_rich_output.py` — uses `MagicMock, patch` - `robot/helper_cli_formats.py` — uses `MagicMock, patch` - `robot/helper_automation_profile_cli.py` — uses `patch` - `robot/helper_a2a_facade_wiring.py` — uses `MagicMock` - `robot/helper_actor_cli_show.py` — uses `MagicMock, patch` - `robot/helper_config_cli.py` — uses `patch` - `robot/helper_cli_lifecycle_e2e.py` — uses `MagicMock, patch` - `robot/helper_cli_lifecycle.py` — uses `MagicMock, patch` - `robot/helper_m1_sourcecode_smoke.py` — uses `MagicMock, patch` - … and 53 more files **Steps to Reproduce:** ```bash grep -r "from unittest.mock import" robot/ --include="*.py" | wc -l # Returns: 68 ``` ## Expected Behavior Integration tests in `robot/` must exercise real services, real endpoints, and real dependencies without any mocking. If a test requires mocking to function, it must be converted to a Behave unit test in `features/` instead. Mocking code belongs only in `features/mocks/`. ## Subtasks - [ ] Audit all 68 affected `robot/` helper files and catalogue each mock usage - [ ] For each file: determine if the test can be rewritten to use real services, or must be moved to `features/` - [ ] Move mock-dependent tests to `features/` as Behave scenarios (with mocks in `features/mocks/`) - [ ] Convert remaining tests to exercise real service dependencies without mocking - [ ] Remove all `unittest.mock` imports from `robot/` directory - [ ] Run `nox -s integration_tests` and verify all pass with no mock usage in `robot/` helpers - [ ] Run `nox -s unit_tests` and verify all pass - [ ] Verify coverage >= 97% via `nox -s coverage_report` ## Definition of Done - [ ] Zero `unittest.mock` imports remain in any file under `robot/` - [ ] All integration tests in `robot/` exercise real services without mocking - [ ] All mock-dependent tests migrated to `features/` as Behave scenarios - [ ] `nox -s integration_tests` passes with no mock usage in `robot/` helpers - [ ] `nox -s unit_tests` passes - [ ] All nox stages pass - [ ] Coverage >= 97% > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.6.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-06 07:05:40 +00:00

#3374 Epic: E2E Workflow Specification Tests & Code Review Tool Examples

freemo added the

Points

label

2026-04-06 07:09:31 +00:00

freemo referenced this issue

2026-04-06 07:12:10 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 07:21:26 +00:00