BUG-HUNT: [spec-alignment] `SkillDefinition._validate_schemas` rejects valid JSON Schemas lacking a `"type"` key — `allOf`, `oneOf`, `$ref` schemas are incorrectly refused #6596

New issue

Open

opened 2026-04-09 21:55:41 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-09 21:55:41 +00:00

Owner

Bug Report: Spec Alignment — Over-Restrictive JSON Schema Validation in `SkillDefinition`

Severity Assessment

Impact: Any skill that defines its input_schema or output_schema using composition keywords (allOf, oneOf, anyOf, $ref, const, enum) without a top-level "type" key will fail Pydantic model validation and cannot be registered. This incorrectly rejects well-formed JSON Schemas.
Likelihood: Medium — any advanced schema author using standard JSON Schema features will hit this. The error message "input_schema must include a 'type' key to be a valid JSON Schema" is misleading because "type" is not required by the JSON Schema specification.
Priority: Medium

Location

File: src/cleveragents/skills/protocol.py
Function: SkillDefinition._validate_schemas (model validator)
Lines: ~253–272

Description

The _validate_schemas model validator enforces that every non-None input/output schema dict must contain a "type" key:

# protocol.py  lines 255–271
@model_validator(mode="after")
def _validate_schemas(self) -> SkillDefinition:
    """Validate that input/output schemas are well-formed JSON Schema.

    A valid JSON Schema object must have ``"type"`` declared.
    This lightweight check catches the most common authoring errors
    without pulling in a full JSON Schema meta-validator.
    """
    for label, schema in [
        ("input_schema", self.input_schema),
        ("output_schema", self.output_schema),
    ]:
        if schema is not None:
            if not isinstance(schema, dict):
                raise ValueError(f"{label} must be a dict, got {type(schema)}")
            if "type" not in schema:
                raise ValueError(
                    f"{label} must include a 'type' key to be a valid JSON Schema"
                )
    return self

The comment "A valid JSON Schema object must have 'type' declared" is incorrect. Per JSON Schema specification (draft-07 and later), the following are all perfectly valid JSON Schemas:

// allOf — no "type" required
{"allOf": [{"type": "string"}, {"minLength": 1}]}

// $ref — no "type" required  
{"$ref": "#/definitions/MyType"}

// anyOf
{"anyOf": [{"type": "string"}, {"type": "integer"}]}

// const — no "type" required
{"const": "fixed_value"}

// empty schema — matches anything
{}

All of the above would be rejected by the validator, raising a ValidationError when trying to construct a SkillDefinition with such schemas.

Expected Behavior

Valid JSON Schema objects should be accepted even without a top-level "type" key.

Actual Behavior

SkillDefinition raises ValidationError with message "input_schema must include a 'type' key to be a valid JSON Schema" for schemas using allOf, oneOf, anyOf, $ref, const, or an empty schema ({}).

Suggested Fix

Either remove the "type" key requirement and only check for isinstance(schema, dict), or if more validation is desired, check for at least one recognized JSON Schema keyword:

_VALID_SCHEMA_KEYWORDS = frozenset({
    "type", "allOf", "anyOf", "oneOf", "not", "$ref",
    "properties", "items", "const", "enum", "format",
    "minimum", "maximum", "minLength", "maxLength",
    "required", "additionalProperties", "description",
})

if schema is not None:
    if not isinstance(schema, dict):
        raise ValueError(f"{label} must be a dict, got {type(schema)}")
    # Allow empty schema {} (matches anything) and any schema with known keywords
    if schema and not _VALID_SCHEMA_KEYWORDS.intersection(schema.keys()):
        raise ValueError(
            f"{label} does not appear to be a valid JSON Schema object"
        )

TDD Note

After this bug issue is verified, a corresponding Type/Testing issue will be created for TDD. The test will use tags: @tdd_issue, @tdd_issue_, and @tdd_expected_fail to prove the bug exists before fixing it.

Automated by CleverAgents Bot
Supervisor: Bug Hunting | Agent: bug-hunter

## Bug Report: Spec Alignment — Over-Restrictive JSON Schema Validation in `SkillDefinition` ### Severity Assessment - **Impact**: Any skill that defines its `input_schema` or `output_schema` using composition keywords (`allOf`, `oneOf`, `anyOf`, `$ref`, `const`, `enum`) without a top-level `"type"` key will fail Pydantic model validation and cannot be registered. This incorrectly rejects well-formed JSON Schemas. - **Likelihood**: Medium — any advanced schema author using standard JSON Schema features will hit this. The error message `"input_schema must include a 'type' key to be a valid JSON Schema"` is misleading because `"type"` is **not** required by the JSON Schema specification. - **Priority**: Medium ### Location - **File**: `src/cleveragents/skills/protocol.py` - **Function**: `SkillDefinition._validate_schemas` (model validator) - **Lines**: ~253–272 ### Description The `_validate_schemas` model validator enforces that every non-None input/output schema dict must contain a `"type"` key: ```python # protocol.py lines 255–271 @model_validator(mode="after") def _validate_schemas(self) -> SkillDefinition: """Validate that input/output schemas are well-formed JSON Schema. A valid JSON Schema object must have ``"type"`` declared. This lightweight check catches the most common authoring errors without pulling in a full JSON Schema meta-validator. """ for label, schema in [ ("input_schema", self.input_schema), ("output_schema", self.output_schema), ]: if schema is not None: if not isinstance(schema, dict): raise ValueError(f"{label} must be a dict, got {type(schema)}") if "type" not in schema: raise ValueError( f"{label} must include a 'type' key to be a valid JSON Schema" ) return self ``` The comment `"A valid JSON Schema object must have 'type' declared"` is **incorrect**. Per [JSON Schema specification (draft-07 and later)](https://json-schema.org/understanding-json-schema/), the following are all perfectly valid JSON Schemas: ```json // allOf — no "type" required {"allOf": [{"type": "string"}, {"minLength": 1}]} // $ref — no "type" required {"$ref": "#/definitions/MyType"} // anyOf {"anyOf": [{"type": "string"}, {"type": "integer"}]} // const — no "type" required {"const": "fixed_value"} // empty schema — matches anything {} ``` All of the above would be rejected by the validator, raising a `ValidationError` when trying to construct a `SkillDefinition` with such schemas. ### Expected Behavior Valid JSON Schema objects should be accepted even without a top-level `"type"` key. ### Actual Behavior `SkillDefinition` raises `ValidationError` with message `"input_schema must include a 'type' key to be a valid JSON Schema"` for schemas using `allOf`, `oneOf`, `anyOf`, `$ref`, `const`, or an empty schema (`{}`). ### Suggested Fix Either remove the `"type"` key requirement and only check for `isinstance(schema, dict)`, or if more validation is desired, check for at least one recognized JSON Schema keyword: ```python _VALID_SCHEMA_KEYWORDS = frozenset({ "type", "allOf", "anyOf", "oneOf", "not", "$ref", "properties", "items", "const", "enum", "format", "minimum", "maximum", "minLength", "maxLength", "required", "additionalProperties", "description", }) if schema is not None: if not isinstance(schema, dict): raise ValueError(f"{label} must be a dict, got {type(schema)}") # Allow empty schema {} (matches anything) and any schema with known keywords if schema and not _VALID_SCHEMA_KEYWORDS.intersection(schema.keys()): raise ValueError( f"{label} does not appear to be a valid JSON Schema object" ) ``` ### Category spec-alignment ### TDD Note After this bug issue is verified, a corresponding Type/Testing issue will be created for TDD. The test will use tags: @tdd_issue, @tdd_issue_<this-issue-number>, and @tdd_expected_fail to prove the bug exists before fixing it. --- **Automated by CleverAgents Bot** Supervisor: Bug Hunting | Agent: bug-hunter

HAL9000 added the

labels

2026-04-09 22:07:20 +00:00

HAL9000 added this to the v3.2.0 milestone

2026-04-09 22:13:20 +00:00

HAL9000 added