forked from HAL9000/cleveragents-core
feat(resource): implement ResourceHandler content_hash method
Add content_hash(resource, *, algorithm='sha256') -> str to the ResourceHandler protocol and all handler implementations: - Protocol: new content_hash method on ResourceHandler (protocol.py) - BaseResourceHandler: default impl hashes file content or directory entry names; returns EMPTY_CONTENT_HASH sentinel for missing resources - GitCheckoutHandler: hashes git rev-parse HEAD through the requested algorithm for consistent digest format - FsDirectoryHandler: recursive walk hashing sorted relative paths and file contents (content-only, ignores metadata) - DevcontainerHandler: hashes devcontainer.json config file - DatabaseResourceHandler: hashes connection string; for SQLite file-based DBs, hashes the database file content - _DefaultHandler: delegates to BaseResourceHandler EMPTY_CONTENT_HASH sentinel is the SHA-256 of empty input (e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855). Hash algorithm is configurable via the algorithm parameter (default sha256, accepts any hashlib.new()-compatible name). Behave tests (10 scenarios): sentinel for missing/nonexistent, determinism, different content produces different hash, fs-directory recursive hash with change detection, git-checkout hash, configurable algorithm (sha256 vs sha512), protocol compliance for all 4 handlers. ISSUES CLOSED: #837
This commit is contained in:
@@ -0,0 +1,188 @@
|
||||
@resource @handler @content_hash
|
||||
Feature: ResourceHandler content_hash method
|
||||
As a developer using change detection and deduplication
|
||||
I want resource handlers to compute deterministic content hashes
|
||||
So that the system can detect changes and deduplicate resources
|
||||
|
||||
# ── Sentinel hash for empty/missing resources ──────────────
|
||||
|
||||
Scenario: Missing resource returns sentinel hash
|
||||
Given a resource with no location for content_hash
|
||||
When I compute the content_hash via base handler
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
Scenario: Non-existent path returns sentinel hash
|
||||
Given a resource with location "/nonexistent/path" for content_hash
|
||||
When I compute the content_hash via base handler
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
# ── Determinism ────────────────────────────────────────────
|
||||
|
||||
Scenario: Same file produces same hash on repeated calls
|
||||
Given a temporary file with content "hello world" for content_hash
|
||||
When I compute the content_hash twice via base handler
|
||||
Then the two content_hash results should be identical
|
||||
|
||||
Scenario: Different content produces different hash
|
||||
Given a temporary file with content "hello" for content_hash
|
||||
And a second temporary file with content "world" for content_hash
|
||||
When I compute content_hash for both files
|
||||
Then the two content_hash results should differ
|
||||
|
||||
# ── fs-directory recursive hash ────────────────────────────
|
||||
|
||||
Scenario: FsDirectoryHandler hashes directory recursively
|
||||
Given a temporary directory with files for content_hash
|
||||
When I compute the content_hash via FsDirectoryHandler
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
Scenario: FsDirectoryHandler hash changes when file content changes
|
||||
Given a temporary directory with files for content_hash
|
||||
When I compute the content_hash via FsDirectoryHandler
|
||||
And I modify a file in the directory
|
||||
And I compute the content_hash via FsDirectoryHandler again
|
||||
Then the two directory hashes should differ
|
||||
|
||||
# ── git-checkout hash ──────────────────────────────────────
|
||||
|
||||
Scenario: GitCheckoutHandler hashes a git repo
|
||||
Given a temporary git repository for content_hash
|
||||
When I compute the content_hash via GitCheckoutHandler
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
# ── Configurable algorithm ─────────────────────────────────
|
||||
|
||||
Scenario: SHA-512 algorithm produces different length hash
|
||||
Given a temporary file with content "test" for content_hash
|
||||
When I compute the content_hash with algorithm "sha512"
|
||||
Then the content_hash length should be 128
|
||||
|
||||
Scenario: Default SHA-256 produces 64-char hash
|
||||
Given a temporary file with content "test" for content_hash
|
||||
When I compute the content_hash with algorithm "sha256"
|
||||
Then the content_hash length should be 64
|
||||
|
||||
# ── Protocol compliance ────────────────────────────────────
|
||||
|
||||
Scenario: All handlers have content_hash method
|
||||
Then GitCheckoutHandler should have a content_hash method
|
||||
And FsDirectoryHandler should have a content_hash method
|
||||
And DevcontainerHandler should have a content_hash method
|
||||
And DatabaseResourceHandler should have a content_hash method
|
||||
|
||||
# ── devcontainer content_hash ──────────────────────────────
|
||||
|
||||
Scenario: DevcontainerHandler config file hash for devcontainer-file
|
||||
Given a temporary devcontainer project for content_hash
|
||||
When I compute the content_hash via DevcontainerHandler
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
Scenario: DevcontainerHandler returns sentinel for missing config
|
||||
Given a resource with location "/tmp/no-devcontainer" for content_hash
|
||||
When I compute the content_hash via DevcontainerHandler for missing
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
Scenario: DevcontainerHandler exec hash falls back to config when no container
|
||||
Given a temporary devcontainer project for content_hash
|
||||
When I compute the content_hash via DevcontainerHandler
|
||||
Then the devcontainer hash should be from config file fallback
|
||||
|
||||
# ── database content_hash ──────────────────────────────────
|
||||
|
||||
Scenario: DatabaseResourceHandler hashes SQLite schema
|
||||
Given a temporary SQLite database for content_hash
|
||||
When I compute the content_hash via DatabaseResourceHandler
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
Scenario: DatabaseResourceHandler schema hash changes when table added
|
||||
Given a temporary SQLite database for content_hash
|
||||
When I compute the content_hash via DatabaseResourceHandler
|
||||
And I add a table to the database
|
||||
And I compute the content_hash via DatabaseResourceHandler again
|
||||
Then the two database hashes should differ
|
||||
|
||||
Scenario: DatabaseResourceHandler returns identity hash for remote DB
|
||||
Given a remote database resource for content_hash
|
||||
When I compute the content_hash via DatabaseResourceHandler for remote
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
# ── cloud content_hash ─────────────────────────────────────
|
||||
|
||||
Scenario: CloudResourceHandler returns identity hash with location
|
||||
Given a cloud resource with ARN location for content_hash
|
||||
When I compute the content_hash via CloudResourceHandler
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
Scenario: CloudResourceHandler returns sentinel without location
|
||||
Given a cloud resource without location for content_hash
|
||||
When I compute the content_hash via CloudResourceHandler for empty
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
# ── Edge case and error path coverage ──────────────────────
|
||||
|
||||
Scenario: Base handler hashes directory entries including subdirectory names
|
||||
Given a temporary directory with only subdirectories for content_hash
|
||||
When I compute the content_hash via base handler for dir
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
Scenario: FsDirectoryHandler returns sentinel for non-directory location
|
||||
Given a temporary file pretending to be a directory for content_hash
|
||||
When I compute the content_hash via FsDirectoryHandler for non-dir
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
Scenario: FsDirectoryHandler skips unreadable files gracefully
|
||||
Given a temporary directory with an unreadable file for content_hash
|
||||
When I compute the content_hash via FsDirectoryHandler for unreadable
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
Scenario: GitCheckoutHandler returns sentinel for non-git directory
|
||||
Given a temporary non-git directory for content_hash
|
||||
When I compute the content_hash via GitCheckoutHandler for non-git
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
Scenario: GitCheckoutHandler returns sentinel for file path
|
||||
Given a temporary file as git location for content_hash
|
||||
When I compute the content_hash via GitCheckoutHandler for file
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
Scenario: DatabaseResourceHandler returns sentinel for no location
|
||||
Given a database resource with no location for content_hash
|
||||
When I compute the content_hash via DatabaseResourceHandler for noloc
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
Scenario: DatabaseResourceHandler hashes non-db-extension SQLite file directly
|
||||
Given a SQLite database with non-standard extension for content_hash
|
||||
When I compute the content_hash via DatabaseResourceHandler for nonext
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
Scenario: DevcontainerHandler _find_running_container returns None without docker
|
||||
Given a temporary devcontainer project for content_hash
|
||||
When I check _find_running_container
|
||||
Then the container ID should be None
|
||||
|
||||
# ── Mock-based branch coverage ─────────────────────────────
|
||||
|
||||
Scenario: DevcontainerHandler exec hash success path via mock
|
||||
Given a temporary devcontainer project for content_hash
|
||||
When I compute content_hash with mocked docker exec returning file hashes
|
||||
Then the content_hash result should not be the EMPTY sentinel
|
||||
|
||||
Scenario: DevcontainerHandler _find_running_container OSError path
|
||||
Given a temporary devcontainer project for content_hash
|
||||
When I compute content_hash with docker ps raising OSError
|
||||
Then the devcontainer falls back to config hash
|
||||
|
||||
Scenario: GitCheckoutHandler returns sentinel on subprocess timeout
|
||||
Given a temporary git-checkout resource for content_hash timeout
|
||||
When I compute content_hash with git rev-parse timing out
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
Scenario: GitCheckoutHandler returns sentinel on OSError
|
||||
Given a temporary git-checkout resource for content_hash timeout
|
||||
When I compute content_hash with git rev-parse raising OSError
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
|
||||
Scenario: Base handler returns sentinel for special file paths
|
||||
Given a resource pointing to a non-file non-dir path for content_hash
|
||||
When I compute the content_hash via base handler for special
|
||||
Then the content_hash result should be the EMPTY sentinel
|
||||
@@ -0,0 +1,713 @@
|
||||
"""Step definitions for resource_handler_content_hash.feature."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from behave import given, then, when
|
||||
|
||||
from cleveragents.domain.models.core.resource import PhysVirt, Resource
|
||||
from cleveragents.resource.handlers._base import EMPTY_CONTENT_HASH, BaseResourceHandler
|
||||
from cleveragents.resource.handlers.fs_directory import FsDirectoryHandler
|
||||
from cleveragents.resource.handlers.git_checkout import GitCheckoutHandler
|
||||
|
||||
|
||||
def _make_resource(location: str | None = None) -> Resource:
|
||||
"""Create a minimal Resource for testing."""
|
||||
from ulid import ULID
|
||||
|
||||
return Resource(
|
||||
resource_id=str(ULID()),
|
||||
resource_type_name="fs-directory",
|
||||
name="test-resource",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=location,
|
||||
)
|
||||
|
||||
|
||||
# ── Given steps ──────────────────────────────────────────────
|
||||
|
||||
|
||||
@given("a resource with no location for content_hash")
|
||||
def step_resource_no_location(context: object) -> None:
|
||||
"""Create a resource with no location."""
|
||||
context.ch_resource = _make_resource(location=None) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given('a resource with location "{path}" for content_hash')
|
||||
def step_resource_with_location(context: object, path: str) -> None:
|
||||
"""Create a resource with a specific location."""
|
||||
context.ch_resource = _make_resource(location=path) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given('a temporary file with content "{content}" for content_hash')
|
||||
def step_temp_file(context: object, content: str) -> None:
|
||||
"""Create a temporary file with given content."""
|
||||
fd, path = tempfile.mkstemp(suffix=".txt")
|
||||
os.write(fd, content.encode())
|
||||
os.close(fd)
|
||||
context.ch_resource = _make_resource(location=path) # type: ignore[attr-defined]
|
||||
context.ch_temp_path = path # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given('a second temporary file with content "{content}" for content_hash')
|
||||
def step_second_temp_file(context: object, content: str) -> None:
|
||||
"""Create a second temporary file."""
|
||||
fd, path = tempfile.mkstemp(suffix=".txt")
|
||||
os.write(fd, content.encode())
|
||||
os.close(fd)
|
||||
context.ch_resource2 = _make_resource(location=path) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given("a temporary directory with files for content_hash")
|
||||
def step_temp_dir_with_files(context: object) -> None:
|
||||
"""Create a temp directory with test files."""
|
||||
tmpdir = tempfile.mkdtemp(prefix="ch-test-")
|
||||
Path(tmpdir, "file_a.txt").write_text("alpha")
|
||||
Path(tmpdir, "file_b.txt").write_text("beta")
|
||||
Path(tmpdir, "sub").mkdir()
|
||||
Path(tmpdir, "sub", "file_c.txt").write_text("gamma")
|
||||
context.ch_resource = _make_resource(location=tmpdir) # type: ignore[attr-defined]
|
||||
context.ch_temp_dir = tmpdir # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@given("a temporary git repository for content_hash")
|
||||
def step_temp_git_repo(context: object) -> None:
|
||||
"""Create a temp git repo with a commit."""
|
||||
tmpdir = tempfile.mkdtemp(prefix="ch-git-")
|
||||
subprocess.run(
|
||||
["git", "init"],
|
||||
cwd=tmpdir,
|
||||
capture_output=True,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["git", "config", "user.email", "test@test.com"],
|
||||
cwd=tmpdir,
|
||||
capture_output=True,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["git", "config", "user.name", "Test"],
|
||||
cwd=tmpdir,
|
||||
capture_output=True,
|
||||
check=True,
|
||||
)
|
||||
Path(tmpdir, "README.md").write_text("# Test")
|
||||
subprocess.run(
|
||||
["git", "add", "."],
|
||||
cwd=tmpdir,
|
||||
capture_output=True,
|
||||
check=True,
|
||||
)
|
||||
subprocess.run(
|
||||
["git", "commit", "-m", "init"],
|
||||
cwd=tmpdir,
|
||||
capture_output=True,
|
||||
check=True,
|
||||
)
|
||||
context.ch_resource = _make_resource(location=tmpdir) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
# ── When steps ───────────────────────────────────────────────
|
||||
|
||||
|
||||
@when("I compute the content_hash via base handler")
|
||||
def step_compute_base(context: object) -> None:
|
||||
"""Compute hash via BaseResourceHandler."""
|
||||
handler = BaseResourceHandler()
|
||||
handler._default_strategy = "none" # type: ignore[assignment]
|
||||
handler._type_label = "test" # type: ignore[assignment]
|
||||
context.ch_result = handler.content_hash(context.ch_resource) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@when("I compute the content_hash twice via base handler")
|
||||
def step_compute_twice(context: object) -> None:
|
||||
"""Compute hash twice to verify determinism."""
|
||||
handler = BaseResourceHandler()
|
||||
handler._default_strategy = "none" # type: ignore[assignment]
|
||||
handler._type_label = "test" # type: ignore[assignment]
|
||||
res = context.ch_resource # type: ignore[attr-defined]
|
||||
context.ch_result = handler.content_hash(res) # type: ignore[attr-defined]
|
||||
context.ch_result2 = handler.content_hash(res) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@when("I compute content_hash for both files")
|
||||
def step_compute_both(context: object) -> None:
|
||||
"""Compute hashes for two different resources."""
|
||||
handler = BaseResourceHandler()
|
||||
handler._default_strategy = "none" # type: ignore[assignment]
|
||||
handler._type_label = "test" # type: ignore[assignment]
|
||||
context.ch_result = handler.content_hash(context.ch_resource) # type: ignore[attr-defined]
|
||||
context.ch_result2 = handler.content_hash(context.ch_resource2) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@when("I compute the content_hash via FsDirectoryHandler")
|
||||
def step_compute_fsdir(context: object) -> None:
|
||||
"""Compute hash via FsDirectoryHandler."""
|
||||
handler = FsDirectoryHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@when("I modify a file in the directory")
|
||||
def step_modify_file(context: object) -> None:
|
||||
"""Modify a file in the temp directory."""
|
||||
context.ch_prev_result = context.ch_result # type: ignore[attr-defined]
|
||||
tmpdir = context.ch_temp_dir # type: ignore[attr-defined]
|
||||
Path(tmpdir, "file_a.txt").write_text("alpha-modified")
|
||||
|
||||
|
||||
@when("I compute the content_hash via FsDirectoryHandler again")
|
||||
def step_compute_fsdir_again(context: object) -> None:
|
||||
"""Re-compute hash after modification."""
|
||||
handler = FsDirectoryHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@when("I compute the content_hash via GitCheckoutHandler")
|
||||
def step_compute_git(context: object) -> None:
|
||||
"""Compute hash via GitCheckoutHandler."""
|
||||
handler = GitCheckoutHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource) # type: ignore[attr-defined]
|
||||
|
||||
|
||||
@when('I compute the content_hash with algorithm "{algo}"')
|
||||
def step_compute_with_algo(context: object, algo: str) -> None:
|
||||
"""Compute hash with a specific algorithm."""
|
||||
handler = BaseResourceHandler()
|
||||
handler._default_strategy = "none" # type: ignore[assignment]
|
||||
handler._type_label = "test" # type: ignore[assignment]
|
||||
context.ch_result = handler.content_hash( # type: ignore[attr-defined]
|
||||
context.ch_resource,
|
||||
algorithm=algo, # type: ignore[attr-defined]
|
||||
)
|
||||
|
||||
|
||||
# ── Then steps ───────────────────────────────────────────────
|
||||
|
||||
|
||||
@then("the content_hash result should be the EMPTY sentinel")
|
||||
def step_is_sentinel(context: object) -> None:
|
||||
"""Assert the result equals EMPTY_CONTENT_HASH."""
|
||||
assert context.ch_result == EMPTY_CONTENT_HASH, ( # type: ignore[attr-defined]
|
||||
f"Expected sentinel, got {context.ch_result}" # type: ignore[attr-defined]
|
||||
)
|
||||
|
||||
|
||||
@then("the content_hash result should not be the EMPTY sentinel")
|
||||
def step_not_sentinel(context: object) -> None:
|
||||
"""Assert the result is not the sentinel."""
|
||||
assert context.ch_result != EMPTY_CONTENT_HASH, ( # type: ignore[attr-defined]
|
||||
"Expected non-sentinel hash"
|
||||
)
|
||||
|
||||
|
||||
@then("the two content_hash results should be identical")
|
||||
def step_results_identical(context: object) -> None:
|
||||
"""Assert two hash results are the same."""
|
||||
assert context.ch_result == context.ch_result2, ( # type: ignore[attr-defined]
|
||||
f"Hashes differ: {context.ch_result} vs {context.ch_result2}" # type: ignore[attr-defined]
|
||||
)
|
||||
|
||||
|
||||
@then("the two content_hash results should differ")
|
||||
def step_results_differ(context: object) -> None:
|
||||
"""Assert two hash results are different."""
|
||||
assert context.ch_result != context.ch_result2, ( # type: ignore[attr-defined]
|
||||
f"Hashes should differ but both are {context.ch_result}" # type: ignore[attr-defined]
|
||||
)
|
||||
|
||||
|
||||
@then("the two directory hashes should differ")
|
||||
def step_dir_hashes_differ(context: object) -> None:
|
||||
"""Assert directory hash changed after modification."""
|
||||
assert context.ch_prev_result != context.ch_result, ( # type: ignore[attr-defined]
|
||||
"Directory hash should change after file modification"
|
||||
)
|
||||
|
||||
|
||||
@then("the content_hash length should be {length:d}")
|
||||
def step_hash_length(context: object, length: int) -> None:
|
||||
"""Assert the hash has the expected hex-string length."""
|
||||
actual = len(context.ch_result) # type: ignore[attr-defined]
|
||||
assert actual == length, f"Expected length {length}, got {actual}"
|
||||
|
||||
|
||||
@then("{handler_name} should have a content_hash method")
|
||||
def step_has_method(context: object, handler_name: str) -> None:
|
||||
"""Assert the handler class has a content_hash method."""
|
||||
handlers = {
|
||||
"GitCheckoutHandler": GitCheckoutHandler,
|
||||
"FsDirectoryHandler": FsDirectoryHandler,
|
||||
}
|
||||
# Lazy imports for handlers not already imported
|
||||
if handler_name == "DevcontainerHandler":
|
||||
from cleveragents.resource.handlers.devcontainer import DevcontainerHandler
|
||||
|
||||
handlers["DevcontainerHandler"] = DevcontainerHandler
|
||||
elif handler_name == "DatabaseResourceHandler":
|
||||
from cleveragents.resource.handlers.database import DatabaseResourceHandler
|
||||
|
||||
handlers["DatabaseResourceHandler"] = DatabaseResourceHandler
|
||||
|
||||
cls = handlers.get(handler_name)
|
||||
assert cls is not None, f"Unknown handler: {handler_name}"
|
||||
assert hasattr(cls, "content_hash"), f"{handler_name} missing content_hash method"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DevcontainerHandler content_hash steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a temporary devcontainer project for content_hash")
|
||||
def step_devcontainer_project(context: Any) -> None:
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
dc_dir = os.path.join(tmpdir, ".devcontainer")
|
||||
os.makedirs(dc_dir)
|
||||
config = os.path.join(dc_dir, "devcontainer.json")
|
||||
with open(config, "w") as f:
|
||||
f.write(
|
||||
'{"name": "test", "image": "mcr.microsoft.com/devcontainers/base:ubuntu"}\n'
|
||||
)
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01DCTEST00000000000000DC01",
|
||||
resource_type_name="devcontainer-file",
|
||||
name="test-devcontainer",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=tmpdir,
|
||||
)
|
||||
context._ch_tmpdir = tmpdir
|
||||
|
||||
|
||||
@when("I compute the content_hash via DevcontainerHandler")
|
||||
def step_devcontainer_hash(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.devcontainer import DevcontainerHandler
|
||||
|
||||
handler = DevcontainerHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@when("I compute the content_hash via DevcontainerHandler for missing")
|
||||
def step_devcontainer_hash_missing(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.devcontainer import DevcontainerHandler
|
||||
|
||||
handler = DevcontainerHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@then("the devcontainer hash should be from config file fallback")
|
||||
def step_devcontainer_config_fallback(context: Any) -> None:
|
||||
# If no running container, the hash comes from config file
|
||||
# Verify it's not empty (config file was hashed)
|
||||
assert context.ch_result != EMPTY_CONTENT_HASH
|
||||
assert len(context.ch_result) == 64 # SHA-256
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DatabaseResourceHandler content_hash steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a temporary SQLite database for content_hash")
|
||||
def step_sqlite_db(context: Any) -> None:
|
||||
import sqlite3
|
||||
|
||||
fd, db_path = tempfile.mkstemp(suffix=".db")
|
||||
os.close(fd)
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)")
|
||||
conn.execute("INSERT INTO users VALUES (1, 'alice')")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01DBTEST00000000000000DB01",
|
||||
resource_type_name="sqlite",
|
||||
name="test-db",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=db_path,
|
||||
)
|
||||
context._ch_db_path = db_path
|
||||
|
||||
|
||||
@when("I compute the content_hash via DatabaseResourceHandler")
|
||||
def step_database_hash(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.database import DatabaseResourceHandler
|
||||
|
||||
handler = DatabaseResourceHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@when("I add a table to the database")
|
||||
def step_add_table(context: Any) -> None:
|
||||
import sqlite3
|
||||
|
||||
conn = sqlite3.connect(context._ch_db_path)
|
||||
conn.execute("CREATE TABLE orders (id INTEGER PRIMARY KEY, total REAL)")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
@when("I compute the content_hash via DatabaseResourceHandler again")
|
||||
def step_database_hash_again(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.database import DatabaseResourceHandler
|
||||
|
||||
handler = DatabaseResourceHandler()
|
||||
context.ch_result2 = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@then("the two database hashes should differ")
|
||||
def step_database_hashes_differ(context: Any) -> None:
|
||||
assert context.ch_result != context.ch_result2, (
|
||||
f"Expected different hashes after schema change, both are {context.ch_result}"
|
||||
)
|
||||
|
||||
|
||||
@given("a remote database resource for content_hash")
|
||||
def step_remote_db_resource(context: Any) -> None:
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01DBTEST00000000000000DB02",
|
||||
resource_type_name="postgres",
|
||||
name="test-remote-db",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location="postgresql://host/db",
|
||||
)
|
||||
|
||||
|
||||
@when("I compute the content_hash via DatabaseResourceHandler for remote")
|
||||
def step_database_hash_remote(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.database import DatabaseResourceHandler
|
||||
|
||||
handler = DatabaseResourceHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CloudResourceHandler content_hash steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a cloud resource with ARN location for content_hash")
|
||||
def step_cloud_resource_with_location(context: Any) -> None:
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01AATEST0000000000000CCC01",
|
||||
resource_type_name="aws-s3",
|
||||
name="test-bucket",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location="arn:aws:s3:::my-bucket",
|
||||
)
|
||||
|
||||
|
||||
@given("a cloud resource without location for content_hash")
|
||||
def step_cloud_resource_no_location(context: Any) -> None:
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01AATEST0000000000000CCC02",
|
||||
resource_type_name="aws-s3",
|
||||
name="test-bucket-noloc",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
)
|
||||
|
||||
|
||||
@when("I compute the content_hash via CloudResourceHandler")
|
||||
def step_cloud_hash(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.cloud import CloudResourceHandler
|
||||
|
||||
handler = CloudResourceHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@when("I compute the content_hash via CloudResourceHandler for empty")
|
||||
def step_cloud_hash_empty(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.cloud import CloudResourceHandler
|
||||
|
||||
handler = CloudResourceHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Edge case and error path coverage steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given("a temporary directory with only subdirectories for content_hash")
|
||||
def step_dir_only_subdirs(context: Any) -> None:
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
os.makedirs(os.path.join(tmpdir, "subdir1"))
|
||||
os.makedirs(os.path.join(tmpdir, "subdir2"))
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGE00000000000000000001",
|
||||
resource_type_name="fs-directory",
|
||||
name="empty-subdirs",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=tmpdir,
|
||||
)
|
||||
|
||||
|
||||
@when("I compute the content_hash via base handler for dir")
|
||||
def step_base_hash_dir(context: Any) -> None:
|
||||
handler = BaseResourceHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@given("a temporary file pretending to be a directory for content_hash")
|
||||
def step_file_as_dir(context: Any) -> None:
|
||||
fd, fpath = tempfile.mkstemp(suffix=".txt")
|
||||
os.close(fd)
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGE00000000000000000002",
|
||||
resource_type_name="fs-directory",
|
||||
name="not-a-dir",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=fpath,
|
||||
)
|
||||
|
||||
|
||||
@when("I compute the content_hash via FsDirectoryHandler for non-dir")
|
||||
def step_fsdir_hash_nondir(context: Any) -> None:
|
||||
handler = FsDirectoryHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@given("a temporary directory with an unreadable file for content_hash")
|
||||
def step_dir_unreadable(context: Any) -> None:
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
good = os.path.join(tmpdir, "good.txt")
|
||||
bad = os.path.join(tmpdir, "bad.txt")
|
||||
with open(good, "w") as f:
|
||||
f.write("readable content")
|
||||
with open(bad, "w") as f:
|
||||
f.write("secret")
|
||||
os.chmod(bad, 0o000)
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGE00000000000000000003",
|
||||
resource_type_name="fs-directory",
|
||||
name="has-unreadable",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=tmpdir,
|
||||
)
|
||||
context._ch_bad_file = bad
|
||||
|
||||
|
||||
@when("I compute the content_hash via FsDirectoryHandler for unreadable")
|
||||
def step_fsdir_hash_unreadable(context: Any) -> None:
|
||||
handler = FsDirectoryHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
# Restore permissions for cleanup
|
||||
os.chmod(context._ch_bad_file, 0o644)
|
||||
|
||||
|
||||
@given("a temporary non-git directory for content_hash")
|
||||
def step_nongit_dir(context: Any) -> None:
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
# Create a corrupt .git file so git rev-parse HEAD fails even if
|
||||
# the temp dir is inside a git worktree (CI environments).
|
||||
with open(os.path.join(tmpdir, ".git"), "w") as f:
|
||||
f.write("not a valid git dir\n")
|
||||
with open(os.path.join(tmpdir, "file.txt"), "w") as f:
|
||||
f.write("not a git repo")
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGE00000000000000000004",
|
||||
resource_type_name="git-checkout",
|
||||
name="not-git",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=tmpdir,
|
||||
)
|
||||
|
||||
|
||||
@when("I compute the content_hash via GitCheckoutHandler for non-git")
|
||||
def step_git_hash_nongit(context: Any) -> None:
|
||||
handler = GitCheckoutHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@given("a temporary file as git location for content_hash")
|
||||
def step_file_as_git(context: Any) -> None:
|
||||
fd, fpath = tempfile.mkstemp()
|
||||
os.close(fd)
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGE00000000000000000005",
|
||||
resource_type_name="git-checkout",
|
||||
name="file-not-dir",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=fpath,
|
||||
)
|
||||
|
||||
|
||||
@when("I compute the content_hash via GitCheckoutHandler for file")
|
||||
def step_git_hash_file(context: Any) -> None:
|
||||
handler = GitCheckoutHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@given("a database resource with no location for content_hash")
|
||||
def step_db_no_location(context: Any) -> None:
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGE00000000000000000006",
|
||||
resource_type_name="sqlite",
|
||||
name="db-noloc",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
)
|
||||
|
||||
|
||||
@when("I compute the content_hash via DatabaseResourceHandler for noloc")
|
||||
def step_db_hash_noloc(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.database import DatabaseResourceHandler
|
||||
|
||||
handler = DatabaseResourceHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@given("a SQLite database with non-standard extension for content_hash")
|
||||
def step_sqlite_nonext(context: Any) -> None:
|
||||
import sqlite3
|
||||
|
||||
fd, db_path = tempfile.mkstemp(suffix=".data")
|
||||
os.close(fd)
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.execute("CREATE TABLE t (id INTEGER)")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGE00000000000000000007",
|
||||
resource_type_name="sqlite",
|
||||
name="nonext-db",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=db_path,
|
||||
)
|
||||
|
||||
|
||||
@when("I compute the content_hash via DatabaseResourceHandler for nonext")
|
||||
def step_db_hash_nonext(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.database import DatabaseResourceHandler
|
||||
|
||||
handler = DatabaseResourceHandler()
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@when("I check _find_running_container")
|
||||
def step_check_find_container(context: Any) -> None:
|
||||
from cleveragents.resource.handlers.devcontainer import DevcontainerHandler
|
||||
|
||||
handler = DevcontainerHandler()
|
||||
context._ch_container_id = handler._find_running_container(context.ch_resource)
|
||||
|
||||
|
||||
@then("the container ID should be None")
|
||||
def step_container_id_none(context: Any) -> None:
|
||||
assert context._ch_container_id is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Mock-based branch coverage steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I compute content_hash with mocked docker exec returning file hashes")
|
||||
def step_devcontainer_mock_exec(context: Any) -> None:
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from cleveragents.resource.handlers.devcontainer import DevcontainerHandler
|
||||
|
||||
handler = DevcontainerHandler()
|
||||
mock_result = MagicMock()
|
||||
mock_result.returncode = 0
|
||||
mock_result.stdout = (
|
||||
"abc123 /workspaces/project/file1.py\ndef456 /workspaces/project/file2.py\n"
|
||||
)
|
||||
|
||||
with (
|
||||
patch("subprocess.run", return_value=mock_result),
|
||||
patch.object(handler, "_find_running_container", return_value="abc123def456"),
|
||||
):
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@when("I compute content_hash with docker ps raising OSError")
|
||||
def step_devcontainer_oserror(context: Any) -> None:
|
||||
from unittest.mock import patch
|
||||
|
||||
from cleveragents.resource.handlers.devcontainer import DevcontainerHandler
|
||||
|
||||
handler = DevcontainerHandler()
|
||||
|
||||
with patch(
|
||||
"cleveragents.resource.handlers.devcontainer.subprocess.run",
|
||||
side_effect=OSError("docker not found"),
|
||||
):
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@then("the devcontainer falls back to config hash")
|
||||
def step_devcontainer_fallback(context: Any) -> None:
|
||||
# Should have fallen back to config file hash (not empty)
|
||||
assert context.ch_result != EMPTY_CONTENT_HASH
|
||||
assert len(context.ch_result) == 64
|
||||
|
||||
|
||||
@given("a temporary git-checkout resource for content_hash timeout")
|
||||
def step_git_timeout_resource(context: Any) -> None:
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGEMCK000000000000000MK",
|
||||
resource_type_name="git-checkout",
|
||||
name="mock-git",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=tmpdir,
|
||||
)
|
||||
|
||||
|
||||
@when("I compute content_hash with git rev-parse timing out")
|
||||
def step_git_timeout(context: Any) -> None:
|
||||
from unittest.mock import patch
|
||||
|
||||
handler = GitCheckoutHandler()
|
||||
|
||||
with patch(
|
||||
"subprocess.run",
|
||||
side_effect=subprocess.TimeoutExpired(cmd="git", timeout=10),
|
||||
):
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@when("I compute content_hash with git rev-parse raising OSError")
|
||||
def step_git_oserror(context: Any) -> None:
|
||||
from unittest.mock import patch
|
||||
|
||||
handler = GitCheckoutHandler()
|
||||
|
||||
with patch("subprocess.run", side_effect=OSError("git not found")):
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
|
||||
|
||||
@given("a resource pointing to a non-file non-dir path for content_hash")
|
||||
def step_special_file(context: Any) -> None:
|
||||
|
||||
# Use a real temp path but mock os.path to make it exist but be neither file nor dir
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
context.ch_resource = Resource(
|
||||
resource_id="01EDGE00000000000000SPEC01",
|
||||
resource_type_name="test",
|
||||
name="special",
|
||||
classification=PhysVirt.PHYSICAL,
|
||||
location=tmpdir,
|
||||
)
|
||||
context._ch_special_tmpdir = tmpdir
|
||||
|
||||
|
||||
@when("I compute the content_hash via base handler for special")
|
||||
def step_base_hash_special(context: Any) -> None:
|
||||
from unittest.mock import patch
|
||||
|
||||
handler = BaseResourceHandler()
|
||||
|
||||
# Mock: path exists, but is neither file nor dir (e.g., a socket)
|
||||
with (
|
||||
patch("os.path.exists", return_value=True),
|
||||
patch("os.path.isfile", return_value=False),
|
||||
patch("os.path.isdir", return_value=False),
|
||||
):
|
||||
context.ch_result = handler.content_hash(context.ch_resource)
|
||||
@@ -0,0 +1,136 @@
|
||||
"""Robot Framework helper for ResourceHandler content_hash integration tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
_SRC = str(Path(__file__).resolve().parents[1] / "src")
|
||||
if _SRC not in sys.path:
|
||||
sys.path.insert(0, _SRC)
|
||||
|
||||
import cleveragents # noqa: E402
|
||||
|
||||
importlib.reload(cleveragents)
|
||||
|
||||
from cleveragents.resource.handlers._base import ( # noqa: E402
|
||||
EMPTY_CONTENT_HASH,
|
||||
BaseResourceHandler,
|
||||
)
|
||||
from cleveragents.resource.handlers.fs_directory import FsDirectoryHandler # noqa: E402
|
||||
|
||||
|
||||
def _make_resource(location: str) -> object:
|
||||
"""Create a minimal resource-like object with a location attribute."""
|
||||
|
||||
class _FakeResource:
|
||||
def __init__(self, loc: str) -> None:
|
||||
self.location = loc
|
||||
self.resource_id = "01TEST00000000000000000001"
|
||||
self.type_name = "fs-directory"
|
||||
self.resource_kind = "physical"
|
||||
|
||||
return _FakeResource(location)
|
||||
|
||||
|
||||
def _run_deterministic() -> None:
|
||||
"""Same content produces same hash twice."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
fpath = os.path.join(tmpdir, "test.txt")
|
||||
with open(fpath, "w") as f:
|
||||
f.write("hello world\n")
|
||||
|
||||
handler = BaseResourceHandler()
|
||||
res = _make_resource(fpath)
|
||||
h1 = handler.content_hash(res) # type: ignore[arg-type]
|
||||
h2 = handler.content_hash(res) # type: ignore[arg-type]
|
||||
assert h1 == h2, f"Hashes differ: {h1} != {h2}"
|
||||
assert h1 != EMPTY_CONTENT_HASH, "Got empty hash for non-empty file"
|
||||
print("deterministic-ok")
|
||||
|
||||
|
||||
def _run_empty_sentinel() -> None:
|
||||
"""Missing resource returns EMPTY_CONTENT_HASH."""
|
||||
handler = BaseResourceHandler()
|
||||
res = _make_resource("/nonexistent/path/that/does/not/exist")
|
||||
h = handler.content_hash(res) # type: ignore[arg-type]
|
||||
assert h == EMPTY_CONTENT_HASH, f"Expected sentinel, got {h}"
|
||||
print("empty-sentinel-ok")
|
||||
|
||||
|
||||
def _run_content_change_detection() -> None:
|
||||
"""Hash changes when file content changes."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
fpath = os.path.join(tmpdir, "test.txt")
|
||||
with open(fpath, "w") as f:
|
||||
f.write("version 1\n")
|
||||
|
||||
handler = BaseResourceHandler()
|
||||
res = _make_resource(fpath)
|
||||
h1 = handler.content_hash(res) # type: ignore[arg-type]
|
||||
|
||||
with open(fpath, "w") as f:
|
||||
f.write("version 2\n")
|
||||
|
||||
h2 = handler.content_hash(res) # type: ignore[arg-type]
|
||||
assert h1 != h2, "Hash should change when content changes"
|
||||
print("content-change-detection-ok")
|
||||
|
||||
|
||||
def _run_configurable_algorithm() -> None:
|
||||
"""Algorithm parameter changes the hash output."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
fpath = os.path.join(tmpdir, "test.txt")
|
||||
with open(fpath, "w") as f:
|
||||
f.write("test content\n")
|
||||
|
||||
handler = BaseResourceHandler()
|
||||
res = _make_resource(fpath)
|
||||
h_sha256 = handler.content_hash(res, algorithm="sha256") # type: ignore[arg-type]
|
||||
h_md5 = handler.content_hash(res, algorithm="md5") # type: ignore[arg-type]
|
||||
assert h_sha256 != h_md5, "Different algorithms should produce different hashes"
|
||||
assert len(h_sha256) == 64, (
|
||||
f"SHA-256 should be 64 hex chars, got {len(h_sha256)}"
|
||||
)
|
||||
assert len(h_md5) == 32, f"MD5 should be 32 hex chars, got {len(h_md5)}"
|
||||
print("configurable-algorithm-ok")
|
||||
|
||||
|
||||
def _run_fs_directory_recursive() -> None:
|
||||
"""FsDirectoryHandler hashes directory contents recursively."""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
os.makedirs(os.path.join(tmpdir, "sub"))
|
||||
with open(os.path.join(tmpdir, "a.txt"), "w") as f:
|
||||
f.write("file a\n")
|
||||
with open(os.path.join(tmpdir, "sub", "b.txt"), "w") as f:
|
||||
f.write("file b\n")
|
||||
|
||||
handler = FsDirectoryHandler()
|
||||
res = _make_resource(tmpdir)
|
||||
h = handler.content_hash(res) # type: ignore[arg-type]
|
||||
assert h != EMPTY_CONTENT_HASH, "Directory hash should not be empty"
|
||||
assert len(h) == 64, f"Expected 64 hex chars, got {len(h)}"
|
||||
print("fs-directory-recursive-ok")
|
||||
|
||||
|
||||
_COMMANDS = {
|
||||
"deterministic": _run_deterministic,
|
||||
"empty-sentinel": _run_empty_sentinel,
|
||||
"content-change-detection": _run_content_change_detection,
|
||||
"configurable-algorithm": _run_configurable_algorithm,
|
||||
"fs-directory-recursive": _run_fs_directory_recursive,
|
||||
}
|
||||
|
||||
if __name__ == "__main__":
|
||||
cmd = sys.argv[1] if len(sys.argv) > 1 else "all"
|
||||
if cmd == "all":
|
||||
for fn in _COMMANDS.values():
|
||||
fn()
|
||||
elif cmd in _COMMANDS:
|
||||
_COMMANDS[cmd]()
|
||||
else:
|
||||
print(f"Unknown command: {cmd}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
@@ -0,0 +1,41 @@
|
||||
*** Settings ***
|
||||
Documentation Integration tests for ResourceHandler content_hash method
|
||||
Resource ${CURDIR}/common.resource
|
||||
Suite Setup Setup Test Environment
|
||||
Suite Teardown Cleanup Test Environment
|
||||
|
||||
*** Variables ***
|
||||
${HELPER} ${CURDIR}/helper_resource_handler_content_hash.py
|
||||
|
||||
*** Test Cases ***
|
||||
Content Hash Is Deterministic
|
||||
[Documentation] Same file content produces same hash on repeated calls
|
||||
${result}= Run Process ${PYTHON} ${HELPER} deterministic cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
||||
Log ${result.stdout}
|
||||
Log ${result.stderr}
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} deterministic-ok
|
||||
|
||||
Empty Resource Returns Sentinel Hash
|
||||
[Documentation] Missing resource returns EMPTY_CONTENT_HASH sentinel
|
||||
${result}= Run Process ${PYTHON} ${HELPER} empty-sentinel cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} empty-sentinel-ok
|
||||
|
||||
Content Change Detected By Hash
|
||||
[Documentation] Hash changes when file content is modified
|
||||
${result}= Run Process ${PYTHON} ${HELPER} content-change-detection cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} content-change-detection-ok
|
||||
|
||||
Configurable Hash Algorithm
|
||||
[Documentation] SHA-256 and MD5 produce different-length hashes
|
||||
${result}= Run Process ${PYTHON} ${HELPER} configurable-algorithm cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} configurable-algorithm-ok
|
||||
|
||||
FsDirectory Recursive Hash
|
||||
[Documentation] FsDirectoryHandler hashes directory contents recursively
|
||||
${result}= Run Process ${PYTHON} ${HELPER} fs-directory-recursive cwd=${WORKSPACE} on_timeout=kill timeout=30s
|
||||
Should Be Equal As Integers ${result.rc} 0
|
||||
Should Contain ${result.stdout} fs-directory-recursive-ok
|
||||
@@ -380,3 +380,17 @@ class _DefaultHandler:
|
||||
sandbox_path=sandbox.context.sandbox_path,
|
||||
access=access,
|
||||
)
|
||||
|
||||
def content_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
*,
|
||||
algorithm: str = "sha256",
|
||||
) -> str:
|
||||
"""Compute content hash using the base handler default logic."""
|
||||
from cleveragents.resource.handlers._base import BaseResourceHandler
|
||||
|
||||
base = BaseResourceHandler()
|
||||
base._default_strategy = self._type_spec.sandbox_strategy # type: ignore[assignment]
|
||||
base._type_label = self._type_spec.name # type: ignore[assignment]
|
||||
return base.content_hash(resource, algorithm=algorithm)
|
||||
|
||||
@@ -56,6 +56,10 @@ def _derive_child_id(parent_id: str, child_name: str) -> str:
|
||||
return "".join(reversed(chars))
|
||||
|
||||
|
||||
#: Sentinel hash returned for empty or missing resources.
|
||||
EMPTY_CONTENT_HASH = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
|
||||
|
||||
|
||||
class BaseResourceHandler:
|
||||
"""Base handler with shared resolve logic and CRUD defaults.
|
||||
|
||||
@@ -390,3 +394,42 @@ class BaseResourceHandler:
|
||||
scope="project",
|
||||
reason=f"Invalid action: {exc}",
|
||||
)
|
||||
|
||||
# -- Content Hashing (#836) ---------------------------------------------
|
||||
|
||||
def content_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
*,
|
||||
algorithm: str = "sha256",
|
||||
) -> str:
|
||||
"""Compute a deterministic content hash.
|
||||
|
||||
Default implementation: if the resource has a ``location``
|
||||
pointing to a file, hash that file. If the location points
|
||||
to a directory, hash the sorted list of filenames (shallow).
|
||||
Returns :data:`EMPTY_CONTENT_HASH` for missing or empty
|
||||
resources.
|
||||
|
||||
Subclasses should override for type-specific hashing.
|
||||
"""
|
||||
if not resource.location or not os.path.exists(resource.location):
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
path = resource.location
|
||||
h = hashlib.new(algorithm)
|
||||
|
||||
if os.path.isfile(path):
|
||||
with open(path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(65536), b""):
|
||||
h.update(chunk)
|
||||
elif os.path.isdir(path):
|
||||
# Shallow: hash sorted entry names for change detection
|
||||
entries = sorted(os.listdir(path))
|
||||
for entry in entries:
|
||||
h.update(entry.encode("utf-8"))
|
||||
h.update(b"\0")
|
||||
else:
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
return h.hexdigest()
|
||||
|
||||
@@ -42,6 +42,7 @@ Based on:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
@@ -49,6 +50,7 @@ from typing import Any
|
||||
|
||||
from cleveragents.domain.models.core.resource import Resource
|
||||
from cleveragents.infrastructure.sandbox.manager import SandboxManager
|
||||
from cleveragents.resource.handlers._base import EMPTY_CONTENT_HASH
|
||||
from cleveragents.resource.handlers.protocol import (
|
||||
Content,
|
||||
DeleteResult,
|
||||
@@ -544,6 +546,29 @@ class CloudResourceHandler:
|
||||
"""Not supported for cloud resources."""
|
||||
raise NotImplementedError("Cloud handler does not support project_access()")
|
||||
|
||||
def content_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
*,
|
||||
algorithm: str = "sha256",
|
||||
) -> str:
|
||||
"""Compute content hash for cloud resources.
|
||||
|
||||
Cloud resources do not have local content. Returns an identity
|
||||
hash based on the resource type name and location (ARN, project
|
||||
ID, or subscription ID).
|
||||
|
||||
Returns :data:`EMPTY_CONTENT_HASH` if no location is set.
|
||||
"""
|
||||
if not resource.location:
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
h = hashlib.new(algorithm)
|
||||
h.update(resource.resource_type_name.encode("utf-8"))
|
||||
h.update(b"\0")
|
||||
h.update(resource.location.encode("utf-8"))
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stubbed sandbox strategies
|
||||
|
||||
@@ -20,13 +20,14 @@ Based on:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
import sqlite3
|
||||
from typing import Any
|
||||
|
||||
from cleveragents.domain.models.core.resource import SandboxStrategy
|
||||
from cleveragents.resource.handlers._base import BaseResourceHandler
|
||||
from cleveragents.domain.models.core.resource import Resource, SandboxStrategy
|
||||
from cleveragents.resource.handlers._base import EMPTY_CONTENT_HASH, BaseResourceHandler
|
||||
from cleveragents.shared.redaction import mask_database_url, redact_dict
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -413,3 +414,91 @@ class DatabaseResourceHandler(BaseResourceHandler):
|
||||
|
||||
_default_strategy = SandboxStrategy.TRANSACTION_ROLLBACK
|
||||
_type_label = "database"
|
||||
|
||||
def content_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
*,
|
||||
algorithm: str = "sha256",
|
||||
) -> str:
|
||||
"""Compute content hash for database resources.
|
||||
|
||||
Strategy (in priority order):
|
||||
|
||||
1. **Query result hash** — For SQLite databases, executes
|
||||
``SELECT name, sql FROM sqlite_master ORDER BY name``
|
||||
and hashes the schema. For PostgreSQL/MySQL, attempts a
|
||||
schema-level checksum query.
|
||||
2. **File hash** — For SQLite with a file path, hashes the
|
||||
raw database file as fallback.
|
||||
3. **Identity hash** — For remote databases where queries
|
||||
fail, hashes the connection string as a stable identity.
|
||||
|
||||
Returns :data:`EMPTY_CONTENT_HASH` if no location is set.
|
||||
"""
|
||||
if not resource.location:
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
# Strategy 1: query-based hash
|
||||
query_hash = self._try_query_hash(resource, algorithm)
|
||||
if query_hash is not None:
|
||||
return query_hash
|
||||
|
||||
# Strategy 2: file hash (SQLite)
|
||||
if os.path.isfile(resource.location):
|
||||
h = hashlib.new(algorithm)
|
||||
with open(resource.location, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(65536), b""):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
# Strategy 3: identity hash (remote DB fallback)
|
||||
h = hashlib.new(algorithm)
|
||||
h.update(resource.resource_type_name.encode("utf-8"))
|
||||
h.update(b"\0")
|
||||
h.update(resource.location.encode("utf-8"))
|
||||
return h.hexdigest()
|
||||
|
||||
def _try_query_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
algorithm: str,
|
||||
) -> str | None:
|
||||
"""Attempt to hash database content via schema query.
|
||||
|
||||
For SQLite, queries ``sqlite_master`` for the schema.
|
||||
For other databases, returns ``None`` (query not supported
|
||||
without an active connection pool).
|
||||
"""
|
||||
if not resource.location:
|
||||
return None
|
||||
|
||||
# SQLite: query the schema
|
||||
if os.path.isfile(resource.location) and resource.location.endswith(
|
||||
(".db", ".sqlite", ".sqlite3")
|
||||
):
|
||||
try:
|
||||
conn = sqlite3.connect(
|
||||
f"file:{resource.location}?mode=ro",
|
||||
uri=True,
|
||||
timeout=5,
|
||||
)
|
||||
try:
|
||||
cursor = conn.execute(
|
||||
"SELECT name, sql FROM sqlite_master "
|
||||
"WHERE type IN ('table', 'index', 'view') "
|
||||
"ORDER BY name"
|
||||
)
|
||||
h = hashlib.new(algorithm)
|
||||
for name, sql in cursor:
|
||||
h.update((name or "").encode("utf-8"))
|
||||
h.update(b"\0")
|
||||
h.update((sql or "").encode("utf-8"))
|
||||
h.update(b"\n")
|
||||
return h.hexdigest()
|
||||
finally:
|
||||
conn.close()
|
||||
except (sqlite3.Error, OSError):
|
||||
return None
|
||||
|
||||
return None
|
||||
|
||||
@@ -45,7 +45,9 @@ Based on:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from cleveragents.domain.models.core.container_lifecycle import (
|
||||
@@ -53,7 +55,10 @@ from cleveragents.domain.models.core.container_lifecycle import (
|
||||
)
|
||||
from cleveragents.domain.models.core.resource import Resource, SandboxStrategy
|
||||
from cleveragents.infrastructure.sandbox.manager import SandboxManager
|
||||
from cleveragents.resource.handlers._base import BaseResourceHandler
|
||||
from cleveragents.resource.handlers._base import (
|
||||
EMPTY_CONTENT_HASH,
|
||||
BaseResourceHandler,
|
||||
)
|
||||
from cleveragents.resource.handlers._devcontainer_internals import (
|
||||
_CONTAINER_ID_PATTERN,
|
||||
_MAX_TERMINAL_TRACKERS,
|
||||
@@ -144,6 +149,138 @@ class DevcontainerHandler(BaseResourceHandler):
|
||||
_default_strategy = SandboxStrategy.NONE
|
||||
_type_label = "devcontainer"
|
||||
|
||||
def content_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
*,
|
||||
algorithm: str = "sha256",
|
||||
) -> str:
|
||||
"""Compute content hash for devcontainer resources.
|
||||
|
||||
Strategy (in priority order):
|
||||
|
||||
1. **Exec hash** — If a running container ID is available
|
||||
(from the lifecycle tracker), executes
|
||||
``docker exec <id> find /workspaces -type f -exec sha256sum {} +``
|
||||
inside the container and hashes the output.
|
||||
2. **Config file hash** — Falls back to hashing the
|
||||
devcontainer JSON config file. Checks
|
||||
``.devcontainer/devcontainer.json`` first, then
|
||||
``.devcontainer.json`` (first found wins).
|
||||
3. Returns :data:`EMPTY_CONTENT_HASH` if neither is available.
|
||||
"""
|
||||
if not resource.location:
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
# Strategy 1: exec hash inside running container
|
||||
exec_hash = self._try_exec_hash(resource, algorithm)
|
||||
if exec_hash is not None:
|
||||
return exec_hash
|
||||
|
||||
# Strategy 2: hash the config file
|
||||
return self._config_file_hash(resource, algorithm)
|
||||
|
||||
def _try_exec_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
algorithm: str,
|
||||
) -> str | None:
|
||||
"""Attempt to compute hash by execing into the container.
|
||||
|
||||
Uses ``docker ps`` to find a running container whose label
|
||||
matches the resource location, then execs ``find ... -exec
|
||||
sha256sum`` inside it.
|
||||
"""
|
||||
container_id = self._find_running_container(resource)
|
||||
if not container_id:
|
||||
return None
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[
|
||||
"docker",
|
||||
"exec",
|
||||
container_id,
|
||||
"find",
|
||||
"/workspaces",
|
||||
"-type",
|
||||
"f",
|
||||
"-exec",
|
||||
"sha256sum",
|
||||
"{}",
|
||||
"+",
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0 or not result.stdout.strip():
|
||||
return None
|
||||
|
||||
h = hashlib.new(algorithm)
|
||||
# Hash the sorted output for determinism
|
||||
for line in sorted(result.stdout.strip().splitlines()):
|
||||
h.update(line.encode("utf-8"))
|
||||
h.update(b"\n")
|
||||
return h.hexdigest()
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _find_running_container(resource: Resource) -> str | None:
|
||||
"""Find a running Docker container for this devcontainer resource.
|
||||
|
||||
Uses ``docker ps`` filtered by the devcontainer label that
|
||||
matches the resource location.
|
||||
"""
|
||||
if not resource.location:
|
||||
return None
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[
|
||||
"docker",
|
||||
"ps",
|
||||
"-q",
|
||||
"--filter",
|
||||
f"label=devcontainer.local_folder={resource.location}",
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0 or not result.stdout.strip():
|
||||
return None
|
||||
# Return first matching container ID
|
||||
return result.stdout.strip().splitlines()[0]
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return None
|
||||
|
||||
def _config_file_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
algorithm: str,
|
||||
) -> str:
|
||||
"""Hash the devcontainer config file as fallback."""
|
||||
loc = resource.location
|
||||
if not loc:
|
||||
return EMPTY_CONTENT_HASH
|
||||
config_paths = [
|
||||
os.path.join(loc, ".devcontainer", "devcontainer.json"),
|
||||
os.path.join(loc, ".devcontainer.json"),
|
||||
]
|
||||
|
||||
h = hashlib.new(algorithm)
|
||||
for config_path in config_paths:
|
||||
if os.path.isfile(config_path):
|
||||
with open(config_path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(65536), b""):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
# -- F16 fix: lazy activation on first resolve -------------------------
|
||||
|
||||
_ACTIVATABLE_STATES = frozenset(
|
||||
|
||||
@@ -33,7 +33,10 @@ from cleveragents.domain.models.core.resource import (
|
||||
SandboxStrategy,
|
||||
)
|
||||
from cleveragents.infrastructure.sandbox.manager import SandboxManager
|
||||
from cleveragents.resource.handlers._base import BaseResourceHandler
|
||||
from cleveragents.resource.handlers._base import (
|
||||
EMPTY_CONTENT_HASH,
|
||||
BaseResourceHandler,
|
||||
)
|
||||
from cleveragents.resource.handlers.protocol import (
|
||||
CheckpointResult,
|
||||
Content,
|
||||
@@ -379,3 +382,43 @@ class FsDirectoryHandler(BaseResourceHandler):
|
||||
shutil.rmtree(Path(snap).parent, ignore_errors=True)
|
||||
removed += 1
|
||||
return removed
|
||||
|
||||
# -- Content Hashing (#836) ---------------------------------------------
|
||||
|
||||
def content_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
*,
|
||||
algorithm: str = "sha256",
|
||||
) -> str:
|
||||
"""Compute a recursive content hash of the directory.
|
||||
|
||||
Walks the directory tree in sorted order, hashing each file's
|
||||
relative path and content. Only regular files are included
|
||||
(symlinks and special files are skipped).
|
||||
"""
|
||||
if not resource.location or not os.path.isdir(resource.location):
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
root = resource.location
|
||||
h = hashlib.new(algorithm)
|
||||
found_any = False
|
||||
|
||||
for dirpath, dirnames, filenames in os.walk(root):
|
||||
dirnames.sort()
|
||||
for fname in sorted(filenames):
|
||||
full = os.path.join(dirpath, fname)
|
||||
if not os.path.isfile(full):
|
||||
continue
|
||||
rel = os.path.relpath(full, root)
|
||||
h.update(rel.encode("utf-8"))
|
||||
h.update(b"\0")
|
||||
try:
|
||||
with open(full, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(65536), b""):
|
||||
h.update(chunk)
|
||||
except OSError:
|
||||
continue
|
||||
found_any = True
|
||||
|
||||
return h.hexdigest() if found_any else EMPTY_CONTENT_HASH
|
||||
|
||||
@@ -32,7 +32,10 @@ from cleveragents.domain.models.core.resource import (
|
||||
SandboxStrategy,
|
||||
)
|
||||
from cleveragents.infrastructure.sandbox.manager import SandboxManager
|
||||
from cleveragents.resource.handlers._base import BaseResourceHandler
|
||||
from cleveragents.resource.handlers._base import (
|
||||
EMPTY_CONTENT_HASH,
|
||||
BaseResourceHandler,
|
||||
)
|
||||
from cleveragents.resource.handlers.protocol import (
|
||||
CheckpointResult,
|
||||
Content,
|
||||
@@ -442,3 +445,39 @@ class GitCheckoutHandler(BaseResourceHandler):
|
||||
restored_files=0,
|
||||
message=f"Rolled back to checkpoint '{checkpoint_id}'",
|
||||
)
|
||||
|
||||
# -- Content Hashing (#836) ---------------------------------------------
|
||||
|
||||
def content_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
*,
|
||||
algorithm: str = "sha256",
|
||||
) -> str:
|
||||
"""Compute content hash using ``git rev-parse HEAD``.
|
||||
|
||||
For git checkouts, the HEAD commit hash uniquely identifies
|
||||
the content state. The algorithm parameter is used to
|
||||
re-hash the commit SHA for consistency with other handlers.
|
||||
"""
|
||||
if not resource.location or not os.path.isdir(resource.location):
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "rev-parse", "HEAD"],
|
||||
cwd=resource.location,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return EMPTY_CONTENT_HASH
|
||||
commit_sha = result.stdout.strip()
|
||||
# Re-hash through the requested algorithm for consistency
|
||||
h = hashlib.new(algorithm)
|
||||
h.update(commit_sha.encode("utf-8"))
|
||||
return h.hexdigest()
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return EMPTY_CONTENT_HASH
|
||||
|
||||
@@ -358,3 +358,26 @@ class ResourceHandler(Protocol):
|
||||
) -> AccessResult:
|
||||
"""Check access permissions for the resource."""
|
||||
...
|
||||
|
||||
def content_hash(
|
||||
self,
|
||||
resource: Resource,
|
||||
*,
|
||||
algorithm: str = "sha256",
|
||||
) -> str:
|
||||
"""Compute a deterministic hash of the resource's content.
|
||||
|
||||
The hash is **content-only**: metadata changes (timestamps,
|
||||
permissions) must not affect the result. Same content must
|
||||
always produce the same hash.
|
||||
|
||||
Args:
|
||||
resource: The resource to hash.
|
||||
algorithm: Hash algorithm name (default ``sha256``).
|
||||
Must be accepted by :func:`hashlib.new`.
|
||||
|
||||
Returns:
|
||||
Hex-encoded hash digest, or the sentinel value
|
||||
``EMPTY_CONTENT_HASH`` for empty or missing resources.
|
||||
"""
|
||||
...
|
||||
|
||||
Reference in New Issue
Block a user