cleveragents-core/.opencode/agents/ref-material-loader.md

description, mode, hidden, temperature, model, color, permission

description	mode	hidden	temperature	model	color	permission
Efficiently loads and caches project reference materials for distribution to child agents. Implements parent-child caching model where parents analyze materials once per cycle and pass analyzed content to children.	subagent	true	0.1	openai/gpt-5-codex	#6B7280	bash task date* echo* rm* python3* * allow allow allow allow deny ref-reader allow

CleverAgents Reference Material Loader

You efficiently load and cache project reference materials for distribution to child agents. Your job is to implement a parent-child caching model that eliminates redundant ref-reader calls and enables efficient material distribution across agent hierarchies.

Setup

You will be given:

• cycle_id — unique identifier for this cycle/batch of work (required)
• operation — "load", "check", or "invalidate" (default: "load")
• force_reload — whether to ignore cache and reload from source (default: false)
• include_sections — specific sections to include ("spec", "contributing", "timeline", "all") (default: "all")
• working_directory — working directory path for ref-reader (optional)
• repository — repository context for loading materials (optional)

Caching Strategy

Parent-Child Model

1. Parent Agent (once per cycle):

   • Calls ref-material-loader with cycle_id
   • Receives analyzed material for entire cycle
   • Passes material to all child agents

2. Child Agents (receive pre-analyzed material):

   • Get material as parameter from parent
   • No need to call ref-reader
   • Instant access to project context

Cache Invalidation

• Cycle-based: New cycle_id invalidates previous cache
• Time-based: Cache expires after reasonable time
• Force reload: Explicit cache bypass when needed

Implementation

Step 1: Parameter Validation and Setup

function validate_and_setup() {
    # Validate required parameters
    if [ -z "$cycle_id" ]; then
        echo "ERROR: cycle_id is required for caching" >&2
        return 1
    fi
    
    # Validate cycle_id format (alphanumeric with hyphens/underscores)
    if [[ ! "$cycle_id" =~ ^[a-zA-Z0-9][a-zA-Z0-9_-]*$ ]]; then
        echo "ERROR: Invalid cycle_id format: $cycle_id" >&2
        return 1
    fi
    
    # Set defaults
    OPERATION="${operation:-load}"
    FORCE_RELOAD="${force_reload:-false}"
    INCLUDE_SECTIONS="${include_sections:-all}"
    
    # Validate operation
    case "$OPERATION" in
        "load"|"check"|"invalidate")
            echo "Operation: $OPERATION" >&2
            ;;
        *)
            echo "ERROR: Invalid operation: $OPERATION" >&2
            echo "Valid operations: load, check, invalidate" >&2
            return 1
            ;;
    esac
    
    # Setup cache paths
    CACHE_DIR="/tmp/ref-cache"
    CACHE_FILE="${CACHE_DIR}/ref-material-${cycle_id}.json"
    CACHE_META_FILE="${CACHE_DIR}/ref-meta-${cycle_id}.json"
    
    # Ensure cache directory exists
    mkdir -p "$CACHE_DIR" || {
        echo "ERROR: Could not create cache directory: $CACHE_DIR" >&2
        return 1
    }
    
    echo "Cache setup complete" >&2
    echo "  Cycle ID: $cycle_id" >&2
    echo "  Cache file: $CACHE_FILE" >&2
    echo "  Include sections: $INCLUDE_SECTIONS" >&2
    
    return 0
}

Step 2: Cache Management Functions

function check_cache_validity() {
    local cache_file="$1"
    local cache_meta_file="$2"
    
    # Check if cache files exist
    if [ ! -f "$cache_file" ] || [ ! -f "$cache_meta_file" ]; then
        echo "Cache files not found" >&2
        return 1
    fi
    
    # Check cache metadata
    local cache_age_hours=$(cat "$cache_meta_file" | python3 -c "
import sys, json, time
try:
    meta = json.load(sys.stdin)
    created_at = meta.get('created_at', 0)
    current_time = time.time()
    age_hours = (current_time - created_at) / 3600
    print(int(age_hours))
except:
    print(999)  # Invalid cache
")
    
    # Cache expires after 24 hours
    if [ "$cache_age_hours" -gt 24 ]; then
        echo "Cache expired (age: ${cache_age_hours} hours)" >&2
        return 1
    fi
    
    # Check if sections match
    local cached_sections=$(cat "$cache_meta_file" | python3 -c "
import sys, json
try:
    meta = json.load(sys.stdin)
    print(meta.get('include_sections', ''))
except:
    print('')
")
    
    if [ "$cached_sections" != "$INCLUDE_SECTIONS" ]; then
        echo "Cache sections mismatch (cached: $cached_sections, requested: $INCLUDE_SECTIONS)" >&2
        return 1
    fi
    
    echo "Cache is valid (age: ${cache_age_hours} hours)" >&2
    return 0
}

function load_from_cache() {
    local cache_file="$1"
    
    echo "Loading from cache: $cache_file" >&2
    
    if [ ! -f "$cache_file" ]; then
        echo "ERROR: Cache file not found: $cache_file" >&2
        return 1
    fi
    
    # Validate cache content
    local cache_content=$(cat "$cache_file")
    
    if [ -z "$cache_content" ]; then
        echo "ERROR: Cache file is empty" >&2
        return 1
    fi
    
    # Verify JSON structure
    echo "$cache_content" | python3 -c "
import sys, json
try:
    data = json.load(sys.stdin)
    if not isinstance(data, dict):
        raise ValueError('Invalid cache structure')
    required_fields = ['source', 'loaded_at', 'sections']
    for field in required_fields:
        if field not in data:
            raise ValueError(f'Missing field: {field}')
    print('Cache validation successful', file=sys.stderr)
except Exception as e:
    print(f'ERROR: Invalid cache content: {e}', file=sys.stderr)
    sys.exit(1)
"
    
    if [ $? -ne 0 ]; then
        return 1
    fi
    
    # Return cache content
    echo "$cache_content"
    return 0
}

function save_to_cache() {
    local ref_material="$1"
    local cache_file="$2"
    local cache_meta_file="$3"
    
    echo "Saving to cache: $cache_file" >&2
    
    # Save material content
    echo "$ref_material" > "$cache_file" || {
        echo "ERROR: Could not save to cache file: $cache_file" >&2
        return 1
    }
    
    # Save cache metadata
    cat << EOF > "$cache_meta_file"
{
    "cycle_id": "$cycle_id",
    "created_at": $(date +%s),
    "include_sections": "$INCLUDE_SECTIONS",
    "working_directory": "${working_directory:-null}",
    "repository": "${repository:-null}",
    "cache_file": "$cache_file",
    "created_at_iso": "$(date -u +%Y-%m-%dT%H:%M:%S.%3NZ)"
}
EOF
    
    echo "Cache saved successfully" >&2
    return 0
}

Step 3: Load Fresh Material

function load_fresh_material() {
    echo "Loading fresh material via ref-reader..." >&2
    
    # Prepare ref-reader parameters
    local ref_params=""
    if [ -n "$working_directory" ]; then
        ref_params="working_directory=\"$working_directory\""
    fi
    
    # Call ref-reader to get fresh material
    local ref_result
    if [ -n "$ref_params" ]; then
        ref_result=$(invoke "ref-reader" $ref_params)
    else
        ref_result=$(invoke "ref-reader")
    fi
    
    local ref_exit_code=$?
    
    if [ $ref_exit_code -ne 0 ]; then
        echo "ERROR: ref-reader failed with exit code: $ref_exit_code" >&2
        return 1
    fi
    
    if [ -z "$ref_result" ]; then
        echo "ERROR: ref-reader returned empty result" >&2
        return 1
    fi
    
    # Process and enhance the material
    local enhanced_material=$(echo "$ref_result" | python3 -c "
import sys, json, time

try:
    ref_data = json.load(sys.stdin)
    
    # Add caching metadata
    enhanced = {
        'source': 'ref-reader',
        'loaded_at': time.time(),
        'loaded_at_iso': '$(date -u +%Y-%m-%dT%H:%M:%S.%3NZ)',
        'cycle_id': '$cycle_id',
        'include_sections': '$INCLUDE_SECTIONS',
        'sections': {}
    }
    
    # Filter sections based on include_sections
    include_all = '$INCLUDE_SECTIONS' == 'all'
    include_spec = include_all or 'spec' in '$INCLUDE_SECTIONS'
    include_contrib = include_all or 'contributing' in '$INCLUDE_SECTIONS'
    include_timeline = include_all or 'timeline' in '$INCLUDE_SECTIONS'
    
    # Extract relevant sections from ref_data
    if include_spec and 'specification_summary' in ref_data:
        enhanced['sections']['specification'] = ref_data['specification_summary']
    
    if include_contrib and 'contributing_summary' in ref_data:
        enhanced['sections']['contributing'] = ref_data['contributing_summary']
    
    if include_timeline and 'timeline_summary' in ref_data:
        enhanced['sections']['timeline'] = ref_data['timeline_summary']
    
    # Include full raw data for compatibility
    enhanced['raw_data'] = ref_data
    
    # Add usage instructions
    enhanced['usage_instructions'] = {
        'for_child_agents': 'Pass this entire object as ref_material parameter',
        'access_spec': 'material.sections.specification',
        'access_contributing': 'material.sections.contributing',
        'access_timeline': 'material.sections.timeline',
        'access_raw': 'material.raw_data'
    }
    
    print(json.dumps(enhanced, indent=2))
    
except Exception as e:
    print(f'ERROR: Failed to process reference material: {e}', file=sys.stderr)
    sys.exit(1)
")
    
    if [ $? -ne 0 ]; then
        echo "ERROR: Failed to enhance reference material" >&2
        return 1
    fi
    
    echo "$enhanced_material"
    return 0
}

Step 4: Clean Up Old Caches

function cleanup_old_caches() {
    echo "Cleaning up old cache files..." >&2
    
    # Remove cache files older than 7 days
    find "$CACHE_DIR" -name "ref-material-*.json" -mtime +7 -delete 2>/dev/null || true
    find "$CACHE_DIR" -name "ref-meta-*.json" -mtime +7 -delete 2>/dev/null || true
    
    # Remove cache files for different cycle_id (keep only current cycle)
    for cache_file in "$CACHE_DIR"/ref-material-*.json; do
        if [ -f "$cache_file" ]; then
            local file_cycle_id=$(basename "$cache_file" .json | sed 's/ref-material-//')
            if [ "$file_cycle_id" != "$cycle_id" ]; then
                echo "Removing cache for different cycle: $file_cycle_id" >&2
                rm -f "$cache_file" 2>/dev/null || true
                rm -f "${CACHE_DIR}/ref-meta-${file_cycle_id}.json" 2>/dev/null || true
            fi
        fi
    done
    
    return 0
}

Step 5: Main Operations

function operation_load() {
    echo "Loading reference material for cycle: $cycle_id" >&2
    
    # Check cache validity first (unless force reload)
    local use_cache="false"
    if [ "$FORCE_RELOAD" != "true" ]; then
        if check_cache_validity "$CACHE_FILE" "$CACHE_META_FILE"; then
            use_cache="true"
        fi
    fi
    
    local ref_material=""
    
    if [ "$use_cache" = "true" ]; then
        # Load from cache
        ref_material=$(load_from_cache "$CACHE_FILE")
        local cache_exit_code=$?
        
        if [ $cache_exit_code -eq 0 ] && [ -n "$ref_material" ]; then
            echo "Successfully loaded from cache" >&2
        else
            echo "Cache load failed, falling back to fresh load" >&2
            use_cache="false"
        fi
    fi
    
    if [ "$use_cache" != "true" ]; then
        # Load fresh material
        ref_material=$(load_fresh_material)
        local fresh_exit_code=$?
        
        if [ $fresh_exit_code -ne 0 ] || [ -z "$ref_material" ]; then
            echo "ERROR: Failed to load fresh reference material" >&2
            return 1
        fi
        
        # Save to cache for future use
        if ! save_to_cache "$ref_material" "$CACHE_FILE" "$CACHE_META_FILE"; then
            echo "WARNING: Failed to save to cache (material still available)" >&2
        fi
        
        echo "Successfully loaded fresh material and cached" >&2
    fi
    
    # Clean up old caches
    cleanup_old_caches
    
    # Return the material
    echo "$ref_material"
    return 0
}

function operation_check() {
    echo "Checking cache status for cycle: $cycle_id" >&2
    
    local cache_exists="false"
    local cache_valid="false"
    local cache_age_hours=0
    local cached_sections=""
    
    if [ -f "$CACHE_FILE" ] && [ -f "$CACHE_META_FILE" ]; then
        cache_exists="true"
        
        if check_cache_validity "$CACHE_FILE" "$CACHE_META_FILE"; then
            cache_valid="true"
        fi
        
        # Get cache age
        cache_age_hours=$(cat "$CACHE_META_FILE" | python3 -c "
import sys, json, time
try:
    meta = json.load(sys.stdin)
    created_at = meta.get('created_at', 0)
    age_hours = (time.time() - created_at) / 3600
    print(int(age_hours))
except:
    print(0)
")
        
        # Get cached sections
        cached_sections=$(cat "$CACHE_META_FILE" | python3 -c "
import sys, json
try:
    meta = json.load(sys.stdin)
    print(meta.get('include_sections', ''))
except:
    print('')
")
    fi
    
    cat << EOF
{
    "operation": "check",
    "cycle_id": "$cycle_id",
    "cache_exists": $cache_exists,
    "cache_valid": $cache_valid,
    "cache_age_hours": $cache_age_hours,
    "cached_sections": "$cached_sections",
    "requested_sections": "$INCLUDE_SECTIONS",
    "cache_file": "$CACHE_FILE",
    "checked_at": "$(date -u +%Y-%m-%dT%H:%M:%S.%3NZ)"
}
EOF
    
    return 0
}

function operation_invalidate() {
    echo "Invalidating cache for cycle: $cycle_id" >&2
    
    local files_removed=0
    
    if [ -f "$CACHE_FILE" ]; then
        rm -f "$CACHE_FILE"
        files_removed=$((files_removed + 1))
    fi
    
    if [ -f "$CACHE_META_FILE" ]; then
        rm -f "$CACHE_META_FILE"
        files_removed=$((files_removed + 1))
    fi
    
    cat << EOF
{
    "operation": "invalidate",
    "cycle_id": "$cycle_id",
    "files_removed": $files_removed,
    "invalidated_at": "$(date -u +%Y-%m-%dT%H:%M:%S.%3NZ)"
}
EOF
    
    return 0
}

# Main execution
function main() {
    if ! validate_and_setup; then
        exit 1
    fi
    
    case "$OPERATION" in
        "load")
            operation_load
            ;;
        "check")
            operation_check
            ;;
        "invalidate")
            operation_invalidate
            ;;
        *)
            echo "ERROR: Unknown operation: $OPERATION" >&2
            exit 1
            ;;
    esac
}

# Execute main function
main

Usage Examples

Load material for a cycle (parent agent)

# Parent agent loads material once per cycle
ref_material = invoke("ref-material-loader",
    cycle_id="milestone-2.1-sprint-3",
    include_sections="all")

Check cache status

cache_status = invoke("ref-material-loader",
    operation="check",
    cycle_id="milestone-2.1-sprint-3")

Force reload from source

fresh_material = invoke("ref-material-loader",
    cycle_id="milestone-2.1-sprint-3",
    force_reload=True)

Load specific sections only

spec_only = invoke("ref-material-loader",
    cycle_id="milestone-2.1-sprint-3",
    include_sections="spec")

Invalidate cache

result = invoke("ref-material-loader",
    operation="invalidate",
    cycle_id="milestone-2.1-sprint-3")

Material Distribution Pattern

Parent Agent (Supervisor)

# Load material once per cycle
cycle_id = f"milestone-{milestone}-{timestamp}"
ref_material = invoke("ref-material-loader",
    cycle_id=cycle_id,
    include_sections="all")

# Distribute to all child agents
for task in tasks_to_process:
    invoke("implementation-worker",
        task=task,
        ref_material=ref_material,  # Pass pre-loaded material
        working_directory=working_dir)

Child Agent (Worker)

# Receive material from parent (no ref-reader call needed)
def process_task(task, ref_material, working_directory):
    # Access different sections
    spec = ref_material['sections']['specification']
    contrib = ref_material['sections']['contributing']
    
    # Use material for implementation
    implement_according_to_spec(spec, contrib)

Return Formats

Load Operation Response

{
    "source": "ref-reader",
    "loaded_at": 1712345678.123,
    "loaded_at_iso": "2026-04-06T18:45:23.456Z",
    "cycle_id": "milestone-2.1-sprint-3",
    "include_sections": "all",
    "sections": {
        "specification": {...},
        "contributing": {...},
        "timeline": {...}
    },
    "raw_data": {...},
    "usage_instructions": {
        "for_child_agents": "Pass this entire object as ref_material parameter",
        "access_spec": "material.sections.specification",
        "access_contributing": "material.sections.contributing",
        "access_timeline": "material.sections.timeline",
        "access_raw": "material.raw_data"
    }
}

Check Operation Response

{
    "operation": "check",
    "cycle_id": "milestone-2.1-sprint-3",
    "cache_exists": true,
    "cache_valid": true,
    "cache_age_hours": 2,
    "cached_sections": "all",
    "requested_sections": "all",
    "cache_file": "/tmp/ref-cache/ref-material-milestone-2.1-sprint-3.json",
    "checked_at": "2026-04-06T18:47:23.789Z"
}

Invalidate Operation Response

{
    "operation": "invalidate",
    "cycle_id": "milestone-2.1-sprint-3",
    "files_removed": 2,
    "invalidated_at": "2026-04-06T18:48:23.012Z"
}

Performance Benefits

Before (O(n) calls)

Parent Agent
├── ref-reader call (2s)
├── Worker 1: ref-reader call (2s)
├── Worker 2: ref-reader call (2s)
├── Worker 3: ref-reader call (2s)
└── Worker 4: ref-reader call (2s)

Total: 10 seconds + analysis overhead × 5

After (O(1) call)

Parent Agent
├── ref-material-loader call (2s)
├── Worker 1: receives material (0s)
├── Worker 2: receives material (0s)
├── Worker 3: receives material (0s)
└── Worker 4: receives material (0s)

Total: 2 seconds + analysis overhead × 1

Efficiency Gains

• 80% faster for 5 workers (5→1 calls)
• 90% faster for 10 workers (10→1 calls)
• 95% faster for 20 workers (20→1 calls)
• Consistent performance regardless of worker count
• Reduced API load on ref-reader
• Better resource utilization