Scale Fixture Generator Instructions

Overview

This document describes how to generate synthetic repositories for scale testing without requiring helper scripts. The process is entirely manual and repeatable using standard Unix tools and Python standard library modules.

Prerequisites

Python 3.13+
Standard Unix tools: mkdir, cp, find
At least 2 GB free disk space for large profile generation

Generating a Small Profile (1,000 files)

Step 1: Create the directory structure

mkdir -p /tmp/scale-test-repo/src/{api,core,models,utils,tests}
mkdir -p /tmp/scale-test-repo/config
mkdir -p /tmp/scale-test-repo/scripts

Step 2: Generate Python files (60% = 600 files)

Create a template Python file and replicate it with unique module names:

for i in $(seq 1 600); do
  dir="src/$(echo api core models utils tests | tr ' ' '\n' | shuf -n1)"
  cat > "/tmp/scale-test-repo/${dir}/module_${i}.py" << 'EOF'
"""Auto-generated module for scale testing."""


class ScaleTestClass:
    """Placeholder class for indexing benchmarks."""

    def __init__(self, name: str) -> None:
        self.name = name

    def process(self) -> str:
        return f"processed-{self.name}"
EOF
done

Step 3: Generate JavaScript files (30% = 300 files)

for i in $(seq 1 300); do
  dir="src/$(echo api core utils | tr ' ' '\n' | shuf -n1)"
  cat > "/tmp/scale-test-repo/${dir}/module_${i}.js" << 'EOF'
// Auto-generated module for scale testing
export class ScaleTestClass {
  constructor(name) {
    this.name = name;
  }
  process() {
    return `processed-${this.name}`;
  }
}
EOF
done

Step 4: Generate YAML files (10% = 100 files)

for i in $(seq 1 100); do
  cat > "/tmp/scale-test-repo/config/config_${i}.yaml" << EOF
name: scale-config-${i}
version: "1.0"
settings:
  enabled: true
  timeout: 30
  retries: 3
EOF
done

Generating a Medium Profile (5,000 files)

Follow the same steps but multiply file counts by 5:

Python: 2,500 files
JavaScript: 1,250 files
TypeScript: 750 files
YAML: 500 files

Add TypeScript generation:

for i in $(seq 1 750); do
  dir="src/$(echo api core models | tr ' ' '\n' | shuf -n1)"
  cat > "/tmp/scale-test-repo/${dir}/module_${i}.ts" << 'EOF'
// Auto-generated TypeScript module for scale testing
export interface ScaleTestData {
  name: string;
  value: number;
}
export function process(data: ScaleTestData): string {
  return `processed-${data.name}`;
}
EOF
done

Generating a Large Profile (10,000 files)

Follow the same steps but multiply file counts by 10:

Python: 4,000 files
JavaScript: 2,000 files
TypeScript: 2,000 files
YAML: 1,000 files
JSON: 1,000 files

Add JSON generation:

for i in $(seq 1 1000); do
  cat > "/tmp/scale-test-repo/config/data_${i}.json" << EOF
{
  "name": "scale-data-${i}",
  "version": "1.0",
  "entries": [
    {"key": "item_1", "value": ${i}},
    {"key": "item_2", "value": $((i * 2))}
  ]
}
EOF
done

Verification

After generation, verify the fixture matches the expected profile:

# Count total files
find /tmp/scale-test-repo -type f | wc -l

# Count by extension
find /tmp/scale-test-repo -name "*.py" | wc -l
find /tmp/scale-test-repo -name "*.js" | wc -l
find /tmp/scale-test-repo -name "*.ts" | wc -l
find /tmp/scale-test-repo -name "*.yaml" | wc -l
find /tmp/scale-test-repo -name "*.json" | wc -l

# Check total size
du -sh /tmp/scale-test-repo

Cleanup

rm -rf /tmp/scale-test-repo

Notes

Generated files are intentionally simple to isolate indexing and decomposition overhead from parsing complexity.
Language mix ratios should match the scale_metadata.json profiles.
For reproducible benchmarks, always generate fresh fixtures before each run.
No external dependencies or helper scripts are required.

3.8 KiB Raw Blame History