# Repository Indexing Workflows with CleverAgents

## Overview

CleverAgents provides a built-in repository indexing engine that scans your
codebase and builds a searchable token-aware index. The `agents repo` command
group lets you index any registered resource, check its status, and trigger
incremental or full re-indexes — all from the command line. This guide walks
through the complete workflow with **real output captured from a live
installation**.

## Prerequisites

- CleverAgents installed (`pip install cleveragents` or from source with `uv sync`)
- Python 3.12 or higher
- At least one resource registered in the resource registry (see
  [Resource and Skill Management](resource-and-skill-management.md))

## What You'll Learn

- How to register a `git-checkout` or `fs-directory` resource for indexing
- How to run an incremental index (fast, only changed files)
- How to run a full re-index (complete rescan)
- How to check index status in both rich text and JSON formats
- How to use `--timeout-seconds` to bound long-running indexes
- How to integrate index status into scripts with `--format json`

---

## Part 1: Explore the `agents repo` Command Group

### Step 1: View Available Commands

```bash
$ agents repo --help
```

**Actual Output:**
```
 Usage: agents repo [OPTIONS] COMMAND [ARGS]...

 Repository indexing management

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --help          Show this message and exit.                                  │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ index   Index or re-index a repository resource.                             │
│ status  Show indexing status for a repository resource.                      │
╰──────────────────────────────────────────────────────────────────────────────╯
```

### Step 2: Explore `agents repo index` Options

```bash
$ agents repo index --help
```

**Actual Output:**
```
 Usage: agents repo index [OPTIONS] RESOURCE_NAME

 Index or re-index a repository resource.

 By default performs an incremental refresh (only changed files).
 Use ``--full`` to force a complete re-index.

 Examples:
     agents repo index local/my-repo
     agents repo index local/my-repo --full
     agents repo index local/my-repo --format json

╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ *    resource_name      TEXT  Resource name or ULID to index (e.g.,          │
│                               local/my-repo)                                 │
│                               [required]                                     │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --full                            Force full re-index (not incremental)      │
│ --timeout-seconds          FLOAT  Abort indexing when elapsed runtime        │
│                                   exceeds this many seconds                  │
│ --format           -f      TEXT   Output format: text or json (default:      │
│                                   text)                                      │
│                                   [default: text]                            │
│ --help                            Show this message and exit.                │
╰──────────────────────────────────────────────────────────────────────────────╯
```

### Step 3: Explore `agents repo status` Options

```bash
$ agents repo status --help
```

**Actual Output:**
```
 Usage: agents repo status [OPTIONS] RESOURCE_NAME

 Show indexing status for a repository resource.

 Displays index metadata including file count, token estimate,
 primary language, status, and last indexed timestamp.

 Examples:
     agents repo status local/my-repo
     agents repo status local/my-repo --format json

╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ *    resource_name      TEXT  Resource name or ULID to check status (e.g.,   │
│                               local/my-repo)                                 │
│                               [required]                                     │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --format  -f      TEXT  Output format: text or json (default: text)          │
│                         [default: text]                                      │
│ --help                  Show this message and exit.                          │
╰──────────────────────────────────────────────────────────────────────────────╯
```

---

## Part 2: Complete End-to-End Workflow

This section demonstrates the full lifecycle: registering a resource, indexing
it, checking its status, and running a full re-index.

### Step 1: Register a Resource

Before indexing, you need a registered resource. Use `agents resource add` to
register a `git-checkout` resource pointing to your repository:

```bash
$ agents resource add git-checkout local/src-only \
    --path /app/src \
    --description "CleverAgents source code only"
```

**Actual Output:**
```
Added resource: local/src-only (id: 01KNM6K5B0E39GK5YXSHGZ4783)
```

> **Tip:** The resource name uses a `namespace/identifier` format. The `local/`
> namespace is the conventional namespace for resources on the local machine.
> You can also use `agents resource add fs-directory` for non-git directories.

### Step 2: Check Status Before Indexing

Before the first index run, the resource has no index:

```bash
$ agents repo status local/src-only
```

**Actual Output:**
```
No index found for resource: local/src-only
```

This is the expected response for a freshly registered resource that has never
been indexed.

### Step 3: Run an Incremental Index

The default index mode is **incremental** — it only processes files that have
changed since the last run (or all files on the first run):

```bash
$ agents repo index local/src-only
```

**Actual Output:**
```
Incremental refresh complete: local/src-only
╭──────────────── Index Result ────────────────╮
│ Status: ready                                │
│ Files: 504                                   │
│ Tokens: 1,467,634                            │
│ Language: python                             │
│ Indexed at: 2026-04-07T14:47:34.444813+00:00 │
╰──────────────────────────────────────────────╯
```

The index completed in under 6 seconds for a 504-file Python codebase,
producing a token estimate of ~1.5M tokens. The primary language is
automatically detected.

### Step 4: Check Status After Indexing

```bash
$ agents repo status local/src-only
```

**Actual Output:**
```
Index status for: local/src-only
╭──────────────── Index Status ────────────────╮
│ Status: ready                                │
│ Files: 504                                   │
│ Tokens: 1,467,634                            │
│ Language: python                             │
│ Indexed at: 2026-04-07T14:47:34.444813+00:00 │
╰──────────────────────────────────────────────╯
```

The status panel shows:
- **Status**: `ready` — the index is complete and available
- **Files**: total number of indexed files
- **Tokens**: estimated token count across all files
- **Language**: auto-detected primary language
- **Indexed at**: ISO 8601 timestamp of the last index run

### Step 5: Run a Full Re-Index

Use `--full` to force a complete rescan of all files, regardless of
modification timestamps:

```bash
$ agents repo index local/src-only --full
```

**Actual Output:**
```
Full index complete: local/src-only
╭──────────────── Index Result ────────────────╮
│ Status: ready                                │
│ Files: 504                                   │
│ Tokens: 1,467,634                            │
│ Language: python                             │
│ Indexed at: 2026-04-07T14:47:50.537996+00:00 │
╰──────────────────────────────────────────────╯
```

The output header changes from `Incremental refresh complete` to
`Full index complete` to confirm the mode used.

### Step 6: Remove the Resource (Cleanup)

```bash
$ agents resource remove local/src-only --yes
```

**Actual Output:**
```
Removed resource: local/src-only
```

---

## Part 3: JSON Output for Scripting

All `agents repo` commands support `--format json` for machine-readable output.
This is ideal for CI pipelines, monitoring scripts, or integrating with other
tools.

### Check Status as JSON

```bash
$ agents repo status local/my-repo --format json
```

**Actual Output:**
```json
{
  "command": "",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "resource": "local/my-repo",
    "resource_id": "01KNKJY4X9K7ZDNEK7M6TKJTDG",
    "index_id": "01KNM6DB6B96BSQ8HN8D5P37PK",
    "status": "ready",
    "file_count": 19136,
    "token_estimate": 210001290,
    "primary_language": "html",
    "indexed_at": "2026-04-07T14:44:25.513344+00:00"
  },
  "timing": {
    "duration_ms": 0
  },
  "messages": [
    {
      "level": "ok",
      "text": "ok"
    }
  ]
}
```

### Run Incremental Index and Capture JSON

```bash
$ agents repo index local/my-repo --format json
```

**Actual Output:**
```json
{
  "command": "",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "resource": "local/my-repo",
    "resource_id": "01KNKJY4X9K7ZDNEK7M6TKJTDG",
    "index_id": "01KNM6DB6B96BSQ8HN8D5P37PK",
    "status": "ready",
    "file_count": 19142,
    "token_estimate": 210003925,
    "primary_language": "html",
    "indexed_at": "2026-04-07T14:45:41.571648+00:00",
    "mode": "incremental"
  },
  "timing": {
    "duration_ms": 0
  },
  "messages": [
    {
      "level": "ok",
      "text": "ok"
    }
  ]
}
```

Note the `"mode": "incremental"` field in the index result — this distinguishes
incremental from full re-index runs in JSON output.

### Run Full Re-Index and Capture JSON

```bash
$ agents repo index local/my-repo --full --format json
```

**Actual Output:**
```json
{
  "command": "",
  "status": "ok",
  "exit_code": 0,
  "data": {
    "resource": "local/my-repo",
    "resource_id": "01KNKJY4X9K7ZDNEK7M6TKJTDG",
    "index_id": "01KNM6GZ0292Z8D1HH8HQHVC9J",
    "status": "ready",
    "file_count": 19145,
    "token_estimate": 210012364,
    "primary_language": "html",
    "indexed_at": "2026-04-07T14:46:14.289238+00:00",
    "mode": "full"
  },
  "timing": {
    "duration_ms": 0
  },
  "messages": [
    {
      "level": "ok",
      "text": "ok"
    }
  ]
}
```

The `"mode": "full"` field confirms a complete re-index was performed. Note
that the `index_id` changes between full re-indexes, while incremental runs
update the same index record.

---

## Part 4: Using `--timeout-seconds`

For large repositories, you can set a time budget for the indexing operation.
If the index exceeds the timeout, it aborts gracefully:

```bash
$ agents repo index local/my-repo --timeout-seconds 60
```

**Actual Output (completed within timeout):**
```
Incremental refresh complete: local/my-repo
╭──────────────── Index Result ────────────────╮
│ Status: ready                                │
│ Files: 19147                                 │
│ Tokens: 210,012,517                          │
│ Language: html                               │
│ Indexed at: 2026-04-07T14:46:33.496394+00:00 │
╰──────────────────────────────────────────────╯
```

If the timeout is exceeded, the command exits with code 1 and prints:
```
Error: Indexing timed out after <N> seconds
```

This is useful in CI environments where you want to bound the indexing step.

---

## Part 5: Error Handling

### Resource Not Found

If you specify a resource that doesn't exist in the registry:

```bash
$ agents repo status nonexistent/repo
```

**Actual Output:**
```
Error: Resource not found: nonexistent/repo
```

Exit code: `1`

```bash
$ agents repo index nonexistent/repo
```

**Actual Output:**
```
Error: Resource not found: nonexistent/repo
```

Exit code: `1`

These errors are clean and scriptable — check the exit code to detect failures
in automation pipelines.

---

## Part 6: Indexing a Large Repository

For reference, here is the status of a large repository (the full CleverAgents
codebase including all build artifacts and documentation):

```bash
$ agents repo status local/my-repo
```

**Actual Output:**
```
Index status for: local/my-repo
╭──────────────── Index Status ────────────────╮
│ Status: ready                                │
│ Files: 19136                                 │
│ Tokens: 210,001,290                          │
│ Language: html                               │
│ Indexed at: 2026-04-07T14:44:25.513344+00:00 │
╰──────────────────────────────────────────────╯
```

A 19,000+ file repository with 210M tokens indexes incrementally in ~10 seconds.

---

## Quick Reference

| Command | Description |
|---------|-------------|
| `agents repo index <resource>` | Incremental index (only changed files) |
| `agents repo index <resource> --full` | Full re-index (all files) |
| `agents repo index <resource> --timeout-seconds N` | Index with time budget |
| `agents repo index <resource> --format json` | Index with JSON output |
| `agents repo status <resource>` | Show index status (rich text) |
| `agents repo status <resource> --format json` | Show index status (JSON) |

### Resource Name Format

Resources use a `namespace/identifier` format:
- `local/my-repo` — a resource in the `local` namespace
- `local/src-only` — another local resource
- You can also use the resource ULID directly (e.g., `01KNKJY4X9K7ZDNEK7M6TKJTDG`)

### Index Status Values

| Status | Meaning |
|--------|---------|
| `ready` | Index is complete and available |
| `indexing` | Index is currently being built |
| `error` | Index failed (check `error_message` field in JSON output) |

### Supported Resource Types for Indexing

The repo indexing commands work with any resource that has a filesystem path:
- `git-checkout` — local git repositories (recommended)
- `fs-directory` — any local filesystem directory

---

## Key Takeaways

1. **Incremental by default** — `agents repo index` only processes changed
   files, making it fast enough to run on every commit.

2. **Auto-detects primary language** — the indexer automatically identifies
   the dominant programming language in the repository.

3. **Token-aware** — the index tracks token estimates, which is critical for
   AI context window management.

4. **JSON output for automation** — `--format json` makes it easy to integrate
   index status into CI pipelines, dashboards, or monitoring scripts.

5. **Graceful timeout** — `--timeout-seconds` lets you set a time budget for
   large repositories without crashing the pipeline.

6. **Clean error messages** — resource-not-found errors exit with code 1 and
   print a clear message, making them easy to handle in scripts.

---

## Try It Yourself

```bash
# 1. Register your project repository
agents resource add git-checkout local/my-project \
    --path /path/to/your/repo \
    --description "My project"

# 2. Check initial status (no index yet)
agents repo status local/my-project

# 3. Run the first index
agents repo index local/my-project

# 4. Check status after indexing
agents repo status local/my-project

# 5. Make some changes to your code, then run incremental refresh
agents repo index local/my-project

# 6. Force a full re-index
agents repo index local/my-project --full

# 7. Get machine-readable status for scripting
agents repo status local/my-project --format json | python3 -c \
    "import sys, json; d=json.load(sys.stdin)['data']; \
     print(f\"Files: {d['file_count']}, Tokens: {d['token_estimate']:,}\")"
```

---

*This example was automatically generated and verified by the CleverAgents UAT system.*
*Feature area: Repo indexing workflows | Test cycle: 1*

---
**Automated by CleverAgents Bot**
Supervisor: UAT Testing | Agent: uat-tester