CleverRDFlib/docs/external_dependency_interceptor.md

# External Dependency Interceptor

The `ExternalDependenciesInterceptor` is a utility module designed to intercept and control HTTP/HTTPS requests made by RDFLib. This ensures that all web requests for resolving external ontologies are exclusively performed by CleverRDFLib, rather than being made autonomously by RDFLib.

## Purpose

RDFLib may autonomously make HTTP/HTTPS requests when loading ontologies, which can lead to:

- **Uncontrolled network access**: RDFLib making requests without CleverRDFLib's knowledge
- **Inconsistent behavior**: Different loading behavior depending on RDFLib's internal mechanisms
- **Security concerns**: Unauthorized or unexpected network requests
- **Testing difficulties**: Unpredictable network behavior during testing

The `ExternalDependenciesInterceptor` prevents RDFLib from making autonomous web requests by intercepting `urllib.request` calls and either blocking them or redirecting them to local files.

## Important Warnings

!!! warning "Client Application Compatibility"
    This interceptor modifies the global `urllib.request` opener, which affects **all** code using `urllib.request` in the same Python process. If your client application or other libraries use `urllib.request` for HTTP/HTTPS requests, this interceptor will block or redirect those requests as well, potentially breaking your application.

!!! warning "Production Use"
    This module is intended for:
    - **Debugging**: Isolating RDFLib's network behavior during development and testing
    - **Production environments**: Only when you can guarantee that no other code in your application uses `urllib.request` for HTTP/HTTPS requests
    
    If your application or its dependencies use `urllib.request`, do **not** use this interceptor in production, as it will interfere with their functionality.

## Handler Types

The module provides several handler types for different use cases:

### Blocking Handlers

These handlers block all HTTP/HTTPS requests unless they are redirected to local files:

- **`ExtDepsRejecterAndRedirectorHTTPHandler`**: Blocks HTTP requests unless mapped to local files
- **`ExtDepsRejecterAndRedirectorHTTPSHandler`**: Blocks HTTPS requests unless mapped to local files

### Redirecting Handlers

These handlers redirect mapped URIs to local files but allow unmapped requests to proceed normally:

- **`LocalFileRedirectorHTTPHandler`**: Redirects HTTP requests to local files, falls back to default behavior
- **`LocalFileRedirectorHTTPSHandler`**: Redirects HTTPS requests to local files, falls back to default behavior

## Basic Usage

### Blocking All External Requests

To block all external HTTP/HTTPS requests (except those mapped to local files):

```python
from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

# Create interceptor (uses blocking handlers by default)
interceptor = ExternalDependenciesInterceptor()

# Register the interceptor
interceptor.register_dependencies_interceptor()

# Now RDFLib cannot make autonomous HTTP/HTTPS requests
# All requests will be blocked unless mapped to local files
```

### Redirecting Requests to Local Files

To redirect specific URIs to local files while blocking others:

```python
from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

# Create interceptor
interceptor = ExternalDependenciesInterceptor()

# Set up URI mappings (web URI -> local file path)
uri_mappings = {
    "http://example.com/ontology.owl": "/path/to/local/ontology.owl",
    "https://example.org/schema.ttl": "/path/to/local/schema.ttl"
}

# Configure mappings
interceptor.set_mappings(uri_mappings)

# Register the interceptor
interceptor.register_dependencies_interceptor()

# Now:
# - Mapped URIs will be served from local files
# - Unmapped URIs will be blocked (PermissionError)
```

### Using Redirecting Handlers (Non-Blocking)

To redirect mapped URIs to local files but allow unmapped requests to proceed:

```python
from cleverrdf_lib.core.external_dep_interceptor import (
    ExternalDependenciesInterceptor,
    LocalFileRedirectorHTTPHandler,
    LocalFileRedirectorHTTPSHandler
)

# Create interceptor
interceptor = ExternalDependenciesInterceptor()

# Replace handlers with redirecting (non-blocking) handlers
interceptor.replace_handlers([
    LocalFileRedirectorHTTPSHandler(),
    LocalFileRedirectorHTTPHandler()
])

# Set up URI mappings
uri_mappings = {
    "http://example.com/ontology.owl": "/path/to/local/ontology.owl"
}
interceptor.set_mappings(uri_mappings)

# Register the interceptor
interceptor.register_dependencies_interceptor()

# Now:
# - Mapped URIs will be served from local files
# - Unmapped URIs will use default HTTP/HTTPS behavior (not blocked)
```

## Use Cases

### Testing and Development

During testing, you may want to ensure that RDFLib doesn't make unexpected network requests:

```python
from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

# Set up interceptor for testing
interceptor = ExternalDependenciesInterceptor()

# Map test URIs to local test files
test_mappings = {
    "http://test.example.com/ontology.owl": "test_data/test_ontology.owl"
}
interceptor.set_mappings(test_mappings)
interceptor.register_dependencies_interceptor()

# Now run your tests - RDFLib cannot make unexpected requests
```

### Production with Controlled Network Access

In production environments where you want to ensure all ontology loading goes through CleverRDFLib:

```python
from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

# Only use this if you're certain no other code uses urllib.request
interceptor = ExternalDependenciesInterceptor()

# Optionally map known URIs to local files
uri_mappings = {
    "http://known-ontology.example.com/owl": "/cached/ontology.owl"
}
interceptor.set_mappings(uri_mappings)

interceptor.register_dependencies_interceptor()

# Now all RDFLib requests are controlled
```

## Handler Details

### Blocking Handlers

**`ExtDepsRejecterAndRedirectorHTTPHandler`** and **`ExtDepsRejecterAndRedirectorHTTPSHandler`**:

- **Behavior**: Block all requests unless mapped to local files
- **Raises**: `PermissionError` for unmapped requests
- **Use case**: Strict control over network access

```python
from cleverrdf_lib.core.external_dep_interceptor import (
    ExtDepsRejecterAndRedirectorHTTPHandler,
    ExtDepsRejecterAndRedirectorHTTPSHandler
)

handler = ExtDepsRejecterAndRedirectorHTTPHandler()
handler.update_mappings({
    "http://example.com/owl": "/local/file.owl"
})

# Mapped URI: Returns file content
# Unmapped URI: Raises PermissionError
```

### Redirecting Handlers

**`LocalFileRedirectorHTTPHandler`** and **`LocalFileRedirectorHTTPSHandler`**:

- **Behavior**: Redirect mapped URIs to local files, allow unmapped requests to proceed
- **Falls back**: Uses default HTTP/HTTPS handler for unmapped requests
- **Use case**: Selective redirection without blocking

```python
from cleverrdf_lib.core.external_dep_interceptor import (
    LocalFileRedirectorHTTPHandler,
    LocalFileRedirectorHTTPSHandler
)

handler = LocalFileRedirectorHTTPHandler()
handler.update_mappings({
    "http://example.com/owl": "/local/file.owl"
})

# Mapped URI: Returns file content
# Unmapped URI: Uses default HTTP handler (normal network request)
```

## Updating Mappings

You can update URI mappings at runtime:

```python
from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

interceptor = ExternalDependenciesInterceptor()

# Initial mappings
interceptor.set_mappings({
    "http://example.com/owl": "/local/file1.owl"
})

interceptor.register_dependencies_interceptor()

# Later, update mappings
interceptor.set_mappings({
    "http://example.com/owl": "/local/file1.owl",
    "http://example.org/schema": "/local/file2.ttl"
})
```

## Custom Handlers

You can create custom handlers by extending the base handlers:

```python
from cleverrdf_lib.core.external_dep_interceptor import (
    ExternalDependenciesInterceptor,
    LocalFileRedirectorHTTPHandler
)
from urllib.request import Request

class CustomHTTPHandler(LocalFileRedirectorHTTPHandler):
    """Custom handler with additional logic."""
    
    def http_open(self, req: Request):
        # Add custom logic before redirection
        url = req.get_full_url()
        if url.endswith(".blocked"):
            raise PermissionError("Blocked URL")
        
        # Use parent implementation
        return super().http_open(req)

# Use custom handler
interceptor = ExternalDependenciesInterceptor()
interceptor.replace_handlers([CustomHTTPHandler()])
interceptor.register_dependencies_interceptor()
```

## Best Practices

1. **Use only when necessary**: Only install the interceptor when you need to control RDFLib's network behavior
2. **Test thoroughly**: If using in production, thoroughly test that no other code uses `urllib.request`
3. **Document usage**: Clearly document in your codebase that the interceptor is installed
4. **Use redirecting handlers**: Prefer redirecting handlers over blocking handlers when possible to avoid breaking other functionality
5. **Map all known URIs**: Pre-map all URIs you know will be requested to avoid blocking legitimate requests

## Limitations

- **Global effect**: The interceptor affects all `urllib.request` usage in the Python process
- **No selective interception**: Cannot selectively intercept only RDFLib requests
- **Process-wide**: Once installed, affects all code in the process
- **Cannot be uninstalled**: Once installed, the opener remains in effect

## When Not to Use

Do **not** use this interceptor if:

- Your application uses `urllib.request` for other purposes
- Your dependencies use `urllib.request` for HTTP/HTTPS requests
- You need RDFLib to make autonomous network requests
- You're unsure whether other code uses `urllib.request`

## Next Steps

- Learn about [Basic Usage](basic_usage.md) for loading ontologies
- Explore [Resolvers](resolvers.md) for controlling reference resolution
- Understand [Error Handling](error_handling.md) for managing loading errors