CleverRDFlib/docs/external_dependency_interceptor.md

External Dependency Interceptor

The ExternalDependenciesInterceptor is a utility module designed to intercept and control HTTP/HTTPS requests made by RDFLib. This ensures that all web requests for resolving external ontologies are exclusively performed by CleverRDFLib, rather than being made autonomously by RDFLib.

Purpose

RDFLib may autonomously make HTTP/HTTPS requests when loading ontologies, which can lead to:

• Uncontrolled network access: RDFLib making requests without CleverRDFLib's knowledge
• Inconsistent behavior: Different loading behavior depending on RDFLib's internal mechanisms
• Security concerns: Unauthorized or unexpected network requests
• Testing difficulties: Unpredictable network behavior during testing

The ExternalDependenciesInterceptor prevents RDFLib from making autonomous web requests by intercepting urllib.request calls and either blocking them or redirecting them to local files.

Important Warnings

!!! warning "Client Application Compatibility" This interceptor modifies the global urllib.request opener, which affects all code using urllib.request in the same Python process. If your client application or other libraries use urllib.request for HTTP/HTTPS requests, this interceptor will block or redirect those requests as well, potentially breaking your application.

!!! warning "Production Use" This module is intended for: - Debugging: Isolating RDFLib's network behavior during development and testing - Production environments: Only when you can guarantee that no other code in your application uses urllib.request for HTTP/HTTPS requests

If your application or its dependencies use `urllib.request`, do **not** use this interceptor in production, as it will interfere with their functionality.

Handler Types

The module provides several handler types for different use cases:

Blocking Handlers

These handlers block all HTTP/HTTPS requests unless they are redirected to local files:

• ExtDepsRejecterAndRedirectorHTTPHandler: Blocks HTTP requests unless mapped to local files
• ExtDepsRejecterAndRedirectorHTTPSHandler: Blocks HTTPS requests unless mapped to local files

Redirecting Handlers

These handlers redirect mapped URIs to local files but allow unmapped requests to proceed normally:

• LocalFileRedirectorHTTPHandler: Redirects HTTP requests to local files, falls back to default behavior
• LocalFileRedirectorHTTPSHandler: Redirects HTTPS requests to local files, falls back to default behavior

Basic Usage

Blocking All External Requests

To block all external HTTP/HTTPS requests (except those mapped to local files):

from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

# Create interceptor (uses blocking handlers by default)
interceptor = ExternalDependenciesInterceptor()

# Register the interceptor
interceptor.register_dependencies_interceptor()

# Now RDFLib cannot make autonomous HTTP/HTTPS requests
# All requests will be blocked unless mapped to local files

Redirecting Requests to Local Files

To redirect specific URIs to local files while blocking others:

from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

# Create interceptor
interceptor = ExternalDependenciesInterceptor()

# Set up URI mappings (web URI -> local file path)
uri_mappings = {
    "http://example.com/ontology.owl": "/path/to/local/ontology.owl",
    "https://example.org/schema.ttl": "/path/to/local/schema.ttl"
}

# Configure mappings
interceptor.set_mappings(uri_mappings)

# Register the interceptor
interceptor.register_dependencies_interceptor()

# Now:
# - Mapped URIs will be served from local files
# - Unmapped URIs will be blocked (PermissionError)

Using Redirecting Handlers (Non-Blocking)

To redirect mapped URIs to local files but allow unmapped requests to proceed:

from cleverrdf_lib.core.external_dep_interceptor import (
    ExternalDependenciesInterceptor,
    LocalFileRedirectorHTTPHandler,
    LocalFileRedirectorHTTPSHandler
)

# Create interceptor
interceptor = ExternalDependenciesInterceptor()

# Replace handlers with redirecting (non-blocking) handlers
interceptor.replace_handlers([
    LocalFileRedirectorHTTPSHandler(),
    LocalFileRedirectorHTTPHandler()
])

# Set up URI mappings
uri_mappings = {
    "http://example.com/ontology.owl": "/path/to/local/ontology.owl"
}
interceptor.set_mappings(uri_mappings)

# Register the interceptor
interceptor.register_dependencies_interceptor()

# Now:
# - Mapped URIs will be served from local files
# - Unmapped URIs will use default HTTP/HTTPS behavior (not blocked)

Use Cases

Testing and Development

During testing, you may want to ensure that RDFLib doesn't make unexpected network requests:

from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

# Set up interceptor for testing
interceptor = ExternalDependenciesInterceptor()

# Map test URIs to local test files
test_mappings = {
    "http://test.example.com/ontology.owl": "test_data/test_ontology.owl"
}
interceptor.set_mappings(test_mappings)
interceptor.register_dependencies_interceptor()

# Now run your tests - RDFLib cannot make unexpected requests

Production with Controlled Network Access

In production environments where you want to ensure all ontology loading goes through CleverRDFLib:

from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

# Only use this if you're certain no other code uses urllib.request
interceptor = ExternalDependenciesInterceptor()

# Optionally map known URIs to local files
uri_mappings = {
    "http://known-ontology.example.com/owl": "/cached/ontology.owl"
}
interceptor.set_mappings(uri_mappings)

interceptor.register_dependencies_interceptor()

# Now all RDFLib requests are controlled

Handler Details

Blocking Handlers

ExtDepsRejecterAndRedirectorHTTPHandler and ExtDepsRejecterAndRedirectorHTTPSHandler:

• Behavior: Block all requests unless mapped to local files
• Raises: PermissionError for unmapped requests
• Use case: Strict control over network access

from cleverrdf_lib.core.external_dep_interceptor import (
    ExtDepsRejecterAndRedirectorHTTPHandler,
    ExtDepsRejecterAndRedirectorHTTPSHandler
)

handler = ExtDepsRejecterAndRedirectorHTTPHandler()
handler.update_mappings({
    "http://example.com/owl": "/local/file.owl"
})

# Mapped URI: Returns file content
# Unmapped URI: Raises PermissionError

Redirecting Handlers

LocalFileRedirectorHTTPHandler and LocalFileRedirectorHTTPSHandler:

• Behavior: Redirect mapped URIs to local files, allow unmapped requests to proceed
• Falls back: Uses default HTTP/HTTPS handler for unmapped requests
• Use case: Selective redirection without blocking

from cleverrdf_lib.core.external_dep_interceptor import (
    LocalFileRedirectorHTTPHandler,
    LocalFileRedirectorHTTPSHandler
)

handler = LocalFileRedirectorHTTPHandler()
handler.update_mappings({
    "http://example.com/owl": "/local/file.owl"
})

# Mapped URI: Returns file content
# Unmapped URI: Uses default HTTP handler (normal network request)

Updating Mappings

You can update URI mappings at runtime:

from cleverrdf_lib.core.external_dep_interceptor import ExternalDependenciesInterceptor

interceptor = ExternalDependenciesInterceptor()

# Initial mappings
interceptor.set_mappings({
    "http://example.com/owl": "/local/file1.owl"
})

interceptor.register_dependencies_interceptor()

# Later, update mappings
interceptor.set_mappings({
    "http://example.com/owl": "/local/file1.owl",
    "http://example.org/schema": "/local/file2.ttl"
})

Custom Handlers

You can create custom handlers by extending the base handlers:

from cleverrdf_lib.core.external_dep_interceptor import (
    ExternalDependenciesInterceptor,
    LocalFileRedirectorHTTPHandler
)
from urllib.request import Request

class CustomHTTPHandler(LocalFileRedirectorHTTPHandler):
    """Custom handler with additional logic."""
    
    def http_open(self, req: Request):
        # Add custom logic before redirection
        url = req.get_full_url()
        if url.endswith(".blocked"):
            raise PermissionError("Blocked URL")
        
        # Use parent implementation
        return super().http_open(req)

# Use custom handler
interceptor = ExternalDependenciesInterceptor()
interceptor.replace_handlers([CustomHTTPHandler()])
interceptor.register_dependencies_interceptor()

Best Practices

1. Use only when necessary: Only install the interceptor when you need to control RDFLib's network behavior
2. Test thoroughly: If using in production, thoroughly test that no other code uses urllib.request
3. Document usage: Clearly document in your codebase that the interceptor is installed
4. Use redirecting handlers: Prefer redirecting handlers over blocking handlers when possible to avoid breaking other functionality
5. Map all known URIs: Pre-map all URIs you know will be requested to avoid blocking legitimate requests

Limitations

• Global effect: The interceptor affects all urllib.request usage in the Python process
• No selective interception: Cannot selectively intercept only RDFLib requests
• Process-wide: Once installed, affects all code in the process
• Cannot be uninstalled: Once installed, the opener remains in effect

When Not to Use

Do not use this interceptor if:

• Your application uses urllib.request for other purposes
• Your dependencies use urllib.request for HTTP/HTTPS requests
• You need RDFLib to make autonomous network requests
• You're unsure whether other code uses urllib.request

Next Steps

• Learn about Basic Usage for loading ontologies
• Explore Resolvers for controlling reference resolution
• Understand Error Handling for managing loading errors