forked from HAL9000/cleveragents-core
e5f75c5c83
- Remove maximum cap (16) on CA_MAX_PARALLEL_WORKERS in resources.yaml - Can now be set to any positive value (32, 64, etc.) - Only minimum validation remains (must be > 0) - Remove dynamic backpressure/throttling from implementation-orchestrator - Dispatch always runs at full configured speed - Resource monitoring remains for visibility only - No automatic reduction of slots_available based on failures - Convert system-watchdog from auto-degradation to monitoring + suggestions - Renamed DEGRADATION_THRESHOLDS to HEALTH_THRESHOLDS - Removed apply_system_degradation() and check_degradation_recovery() - Changed findings to include suggestions instead of actions - Watchdog now reports issues with fix recommendations - No automatic throttling or pausing of agents The system now operates at maximum configured speed at all times, with the watchdog providing diagnostic insights when issues arise.
10 KiB
10 KiB
Credential Security Standards
CRITICAL: Credentials are sensitive data that must be handled with extreme care. Violations of these standards are security incidents.
Core Principles
- Never log credentials - Not even partially or obfuscated
- Use environment variables exclusively - Never hardcode or pass in prompts
- Validate without exposing - Check format without revealing content
- Fail securely - Missing credentials should halt execution
- Clean up after use - Remove any temporary credential storage
Required Credentials
REQUIRED_CREDENTIALS = {
'FORGEJO_PAT': {
'description': 'Forgejo Personal Access Token',
'format': r'^[a-f0-9]{40}$', # 40 hex characters
'usage': 'HTTPS git authentication and API access'
},
'GIT_USER_NAME': {
'description': 'Git commit author name',
'format': r'^.+$', # Non-empty
'usage': 'Git commit attribution'
},
'GIT_USER_EMAIL': {
'description': 'Git commit author email',
'format': r'^[^@]+@[^@]+\.[^@]+$', # Basic email format
'usage': 'Git commit attribution'
},
'FORGEJO_USERNAME': {
'description': 'Forgejo username',
'format': r'^[a-zA-Z0-9_-]+$', # Alphanumeric with _ and -
'usage': 'Forgejo API operations and issue assignment'
},
'FORGEJO_PASSWORD': {
'description': 'Forgejo password',
'format': r'^.{8,}$', # At least 8 characters
'usage': 'Web UI access for CI logs when API unavailable'
}
}
OPTIONAL_CREDENTIALS = {
'CA_MAX_PARALLEL_WORKERS': {
'description': 'Maximum parallel workers',
'format': r'^\d+$', # Positive integer
'default': '4',
'usage': 'Parallelism control'
},
'OPENCODE_SERVER_PASSWORD': {
'description': 'OpenCode server password',
'format': r'^.+$', # Non-empty if set
'usage': 'OpenCode API authentication'
}
}
Credential Validation
def validate_credentials(required_only=False):
"""
Validate all required (and optionally optional) credentials.
Returns (valid: bool, missing: list, malformed: list)
CRITICAL: This function validates WITHOUT logging credential values.
"""
missing = []
malformed = []
# Check required credentials
for var_name, config in REQUIRED_CREDENTIALS.items():
value = os.environ.get(var_name, '').strip()
if not value:
missing.append(var_name)
elif config.get('format'):
# Validate format without logging the value
if not re.match(config['format'], value):
malformed.append(var_name)
# Log the issue without the value
print(f"[CREDENTIAL ERROR] {var_name} format validation failed")
# Check optional credentials if requested
if not required_only:
for var_name, config in OPTIONAL_CREDENTIALS.items():
value = os.environ.get(var_name, '').strip()
if value and config.get('format'):
if not re.match(config['format'], value):
malformed.append(var_name)
print(f"[CREDENTIAL WARNING] {var_name} format validation failed")
# Report results without exposing values
if missing:
print(f"[CREDENTIAL ERROR] Missing required credentials: {missing}")
if malformed:
print(f"[CREDENTIAL ERROR] Malformed credentials: {malformed}")
return (not missing and not malformed, missing, malformed)
def get_credential(var_name, default=None):
"""
Safely retrieve a credential value.
NEVER use this in print statements or logs!
"""
if var_name not in REQUIRED_CREDENTIALS and var_name not in OPTIONAL_CREDENTIALS:
raise ValueError(f"Unknown credential: {var_name}")
value = os.environ.get(var_name, '').strip()
if not value:
if var_name in OPTIONAL_CREDENTIALS:
return OPTIONAL_CREDENTIALS[var_name].get('default', default)
return None
return value
def check_credential_exists(var_name):
"""Check if a credential is set without retrieving its value."""
return bool(os.environ.get(var_name, '').strip())
Safe Usage Patterns
Git Configuration
def configure_git_safely(clone_dir):
"""Configure git identity without logging credentials."""
name = get_credential('GIT_USER_NAME')
email = get_credential('GIT_USER_EMAIL')
if not name or not email:
raise ValueError("Git identity credentials not available")
# Configure without echoing values
os.chdir(clone_dir)
subprocess.run(['git', 'config', 'user.name', name],
capture_output=True, check=True)
subprocess.run(['git', 'config', 'user.email', email],
capture_output=True, check=True)
print("[GIT] Configured user identity") # Note: no values logged
HTTPS Authentication
def get_authenticated_url(base_url):
"""Build authenticated URL without exposing the token."""
pat = get_credential('FORGEJO_PAT')
if not pat:
raise ValueError("FORGEJO_PAT not available")
# Parse URL
parsed = urllib.parse.urlparse(base_url)
# Insert token (never log this URL!)
auth_url = f"{parsed.scheme}://{pat}@{parsed.netloc}{parsed.path}"
return auth_url
def clone_with_auth(repo_url, target_dir):
"""Clone repository with authentication."""
auth_url = get_authenticated_url(repo_url)
# Clone without echoing the URL
result = subprocess.run(
['git', 'clone', auth_url, target_dir],
capture_output=True,
text=True
)
if result.returncode == 0:
print(f"[GIT] Successfully cloned to {target_dir}")
else:
# Log error without exposing URL
print(f"[GIT] Clone failed: {result.stderr}")
# Clean up any partial clone
if os.path.exists(target_dir):
shutil.rmtree(target_dir)
raise RuntimeError("Git clone failed")
Web Authentication
def web_login_safely():
"""Login to Forgejo web UI without exposing credentials."""
username = get_credential('FORGEJO_USERNAME')
password = get_credential('FORGEJO_PASSWORD')
if not username or not password:
raise ValueError("Web login credentials not available")
cookie_file = f"/tmp/cookies_{os.getpid()}.txt"
try:
# Get CSRF token
csrf_result = subprocess.run(
['curl', '-s', '-c', cookie_file, 'https://git.cleverthis.com/user/login'],
capture_output=True, text=True
)
csrf_match = re.search(r'name="_csrf" value="([^"]+)"', csrf_result.stdout)
if not csrf_match:
raise RuntimeError("Could not get CSRF token")
csrf_token = csrf_match.group(1)
# Login (credentials in POST data, not logged)
login_result = subprocess.run([
'curl', '-s', '-b', cookie_file, '-c', cookie_file,
'-X', 'POST', 'https://git.cleverthis.com/user/login',
'-d', f'user_name={username}',
'-d', f'password={password}',
'-d', f'_csrf={csrf_token}',
'-L'
], capture_output=True)
# Check login success without exposing response
if 'user/settings' in login_result.stdout.decode('utf-8', errors='ignore'):
print("[AUTH] Web login successful")
return cookie_file
else:
print("[AUTH] Web login failed")
raise RuntimeError("Web authentication failed")
except Exception as e:
# Clean up on any error
if os.path.exists(cookie_file):
os.remove(cookie_file)
raise
Startup Validation Template
Every agent MUST validate credentials at startup:
def agent_startup():
"""Standard agent startup with credential validation."""
print("[STARTUP] Validating credentials...")
# Validate all required credentials
valid, missing, malformed = validate_credentials(required_only=True)
if not valid:
error_msg = "Cannot start: credential validation failed\n"
if missing:
error_msg += f"Missing: {', '.join(missing)}\n"
if malformed:
error_msg += f"Malformed: {', '.join(malformed)}\n"
print(f"[FATAL] {error_msg}")
# Provide helpful information without exposing values
print("\nRequired environment variables:")
for var, config in REQUIRED_CREDENTIALS.items():
status = "✓ Set" if check_credential_exists(var) else "✗ Missing"
print(f" {var}: {status} - {config['description']}")
raise ValueError("Credential validation failed")
print("[STARTUP] All credentials validated successfully")
# Continue with agent initialization...
Security Rules
- NO credential logging - Never print, log, or echo credential values
- NO credentials in prompts - All credentials via environment only
- NO credentials in comments - Don't post credentials to Forgejo
- NO credentials in temp files - Except cookies, which must be cleaned up
- NO partial credentials - Don't log even parts of tokens
- NO credential defaults - Required credentials have no defaults
- Validate early - Check credentials at startup, not during work
Common Mistakes to Avoid
# ❌ WRONG - Logs the token
print(f"Using PAT: {os.environ['FORGEJO_PAT']}")
# ✓ CORRECT - Confirms presence without exposing
print("FORGEJO_PAT is configured" if check_credential_exists('FORGEJO_PAT') else "FORGEJO_PAT missing")
# ❌ WRONG - Includes token in URL that might be logged
clone_url = f"https://{pat}@git.cleverthis.com/repo.git"
print(f"Cloning from {clone_url}")
# ✓ CORRECT - Never logs authenticated URLs
auth_url = get_authenticated_url("https://git.cleverthis.com/repo.git")
subprocess.run(['git', 'clone', auth_url, target], capture_output=True)
print(f"Cloning repository to {target}")
# ❌ WRONG - Credential in error message
try:
api_call(token)
except:
print(f"API call failed with token {token}")
# ✓ CORRECT - Generic error without credential
try:
api_call(token)
except:
print("API authentication failed")
Enforcement
The system-watchdog will monitor for:
- Credentials appearing in Forgejo comments
- Credentials in log output
- Git commits containing credentials
- Temp files with credentials not cleaned up
- Agents starting without validation