[CHORE]: SAST (Semgrep) and DAST (OWASP ZAP) automated security testing Makefile targets and GitHub Actions #259
Description
🛡️ Chore Summary
Implement comprehensive SAST and DAST security testing automation across the entire MCP Gateway codebase: establish robust static and dynamic application security testing with make security-all, make security-sast, make security-dast, and make security-report targets, achieving deep security coverage of source code patterns, runtime vulnerabilities, API endpoint security, and authentication flows to uncover security issues, injection attacks, and configuration weaknesses before they reach production.
🧱 Areas Affected
- Security testing infrastructure / SAST & DAST framework setup
- Build system / Make targets (
make security-all,make security-sast,make security-dast,make security-report) - GitHub Actions / CI pipeline (PR-based and nightly security scanning)
- Static code analysis (SQL injection, XSS, hard-coded secrets, insecure patterns)
- Dynamic API security testing (authentication bypasses, injection attacks, headers)
- FastAPI endpoint security (JSON-RPC, admin endpoints, authentication flows)
- Configuration security (TLS, authentication, environment variables)
- Dependency vulnerability scanning (known CVEs, outdated packages)
⚙️ Context / Rationale
Security testing ensures that every line of code and running endpoint is protected against common vulnerabilities including OWASP Top 10 issues, injection attacks, authentication bypasses, and configuration weaknesses. By combining static analysis (SAST) and dynamic testing (DAST), we create comprehensive security coverage that catches vulnerabilities at both code-time and runtime. This is critical for a gateway that processes authentication, handles arbitrary JSON-RPC requests, and manages sensitive configuration data.
What is SAST vs DAST?
| Phase | What it analyzes | Primary goal | Tool to use |
|---|---|---|---|
| SAST (before the code ever runs) | Source code & config files | Catch insecure patterns (e.g. string-building SQL) early and cheaply | Semgrep |
| DAST (while the app is running) | A live instance of your API/site | Provoke the server with evil requests & check responses/headers | OWASP ZAP |
MCP Gateway Security Testing Architecture:
graph TD
A[Security Testing Suite] --> B[Static Analysis - SAST]
A --> C[Dynamic Analysis - DAST]
A --> D[Dependency Scanning]
A --> E[Configuration Auditing]
B --> B1[Semgrep - Code Patterns]
B --> B2[SQL Injection Detection]
B --> B3[XSS Prevention Check]
B --> B4[Hard-coded Secrets]
B --> B5[Authentication Bypasses]
C --> C1[OWASP ZAP - Runtime]
C --> C2[API Endpoint Fuzzing]
C --> C3[Authentication Testing]
C --> C4[Header Security Checks]
C --> C5[Input Validation Testing]
D --> D1[pip-audit - Known CVEs]
D --> D2[Safety - PyPI Vulnerabilities]
D --> D3[Outdated Dependencies]
E --> E1[TLS Configuration]
E --> E2[Environment Variables]
E --> E3[Default Credentials]
E --> E4[Secure Headers]
Semgrep SAST Rules Example:
# security/semgrep-rules.yml
rules:
- id: hardcoded-password
patterns:
- pattern: password = "$PASSWORD"
- pattern: PASSWORD = "$PASSWORD"
- metavariable-regex:
metavariable: $PASSWORD
regex: ^.{8,}$
message: "Hard-coded password detected"
languages: [python]
severity: ERROR
- id: sql-injection-risk
patterns:
- pattern: $QUERY = f"SELECT ... {$USER_INPUT}"
- pattern: $QUERY = "SELECT ... " + $USER_INPUT
- pattern: cursor.execute($QUERY % $USER_INPUT)
message: "Potential SQL injection vulnerability"
languages: [python]
severity: ERROR
- id: jwt-hardcoded-secret
patterns:
- pattern: jwt.encode($PAYLOAD, "$SECRET", ...)
- metavariable-regex:
metavariable: $SECRET
regex: ^[a-zA-Z0-9]{8,}$
message: "Hard-coded JWT secret key"
languages: [python]
severity: ERROR
- id: insecure-random
patterns:
- pattern: random.random()
- pattern: random.randint(...)
message: "Use secrets module for cryptographic randomness"
languages: [python]
severity: WARNING
- id: debug-mode-enabled
patterns:
- pattern: debug=True
- pattern: DEBUG = True
message: "Debug mode should not be enabled in production"
languages: [python]
severity: WARNINGCustom MCP Gateway Security Rules:
# security/mcp-gateway-rules.yml
rules:
- id: mcp-auth-bypass
patterns:
- pattern: app.dependency_overrides[$AUTH_FUNC] = lambda: $USER
message: "Authentication bypass detected - ensure this is test-only"
languages: [python]
severity: WARNING
- id: mcp-unsafe-jsonpath
patterns:
- pattern: jsonpath_ng.parse($USER_INPUT)
- pattern: jq.compile($USER_INPUT)
message: "JSONPath/JQ expressions from user input may be unsafe"
languages: [python]
severity: WARNING
- id: mcp-insecure-transport
patterns:
- pattern: skip_ssl_verify=True
- pattern: verify=False
message: "SSL verification disabled - security risk"
languages: [python]
severity: ERROR
- id: mcp-weak-auth
patterns:
- pattern: basic_auth_password = "changeme"
- pattern: jwt_secret_key = "my-test-key"
message: "Default/weak authentication credentials"
languages: [python]
severity: ERROROWASP ZAP DAST Configuration:
# security/zap-config.yml
zap:
spider:
maxDuration: 5 # minutes
maxDepth: 5
maxChildren: 10
activeScan:
policy: "Default Policy"
maxRuleDurationInMins: 5
maxScanDurationInMins: 15
authentication:
method: "httpAuthentication"
httpAuthentication:
hostname: "localhost"
port: 4444
realm: ""
username: "admin"
password: "changeme"
globalExcludeUrl:
- ".*logout.*"
- ".*\\.css"
- ".*\\.js"
- ".*\\.png"
- ".*\\.jpg"
rules:
- id: 10035 # Strict-Transport-Security missing
action: "FAIL"
- id: 10036 # Server header information leak
action: "WARN"
- id: 40012 # Cross-Site Scripting (Reflected)
action: "FAIL"
- id: 40014 # Cross-Site Scripting (Persistent)
action: "FAIL"
- id: 40018 # SQL Injection
action: "FAIL"
- id: 40019 # SQL Injection (MySQL)
action: "FAIL"ZAP Custom Scripts for MCP Gateway:
# security/zap-scripts/mcp-auth-test.py
"""Custom ZAP script for testing MCP Gateway authentication."""
def scanNode(sas, msg):
"""Test authentication bypass attempts."""
# Test endpoints without auth headers
endpoints = [
"/admin/tools",
"/admin/servers",
"/admin/resources",
"/rpc",
"/metrics"
]
for endpoint in endpoints:
# Test without Authorization header
test_msg = msg.cloneRequest()
test_msg.getRequestHeader().setURI(endpoint)
test_msg.getRequestHeader().setHeader("Authorization", None)
sas.sendAndReceive(test_msg)
if test_msg.getResponseHeader().getStatusCode() == 200:
sas.raiseAlert(
risk=3, # High
confidence=3, # High
name="Authentication Bypass",
description=f"Endpoint {endpoint} accessible without authentication",
uri=endpoint,
param="",
attack="No Authorization header",
otherInfo="",
solution="Ensure all sensitive endpoints require authentication",
reference="",
evidence="",
cweId=287, # CWE-287: Improper Authentication
wascId=1, # WASC-1: Insufficient Authentication
msg=test_msg
)📦 Related Make Targets
| Target | Purpose |
|---|---|
make security-all |
Run complete security testing suite (SAST + DAST + deps) |
make security-sast |
Static Application Security Testing with Semgrep |
make security-dast |
Dynamic Application Security Testing with OWASP ZAP |
make security-deps |
Dependency vulnerability scanning |
make security-install |
Install all security testing dependencies |
make security-report |
Generate comprehensive security testing report |
make security-baseline |
Quick security baseline scan for CI/PR validation |
make security-full |
Extended security testing for nightly runs |
make security-clean |
Clean security testing artifacts and reports |
make security-config |
Validate security configuration and settings |
make security-headers |
Test HTTP security headers configuration |
make security-auth |
Test authentication and authorization mechanisms |
Bold targets are mandatory; CI must fail if critical or high-severity security issues are discovered.
📋 Acceptance Criteria
-
make security-allcompletes successfully with 0 critical and 0 high-severity security findings. - Semgrep SAST scans detect SQL injection, XSS, hard-coded secrets, and authentication bypass patterns.
- OWASP ZAP DAST validates all API endpoints against injection attacks, authentication bypasses, and header security.
- Dependency scanning identifies and reports known CVEs in Python packages.
- Authentication testing verifies proper access controls on admin endpoints and RPC calls.
- Security headers validation ensures HSTS, CSP, X-Frame-Options, and other protective headers.
- Configuration auditing checks for default credentials, debug modes, and insecure settings.
- CI integration runs baseline security scans on PRs and comprehensive scans nightly.
- Security reports provide actionable findings with remediation guidance and severity ratings.
- False positive management includes suppressions for legitimate patterns and test code.
- Regression prevention ensures previously fixed security issues don't reappear.
- Documentation provides clear security testing procedures and vulnerability response workflow.
🛠️ Task List (suggested flow)
-
Install security testing dependencies and setup
# Install core security testing tools pip install semgrep bandit safety pip-audit # Install OWASP ZAP (requires Java) # Option 1: Docker (recommended) docker pull ghcr.io/zaproxy/zaproxy:stable # Option 2: Direct install # Download from https://www.zaproxy.org/download/
Create
requirements-security.txt:semgrep>=1.45.0 bandit>=1.7.5 safety>=2.3.0 pip-audit>=2.6.0 requests>=2.31.0 pyyaml>=6.0
-
Makefile integration
.PHONY: security-all security-sast security-dast security-deps \ security-install security-report security-baseline security-clean # Install all security testing dependencies security-install: @echo "🔧 Installing security testing dependencies..." pip install -r requirements-security.txt @echo "✅ Security tools installed" # Run complete security testing suite security-all: security-sast security-dast security-deps @echo "🛡️ Complete security testing suite finished" make security-report # Static Application Security Testing security-sast: @echo "🔍 Running Semgrep SAST analysis..." @mkdir -p reports/security semgrep scan \ --config=security/semgrep-rules.yml \ --config=security/mcp-gateway-rules.yml \ --config=p/security-audit \ --config=p/secrets \ --config=p/sql-injection \ --config=p/xss \ --sarif \ --output=reports/security/semgrep.sarif \ --error \ . @echo "📊 Running Bandit security linter..." bandit -r mcpgateway/ -f json -o reports/security/bandit.json || true # Dynamic Application Security Testing security-dast: @echo "🌐 Running OWASP ZAP DAST analysis..." @mkdir -p reports/security @echo "Starting test server..." make run-dev & sleep 15 # Baseline scan (passive) docker run --rm -v $(PWD)/reports/security:/zap/wrk/ \ --network host \ ghcr.io/zaproxy/zaproxy:stable \ zap-baseline.py \ -t http://localhost:4444 \ -J zap-baseline.json \ -r zap-baseline.html \ -m 5 -T 300 \ -z "-configfile /zap/wrk/zap-config.yml" # Full active scan docker run --rm -v $(PWD)/reports/security:/zap/wrk/ \ --network host \ ghcr.io/zaproxy/zaproxy:stable \ zap-full-scan.py \ -t http://localhost:4444 \ -J zap-full.json \ -r zap-full.html \ -m 10 -T 600 \ -z "-configfile /zap/wrk/zap-config.yml" pkill -f "make run-dev" || true # Dependency vulnerability scanning security-deps: @echo "📦 Scanning dependencies for vulnerabilities..." @mkdir -p reports/security pip-audit --format=json --output=reports/security/pip-audit.json safety check --json --output reports/security/safety.json || true # Quick security baseline for CI security-baseline: @echo "⚡ Running quick security baseline..." semgrep scan \ --config=p/security-audit \ --config=p/secrets \ --error \ --quiet \ . pip-audit --format=cyclonedx --output=/dev/null # Generate security report security-report: @echo "📊 Generating security testing report..." python scripts/generate_security_report.py # Clean security artifacts security-clean: @echo "🧹 Cleaning security testing artifacts..." rm -rf reports/security/ rm -f zap-*.json zap-*.html
-
Directory structure setup
security/ ├── semgrep-rules.yml # General security rules ├── mcp-gateway-rules.yml # MCP Gateway specific rules ├── zap-config.yml # OWASP ZAP configuration ├── zap-scripts/ # Custom ZAP testing scripts │ ├── mcp-auth-test.py # Authentication testing │ ├── mcp-jsonrpc-test.py # JSON-RPC endpoint testing │ └── mcp-admin-test.py # Admin UI security testing ├── suppressions/ # False positive suppressions │ ├── semgrep.yml # Semgrep suppressions │ └── bandit.yml # Bandit suppressions └── policies/ # Security policies and guidelines ├── security-policy.md # Security testing policy └── vulnerability-response.md # Incident response procedures reports/security/ ├── semgrep.sarif # Semgrep SAST results ├── bandit.json # Bandit security linting ├── zap-baseline.json # ZAP baseline scan ├── zap-full.json # ZAP active scan ├── pip-audit.json # Dependency vulnerabilities ├── safety.json # PyPI security database └── security-report.html # Comprehensive report -
Enhanced Semgrep rules for MCP Gateway
# security/mcp-gateway-rules.yml rules: - id: mcp-gateway-weak-jwt message: "Weak JWT configuration detected" languages: [python] severity: ERROR patterns: - pattern-either: - pattern: jwt_secret_key = "my-test-key" - pattern: JWT_SECRET_KEY = "my-test-key" - pattern: jwt.encode($PAYLOAD, "test", ...) - pattern: jwt.encode($PAYLOAD, "secret", ...) - id: mcp-gateway-debug-mode message: "Debug mode enabled - security risk in production" languages: [python] severity: WARNING patterns: - pattern-either: - pattern: uvicorn.run(..., debug=True, ...) - pattern: app.debug = True - pattern: DEBUG = True - id: mcp-gateway-ssl-disabled message: "SSL verification disabled" languages: [python] severity: ERROR patterns: - pattern-either: - pattern: skip_ssl_verify = True - pattern: verify=False - pattern: ssl.create_default_context(check_hostname=False) - id: mcp-gateway-default-credentials message: "Default credentials detected" languages: [python] severity: ERROR patterns: - pattern-either: - pattern: basic_auth_password = "changeme" - pattern: BASIC_AUTH_PASSWORD = "changeme" - pattern: password = "admin" - id: mcp-gateway-insecure-session message: "Insecure session configuration" languages: [python] severity: WARNING patterns: - pattern-either: - pattern: SessionMiddleware(..., https_only=False, ...) - pattern: session.configure(..., secure=False, ...) - id: mcp-gateway-path-traversal message: "Potential path traversal vulnerability" languages: [python] severity: ERROR patterns: - pattern-either: - pattern: open($USER_INPUT, ...) - pattern: Path($USER_INPUT) - pattern: os.path.join(..., $USER_INPUT) - pattern-not: - pattern: open($SAFE_PATH, ...) - pattern-where-python: | isinstance($USER_INPUT, str) and not ".." in $USER_INPUT - id: mcp-gateway-command-injection message: "Potential command injection" languages: [python] severity: ERROR patterns: - pattern-either: - pattern: subprocess.run($USER_INPUT, ...) - pattern: os.system($USER_INPUT) - pattern: subprocess.call($USER_INPUT, ...) - pattern-not: - pattern: subprocess.run([$SAFE_COMMAND, ...], ...)
-
OWASP ZAP configuration and custom scripts
# security/zap-config.yml # ZAP configuration for MCP Gateway testing zap: # Spider configuration spider: maxDuration: 5 maxDepth: 5 maxChildren: 10 acceptCookies: true handleParameters: true parseComments: true parseRobotsTxt: true parseSitemapXml: true postForm: true processForm: true requestWaitTime: 200 sendRefererHeader: true skipURLString: "logout" userAgent: "ZAP Security Scanner" # Active scan configuration activeScan: policy: "Default Policy" maxRuleDurationInMins: 5 maxScanDurationInMins: 15 addQueryParam: false defaultPolicy: true delayInMs: 0 handleAntiCSRFTokens: true inScopeOnly: true scanHeadersAllRequests: true scanNullJsonValues: true # Authentication configuration for MCP Gateway authentication: method: "httpAuthentication" httpAuthentication: hostname: "localhost" port: 4444 realm: "" username: "admin" password: "changeme" # Session management sessionManagement: method: "cookieBasedSessionManagement" # User management for authenticated scanning users: - name: "admin_user" credentials: username: "admin" password: "changeme" # URLs to exclude from scanning globalExcludeUrl: - ".*logout.*" - ".*\\.css$" - ".*\\.js$" - ".*\\.png$" - ".*\\.jpg$" - ".*\\.ico$" - ".*\\.woff.*" - ".*/static/.*" # Custom rules configuration rules: - id: 10035 # Strict-Transport-Security missing action: "FAIL" - id: 10036 # Server header information leak action: "WARN" - id: 10038 # Content Security Policy missing action: "FAIL" - id: 40012 # Cross-Site Scripting (Reflected) action: "FAIL" - id: 40014 # Cross-Site Scripting (Persistent) action: "FAIL" - id: 40016 # Cross-Site Scripting (Persistent) - Prime action: "FAIL" - id: 40017 # Cross-Site Scripting (Persistent) - Spider action: "FAIL" - id: 40018 # SQL Injection action: "FAIL" - id: 40019 # SQL Injection (MySQL) action: "FAIL" - id: 40020 # SQL Injection (Hypersonic) action: "FAIL" - id: 40021 # SQL Injection (Oracle) action: "FAIL" - id: 40022 # SQL Injection (PostgreSQL) action: "FAIL" - id: 90019 # Code Injection action: "FAIL" - id: 90020 # Command Injection action: "FAIL"
# security/zap-scripts/mcp-jsonrpc-test.py """Custom ZAP script for testing JSON-RPC endpoint security.""" def scanNode(sas, msg): """Test JSON-RPC endpoint for injection vulnerabilities.""" # Test malicious JSON-RPC payloads malicious_payloads = [ # SQL injection attempts '{"jsonrpc":"2.0","method":"tools/list","params":{"filter":"1\' OR 1=1--"},"id":1}', # XSS attempts '{"jsonrpc":"2.0","method":"ping","params":{"message":"<script>alert(1)</script>"},"id":1}', # Command injection '{"jsonrpc":"2.0","method":"tools/call","params":{"name":"test; rm -rf /"},"id":1}', # Path traversal '{"jsonrpc":"2.0","method":"resources/read","params":{"uri":"../../../etc/passwd"},"id":1}', # XXE injection '{"jsonrpc":"2.0","method":"initialize","params":{"name":"<?xml version=\\"1.0\\"?><!DOCTYPE foo [<!ENTITY xxe SYSTEM \\"file:///etc/passwd\\">]><foo>&xxe;</foo>"},"id":1}', # Large payload (DoS) '{"jsonrpc":"2.0","method":"ping","params":{"data":"' + 'A' * 100000 + '"},"id":1}', # Invalid method names '{"jsonrpc":"2.0","method":"../admin/users","params":{},"id":1}', '{"jsonrpc":"2.0","method":"system.exit","params":{},"id":1}', # Protocol confusion '{"jsonrpc":"1.0","method":"ping","params":{},"id":1}', '{"jsonrpc":"3.0","method":"ping","params":{},"id":1}', ] original_uri = msg.getRequestHeader().getURI().toString() if "/rpc" not in original_uri: return for payload in malicious_payloads: test_msg = msg.cloneRequest() test_msg.getRequestHeader().setMethod("POST") test_msg.getRequestHeader().setHeader("Content-Type", "application/json") test_msg.getRequestHeader().setHeader("Authorization", "Basic YWRtaW46Y2hhbmdlbWU=") test_msg.setRequestBody(payload) sas.sendAndReceive(test_msg) response_code = test_msg.getResponseHeader().getStatusCode() response_body = test_msg.getResponseBody().toString() # Check for potential vulnerabilities if response_code == 200 and "error" not in response_body.lower(): sas.raiseAlert( risk=2, # Medium confidence=2, # Medium name="JSON-RPC Injection Vulnerability", description=f"JSON-RPC endpoint may be vulnerable to injection attacks", uri=original_uri, param="JSON-RPC payload", attack=payload[:100] + "..." if len(payload) > 100 else payload, otherInfo=f"Response: {response_body[:200]}", solution="Implement proper input validation and sanitization", reference="https://owasp.org/www-project-top-ten/", evidence=response_body[:500], cweId=89, # CWE-89: SQL Injection wascId=19, # WASC-19: SQL Injection msg=test_msg )
-
Security configuration validation tests
# tests/security/test_config_security.py """Security configuration validation tests.""" import pytest from mcpgateway.config import Settings import os class TestSecurityConfiguration: """Test security-related configuration settings.""" def test_default_credentials_detection(self): """Test detection of default/weak credentials.""" settings = Settings() # These should trigger warnings in production weak_indicators = [ settings.basic_auth_password == "changeme", settings.jwt_secret_key == "my-test-key", settings.auth_encryption_secret == "my-test-salt" ] # In test environment, these are acceptable # In production, they should be changed if os.getenv("ENV") == "production": assert not any(weak_indicators), "Weak credentials detected in production" def test_ssl_configuration(self): """Test SSL/TLS security configuration.""" settings = Settings() # SSL verification should be enabled in production if os.getenv("ENV") == "production": assert not settings.skip_ssl_verify, "SSL verification disabled in production" def test_debug_mode_disabled(self): """Test that debug mode is disabled in production.""" # This would be checked during deployment assert os.getenv("DEBUG") != "True", "Debug mode should not be enabled" def test_authentication_required(self): """Test that authentication is required by default.""" settings = Settings() assert settings.auth_required, "Authentication should be required by default" def test_secure_random_generation(self): """Test that secure random generation is used.""" import secrets import random # Test that we're using cryptographically secure random secure_token = secrets.token_hex(16) assert len(secure_token) == 32 # Verify we're not using predictable random insecure_random = random.random() assert isinstance(insecure_random, float) # Just ensure it works
-
Dependency vulnerability scanning
# scripts/security_deps_check.py #!/usr/bin/env python3 """Enhanced dependency vulnerability checking.""" import json import subprocess import sys from pathlib import Path def run_pip_audit(): """Run pip-audit and return results.""" try: result = subprocess.run([ "pip-audit", "--format=json", "--progress-spinner=off" ], capture_output=True, text=True, check=False) if result.returncode != 0 and result.stdout: # pip-audit found vulnerabilities return json.loads(result.stdout) elif result.returncode == 0: # No vulnerabilities found return {"vulnerabilities": []} else: print(f"pip-audit failed: {result.stderr}") return None except Exception as e: print(f"Error running pip-audit: {e}") return None def run_safety_check(): """Run safety check and return results.""" try: result = subprocess.run([ "safety", "check", "--json" ], capture_output=True, text=True, check=False) if result.stdout: return json.loads(result.stdout) else: return [] except Exception as e: print(f"Error running safety: {e}") return [] def analyze_vulnerabilities(pip_audit_results, safety_results): """Analyze vulnerability results and determine severity.""" critical_count = 0 high_count = 0 medium_count = 0 low_count = 0 findings = [] # Process pip-audit results if pip_audit_results and "vulnerabilities" in pip_audit_results: for vuln in pip_audit_results["vulnerabilities"]: severity = "unknown" if "aliases" in vuln: # Try to determine severity from CVE info for alias in vuln["aliases"]: if alias.startswith("CVE-"): # In real implementation, you'd look up CVSS scores severity = "medium" # Default assumption findings.append({ "source": "pip-audit", "package": vuln.get("package", "unknown"), "version": vuln.get("installed_version", "unknown"), "vulnerability": vuln.get("id", "unknown"), "severity": severity, "description": vuln.get("description", "") }) if severity == "critical": critical_count += 1 elif severity == "high": high_count += 1 elif severity == "medium": medium_count += 1 else: low_count += 1 # Process safety results for vuln in safety_results: severity = "medium" # Safety doesn't provide severity, assume medium findings.append({ "source": "safety", "package": vuln.get("package_name", "unknown"), "version": vuln.get("installed_version", "unknown"), "vulnerability": vuln.get("vulnerability_id", "unknown"), "severity": severity, "description": vuln.get("advisory", "") }) medium_count += 1 return { "summary": { "critical": critical_count, "high": high_count, "medium": medium_count, "low": low_count, "total": len(findings) }, "findings": findings } def main(): """Main dependency vulnerability checking function.""" print("🔍 Scanning dependencies for vulnerabilities...") # Run vulnerability scans pip_audit_results = run_pip_audit() safety_results = run_safety_check() # Analyze results analysis = analyze_vulnerabilities(pip_audit_results, safety_results) # Create reports directory reports_dir = Path("reports/security") reports_dir.mkdir(parents=True, exist_ok=True) # Write detailed report with open(reports_dir / "dependency-vulnerabilities.json", "w") as f: json.dump(analysis, f, indent=2) # Print summary summary = analysis["summary"] print(f"📊 Vulnerability Summary:") print(f" Critical: {summary['critical']}") print(f" High: {summary['high']}") print(f" Medium: {summary['medium']}") print(f" Low: {summary['low']}") print(f" Total: {summary['total']}") # Exit with error code if critical or high vulnerabilities found if summary["critical"] > 0 or summary["high"] > 0: print("❌ Critical or high-severity vulnerabilities found!") sys.exit(1) elif summary["medium"] > 5: # Threshold for medium severity print("⚠️ Too many medium-severity vulnerabilities found!") sys.exit(1) else: print("✅ No critical security issues found in dependencies") sys.exit(0) if __name__ == "__main__": main()
-
Comprehensive security report generation
# scripts/generate_security_report.py #!/usr/bin/env python3 """Generate comprehensive security testing report.""" import json import os from datetime import datetime from pathlib import Path import xml.etree.ElementTree as ET def parse_semgrep_sarif(sarif_file): """Parse Semgrep SARIF results.""" if not Path(sarif_file).exists(): return {"findings": [], "summary": {"error": 0, "warning": 0, "info": 0}} try: with open(sarif_file) as f: sarif_data = json.load(f) findings = [] summary = {"error": 0, "warning": 0, "info": 0} for run in sarif_data.get("runs", []): for result in run.get("results", []): rule_id = result.get("ruleId", "unknown") level = result.get("level", "info") message = result.get("message", {}).get("text", "") for location in result.get("locations", []): physical_location = location.get("physicalLocation", {}) artifact_location = physical_location.get("artifactLocation", {}) file_path = artifact_location.get("uri", "unknown") finding = { "rule_id": rule_id, "severity": level, "message": message, "file": file_path, "tool": "semgrep" } findings.append(finding) summary[level] = summary.get(level, 0) + 1 return {"findings": findings, "summary": summary} except Exception as e: print(f"Error parsing Semgrep SARIF: {e}") return {"findings": [], "summary": {"error": 0}} def parse_bandit_json(bandit_file): """Parse Bandit JSON results.""" if not Path(bandit_file).exists(): return {"findings": [], "summary": {"HIGH": 0, "MEDIUM": 0, "LOW": 0}} try: with open(bandit_file) as f: bandit_data = json.load(f) findings = [] summary = {"HIGH": 0, "MEDIUM": 0, "LOW": 0} for result in bandit_data.get("results", []): finding = { "rule_id": result.get("test_id", "unknown"), "severity": result.get("issue_severity", "UNKNOWN"), "message": result.get("issue_text", ""), "file": result.get("filename", "unknown"), "line": result.get("line_number", 0), "tool": "bandit" } findings.append(finding) severity = result.get("issue_severity", "LOW") summary[severity] = summary.get(severity, 0) + 1 return {"findings": findings, "summary": summary} except Exception as e: print(f"Error parsing Bandit JSON: {e}") return {"findings": [], "summary": {}} def parse_zap_json(zap_file): """Parse OWASP ZAP JSON results.""" if not Path(zap_file).exists(): return {"findings": [], "summary": {"High": 0, "Medium": 0, "Low": 0}} try: with open(zap_file) as f: zap_data = json.load(f) findings = [] summary = {"High": 0, "Medium": 0, "Low": 0, "Informational": 0} for site in zap_data.get("site", []): for alert in site.get("alerts", []): risk = alert.get("risk", "Low") for instance in alert.get("instances", []): finding = { "rule_id": alert.get("pluginid", "unknown"), "severity": risk, "message": alert.get("name", ""), "description": alert.get("desc", ""), "uri": instance.get("uri", ""), "method": instance.get("method", ""), "evidence": instance.get("evidence", ""), "tool": "zap" } findings.append(finding) summary[risk] = summary.get(risk, 0) + 1 return {"findings": findings, "summary": summary} except Exception as e: print(f"Error parsing ZAP JSON: {e}") return {"findings": [], "summary": {}} def generate_html_report(report_data): """Generate HTML security report.""" html_template = """ <!DOCTYPE html> <html> <head> <title>Security Testing Report</title> <style> body { font-family: Arial, sans-serif; margin: 40px; } .header { background: #f8f9fa; padding: 20px; border-radius: 5px; margin-bottom: 30px; } .summary { display: flex; gap: 20px; margin-bottom: 30px; } .summary-card { background: #fff; border: 1px solid #ddd; padding: 15px; border-radius: 5px; flex: 1; } .critical { border-left: 4px solid #dc3545; } .high { border-left: 4px solid #fd7e14; } .medium { border-left: 4px solid #ffc107; } .low { border-left: 4px solid #28a745; } .findings { margin-top: 30px; } .finding { background: #f8f9fa; margin: 10px 0; padding: 15px; border-radius: 5px; border-left: 4px solid #ddd; } .finding.error { border-left-color: #dc3545; } .finding.warning { border-left-color: #ffc107; } .finding.info { border-left-color: #17a2b8; } .tool-badge { background: #007bff; color: white; padding: 2px 8px; border-radius: 3px; font-size: 0.8em; } </style> </head> <body> <div class="header"> <h1>🛡️ Security Testing Report</h1> <p>Generated: {timestamp}</p> <p>MCP Gateway Security Analysis</p> </div> <div class="summary"> <div class="summary-card critical"> <h3>Critical</h3> <div style="font-size: 2em; font-weight: bold;">{critical_count}</div> </div> <div class="summary-card high"> <h3>High</h3> <div style="font-size: 2em; font-weight: bold;">{high_count}</div> </div> <div class="summary-card medium"> <h3>Medium</h3> <div style="font-size: 2em; font-weight: bold;">{medium_count}</div> </div> <div class="summary-card low"> <h3>Low</h3> <div style="font-size: 2em; font-weight: bold;">{low_count}</div> </div> </div> <h2>Security Findings</h2> <div class="findings"> {findings_html} </div> </body> </html> """ # Count findings by severity critical_count = sum(1 for f in report_data["all_findings"] if f.get("severity") in ["error", "HIGH", "High"]) high_count = sum(1 for f in report_data["all_findings"] if f.get("severity") in ["High"]) medium_count = sum(1 for f in report_data["all_findings"] if f.get("severity") in ["warning", "MEDIUM", "Medium"]) low_count = sum(1 for f in report_data["all_findings"] if f.get("severity") in ["info", "LOW", "Low"]) # Generate findings HTML findings_html = "" for finding in report_data["all_findings"][:50]: # Limit to first 50 findings severity_class = "info" if finding.get("severity") in ["error", "HIGH", "High"]: severity_class = "error" elif finding.get("severity") in ["warning", "MEDIUM", "Medium"]: severity_class = "warning" findings_html += f""" <div class="finding {severity_class}"> <div style="display: flex; justify-content: space-between; align-items: center;"> <h4>{finding.get('message', 'Unknown issue')}</h4> <span class="tool-badge">{finding.get('tool', 'unknown')}</span> </div> <p><strong>File:</strong> {finding.get('file', 'unknown')}</p> <p><strong>Rule:</strong> {finding.get('rule_id', 'unknown')}</p> <p><strong>Severity:</strong> {finding.get('severity', 'unknown')}</p> {f"<p><strong>Description:</strong> {finding.get('description', '')}</p>" if finding.get('description') else ""} </div> """ return html_template.format( timestamp=datetime.now().strftime("%Y-%m-%d %H:%M:%S"), critical_count=critical_count, high_count=high_count, medium_count=medium_count, low_count=low_count, findings_html=findings_html ) def main(): """Generate comprehensive security report.""" print("📊 Generating security testing report...") reports_dir = Path("reports/security") reports_dir.mkdir(parents=True, exist_ok=True) # Parse all security tool results semgrep_results = parse_semgrep_sarif(reports_dir / "semgrep.sarif") bandit_results = parse_bandit_json(reports_dir / "bandit.json") zap_baseline_results = parse_zap_json(reports_dir / "zap-baseline.json") zap_full_results = parse_zap_json(reports_dir / "zap-full.json") # Load dependency scan results deps_file = reports_dir / "dependency-vulnerabilities.json" if deps_file.exists(): with open(deps_file) as f: deps_results = json.load(f) else: deps_results = {"summary": {}, "findings": []} # Combine all findings all_findings = [] all_findings.extend(semgrep_results["findings"]) all_findings.extend(bandit_results["findings"]) all_findings.extend(zap_baseline_results["findings"]) all_findings.extend(zap_full_results["findings"]) all_findings.extend(deps_results["findings"]) # Create comprehensive report report_data = { "timestamp": datetime.now().isoformat(), "summary": { "semgrep": semgrep_results["summary"], "bandit": bandit_results["summary"], "zap_baseline": zap_baseline_results["summary"], "zap_full": zap_full_results["summary"], "dependencies": deps_results["summary"] }, "all_findings": all_findings, "total_findings": len(all_findings) } # Write JSON report with open(reports_dir / "security-report.json", "w") as f: json.dump(report_data, f, indent=2) # Generate HTML report html_report = generate_html_report(report_data) with open(reports_dir / "security-report.html", "w") as f: f.write(html_report) print(f"📊 Security report generated:") print(f" JSON: {reports_dir / 'security-report.json'}") print(f" HTML: {reports_dir / 'security-report.html'}") print(f" Total findings: {len(all_findings)}") # Count critical/high findings critical_high_count = sum(1 for f in all_findings if f.get("severity") in ["error", "HIGH", "High"]) if critical_high_count > 0: print(f"❌ {critical_high_count} critical/high severity findings detected!") return 1 else: print("✅ No critical or high-severity security issues found") return 0 if __name__ == "__main__": import sys sys.exit(main())
-
CI/CD integration
# .github/workflows/security-testing.yml name: Security Testing on: push: branches: [main] pull_request: branches: [main] schedule: - cron: '0 3 * * *' # Nightly at 3 AM jobs: security-baseline: name: Security Baseline (PR) runs-on: ubuntu-latest if: github.event_name == 'pull_request' steps: - name: ⬇️ Checkout source uses: actions/checkout@v4 - name: 🐍 Set up Python uses: actions/setup-python@v5 with: python-version: "3.12" cache: pip - name: 📦 Install dependencies run: | python -m pip install --upgrade pip pip install -e .[dev] make security-install - name: ⚡ Run security baseline run: make security-baseline timeout-minutes: 10 - name: 📊 Upload baseline results uses: actions/upload-artifact@v4 if: always() with: name: security-baseline-results path: reports/security/ security-sast: name: Static Security Analysis (SAST) runs-on: ubuntu-latest steps: - name: ⬇️ Checkout source uses: actions/checkout@v4 - name: 🐍 Set up Python uses: actions/setup-python@v5 with: python-version: "3.12" cache: pip - name: 📦 Install dependencies run: | python -m pip install --upgrade pip pip install -e .[dev] make security-install - name: 🔍 Run SAST analysis run: make security-sast timeout-minutes: 15 - name: 📊 Upload SAST results to GitHub Security uses: github/codeql-action/upload-sarif@v3 if: always() with: sarif_file: reports/security/semgrep.sarif category: semgrep - name: 📋 Upload SAST artifacts uses: actions/upload-artifact@v4 if: always() with: name: sast-results path: reports/security/ security-dast: name: Dynamic Security Analysis (DAST) runs-on: ubuntu-latest if: github.event_name == 'schedule' || github.event_name == 'push' steps: - name: ⬇️ Checkout source uses: actions/checkout@v4 - name: 🐍 Set up Python uses: actions/setup-python@v5 with: python-version: "3.12" cache: pip - name: 📦 Install dependencies run: | python -m pip install --upgrade pip pip install -e .[dev] - name: 🌐 Run DAST analysis run: make security-dast timeout-minutes: 30 - name: 📋 Upload DAST artifacts uses: actions/upload-artifact@v4 if: always() with: name: dast-results path: | reports/security/ zap-*.json zap-*.html security-dependencies: name: Dependency Security Scan runs-on: ubuntu-latest steps: - name: ⬇️ Checkout source uses: actions/checkout@v4 - name: 🐍 Set up Python uses: actions/setup-python@v5 with: python-version: "3.12" cache: pip - name: 📦 Install dependencies run: | python -m pip install --upgrade pip pip install -e .[dev] make security-install - name: 📦 Run dependency scan run: make security-deps timeout-minutes: 10 - name: 📋 Upload dependency results uses: actions/upload-artifact@v4 if: always() with: name: dependency-scan-results path: reports/security/ security-report: name: Generate Security Report runs-on: ubuntu-latest needs: [security-sast, security-dast, security-dependencies] if: always() && (github.event_name == 'schedule' || github.event_name == 'push') steps: - name: ⬇️ Checkout source uses: actions/checkout@v4 - name: 🐍 Set up Python uses: actions/setup-python@v5 with: python-version: "3.12" - name: 📥 Download all security artifacts uses: actions/download-artifact@v4 with: path: artifacts/ - name: 📁 Organize security results run: | mkdir -p reports/security find artifacts/ -name "*.json" -o -name "*.sarif" -o -name "*.html" | \ xargs -I {} cp {} reports/security/ - name: 📊 Generate comprehensive report run: | pip install -r requirements-security.txt python scripts/generate_security_report.py - name: 🚨 Check for critical findings run: | python -c " import json with open('reports/security/security-report.json') as f: report = json.load(f) critical_findings = [f for f in report.get('all_findings', []) if f.get('severity') in ['error', 'HIGH', 'High']] if critical_findings: print(f'🚨 Found {len(critical_findings)} critical findings!') for finding in critical_findings[:5]: # Show first 5 print(f' - {finding.get(\"message\", \"Unknown\")} in {finding.get(\"file\", \"unknown\")}') exit(1) else: print('✅ No critical security findings detected') " - name: 📋 Upload final security report uses: actions/upload-artifact@v4 if: always() with: name: comprehensive-security-report path: reports/security/security-report.*
-
Documentation and security policies
# security/policies/security-policy.md # Security Testing Policy ## Overview This document outlines the security testing requirements and procedures for MCP Gateway. ## Security Testing Requirements ### Static Analysis (SAST) - **Semgrep** scans must pass with 0 critical and 0 high-severity findings - **Bandit** security linting must pass with minimal suppressions - Custom rules must cover MCP Gateway-specific security patterns - Hard-coded secrets, SQL injection, and XSS patterns must be detected ### Dynamic Analysis (DAST) - **OWASP ZAP** baseline and full scans must complete successfully - API endpoints must resist injection attacks and authentication bypasses - Security headers must be properly configured (HSTS, CSP, X-Frame-Options) - Authentication mechanisms must be properly tested ### Dependency Security - **pip-audit** and **safety** scans must identify known vulnerabilities - Critical and high-severity CVEs must be addressed within 7 days - Dependencies must be kept current with security patches ## Security Testing Workflow ### Pull Request Testing 1. Security baseline scan runs on every PR 2. Critical findings block merge 3. New security issues compared to baseline must be addressed ### Nightly Testing 1. Comprehensive SAST, DAST, and dependency scans 2. Full security report generated and reviewed 3. Security metrics tracked over time ### Release Testing 1. Complete security test suite must pass 2. Security sign-off required for production releases 3. Penetration testing for major releases ## Vulnerability Response ### Severity Levels - **Critical**: Immediate fix required, deployment blocked - **High**: Fix within 7 days, monitor closely - **Medium**: Fix within 30 days, include in next release - **Low**: Fix when convenient, track in backlog ### Response Process 1. **Detection**: Automated scanning or manual reporting 2. **Assessment**: Verify and classify severity 3. **Remediation**: Develop and test fix 4. **Validation**: Confirm fix resolves issue 5. **Documentation**: Update tests and documentation ## Tool Configuration ### Semgrep Rules - OWASP Top 10 coverage required - MCP Gateway-specific patterns - False positive suppressions documented - Custom rules for authentication and JSONPath handling ### OWASP ZAP Configuration - Authenticated scanning with admin credentials - Custom scripts for JSON-RPC testing - Comprehensive endpoint coverage - Security header validation ### Dependency Scanning - Daily automated scans - Integration with GitHub Security Advisories - Automated PR creation for security updates
📖 References
- Semgrep Documentation – Static application security testing · https://semgrep.dev/docs/
- OWASP ZAP User Guide – Dynamic application security testing · https://www.zaproxy.org/docs/
- OWASP Top 10 – Most critical web application security risks · https://owasp.org/www-project-top-ten/
- GitHub Security Features – SARIF upload and code scanning · https://docs.github.com/en/code-security
- pip-audit Documentation – Python package vulnerability scanner · https://github.com/pypa/pip-audit
- Bandit Security Linter – Python AST security analysis · https://bandit.readthedocs.io/
🧩 Additional Notes
- Start with baseline: Begin with Semgrep SAST and ZAP baseline scans, then expand to comprehensive testing.
- CI integration: Run quick scans on PRs and comprehensive scans nightly to balance speed and coverage.
- False positive management: Maintain suppressions for legitimate patterns and test code to reduce noise.
- Security headers: Ensure proper HSTS, CSP, X-Frame-Options, and other security headers are configured.
- Authentication testing: Focus on JWT handling, basic auth, and session management security.
- Configuration security: Check for default credentials, debug modes, and insecure settings.
- Dependency monitoring: Keep dependencies current and monitor for new vulnerabilities.
- Documentation maintenance: Keep security policies and procedures updated with tool configurations.
Security Testing Best Practices for MCP Gateway:
- Focus on authentication, JSON-RPC validation, and admin endpoint security as high-value targets
- Use both SAST and DAST to catch issues at code-time and runtime
- Integrate security testing into CI/CD pipeline with appropriate gates
- Maintain comprehensive security rules for MCP Gateway-specific patterns
- Monitor and respond to security findings with defined SLAs
- Document security testing procedures and vulnerability response workflows
- Regularly update security tools and rules to catch new vulnerability patterns
Metadata
Metadata
Assignees
Labels
Linting, formatting, dependency hygiene, or project maintenance choresIssue with CI/CD process (GitHub Actions, scaffolding)DevOps activities (containers, automation, deployment, makefiles, etc)Extra attention is neededImproves securityTesting (unit, e2e, manual, automated, etc)Issues / Features awaiting triage