[CHORE]: Implement 90% Test Coverage Quality Gate and automatic badge and coverage html / markdown report publication #261
Description
🧪 Chore Summary
Implement comprehensive 90% test coverage quality gate enforcement across the entire MCP Gateway codebase: establish robust pytest coverage measurement with make coverage, make test, make coverage-report, and make coverage-gaps targets, achieving 90% total and per-file coverage requirements with automated badge generation, comprehensive coverage documentation under docs/docs/coverage/, detailed unit test reporting in docs/docs/test/unittest.md, and GitHub Actions quality gates that block PRs below the 90% threshold to ensure high code quality and thorough testing before production deployment.
🧱 Areas Affected
- Test coverage infrastructure / pytest configuration and enforcement
- Build system / Make targets (
make coverage,make test,make coverage-report,make coverage-gaps) - GitHub Actions / CI pipeline (PR-based coverage quality gates)
- Coverage reporting (HTML, XML, JSON, markdown formats)
- Documentation generation (coverage docs, badges, unit test reports)
- Quality gates (90% total coverage, 90% per-file coverage, branch coverage)
- Developer tooling (coverage gap analysis, local development support)
- Badge automation (SVG coverage badge generation and commit)
⚙️ Context / Rationale
Test coverage ensures that every critical line of code is validated and protected against regressions through comprehensive unit testing. By enforcing 90% coverage at both total and per-file levels, we create a quality gate that catches bugs early, improves code confidence, and maintains high development standards. This is critical for a gateway that processes authentication, handles arbitrary JSON-RPC requests, manages server configurations, and provides admin functionality.
Current State vs Target:
- Current: 72% total coverage (6435 statements, 1622 missing)
- Current Quality Gate: 40% (too permissive)
- Target: 90% total + 90% per-file coverage
- Target Quality Gate: Strict 90% enforcement in CI/CD
Coverage Testing Architecture:
graph TD
A[Coverage Testing Suite] --> B[pytest Execution]
A --> C[Coverage Measurement]
A --> D[Quality Gate Enforcement]
A --> E[Reporting & Documentation]
B --> B1[Unit Tests]
B --> B2[Integration Tests]
B --> B3[Async Tests]
B --> B4[Doctest Validation]
C --> C1[Line Coverage - mcpgateway/]
C --> C2[Branch Coverage]
C --> C3[Per-File Analysis]
C --> C4[Missing Lines Detection]
D --> D1[90% Total Threshold]
D --> D2[90% Per-File Threshold]
D --> D3[CI/CD Gate Enforcement]
D --> D4[PR Blocking on Failure]
E --> E1[HTML Reports - docs/coverage/]
E --> E2[Coverage Badge - SVG]
E --> E3[Markdown Documentation]
E --> E4[JSON/XML Exports]
Enhanced pytest Configuration:
# pyproject.toml - Coverage Configuration
[tool.coverage.run]
source = ["mcpgateway"]
branch = true
omit = [
"*/tests/*",
"*/test_*",
"*/__main__.py",
"*/venv/*",
"*/.venv/*",
]
[tool.coverage.report]
show_missing = true
skip_covered = false
fail_under = 90
exclude_lines = [
"pragma: no cover",
"def __repr__",
"if self.debug:",
"if settings.DEBUG",
"raise AssertionError",
"raise NotImplementedError",
"if 0:",
"if __name__ == .__main__.:",
"class .*\\bProtocol\\):",
"@(abc\\.)?abstractmethod",
]
[tool.coverage.html]
directory = "docs/docs/coverage"
show_contexts = true
[tool.coverage.xml]
output = "coverage.xml"
[tool.coverage.json]
output = "coverage.json"
show_contexts = true
[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra -q --cov=mcpgateway --cov-branch --cov-fail-under=90"
testpaths = ["tests"]
asyncio_mode = "auto"Coverage Gap Analysis Script:
# Coverage gap analysis for targeted improvements
import json, pathlib
def analyze_coverage_gaps():
"""Identify specific files and lines needing test coverage."""
with open('coverage.json') as f:
data = json.load(f)
gaps = []
for filename, file_data in data['files'].items():
coverage_pct = file_data['summary']['percent_covered']
if coverage_pct < 90.0:
missing_lines = file_data['missing_lines']
gaps.append({
'file': pathlib.Path(filename).name,
'coverage': coverage_pct,
'missing_lines': len(missing_lines),
'lines': missing_lines[:10] # First 10 missing lines
})
# Sort by priority (lowest coverage first)
gaps.sort(key=lambda x: x['coverage'])
return gapsGitHub Actions Coverage Enforcement:
# Enhanced coverage quality gate with per-file validation
- name: 🧪 Run pytest with 90% coverage requirement
run: |
pytest \
--cov=mcpgateway \
--cov-report=xml \
--cov-report=html \
--cov-report=json \
--cov-report=term \
--cov-branch \
--cov-fail-under=90
- name: 📊 Enforce per-file 90% coverage
run: |
python - <<'PY'
import json, sys
with open('coverage.json') as f:
data = json.load(f)
failed_files = []
for filename, file_data in data['files'].items():
coverage_pct = file_data['summary']['percent_covered']
if coverage_pct < 90.0:
failed_files.append(f"{filename}: {coverage_pct:.1f}%")
if failed_files:
print("❌ Files below 90% coverage:")
for file_info in failed_files:
print(f" • {file_info}")
sys.exit(1)
else:
print("✅ All files have ≥90% coverage")
PY📦 Related Make Targets
| Target | Purpose |
|---|---|
make coverage |
Run comprehensive coverage analysis with 90% enforcement |
make test |
Run unit tests with coverage quality gate |
make coverage-report |
Generate quick coverage summary and analysis |
make coverage-gaps |
Identify files and lines needing test coverage |
make htmlcov |
Generate HTML coverage report for detailed analysis |
make coverage-badge |
Generate/update SVG coverage badge |
make coverage-docs |
Build comprehensive coverage documentation |
make coverage-clean |
Clean all coverage artifacts and reports |
make coverage-validate |
Validate coverage configuration and thresholds |
Bold targets are mandatory; CI must fail if coverage drops below 90% total or any file drops below 90%.
📋 Acceptance Criteria
-
make coveragecompletes successfully with 90% total coverage and 90% per-file coverage. - GitHub Actions quality gate blocks PRs that drop below 90% coverage threshold.
- Coverage badge automatically generated and committed to
docs/docs/images/coverage.svg. - HTML coverage report available at
docs/docs/coverage/index.htmlwith interactive analysis. - Coverage documentation provides comprehensive coverage metrics and gap analysis.
- Unit test documentation generated at
docs/docs/test/unittest.mdwith pytest results. - Per-file coverage validation ensures no individual file drops below 90%.
- Branch coverage enabled and measured alongside line coverage.
- Coverage gap analysis identifies specific files and lines needing attention.
- Developer tooling provides actionable feedback for improving coverage locally.
- False positive exclusions properly configured for protocol classes and error handling.
- CI integration runs coverage checks on every PR with clear pass/fail feedback.
🛠️ Task List (suggested flow)
-
Update GitHub Actions workflow with 90% enforcement
Replace existing
.github/workflows/pytest.yml:# =============================================================== # 🧪 PyTest & Coverage - Quality Gate (90% minimum) # =============================================================== name: Tests & Coverage on: push: branches: ["main"] pull_request: branches: ["main"] schedule: - cron: '42 3 * * 1' # Monday 03:42 UTC permissions: contents: write checks: write actions: read jobs: test: name: pytest (py${{ matrix.python }}) runs-on: ubuntu-latest strategy: fail-fast: false matrix: python: ["3.11", "3.12"] env: PYTHONUNBUFFERED: "1" PIP_DISABLE_PIP_VERSION_CHECK: "1" steps: - name: ⬇️ Checkout code uses: actions/checkout@v4 with: fetch-depth: 1 - name: 🐍 Setup Python ${{ matrix.python }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python }} cache: pip - name: 📦 Install dependencies (editable + dev extra) run: | python -m pip install --upgrade pip pip install -e .[dev] pip install pytest pytest-cov pytest-asyncio coverage[toml] coverage-badge - name: 🧪 Run pytest with 90% coverage requirement run: | pytest \ --cov=mcpgateway \ --cov-report=xml \ --cov-report=html \ --cov-report=term \ --cov-report=json \ --cov-branch \ --cov-fail-under=90 - name: 📊 Enforce per-file 90% coverage run: | python - <<'PY' import json, sys with open('coverage.json') as f: data = json.load(f) failed_files = [] for filename, file_data in data['files'].items(): coverage_pct = file_data['summary']['percent_covered'] if coverage_pct < 90.0: failed_files.append(f"{filename}: {coverage_pct:.1f}%") if failed_files: print("❌ Files below 90% coverage:") for file_info in failed_files: print(f" • {file_info}") sys.exit(1) else: print("✅ All files have ≥90% coverage") PY - name: 📤 Upload coverage reports uses: actions/upload-artifact@v4 with: name: coverage-reports-py${{ matrix.python }} path: | coverage.xml htmlcov/ coverage.json retention-days: 30 - name: 📊 Create coverage badge if: matrix.python == '3.11' && github.ref == 'refs/heads/main' run: | mkdir -p docs/docs/images coverage-badge -f -o docs/docs/images/coverage.svg - name: 🚀 Commit coverage badge if: matrix.python == '3.11' && github.ref == 'refs/heads/main' uses: stefanzweifel/git-auto-commit-action@v5 with: commit_message: "docs(coverage): update coverage badge [skip ci]" file_pattern: "docs/docs/images/coverage.svg" - name: 📝 Generate coverage documentation if: matrix.python == '3.11' run: | mkdir -p docs/docs/coverage docs/docs/test cat > docs/docs/coverage/index.md << 'EOF' # Test Coverage Report This page provides comprehensive test coverage metrics for the MCP Gateway project. ## Coverage Requirements - **Total Coverage**: ≥90% - **Per-File Coverage**: ≥90% - **Branch Coverage**: Enabled ## Current Coverage Status  ## Reports - [Detailed Coverage Report](./detailed.md) - [HTML Coverage Report](../../coverage/index.html) (available in CI artifacts) - [Unit Test Results](../test/unittest.md) ## Coverage Trends Coverage is automatically measured on every pull request and must meet the 90% threshold for both total and per-file coverage. EOF echo "# Detailed Coverage Report" > docs/docs/coverage/detailed.md echo "" >> docs/docs/coverage/detailed.md echo "Generated on: $(date -u '+%Y-%m-%d %H:%M:%S UTC')" >> docs/docs/coverage/detailed.md echo "" >> docs/docs/coverage/detailed.md coverage report --format=markdown >> docs/docs/coverage/detailed.md - name: 📋 Coverage summary to job output if: always() run: | echo "### 📊 Coverage Report - Python ${{ matrix.python }}" >> "$GITHUB_STEP_SUMMARY" echo "" >> "$GITHUB_STEP_SUMMARY" TOTAL_COV=$(coverage report --format=total) echo "**Total Coverage: ${TOTAL_COV}%**" >> "$GITHUB_STEP_SUMMARY" echo "" >> "$GITHUB_STEP_SUMMARY" echo "| File | Statements | Missing | Branches | Partial | Coverage |" >> "$GITHUB_STEP_SUMMARY" echo "|------|----------:|--------:|---------:|--------:|---------:|" >> "$GITHUB_STEP_SUMMARY" coverage json -q -o cov_summary.json python - <<'PY' >> "$GITHUB_STEP_SUMMARY" import json, pathlib data = json.load(open("cov_summary.json")) root = pathlib.Path().resolve() for filename, file_data in data["files"].items(): try: rel_path = pathlib.Path(filename).resolve().relative_to(root) except ValueError: rel_path = filename s = file_data["summary"] coverage_pct = s["percent_covered"] status_emoji = "✅" if coverage_pct >= 90 else "⚠️" if coverage_pct >= 75 else "❌" print(f"| {status_emoji} {rel_path} | {s['num_statements']} | {s['missing_lines']} | " f"{s['num_branches']} | {s['missing_branches']} | {coverage_pct:.1f}% |") PY
-
Update pyproject.toml with comprehensive coverage configuration
[tool.coverage.run] source = ["mcpgateway"] branch = true omit = [ "*/tests/*", "*/test_*", "*/__main__.py", "*/venv/*", "*/.venv/*", ] [tool.coverage.report] show_missing = true skip_covered = false fail_under = 90 exclude_lines = [ "pragma: no cover", "def __repr__", "if self.debug:", "if settings.DEBUG", "raise AssertionError", "raise NotImplementedError", "if 0:", "if __name__ == .__main__.:", "class .*\\bProtocol\\):", "@(abc\\.)?abstractmethod", ] [tool.coverage.html] directory = "docs/docs/coverage" show_contexts = true [tool.coverage.xml] output = "coverage.xml" [tool.coverage.json] output = "coverage.json" show_contexts = true [tool.pytest.ini_options] minversion = "6.0" addopts = "-ra -q --cov=mcpgateway --cov-branch --cov-fail-under=90" testpaths = ["tests"] asyncio_mode = "auto" filterwarnings = [ "ignore:Passing 'msg' argument to .*.cancel\\(\\) is deprecated:DeprecationWarning", ]
-
Enhanced Makefile coverage targets with 90% enforcement
Replace existing coverage targets in Makefile:
# ============================================================================= # 🧪 TESTING (Enhanced with 90% Coverage Enforcement) # ============================================================================= test: @echo "🧪 Running tests with coverage enforcement..." @test -d "$(VENV_DIR)" || $(MAKE) venv @/bin/bash -c "source $(VENV_DIR)/bin/activate && \ python3 -m pip install -q pytest pytest-asyncio pytest-cov coverage[toml] coverage-badge && \ python3 -m pytest --maxfail=0 --disable-warnings -v \ --cov=mcpgateway \ --cov-report=term \ --cov-fail-under=90" coverage: @echo "🧪 Running comprehensive coverage analysis..." @test -d "$(VENV_DIR)" || $(MAKE) venv @mkdir -p $(TEST_DOCS_DIR) $(COVERAGE_DIR) $(DOCS_DIR)/docs/coverage $(DOCS_DIR)/docs/images @# Run tests with full coverage reporting @/bin/bash -c "source $(VENV_DIR)/bin/activate && \ python3 -m pip install -q pytest pytest-cov pytest-asyncio coverage[toml] coverage-badge pytest-md-report && \ python3 -m pytest --reruns=1 --reruns-delay=30 \ --md-report --md-report-output=$(DOCS_DIR)/docs/test/unittest.md \ --cov=mcpgateway \ --cov-report=xml \ --cov-report=html:$(COVERAGE_DIR) \ --cov-report=json \ --cov-report=term \ --cov-branch \ --cov-fail-under=90 \ -v" @# Generate coverage badge @/bin/bash -c "source $(VENV_DIR)/bin/activate && \ coverage-badge -f -o $(DOCS_DIR)/docs/images/coverage.svg" @# Check per-file coverage (90% minimum) @/bin/bash -c "source $(VENV_DIR)/bin/activate && python - <<'PY' import json, sys try: with open('coverage.json') as f: data = json.load(f) failed_files = [] for filename, file_data in data['files'].items(): coverage_pct = file_data['summary']['percent_covered'] if coverage_pct < 90.0: failed_files.append(f'{filename}: {coverage_pct:.1f}%') if failed_files: print('❌ Files below 90% coverage:') for file_info in failed_files: print(f' • {file_info}') print('\n💡 Run \'make coverage-gaps\' to see detailed analysis') sys.exit(1) else: print('✅ All files have ≥90% coverage') except FileNotFoundError: print('⚠️ coverage.json not found - run coverage first') sys.exit(1) PY" @# Generate coverage documentation @printf '# Test Coverage Report\n\n' > $(DOCS_DIR)/docs/coverage/index.md @printf 'Generated on: %s\n\n' "$$(date -u '+%Y-%m-%d %H:%M:%S UTC')" >> $(DOCS_DIR)/docs/coverage/index.md @printf '\n\n' >> $(DOCS_DIR)/docs/coverage/index.md @printf '## Coverage Requirements\n\n- **Total Coverage**: ≥90%%\n- **Per-File Coverage**: ≥90%%\n- **Branch Coverage**: Enabled\n\n' >> $(DOCS_DIR)/docs/coverage/index.md @printf '## Detailed Report\n\n' >> $(DOCS_DIR)/docs/coverage/index.md @/bin/bash -c "source $(VENV_DIR)/bin/activate && coverage report --format=markdown >> $(DOCS_DIR)/docs/coverage/index.md" @printf '\n## Unit Tests Report\n\n' >> $(DOCS_DIR)/docs/test/unittest.md @/bin/bash -c "source $(VENV_DIR)/bin/activate && \ coverage report --format=markdown -m --no-skip-covered >> $(DOCS_DIR)/docs/test/unittest.md" @echo "✅ Coverage artifacts generated:" @echo " • HTML report: $(COVERAGE_DIR)/index.html" @echo " • Coverage docs: $(DOCS_DIR)/docs/coverage/" @echo " • Badge: $(DOCS_DIR)/docs/images/coverage.svg" @echo " • Unit tests: $(DOCS_DIR)/docs/test/unittest.md" coverage-report: @echo "📊 Generating coverage analysis report..." @test -d "$(VENV_DIR)" || $(MAKE) venv @/bin/bash -c "source $(VENV_DIR)/bin/activate && \ coverage report --show-missing --format=markdown && \ echo && \ coverage report --show-missing" coverage-gaps: @echo "🔍 Identifying coverage gaps..." @test -d "$(VENV_DIR)" || $(MAKE) venv @/bin/bash -c "source $(VENV_DIR)/bin/activate && python - <<'PY' import json, pathlib try: with open('coverage.json') as f: data = json.load(f) print('📊 Coverage Analysis by File\n') print('Files needing attention (sorted by coverage %):') file_coverage = [] for filename, file_data in data['files'].items(): rel_path = pathlib.Path(filename).name summary = file_data['summary'] coverage_pct = summary['percent_covered'] missing_lines = summary['missing_lines'] file_coverage.append((coverage_pct, rel_path, missing_lines)) # Sort by coverage percentage (lowest first) file_coverage.sort() for coverage_pct, filename, missing_lines in file_coverage: status = '✅' if coverage_pct >= 90 else '⚠️' if coverage_pct >= 75 else '❌' print(f'{status} {filename}: {coverage_pct:.1f}% ({missing_lines} missing lines)') total_pct = data['totals']['percent_covered'] print(f'\n📈 Total Coverage: {total_pct:.1f}%') except FileNotFoundError: print('❌ coverage.json not found. Run \"make coverage\" first.') PY" coverage-badge: @echo "📊 Generating coverage badge..." @test -d "$(VENV_DIR)" || $(MAKE) venv @mkdir -p $(DOCS_DIR)/docs/images @/bin/bash -c "source $(VENV_DIR)/bin/activate && \ coverage-badge -f -o $(DOCS_DIR)/docs/images/coverage.svg" @echo "✅ Coverage badge updated → $(DOCS_DIR)/docs/images/coverage.svg" htmlcov: @echo "📊 Generating HTML coverage report..." @test -d "$(VENV_DIR)" || $(MAKE) venv @mkdir -p $(COVERAGE_DIR) @if [ ! -f .coverage ]; then \ echo "ℹ️ No .coverage file found - running full coverage first..."; \ $(MAKE) --no-print-directory coverage; \ fi @/bin/bash -c "source $(VENV_DIR)/bin/activate && coverage html -i -d $(COVERAGE_DIR)" @echo "✅ HTML coverage report ready → $(COVERAGE_DIR)/index.html" coverage-clean: @echo "🧹 Cleaning coverage artifacts..." @rm -rf $(COVERAGE_DIR) .coverage coverage.xml coverage.json @rm -f $(DOCS_DIR)/docs/images/coverage.svg @rm -f $(DOCS_DIR)/docs/coverage/*.md $(DOCS_DIR)/docs/test/unittest.md @echo "✅ Coverage artifacts cleaned" coverage-validate: @echo "🔍 Validating coverage configuration..." @test -d "$(VENV_DIR)" || $(MAKE) venv @/bin/bash -c "source $(VENV_DIR)/bin/activate && python - <<'PY' import coverage import pytest import toml # Validate pyproject.toml coverage configuration with open('pyproject.toml') as f: config = toml.load(f) cov_config = config.get('tool', {}).get('coverage', {}) if cov_config.get('report', {}).get('fail_under', 0) >= 90: print('✅ Coverage threshold configured correctly (≥90%)') else: print('❌ Coverage threshold too low - should be ≥90%') if cov_config.get('run', {}).get('branch', False): print('✅ Branch coverage enabled') else: print('❌ Branch coverage not enabled') pytest_config = config.get('tool', {}).get('pytest', {}).get('ini_options', {}) addopts = pytest_config.get('addopts', '') if '--cov-fail-under=90' in addopts: print('✅ pytest coverage threshold configured correctly') else: print('❌ pytest coverage threshold not configured') PY"
-
Create comprehensive coverage documentation structure
# docs/docs/coverage/index.md # Test Coverage Documentation This section contains comprehensive test coverage reports and analysis for the MCP Gateway project. ## Coverage Requirements - **Total Coverage**: ≥90% - **Per-File Coverage**: ≥90% - **Branch Coverage**: Enabled - **Quality Gate**: Enforced in CI/CD ## Current Status  ## Reports - [Detailed Coverage Report](./detailed.md) - Line-by-line coverage analysis - [HTML Coverage Report](./htmlcov/index.html) - Interactive coverage report - [Unit Test Results](../test/unittest.md) - Pytest execution summary ## Make Targets - `make coverage` - Full coverage analysis with 90% enforcement - `make coverage-report` - Quick coverage summary - `make coverage-gaps` - Identify files needing attention - `make coverage-badge` - Generate SVG coverage badge - `make htmlcov` - Generate HTML coverage report ## CI/CD Integration Coverage is automatically measured on every pull request and must meet the 90% threshold for: - Total project coverage - Individual file coverage - Branch coverage where applicable Pull requests failing coverage requirements will be blocked from merging. ## Coverage Analysis Files are analyzed for: - **Line Coverage**: Percentage of executable lines tested - **Branch Coverage**: Percentage of conditional branches tested - **Missing Lines**: Specific lines not covered by tests - **Exclusions**: Lines excluded from coverage requirements ## Improving Coverage To improve coverage in specific files: 1. Run `make coverage-gaps` to identify problematic files 2. Use `make htmlcov` to see detailed line-by-line analysis 3. Add tests for uncovered lines and branches 4. Verify improvements with `make coverage` ## Exclusions The following patterns are excluded from coverage requirements: - Abstract methods and protocols - Debug-only code paths - Error handling for impossible conditions - Development-only utilities
-
Coverage gap analysis script for targeted improvements
# scripts/coverage_analysis.py #!/usr/bin/env python3 """Advanced coverage gap analysis and recommendations.""" import json import ast import pathlib from typing import Dict, List, Tuple def analyze_missing_coverage(coverage_file: str = "coverage.json") -> Dict: """Analyze coverage gaps and provide improvement recommendations.""" try: with open(coverage_file) as f: data = json.load(f) except FileNotFoundError: return {"error": "Coverage file not found. Run 'make coverage' first."} analysis = { "total_coverage": data["totals"]["percent_covered"], "files_analysis": [], "priority_files": [], "recommendations": [] } for filename, file_data in data["files"].items(): file_path = pathlib.Path(filename) summary = file_data["summary"] coverage_pct = summary["percent_covered"] file_analysis = { "file": file_path.name, "full_path": str(file_path), "coverage": coverage_pct, "missing_lines": summary["missing_lines"], "total_statements": summary["num_statements"], "missing_count": len(file_data.get("missing_lines", [])), "priority": "high" if coverage_pct < 75 else "medium" if coverage_pct < 90 else "low" } analysis["files_analysis"].append(file_analysis) if coverage_pct < 90: analysis["priority_files"].append(file_analysis) # Sort priority files by coverage (lowest first) analysis["priority_files"].sort(key=lambda x: x["coverage"]) # Generate recommendations total_missing = sum(f["missing_count"] for f in analysis["priority_files"]) if total_missing > 0: analysis["recommendations"].extend([ f"Focus on {len(analysis['priority_files'])} files below 90% coverage", f"Add tests for {total_missing} missing lines total", "Run 'make htmlcov' for detailed line-by-line analysis", "Use 'make coverage-gaps' for file-by-file breakdown" ]) return analysis def generate_coverage_plan(analysis: Dict) -> str: """Generate a structured plan for improving coverage.""" if analysis.get("error"): return f"Error: {analysis['error']}" plan = f""" # Coverage Improvement Plan ## Current Status - **Total Coverage**: {analysis['total_coverage']:.1f}% - **Files Below 90%**: {len(analysis['priority_files'])} - **Target**: 90% total and per-file coverage ## Priority Files (Lowest Coverage First) """ for i, file_info in enumerate(analysis["priority_files"][:10], 1): plan += f""" ### {i}. {file_info['file']} ({file_info['coverage']:.1f}% coverage) - **Missing Lines**: {file_info['missing_count']} out of {file_info['total_statements']} - **Priority**: {file_info['priority'].upper()} - **Path**: `{file_info['full_path']}` """ plan += f""" ## Recommended Actions """ for rec in analysis["recommendations"]: plan += f"- {rec}\n" plan += f""" ## Quick Commands ```bash # Analyze specific files make htmlcov # Check current status make coverage-gaps # Run tests with coverage make coverage
"""
return plandef main():
"""Main coverage analysis function."""
print("🔍 Analyzing coverage gaps...")analysis = analyze_missing_coverage() if analysis.get("error"): print(f"❌ {analysis['error']}") return 1 # Generate improvement plan plan = generate_coverage_plan(analysis) # Write plan to file plan_file = pathlib.Path("reports/coverage-improvement-plan.md") plan_file.parent.mkdir(exist_ok=True) plan_file.write_text(plan) # Print summary total_coverage = analysis["total_coverage"] files_below_90 = len(analysis["priority_files"]) print(f"📊 Coverage Analysis Summary:") print(f" Total Coverage: {total_coverage:.1f}%") print(f" Files Below 90%: {files_below_90}") if total_coverage >= 90 and files_below_90 == 0: print("✅ Coverage targets met!") return 0 else: print(f"📋 Improvement plan written to: {plan_file}") print(f"🎯 Focus on {files_below_90} files to reach 90% target") return 1if name == "main":
import sys
sys.exit(main()) -
README.md coverage badge integration
Add to README.md badges section:
<!-- Coverage Badge --> 
-
Pre-commit hook for coverage validation
# .pre-commit-config.yaml - Add coverage check repos: - repo: local hooks: - id: coverage-check name: Coverage Check entry: make args: [coverage-validate] language: system pass_filenames: false stages: [pre-push]
-
Documentation navigation updates
Add to documentation navigation:
# docs/docs/nav.md or mkdocs.yml - Coverage: - Overview: coverage/index.md - Detailed Report: coverage/detailed.md - Unit Tests: test/unittest.md
📖 References
- pytest Coverage Documentation – Comprehensive testing coverage measurement · https://pytest-cov.readthedocs.io/
- Coverage.py Documentation – Python code coverage measurement · https://coverage.readthedocs.io/
- GitHub Actions Python – CI/CD for Python projects · https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
- pytest Documentation – Testing framework best practices · https://docs.pytest.org/
- coverage-badge – SVG badge generation · https://github.com/dbrgn/coverage-badge
🧩 Additional Notes
- Start with analysis: Use
make coverage-gapsto identify specific files needing attention before writing new tests. - HTML reports: Interactive HTML coverage reports provide line-by-line analysis for targeted test improvements.
- Branch coverage: Enable branch coverage to catch untested conditional logic and error handling paths.
- Quality gate timing: Run full coverage analysis on main branch pushes, quick validation on PRs.
- Badge automation: Coverage badge automatically updates on main branch with latest coverage percentage.
- Developer workflow: Local
make coverageprovides immediate feedback during development. - CI optimization: Upload coverage artifacts for detailed analysis without blocking PR velocity.
- Documentation integration: Coverage reports integrate seamlessly with existing docs structure.
Coverage Testing Best Practices for MCP Gateway:
- Focus on critical paths: authentication, JSON-RPC handling, and server management as high-value targets
- Use both line and branch coverage to catch logical edge cases
- Integrate coverage checking into development workflow with make targets
- Maintain comprehensive coverage documentation for team visibility
- Monitor coverage trends over time with automated badge updates
- Balance coverage requirements with development velocity through appropriate CI gates