Auto-commit: 2025-07-29 21:43:14

Author: Yaniv Ben Hemo

dashboard/README.md

@@ -0,0 +1,321 @@
+# 🚀 Kafka Dashboard
+
+Interactive web dashboard for monitoring and analyzing Kafka cluster health and performance metrics.
+
+## 📊 Features
+
+- **Real-time Health Monitoring** - Live cluster health score and status indicators
+- **Interactive Charts** - Plotly-powered visualizations with hover details and zoom
+- **Health Checks Analysis** - Detailed breakdown of all health check results
+- **Topics Overview** - Partition distribution, replication factors, and topic analytics
+- **Consumer Groups Monitoring** - Active/inactive groups and member status
+- **Broker Information** - Cluster metadata and broker details
+- **Auto-refresh** - Automatically updates every 30 seconds
+- **Responsive Design** - Works on desktop, tablet, and mobile devices
+
+## 🎯 Screenshots
+
+### Main Dashboard
+
+- Health score gauge with color-coded status
+- Metrics cards showing key cluster statistics
+- Distribution charts for topics and consumer groups
+
+### Health Checks
+
+- Comprehensive table of all health check results
+- Color-coded status indicators (✅ Passed, ❌ Failed, ⚠️ Warning)
+- Recommendations for each check
+
+## 🛠️ Installation
+
+### Prerequisites
+
+- Python 3.8+
+- Kafka analysis reports (generated by the main analyzer)
+
+### Quick Start
+
+1. **Install dependencies:**
+
+   ```bash
+   cd dashboard
+   pip install -r requirements.txt
+   ```
+
+2. **Generate analysis data (from parent directory):**
+
+   ```bash
+   cd ..
+   npx superstream-kafka-analyzer --config kraft-config.json
+   ```
+
+3. **Run dashboard:**
+
+   ```bash
+   cd dashboard
+   python run_dashboard.py
+   ```
+
+4. **Open browser:**
+   ```
+   http://localhost:8050
+   ```
+
+### Advanced Installation
+
+```bash
+# Install with auto-dependency installation
+python run_dashboard.py --install
+
+# Run on different port
+python run_dashboard.py --port 8080
+
+# Use custom data directory
+python run_dashboard.py --data-dir /path/to/kafka-reports
+
+# Run in debug mode
+python run_dashboard.py --debug
+
+# Bind to all interfaces (accessible from other machines)
+python run_dashboard.py --host 0.0.0.0
+```
+
+## 📁 Project Structure
+
+```
+dashboard/
+├── app.py                  # Main dashboard application
+├── run_dashboard.py        # Launcher script with dependency checking
+├── requirements.txt        # Python dependencies
+├── README.md              # This file
+├── components/
+│   ├── charts.py          # Chart generation components
+│   └── layout.py          # UI layout components
+├── utils/
+│   └── data_loader.py     # Data loading and processing utilities
+└── assets/                # Static assets (CSS, images)
+```
+
+## 🔧 Configuration
+
+### Data Directory
+
+The dashboard looks for analysis files in these locations (in order):
+
+1. `./kafka-analysis` (current directory)
+2. `../kafka-analysis` (parent directory)
+3. `../../kafka-analysis` (grandparent directory)
+4. `~/kafka-analysis` (home directory)
+
+### File Format
+
+The dashboard expects JSON files with this naming pattern:
+
+```
+kafka-analysis-{timestamp}.json
+```
+
+## 📊 Dashboard Sections
+
+### 1. **Metrics Cards**
+
+- Total Brokers
+- Total Topics
+- Total Partitions
+- Consumer Groups
+- Active Groups
+- Average Partitions per Topic
+
+### 2. **Health Score Gauge**
+
+- Overall cluster health percentage
+- Color-coded indicator (Red < 70%, Yellow 70-90%, Green > 90%)
+- Delta from target (90%)
+
+### 3. **Health Checks Summary**
+
+- Bar chart showing passed/failed/warning counts
+- Interactive tooltips with details
+
+### 4. **Topics Distribution**
+
+- Pie chart of user vs internal topics
+- Total count in center
+
+### 5. **Partitions per Topic**
+
+- Bar chart showing partition distribution
+- Excludes internal topics for clarity
+
+### 6. **Consumer Groups Status**
+
+- Pie chart of active vs inactive groups
+- Helps identify unused consumer groups
+
+### 7. **Replication Factor Distribution**
+
+- Bar chart showing RF distribution across topics
+- Helps identify replication inconsistencies
+
+### 8. **Cluster Information Card**
+
+- Vendor information
+- Cluster ID and controller
+- Last analysis timestamp
+- Broker details with badges
+
+### 9. **Detailed Health Checks Table**
+
+- Comprehensive table of all health checks
+- Sortable and filterable columns
+- Color-coded rows by status
+- Recommendations for each check
+
+## 🔄 Data Refresh
+
+### Automatic Refresh
+
+- Dashboard auto-refreshes every 30 seconds
+- Loads the latest analysis file automatically
+- Updates timestamp in top-right corner
+
+### Manual Refresh
+
+- Click "🔄 Refresh Now" button
+- Immediately loads latest data
+- Useful for testing or immediate updates
+
+## 🎨 Customization
+
+### Themes
+
+The dashboard uses Bootstrap themes and can be customized by modifying:
+
+- `external_stylesheets` in `app.py`
+- Custom CSS in the `app.index_string`
+
+### Charts
+
+Chart appearance can be modified in `components/charts.py`:
+
+- Colors and styling
+- Chart types and layouts
+- Hover templates and annotations
+
+### Layout
+
+UI components can be customized in `components/layout.py`:
+
+- Card designs and layouts
+- Metrics and badges
+- Table styling
+
+## 🐛 Troubleshooting
+
+### Common Issues
+
+**No data available:**
+
+```bash
+# Generate analysis reports first
+cd /path/to/kafka-analyzer
+npx superstream-kafka-analyzer --config config.json
+```
+
+**Dependencies missing:**
+
+```bash
+# Install required packages
+pip install -r requirements.txt
+# Or use auto-install
+python run_dashboard.py --install
+```
+
+**Port already in use:**
+
+```bash
+# Use different port
+python run_dashboard.py --port 8080
+```
+
+**Permission denied:**
+
+```bash
+# Make launcher executable (Linux/Mac)
+chmod +x run_dashboard.py
+```
+
+### Debug Mode
+
+Run with debug flag for detailed error information:
+
+```bash
+python run_dashboard.py --debug
+```
+
+## 📈 Performance
+
+### Optimization Tips
+
+- Dashboard is optimized for files up to 100MB
+- For large clusters (1000+ topics), consider:
+  - Filtering internal topics in visualizations
+  - Pagination for large tables
+  - Caching for historical data
+
+### Memory Usage
+
+- Typical memory usage: 50-100MB
+- Scales with number of topics and historical reports
+- Auto-cleanup of old data stores
+
+## 🔒 Security
+
+### Network Access
+
+- Default binding: `127.0.0.1` (localhost only)
+- For network access: use `--host 0.0.0.0`
+- Consider reverse proxy for production use
+
+### Data Privacy
+
+- All processing happens locally
+- No external API calls (except CDN for fonts)
+- Analysis data stays on your machine
+
+## 🤝 Contributing
+
+### Development Setup
+
+```bash
+# Clone and setup
+git clone <repository>
+cd dashboard
+
+# Install dev dependencies
+pip install -r requirements.txt
+
+# Run in debug mode
+python run_dashboard.py --debug
+```
+
+### Adding Features
+
+1. Create new components in `components/`
+2. Add data processing in `utils/`
+3. Register callbacks in `app.py`
+4. Update layout as needed
+
+## 📄 License
+
+MIT License - see parent project for details.
+
+## 🆘 Support
+
+For issues and questions:
+
+- Check troubleshooting section above
+- Review console output with `--debug` flag
+- Ensure analysis files are generated correctly
+- Verify Python version (3.8+ required)