Version 0.2.0 #467
Open
Version 0.2.0 #467
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Reduce healthcheck interval from 30s to 10s for faster feedback - Add 15s start period to prevent failures during startup - Reduce timeout from 3s to 2s (health endpoint responds in ~20ms) - Add explicit retries=3 for clarity This improves Docker Compose startup time and provides more responsive health status feedback during development and deployment.
…tion - Implement hybrid OAuth approach (Option C) with temporary session exchange - Add OAuth session storage with 5-minute expiry for security - Create `/oauth/exchange` API endpoint for frontend token retrieval - Update OAuth callback to redirect to frontend with session ID - Separate API and frontend OAuth callback paths to prevent conflicts - Modify frontend to handle session exchange flow instead of direct tokens - Add comprehensive OAuth state management and validation - Ensure no sensitive tokens appear in URLs or browser history - Support configurable frontend callback URLs via redirect_uri parameter This approach provides secure OAuth authentication while maintaining flexibility for different deployment configurations and prevents common security issues like token exposure in URLs.
…Gravatar support This comprehensive refactoring transforms the OAuth implementation from GitLab-specific to fully OIDC-compliant and provider-agnostic, while adding modern UI enhancements. **OIDC Standards Compliance:** - Replace GitLab-specific `gitlab_url` with generic `oidc_issuer_url` configuration - Update `GitLabUser` to `OidcUser` with OIDC standard fields (`sub`, `preferred_username`) - Use OIDC `/oauth/userinfo` endpoint instead of provider-specific endpoints - Support optional OIDC claims with intelligent fallbacks **Provider Interchangeability:** - Support GitLab, Auth0, Keycloak, Google, and other OIDC providers - Provider-agnostic configuration and documentation - Remove all GitLab-specific references from codebase **Enhanced Authentication UX:** - Add reactive user store for immediate UI updates after OAuth login - Implement session exchange flow to avoid exposing tokens in URLs - Fix user info display appearing only after page reload - Add proper error handling and debugging for OAuth flows **Modern Avatar System:** - Add Gravatar support with MD5 hashing for user avatars - Implement smart fallback system: Gravatar → initials → generic avatar - Create reusable UserAvatar component with multiple sizes and shapes - Enhanced user dropdown with professional styling and logout icon **Updated Documentation:** - Comprehensive OIDC authentication guide with provider examples - Updated CLI documentation with new command structure - Provider setup instructions for GitLab, Auth0, Keycloak, and Google - Migration guide from oauth2-proxy to native OIDC implementation **Frontend Improvements:** - Reactive authentication state management with Svelte stores - DaisyUI integration for consistent avatar styling - Enhanced user interface with Gravatar images and rich user dropdowns - Improved error handling and user feedback **Backend Enhancements:** - Better OIDC user field handling with meaningful fallbacks - Improved session management and token validation - Enhanced OAuth error handling and debugging capabilities The system now works seamlessly with any OIDC-compliant provider while providing a modern, professional user experience with Gravatar support and reactive UI updates.
Complete end-to-end OAuth device flow implementation enabling CLI authentication with OIDC providers like GitLab. This adds native device flow support alongside the existing web-based OAuth flow. Key improvements: - Implement full device flow token polling and exchange in Scotty server - Add proper OIDC provider integration with device authorization grant - Fix server info endpoint to use OIDC-compliant field names - Resolve server URL mismatch between localhost and 127.0.0.1 in token storage - Update scottyctl to handle device flow authentication properly - Add comprehensive error handling for OAuth flow states Technical changes: - Server: Add exchange_device_code_for_token() method for GitLab token polling - Server: Store device flow session interval for proper polling cadence - Server: Update info handler to return oidc_issuer_url instead of gitlab_url - scottyctl: Fix token storage to use user-provided server URL - scottyctl: Remove placeholder user info and use actual token response data - scottyctl: Update OAuth structures to be fully OIDC-compliant The device flow now supports the complete OAuth 2.0 Device Authorization Grant flow (RFC 8628) with proper error handling for authorization_pending, access_denied, and expired_token scenarios. Tested with GitLab OIDC provider - full authentication and API access working.
- Remove unused variables in frontend components - Remove imageLoaded variable from user-avatar component - Remove oauthRedirectUrl variable from login page - Apply Prettier formatting across all frontend files - Fix arrow function formatting in stores and components - Improve code readability with consistent indentation These changes resolve ESLint warnings and ensure consistent code style across the frontend codebase.
# Conflicts: # Cargo.lock
Update all documentation to use correct CLI command syntax: - Change apps subcommand to app:subcommand format - Change blueprints list to blueprint:list - Change notifications add/remove to notify:add/notify:remove This aligns documentation with actual CLI implementation.
Implement complete test suite for bearer token and OAuth authentication flows: **Bearer Authentication Tests (14 tests)** - Valid/invalid/missing token authentication scenarios - Malformed headers and public endpoint access validation - Configuration-dependent behavior testing - Cross-authentication mode validation **OAuth Authentication Tests (8 tests)** - Complete OAuth device flow: authorization → token → protected endpoint access - OAuth web flow: authorization URL generation, callback handling, session exchange - Mock OAuth provider integration with exact API format matching - OAuth provider error handling and authorization pending states **Key Technical Achievements** - Tests actual Scotty application router with complete middleware stack - Uses AppState access for OAuth session store manipulation to test complete flows - Mock OAuth provider with wiremock exactly matches implementation request formats - Validates end-to-end authentication: auth flow → token → protected API access - All 22 tests validate real implementation behavior, not isolated components The AppState approach solves OAuth web flow testing complexity by directly populating session stores, enabling complete flow validation without complex callback coordination between HTTP requests. Dependencies: Add axum-test, tokio-test, wiremock for testing infrastructure.
Removed assert!(true) statements that would be optimized out by the compiler, as flagged by clippy's assertions_on_constants lint. These assertions provided no actual test value and were replaced with comments explaining the test flow.
- Add preflight version check using semver to ensure major/minor compatibility - Create shared ServerInfo and OAuthConfig types in scotty_core for consistency - Add --bypass-version-check flag for emergency situations - Update AuthMode enum to support serialization with proper defaults - Version check runs before commands requiring server connection - Show clear error messages when versions are incompatible - Skip version check for auth commands (login/logout) and completion This prevents backwards compatibility issues by ensuring scottyctl and scotty server versions are compatible before executing operations.
- Replace direct println! calls with app_context.ui() methods - Use proper success/failed status methods for better terminal integration - Reduce excessive emoji usage for cleaner, more professional output - Maintain colored text for important information (URLs, usernames, servers) - Ensure consistent UI behavior with status line management
This commit centralizes shared functionality in scotty-core and significantly improves OAuth error handling with type-safe enums. ## New shared modules in scotty-core: - Add HTTP client with retry logic and exponential backoff - Add unified OAuth types with type-safe error enums - Add version management utilities with compatibility checking - Move retry logic from scottyctl to shared location ## OAuth improvements: - Replace string literals with OAuthErrorCode enum for type safety - Add built-in error descriptions to OAuth error codes - Update all OAuth handlers to use type-safe error responses - Maintain backward compatibility with legacy error formats ## HTTP client consolidation: - Create shared HttpClient with builder pattern and timeout support - Replace scattered reqwest::Client usage with shared implementation - Add proper error handling and retry policies across the workspace - Update OAuth flows to use shared HTTP client ## Version management: - Add comprehensive version comparison and compatibility utilities - Update preflight checker to use shared version management - Add user-friendly version formatting and update recommendations - Include extensive test coverage for version handling ## Code cleanup: - Remove unused dependencies (utoipa-axum) - Fix clippy warnings and improve code consistency - Update imports to use shared types across workspace - Maintain full backward compatibility All tests pass, ensuring no regressions were introduced.
This commit consolidates the OAuth error handling across the Scotty project by: - Unifying OAuthError and OAuthErrorCode into a single comprehensive error type in scotty-core - Implementing smart IntoResponse for AppError that returns OAuth-compliant ErrorResponse format for OAuth errors - Adding proper HTTP status code mappings for all OAuth error types (400, 401, 403, 404, 429) - Fixing device flow "Server error" issue by improving scottyctl error handling to process all OAuth status codes - Adding SlowDown variant for handling OAuth2 "slow_down" error during device flow polling - Maintaining OAuth2 RFC 6749 compliance while simplifying the error architecture - Updating all components (scotty, scottyctl, scotty-core) to use the unified system with proper error conversions The device flow now properly handles polling rate limiting and provides specific error messages instead of generic "Server error" responses.
Add role-based access control (RBAC) system using Casbin for granular permission management across apps and groups. Key features: - Group-based app organization (development, staging, production, default) - Role-based permissions (viewer, operator, developer, admin) - Per-app permission checks (view, manage, shell, logs, create, destroy) - Universal default group access for all users via wildcard assignment - Authorization middleware for API endpoints - Groups list endpoint (/api/v1/authenticated/groups/list) - Seamless fallback when authorization config unavailable This enables secure multi-tenant app management while maintaining backward compatibility.
- Fix token bounds checking in basic_auth.rs to prevent panics with short tokens - Implement proper get_user_permissions method in authorization service - Update authorization middleware to extract app names from API v1 paths - Fix middleware ordering by using State extractor instead of request extensions - Update router to use from_fn_with_state for require_permission middleware These changes resolve authentication failures and "App state not found" errors in the RBAC authorization system.
- Remove references to legacy api.access_token fallback - Add migration instructions for existing bearer token installations - Update PRD to reflect completed Phase 4 enforcement - Document middleware architecture improvements - Add warnings about breaking changes for bearer token authentication The authorization system now requires explicit RBAC assignments for all bearer tokens, removing the legacy fallback behavior.
Updates scottyctl to check server auth mode before using stored OAuth tokens. When server is in bearer mode, prioritizes --access-token parameter and SCOTTY_ACCESS_TOKEN environment variable over stored OAuth credentials. Resolves authentication failures where scottyctl would attempt to use invalid OAuth tokens against servers configured for bearer authentication.
- Remove fallback authorization service - RBAC is now required - Update tests to use actual RBAC configuration instead of fallback - Add test bearer token to policy configuration - Remove obsolete test for no-token configuration - Improve log format with timestamp, level, target, and message - Clean up telemetry configuration and reduce verbose span output
…ystem Oauth2 support
# Conflicts: # Cargo.lock
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
- Enable crossterm event-stream feature for keyboard event handling - Implement raw terminal mode for character-by-character input - Add bidirectional I/O between stdin and WebSocket - Handle keyboard events: arrow keys, backspace, enter, etc. - Implement Ctrl+C (send SIGINT to container) and Ctrl+D (exit) - Support both interactive mode and command mode (-c flag) - Add proper terminal cleanup on exit (disable raw mode) - Handle terminal resize events (detection, sending to server) - Use tokio::select! for concurrent WebSocket and stdin handling The shell is now fully interactive with proper TTY support. Related: scotty-2
- Create shell.rs WebSocket handler module - Handle ShellSessionData messages with Input type - Forward client input to shell sessions via ShellService - Add proper error handling for invalid sessions - Register shell handler in WebSocket message dispatcher This fixes the "Unhandled WebSocket message type: ShellSessionData" warnings and enables bidirectional shell communication. Related: scotty-2
CRITICAL BUG FIX: Shell sessions were not found because each handler created its own ShellService instance with separate session storage. Changes: - Add shell_service field to SharedAppState - Initialize ShellService once in AppState::new() - Update REST API handlers to use shared shell_service - Update WebSocket handler to use shared shell_service - Add Debug derive to ShellService for AppState compatibility Before: Each handler had its own HashMap of sessions After: All handlers share the same session storage This fixes "Session not found" errors when sending input to shells. Related: scotty-2
- Create AGENTS.md with detailed architecture and development guide - Project overview and workspace structure - Development commands for server, scottyctl, and frontend - Architecture details (API, Docker orchestration, authorization) - Configuration options for server and CLI - Testing, observability, and release process - Create CLAUDE.md that references AGENTS.md This documentation provides future developers with the big-picture architecture understanding needed to be productive quickly.
The server now automatically loads .env and .env.local files on startup to support local development configuration. Changes: - Add dotenvy dependency to scotty package - Load .env files in main.rs before configuration initialization - Add .env.local to .gitignore to prevent accidental commits - Implement comprehensive tests for .env loading behavior File precedence (highest to lowest): 1. Environment variables (always take precedence) 2. .env.local (local overrides) 3. .env (shared defaults) Closes scotty-4443
Add comprehensive documentation for the new .env file loading feature: - Add dedicated section in configuration.md explaining .env usage - Document configuration precedence (.env vs .env.local vs environment) - Include examples and best practices - Update AGENTS.md development guide with .env example Related to scotty-4443
Fixed four critical bugs that prevented interactive shell from working:
1. Console output not displaying (shell.rs)
- Problem: TTY mode sends output as LogOutput::Console, but code
was skipping Console variant with continue statement
- Fix: Added Console output handling to process and display TTY output
- Impact: Shell output now displays correctly in interactive mode
2. Commands not executing (shell.rs)
- Problem: Docker exec received "/bin/bash -c 'cmd'" as single arg,
tried to execute it as a file path
- Fix: Changed to vec!["sh", "-c", cmd] to properly invoke shell
- Impact: Commands now execute correctly via sh -c
3. UI spinner interfering with shell (shell.rs, ui.rs)
- Problem: Status line animation threads continued running in raw
terminal mode, causing visual interference
- Fix: Added ui.clear() that calls stop_animation() to properly
stop both animation and render threads before raw mode
- Impact: Clean interactive shell without UI elements overlaying output
4. Shell not exiting after "exit" command (shell.rs)
- Problem: Docker exec output stream doesn't close cleanly in TTY
mode when shell exits, leaving client waiting indefinitely
- Fix: Added periodic exec status polling (500ms) using
docker.inspect_exec() to detect process completion
- Impact: Shell sessions now exit cleanly within 500ms of typing "exit"
All fixes target TTY-specific behavior. Interactive shell now works
correctly with proper output display, command execution, clean UI,
and exit handling.
Related: scotty-2
Implemented terminal dimension handling to fix full-screen TUI applications: 1. Send initial terminal size on session creation - Get terminal dimensions using crossterm's terminal_size() - Send ResizeShellTty message before entering interactive mode - Docker TTY is resized to match client terminal 2. Handle terminal resize events - Implemented Event::Resize handler (was TODO) - Forwards resize events to server via ResizeShellTty message - Server calls docker.resize_exec() to update container TTY Impact: - vim, htop, top now work correctly with proper display - Programs can query terminal size (COLUMNS, LINES env vars work) - Terminal resize during session updates container TTY Note: Basic vi (busybox vi) still crashes - likely expects different terminal capabilities. vim (full implementation) works fine. Related: scotty-2
Problem: Client was wrapping commands with '/bin/bash -c "cmd"' and server was wrapping again with 'sh -c', causing failures in containers that don't have bash installed (like many minimal Alpine images). Solution: Let server handle all command wrapping. Client now: - Sends raw command for command mode (server wraps with 'sh -c') - Sends custom shell path or None for interactive mode This ensures compatibility with all container types since 'sh' is universally available. Tested on: - Alpine 3.18 (logdemo container) - works - Alpine 3.20 with OpenResty (nginx container) - works - Complex commands with pipes and multiple statements - works
Removed REST shell endpoint in favor of WebSocket-based shell sessions for better real-time bidirectional communication. Changes: - Removed REST endpoint handlers/apps/shell.rs - Implemented WebSocket shell session handlers - Added shell session types: CreateShellSession, ShellSessionCreated, ShellSessionData, ShellSessionEnded, ShellSessionError, ResizeShellTty - Updated Casbin policy to include shell permissions - Removed OpenAPI documentation for REST shell endpoint WebSocket provides better support for: - Real-time terminal I/O without polling - Terminal resize events - Clean session lifecycle management - Lower latency for interactive shells The WebSocket handler integrates with the existing ShellService singleton for session management, metrics, and Docker exec handling.
Add extensive unit test coverage for the shell CLI functionality: - scottyctl: Add 8 tests for key conversion logic (printable chars, arrow keys, special keys, control modifiers, ANSI sequences) - scotty-types: Add 7 tests for WebSocket message serialization and shell session data types - scotty: Add 7 tests for shell service (session expiration, error variants, UUID uniqueness, session info validation, command channel communication) Create shared test utilities module (scotty/src/api/test_utils.rs) to eliminate code duplication: - create_test_websocket_messenger(): Shared WebSocket messenger creation - create_test_app_state_with_config(): AppState from config file - create_test_app_state_with_settings(): AppState from Settings object Refactor existing test files to use shared utilities, reducing ~150 lines of duplicated test setup code across bearer_auth_tests, oauth_flow_tests, login_test, rate_limiting/tests, and apps/list tests. All 22 new unit tests passing. Tests focus on business logic without requiring Docker or external dependencies.
Modified run_command_mode() to capture and propagate exit codes from shell sessions. The CLI now exits with the actual exit code from the executed command instead of always returning 0. - Capture exit_code from ShellSessionEnded message - Set exit code to 1 for error cases (ShellSessionError, WebSocket errors) - Use std::process::exit() to propagate the code to the parent process Closes scotty-93c2
Implemented Drop-based SessionGuard to ensure shell sessions are removed from the active_sessions map even if the spawned task panics. - Created SessionGuard struct with Drop trait implementation - Uses tokio::task::block_in_place for async cleanup in Drop - Instantiated guard at start of spawned task for RAII pattern - Removed manual cleanup code (now handled automatically) - All existing shell tests pass This defensive programming practice prevents session leaks on panic. Closes scotty-25ce
Added comprehensive end-to-end WebSocket integration tests for shell sessions. Tests are marked with #[ignore] to prevent execution in CI environments that lack Docker or test containers. Test coverage: - Shell session creation and termination flow - Command execution with exit code verification - Bidirectional I/O (input/output streaming) - TTY resize message handling Changes: - Created scotty/src/lib.rs to expose public API for integration tests - Added tokio-tungstenite as dev-dependency - Made test_utils public for integration test access - Created 4 comprehensive WebSocket test cases - All tests properly skip in CI (marked with #[ignore]) Tests can be run explicitly with: cargo test --test test_shell_websocket_e2e -- --ignored --nocapture Closes scotty-0ca3
Implemented comprehensive audit logging for shell sessions to support compliance and security monitoring requirements. Audit logging features: - Session creation logged with: session_id, client_id, app_name, service_name, container_id, shell_command - Session termination logged with: duration, reason, exit_code - Structured logging using tracing with named fields - Logs at all termination points: timeout, normal exit, user termination, and error cases The structured logs can be exported to OTLP/JSON for log aggregation systems (Elasticsearch, Splunk, etc.) to support: - Compliance audits (who accessed which containers, when, for how long) - Security monitoring (detect suspicious shell access patterns) - Usage analytics (track shell session duration and frequency) Example audit log fields: event: "shell_session_created" | "shell_session_ended" session_id: UUID client_id: UUID (WebSocket client identifier) app_name: string service_name: string container_id: string shell_command: string duration_seconds: f64 reason: string exit_code: Option<i32> All existing tests pass. Closes scotty-ccf1
Added payload preview to ShellSessionData Display implementation to improve log readability. Now shows first 50 characters of data payload with byte count instead of no preview. Before: "Shell session xxx data (Input)" After: "Shell session xxx data (Input): echo hello... (15 bytes)" This makes debug logs more useful while preventing log spam from large payloads. The Display trait is only used for logging - actual WebSocket transmission uses serde serialization and is unaffected.
Updated Cargo.lock with new dev-dependency (tokio-tungstenite). Updated beads tracking with closed issues: - scotty-93c2: Exit code propagation - scotty-25ce: SessionGuard for panic safety - scotty-0ca3: E2E WebSocket tests - scotty-ccf1: Audit logging for shell sessions
- Applied cargo fmt formatting to E2E test file - Fixed clippy::collapsible_match warning by combining nested if let patterns
- Added Default implementations for NotifyLog, StopFlag, and TaskOutputStreamingService - Added #[allow(private_interfaces)] for functions with pub(crate) return types - Added #[allow(clippy::should_implement_trait)] for Permission::from_str These are pre-existing clippy warnings that prevent the pre-push hook from passing.
Added #[allow(dead_code)] to test_utils functions that are used by integration tests. The binary build doesn't see integration test usage, causing false positive warnings in the pre-push hook.
Fixed doctests that were broken by the addition of lib.rs which exposed them to compilation. All examples now have proper imports and are marked as no_run since they're illustrative examples, not actual tests. - create_app: Added imports and async wrapper - inspect_app: Added imports and Result return - wait_for_containers_ready: Added SharedAppState import
Added comprehensive documentation for two previously undocumented CLI commands: - app:logs: View and follow logs from app services * Options: --follow, --lines, --since, --until, --timestamps * Examples showing common usage patterns - app:shell: Open interactive shell or execute commands in containers * Options: --command, --shell, --workdir * Examples including exit code propagation feature * Note about required shell permission Both commands are placed logically after app:info since they are used for inspecting and interacting with running applications.
The --workdir option was defined in the CLI interface but was never implemented in the WebSocket protocol or shell service. Removed the option and updated documentation to show the correct pattern of using 'cd' within shell commands to change directories.
feat(scotty-2): implement interactive shell CLI command
…cated metric group dashboards Replace single large dashboard (3874 lines) with 11 focused dashboards organized by metric category for better maintainability and navigation. Each dashboard covers a specific subsystem: HTTP server, log streaming, shell sessions, WebSocket, tasks, memory, Tokio runtime, applications, OAuth, and rate limiting. Overview dashboard provides high-level metrics and quick links to all specialized dashboards.
Add explicit min: 0 configuration to memory dashboard panels to prevent Y-axis from showing duplicate values when memory usage is constant. This fixes the visualization issue where all Y-axis tick marks displayed the same value (e.g., all showing 446 GB).
…hreshold Reduce data retention from 30d to 7d and lower minimum free disk space requirement to 1GB to prevent read-only mode errors when disk space is limited. This fixes the "storage is in read-only mode" error that occurs when available disk space falls below the threshold. Changes: - Set retentionPeriod to 7d (was 30d) - Add storage.minFreeDiskSpaceBytes=1GB flag
Fix three panels showing "No data" due to incorrect metric names: - WebSocket: scotty_websocket_connections_active → scotty_websocket_connections - Memory RSS: process_resident_memory_bytes → scotty_memory_rss_bytes - Tokio Workers: tokio_worker_threads → scotty_tokio_workers_count The metric names now match the actual metrics exported by the Scotty application.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.