feat: scenario showing a polyline in Boston #6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Uses Firebase ai to send prompts to GenAI module to let the user navigate using a set of built in commands.
This commit introduces several improvements to the AI Navigator:
- **User Messages:** The AI can now send short messages to the user during an animation sequence (e.g., "Eiffel Tower"). These messages are displayed as toasts.
- **Improved Prompting:**
- The core prompt for the AI has been refined to better guide it in generating appropriate camera altitudes, especially for the `alt` parameter of the camera's focal point.
- Specific instructions added for including delays after `flyTo` commands to allow map tiles to load and for user pacing.
- The `message` command has been added to the AI's repertoire.
- **Example Prompts:** A list of diverse example user prompts has been added. Users can now click an "I'm feeling lucky" button to populate the input field with a random example.
- **UI Enhancements:**
- The user input field now allows for multi-line input.
- An "I'm feeling lucky" button has been added next to the "Go" button.
- **Bug Fix:** Fixed an issue where escaped quotes in the AI's animation string response were not being correctly unescaped.
- **Code Refactoring:**
- `ScenarioBaseViewModel` now includes a `showMessage` function.
- `AiNavigatorViewModel` implements `showMessage` to display toasts.
- `ScenarioMapper` now correctly parses the new `message` animation step.
This commit significantly refines the AI Navigator feature by improving the user experience, providing more robust animation control, and hardening the backend prompt engineering. ### UI/UX Enhancements 🎨 - **Replaced `Toast` with `Snackbar`**: User messages are now displayed in a `SnackbarHost` for a more integrated and less intrusive experience. - **Improved Layout**: The UI is rebuilt using a `Scaffold` and `Box` layout for better structure and to ensure control panels overlay the map correctly. - **Added Animation Controls**: A new row of `IconButtons` is overlaid on the map, providing direct control to **play**, **stop**, and **restart** the camera animation. - **Visual Feedback**: A `LinearProgressIndicator` is now displayed during API calls, and buttons are disabled to provide clear feedback on the app's state. ### ViewModel and State Management ⚙️ - **Stateful Animation**: `AiNavigatorViewModel` now maintains the state of the current animation sequence and its playback status (`isAnimating`). - **Granular Control**: New public functions (`playAnimation`, `stopAnimation`, `restartAnimation`) are added to the ViewModel, allowing the UI to precisely control the animation lifecycle. ### Prompt and Parsing Fixes 🤖 - **Refined Prompt Engineering**: The system prompt sent to the Gemini model has been improved with more explicit negative constraints (e.g., no sky, weather, or night scenes) and clearer rules for camera parameters and delay logic to improve the reliability of the generated animations. - **Robust Parsing**: The `message` command parser now correctly trims quotation marks, and the service's sanitize function is updated to handle more markdown block variations from the model's response.
This commit introduces the ability for users to generate new sample prompts for the AI Navigator feature. Key changes: - Added a `getNewPrompts()` function to `NavigatorService` to fetch new prompts from Firebase Vertex AI. - Implemented a "Shuffle" button in the UI (`AiNavigatorActivity`) to trigger prompt generation. - Added a `generateNewPrompts()` function in `AiNavigatorViewModel` to handle the prompt generation logic, including updating the list of available prompts. - Refined the `promptGeneratorPrompt` in `Prompts.kt` to improve the quality and diversity of generated prompts, emphasizing unique locations and varied request types. - The "I'm feeling lucky" button now uses `viewModel.getRandomPrompt()` which includes both the initial and newly generated prompts. - Added a short delay to the map mode button to prevent rapid toggling which could cause issues.
- Moves "New Prompts" button to the bottom row of UI - Adds snackbar notifications when user requests new prompts - Replace the current prompts with the new prompts
This commit introduces the "What am I looking at?" functionality to the AI Navigator. When the user taps the corresponding button, the current camera parameters (latitude, longitude, altitude, heading, tilt, and range) are sent to a Vertex AI model. The model uses these parameters, along with a detailed prompt, to identify the likely subject of the user's view (e.g., landmark, city, geographical feature) and returns a concise, 1-2 sentence description. This description is then displayed to the user as a snackbar message. Key changes: - Added `whatAmILookingAtPrompt` to `Prompts.kt` to guide the AI in interpreting camera parameters and generating descriptions. - Implemented `whatAmILookingAt()` method in `NavigatorService.kt` to call the Vertex AI model with the camera parameters and prompt. - Added `whatAmILookingAt()` function in `AiNavigatorViewModel.kt` to handle the request, manage loading state, and send the AI's response to the UI. - Updated `AiNavigatorActivity.kt` to trigger the `whatAmILookingAt()` ViewModel function when the "What am I looking at?" button is pressed. - Added a utility function `toCameraString()` to format `CameraPosition` into the string expected by the AI prompt.
This commit updates the AiNavigator sample to: - Pass a screenshot of the 3D map view to the Gemini API. - Update prompts for multimodal input. - Remove the `firebase-vertexai` dependency as it has been replaced with `firebase-ai`.
This commit introduces a compass to the AI Navigator UI. The compass displays the current heading of the map and updates as the camera moves.
This commit introduces several enhancements to the AI Navigator feature: - **Contextual AI Prompts:** The AI now receives current camera parameters, enabling it to generate animations relative to the user's current view (e.g., "show me nearby castles"). This allows for more intuitive and contextual interactions. - **Image Scaling for AI:** Images captured for the "What am I looking at?" feature are now scaled down by 50% before being sent to the AI. This reduces payload size and processing time without significant loss of information for the AI's understanding. - **Enhanced Compass UI:** Replaced the previous compass with a new "Whiskey Compass" style display. This compass shows the precise numerical heading and cardinal direction fixed in the center, with a rotating dial of tick marks behind it, offering a more professional and informative look. - **Updated Prompts:** The AI prompt has been significantly updated to guide the AI in utilizing the new current camera parameters and to provide more nuanced instructions on altitude selection for camera focus, tile loading delays, and avoiding animations that are not renderable (e.g., sky, specific times of day). - **Unit Tests:** Added unit tests for the new cardinal direction conversion logic used in the Whiskey Compass. Overall, these changes aim to make the AI Navigator more intelligent, responsive, and visually appealing.
This commit significantly enhances the `WhiskeyCompass` composable and integrates it into the `AiNavigatorActivity`. Key changes: - **Customization:** `WhiskeyCompass` now offers extensive customization options, including text styles for cardinal and numeric headings, tick sizes (major/minor height), tick area height, and stroke widths for ticks and the lubber line. - **Styling in `AiNavigatorActivity`:** The compass in `AiNavigatorActivity` is now styled for a more compact and integrated look. It's made slightly transparent, positioned considering the status bar, and uses smaller text and denser tick marks. - **Improved Preview:** The `WhiskeyCompassPreview` has been updated to showcase various styling options, including default, custom scaled/styled, and a minimalist small version. - **Cardinal Direction Logic:** Refined the logic in `toCardinalDirection()` for more accurate centering of the 45-degree segments. - **Drawing Optimization:** The compass drawing logic now draws tick marks across a wider range (-360 to 720 degrees internally) to ensure seamless wrapping and visibility. - **Layout Adjustments:** Minor layout adjustments in `AiNavigatorActivity` to accommodate the styled compass.
This commit significantly enhances the `WhiskeyCompass` composable and integrates it into the `AiNavigatorActivity`. Key changes: - **Customization:** `WhiskeyCompass` now offers extensive customization options, including text styles for cardinal and numeric headings, tick sizes (major/minor height), tick area height, and stroke widths for ticks and the lubber line. - **Styling in `AiNavigatorActivity`:** The compass in `AiNavigatorActivity` is now styled for a more compact and integrated look. It's made slightly transparent, positioned considering the status bar, and uses smaller text and denser tick marks. - **Improved Preview:** The `WhiskeyCompassPreview` has been updated to showcase various styling options, including default, custom scaled/styled, and a minimalist small version. - **Cardinal Direction Logic:** Refined the logic in `toCardinalDirection()` for more accurate centering of the 45-degree segments. - **Drawing Optimization:** The compass drawing logic now draws tick marks across a wider range (-360 to 720 degrees internally) to ensure seamless wrapping and visibility. - **Layout Adjustments:** Minor layout adjustments in `AiNavigatorActivity` to accommodate the styled compass.
The WhiskeyCompass will now correctly account for system elements like display cutouts by applying `safeDrawingPadding`. Additionally, its opacity has been reduced to `0.55f`.
This commit introduces a fade animation for the compass in the AiNavigator feature. The compass will now fade to a lower opacity after 2 seconds of inactivity (no change in camera heading) and will return to its original opacity when the camera heading changes again. The following changes were made: - Added `Animatable` to control the compass alpha. - Implemented `LaunchedEffect` to trigger the fade animation after 2 seconds of inactivity and reset the alpha when the camera heading changes. - Updated the `Compass` composable to use the animated alpha value.
This commit refactors the coroutine scope management in `AiNavigatorActivity.kt` for better clarity and robustness. Specifically: - Uses a consistent `scope` (rememberCoroutineScope()) for UI-triggered actions. - Leverages `LaunchedEffect`'s own scope for operations tied to its lifecycle (e.g., collecting `userMessage`, animating `compassAlpha`). This ensures these operations are automatically cancelled and restarted when the `LaunchedEffect`'s keys change, preventing potential leaks or unintended behavior. - Removes the unused `Compass` composable and its related `toCardinalDirection` helper function. - Simplifies comments related to system UI and screen dimming. - Uses `collectAsStateWithLifecycle()` for `viewModel.isRequestInflight` for better lifecycle awareness.
This commit introduces the following changes:
- **New AI Navigator Commands:**
- `addMarker`: Allows the AI to place markers on the map.
- `addPolyline`: Enables the AI to draw polylines (routes, paths) on the map using Google Encoded Polyline Algorithm strings.
- **UI Enhancement:**
- Added a "Clear Map Objects" button to the AI Navigator interface, allowing users to remove all markers and polylines added by the AI.
- **Prompt Updates:**
- The AI prompt (`promptWithCamera`) has been updated to include instructions and examples for using the new `addMarker` and `addPolyline` commands.
- Expanded the list of example prompts to include scenarios utilizing markers and polylines.
- **ViewModel and Scenario Integration:**
- `AiNavigatorViewModel` now includes a `clearMapObjects` function to handle the removal of map objects.
- `ScenariosViewModel` interface and `ScenarioBaseViewModel` are updated to support adding markers and polylines.
- New `AnimationStep` types (`AddMarkerStep`, `AddPolylineStep`) and corresponding parsing logic in `ScenarioMapper.kt` have been implemented to handle these commands from the AI's animation string.
- **Utility Function:**
- Made `Float.toCardinalDirection()` in `AviationCompass.kt` public for potential external use.
This commit refactors the AI navigator's polyline functionality by:
- Changing the `addPolyline` command to accept a list of points (`lat,lng;lat,lng;...`) instead of an encoded polyline string. This simplifies prompt generation and AI output.
- Updating the `promptWithCamera` to reflect this change, including examples and constraints on the number of points.
- Modifying `ScenarioMapper.kt` to:
- Parse the new `points` format for `addPolyline`.
- Improve the robustness of `toAttributesMap()` to correctly handle quoted values containing commas or semicolons, which is necessary for the new `points` format and `message` commands.
- Enhance `toAnimation()` to correctly parse command strings, especially those with quoted values that might contain command delimiters (`;`).
- Adjust `toPolylineOptionsFromAi()` to process the list of points and correctly set altitude mode.
- Adding a new example prompt demonstrating the creation of a polyline with multiple markers.
This commit introduces a new `addPolygon` command to the AI Navigator functionality.
Key changes include:
- **New `addPolygon` command:** Allows AI to request drawing filled polygons on the map.
- Supports `id`, `outerPoints`, `fillColor`, `strokeColor`, `strokeWidth`, and `altMode` parameters.
- Limits outer points to a maximum of 100.
- Inner holes are not currently supported by this command.
- **Updated `promptWithCamera`:**
- Includes detailed instructions and examples for using the new `addPolygon` command.
- Specifies point limits for polygons.
- Updates polyline point limit to 100 (previously 20-30).
- Default polyline color changed to opaque blue.
- **`ScenarioMapper` enhancements:**
- Added `toPolygonOptionsFromAi()` to parse polygon parameters from the AI's string.
- `toAnimation()` now correctly parses `addPolygon` commands.
- The animation string parser was improved to handle quoted values within commands more robustly.
- **New `AddPolygonStep` animation class:** Handles the logic for adding a polygon to the map via the view model.
- **`ScenariosViewModel` interface updated:** Added `addPolygon` function.
- **Removed `whatAmILookingAtPromptOld`:** The older, non-image-based "what am I looking at" prompt has been removed as it's superseded by the image-based version.
- **Removed unused `android.graphics.Bitmap` import** in `NavigatorService`.
This commit refines the user prompts and the guidelines for the AI Navigator's animation generation.
Key changes:
* **Updated `promptWithCamera`:**
* Clarified instructions for using `currentCameraParams` for relative vs. absolute user requests.
* Introduced a crucial guideline: add map objects (`addMarker`, `addPolyline`, `addPolygon`) *before* camera movements (`flyTo`, `flyAround`) that focus on them, unless the prompt dictates otherwise.
* Emphasized focusing on Earth's surface and avoiding sky/weather/time-of-day specific requests.
* Revised delay logic for tile loading.
* Updated `addPolygon` description for brevity.
* Refreshed examples to reflect the new order of operations.
* **Revised `examplePrompts` list:**
* Updated the list of example user prompts to be more diverse and to better showcase the capabilities of drawing markers, polylines, and polygons in conjunction with camera movements.
* **Refined `promptGeneratorPrompt`:**
* Updated the prompt for generating example user prompts to emphasize the use of `addMarker`, `addPolyline`, and `addPolygon` commands.
* Reinforced guidance on geographic diversity and avoiding overly common landmarks.
* Updated example output to reflect new capabilities.
This commit introduces a new scenario titled "Lost in Boston" to the advanced maps sample application.
This commit introduces several enhancements to the `ScenariosActivity`:
- **Immersive Mode:** The activity now hides system UI bars (status and navigation) for a more immersive experience. The screen is also kept on.
- **Whiskey Compass:** A compass UI element (`WhiskeyCompass`) is added to the map view.
- The compass's alpha (transparency) fades in when the camera heading changes and fades out after a period of inactivity.
- **Conditional Top App Bar:** The top app bar is now only displayed when no specific scenario is active (i.e., on the scenario selection screen).
- Minor UI adjustments for the close button.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Show some possibilities around drawing a polyline in in Boston