Added Voxtral backbone and it's test

[sanskarmodi8]

In accordance with issue #2349 , I have added the Voxtral backbone and it's test as a first step towards adding the model to Keras-Hub.

Here's the colab link where I have used the implemented backbone along with the original Voxtral model from HF and compared outputs -https://colab.research.google.com/drive/1KVvFko4LwzTH-KhJzKPCGuISrtElaMNc?usp=sharing

[gemini-code-assist]

Summary of Changes

Hello @sanskarmodi8, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates the VoxTral audio processing backbone into the Keras-Hub library. This addition provides a new capability for handling audio data by introducing a specialized model architecture designed for audio encoding and adaptation. The changes lay the groundwork for future full model integration and ensure the new component is properly tested and accessible within the framework.

Highlights

• New Model Backbone Added: Introduced the VoxTralBackbone model, an audio encoder and adapter, as a foundational step towards integrating the full VoxTral model into Keras-Hub.
• Core Components for Audio Processing: Implemented custom Keras layers: ChunkAndPad for spectrogram preparation, PositionalEmbedding for learnable positional embeddings, and ReassembleChunks for output reconstruction, all integral to the VoxTralBackbone.
• Comprehensive Testing: Added a dedicated test file (voxtral_backbone_test.py) to ensure the correct functionality, output shape, and savability of the VoxTralBackbone.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces the VoxTralBackbone and its associated tests, which is a great first step. However, there are several critical issues that need to be addressed. The implementation currently uses tensorflow specific APIs, which violates the backend-agnostic principle of KerasHub. All backend-dependent code should be migrated to use keras.ops. Additionally, the custom layers are missing get_config methods, which is essential for model serialization. The docstrings for the new backbone and layers are also incomplete and should be expanded to match the project's style guide. Finally, the tests should be updated to use the standard helper functions provided by the framework for better coverage and consistency.

[sanskarmodi8]

I’ve added the backbone for the Voxtral model, but I’m currently seeing an MAE of 0.4 when comparing my implementation with the original Voxtral model from Hugging Face (as shown in the Colab notebook).

I’m a bit unsure how to improve my implementation and finalize it, since there aren’t any reliable implementations available online. Any guidance or suggestions on how to proceed would be really helpful.

[sanskarmodi8]

I'm pretty confused about the 4 failing test cases, and how to fix it. Some guidance on what the errors are about and how to fix it would really help.