Animalkingdom pr #833
Open
Animalkingdom pr #833
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nition on wildlife videos - Implements animal, action, and activity recognition subtasks for the AnimalKingdom dataset - Includes dataset builder, YAML configs, and strict Jaccard metric evaluation - Utilities for prompt/answer extraction, result processing, and HuggingFace video download - See animalkingdom/README.md for details and usage instructions
kcz358
reviewed
Sep 26, 2025
Comment on lines
+70
to
+84
| try: | ||
| from huggingface_hub import hf_hub_download | ||
|
|
||
| # Download the video file from the HuggingFace dataset repository | ||
| local_path = hf_hub_download(repo_id="luciehmct/animalkingdom-test", filename=clip_path, repo_type="dataset") | ||
|
|
||
| if os.path.exists(local_path): | ||
| return [local_path] | ||
| else: | ||
| eval_logger.error(f"Downloaded file does not exist: {local_path}") | ||
| return [clip_path] | ||
|
|
||
| except Exception as e: | ||
| eval_logger.error(f"Failed to download video {clip_path}: {str(e)}") | ||
| return [clip_path] |
There was a problem hiding this comment.
This part is a bit hacky. Should not really get clip path like this
kcz358
reviewed
Sep 26, 2025
|
I think the major problem is the path and the configuration though. Maybe for most of the PR, you can again check what is the recommended way to pass the video path through doc_to_visual. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
_cot.jsonlannotations into Hugging Face–style datasets via the same dataset_builder.py used for MammAlps (--dataset animalkingdom).doc_to_visual,doc_to_text,doc_to_target,process_results) and strict Jaccard scoring/aggregation pipeline introduced for MammAlps.num_frame=32,use_temporal_context=True).Details
luciehmct/animalkingdom-test, covering diverse YouTube wildlife clips. While MammAlps comes from alpine camera-trap footage, both datasets are standardized by the same builder, ensuring consistent JSON records and folder layouts.animalkingdom_animal.yaml,animalkingdom_action.yaml,animalkingdom_activity.yaml) are direct analogs of the MammAlps configs.results/<model>_<timestamp>/animalkingdom_<subtask>.jsonlwith prompt, response, parsed predictions, ground truth, and per-example Jaccard score.lmms_eval/api/metrics.py, identical to MammAlps.use_temporal_context=Trueflag enriches prompts with frame timestamps, exactly as documented for MammAlps.Testing