CORE-1219 - upload latest ancillary version #423
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Includes if the entry is the latest one or not
TylerZeroMaster
changed the title
* Added `get_latest_code_version(api_root)` – fetches `/api/version/` and returns the JSON payload. * Added `get_is_latest_code_version(api_root, code_version)` – compares the `tag` from the version endpoint to the supplied `code_version`. * Updated `get_abl` to call the new helper and to set `metadata["is_latest"]` only when the code version is the latest **and** the commit SHA matches the expected latest commit. * Refactored tests: * Introduced `MockJsonResponse` and `MockApi` to stub both `/api/abl` and `/api/version` endpoints. * Adjusted expected `is_latest` values to match the new logic. * Updated mock responses to include version information. This change ensures that the pipeline only marks a book as the latest when the build’s code version is actually the most recent, preventing stale data from being uploaded.
I must have forgotten to remove it somehow
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://openstax.atlassian.net/browse/CORE-1219
Summary via Gemini:
This
diffshows several changes to a Python script and a few shell scripts related to a "bakery" system, which seems to process and build books. The main purpose of these changes is to introduce a new feature that identifies and processes only the latest version of a book based on its publication date.Key Changes
The most significant change is the modification of the
get_ablfunction incheck_feed.py. Previously, it would sort all unique book entries by their commit date and return a list of{"repo": ..., "version": ...}pairs. The updated version now:itertools.groupby: This new library is used to group the entries.repository_nameanduuidto uniquely identify each book edition. It then sorts these groups bycommitted_atand determines which version is the most recent.metadatadictionary: The function now includes a newmetadatakey in the output, which contains the boolean flagis_latest. This flag indicates whether the current version of the book being processed is the most recent one.Supporting Changes
test_bakery_scripts.pyhas been updated to reflect the newmetadataandis_latestkey in the expected output, ensuring the new logic is correctly tested.dockerfileshave been updated to handle the newmetadataandis_latestflag.docker-entrypoint.sh: This script now reads theis_latestflag from a newmetadatafile. It also copies this metadata file into the working directory (edit: only when running on local).git-dequeue-book.bash: This script now extracts and stores themetadatafrom the book's data feed into a new file.step-upload-book.bash: This script now uses theARG_IS_LATESTflag to conditionally upload ancillary files (like images or other supplementary materials), but only if the book being processed is the latest version. This prevents unnecessary uploads of older bookeditionsversions.Summary
In short, these changes introduce a new mechanism to track the latest version of a book within the system. Instead of simply processing all unique book entries, the system now identifies the most recent version of each book edition and uses this information to control subsequent actions, such as whether to upload ancillary files. This likely improves efficiency by avoiding redundant uploads for older book versions. 📚
Human note
This is required to avoid uploading two different versions of the same ancillary. We only care about the latest one. It is more efficient since we avoid excessive uploading, but the main goal was to only use the latest version.