[GSoC 2025] Load model JSON files from URLs #5137

medha-14 · 2025-07-29T19:02:45Z

Description

This PR is based on #5056 and contains changes that build upon it.
Once #5056 is merged, the diff for this PR will automatically update to show only the relevant changes.

Type of change

Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #)

Important checks:

Please confirm the following before marking the PR as ready for review:

No style issues: nox -s pre-commit
All tests pass: nox -s tests
The documentation builds: nox -s doctests
Code is commented for hard-to-understand areas
Tests added that prove fix is effective or that feature works

medha-14 · 2025-07-29T19:08:01Z

I've updated the entry_points.py so that models can now be loaded directly from a URL. This is how it is working at the moment.

import pybamm
url = "https://raw.githubusercontent.com/medha-14/model_json/refs/heads/main/dfn.json"
model =  pybamm.Model(url = url,battery_model=pybamm.lithium_ion.BaseModel())
sim = pybamm.Simulation(model)
sim.solve([0, 3600])
sim.plot(show_plot=False)

Please let me know if any part of this approach needs changing or improvisation.

Saransh-cpp · 2025-07-30T13:17:24Z

@medha-14 can you change the base branch here? Would be much easier to review

agriyakhetarpal · 2025-07-31T00:27:28Z

src/pybamm/dispatch/entry_points.py

+def get_cache_path(url):
+    cache_dir = Path.home() / ".pybamm_cache" / "pybamm" / "models"
+    cache_dir.mkdir(parents=True, exist_ok=True)
+    file_hash = hashlib.md5(url.encode()).hexdigest()
+    return cache_dir / f"{file_hash}.json"
+
+
+def clear_model_cache():
+    cache_dir = Path.home() / ".pybamm_cache" / "pybamm" / "models"
+    if cache_dir.exists():
+        for file in cache_dir.glob("*.json"):
+            file.unlink()


This should be handled using platformdirs.user_cache_dir: https://platformdirs.readthedocs.io/en/latest/api.html#cache-directory

agriyakhetarpal · 2025-08-04T19:50:26Z

@medha-14 can you change the base branch here? Would be much easier to review

Hi @medha-14, could you please create a PR within your fork from this link? medha-14/PyBaMM@GSoC...medha-14:PyBaMM:load_json_from_url. I see a better diff there. We could add comments on that PR until #5056 is merged. Otherwise, I think it is on the right track, modulo that we need to use platformdirs and we need tests.

codecov · 2025-08-19T21:30:14Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.86%. Comparing base (a1aa02c) to head (c39d1c0).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #5137      +/-   ##
===========================================
- Coverage    98.88%   98.86%   -0.03%     
===========================================
  Files          320      320              
  Lines        26949    26988      +39     
===========================================
+ Hits         26648    26681      +33     
- Misses         301      307       +6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

agriyakhetarpal · 2025-09-03T12:04:21Z

src/pybamm/dispatch/entry_points.py

+from pybamm.expression_tree.operations.serialise import Serialise
+
+APP_NAME = "pybamm"
+APP_AUTHOR = "pybamm"


I think we can keep just APP_NAME and drop APP_AUTHOR, and we can set Path(user_cache_dir(APP_NAME)) / "models" as a constant here instead.

agriyakhetarpal · 2025-09-03T12:08:22Z

src/pybamm/dispatch/entry_points.py

+def get_cache_path(url: str) -> Path:
+    cache_dir = _get_cache_dir()
+    file_hash = hashlib.md5(url.encode()).hexdigest()
+    return cache_dir / f"{file_hash}.json"


Why MD5 and not SHA-256? :)

agriyakhetarpal · 2025-09-03T12:08:41Z

src/pybamm/dispatch/entry_points.py

+        try:
+            file.unlink()
+        except Exception as e:
+            # Optional: log error instead of failing silently


Suggested change

# Optional: log error instead of failing silently

agriyakhetarpal · 2025-09-03T12:14:04Z

src/pybamm/dispatch/entry_points.py

+def clear_model_cache() -> None:
+    cache_dir = _get_cache_dir()
+    for file in cache_dir.glob("*.json"):
+        try:
+            file.unlink()
+        except Exception as e:


Note, we also store some of PyBaMM's data files in the cache dir using a pooch registry:

PyBaMM/src/pybamm/pybamm_data.py

Lines 126 to 139 in a1aa02c

def get_data(self, filename: str):

"""

Fetches the data file from upstream and stores it in the local cache directory under pybamm directory.

Parameters

----------

filename : str

Name of the data file to be fetched from the registry.

Returns

-------

pathlib.PurePath

"""

self.registry.fetch(filename)

return pathlib.Path(f"{self.path}/{self.version}/{filename}")

I wonder if we could reuse some of the code here, because clearing the cache directory of JSON files could have unintended side effects. We do have JSON files there: https://github.com/pybamm-team/pybamm-data/releases/tag/v1.0.1

Could we perhaps rewrite the PR to use pooch instead? That will also provide safer defaults than using urllib.request.urlretrieve() directly, and we could rely on it for the checksum/caching bits to see if we need to download the model JSON again or not.

agriyakhetarpal · 2025-09-03T12:14:26Z

src/pybamm/dispatch/entry_points.py

+def Model(
+    model=None,
+    url=None,
+    force_download=False,
+    *args,
+    **kwargs,
+):


I think we can have better typing here.

load json from web

f55a0a9

medha-14 requested a review from a team as a code owner July 29, 2025 19:02

agriyakhetarpal reviewed Jul 31, 2025

View reviewed changes

santacodes added the GSoC 2025 Items being done as part of GSoC 2025 label Aug 1, 2025

agriyakhetarpal mentioned this pull request Aug 4, 2025

[GSOC 2025] Saving and Loading Parameter Sets #5134

Closed

medha-14 and others added 5 commits August 5, 2025 13:50

Merge branch 'develop' into load_json_from_url

0b3d671

Merge branch 'develop' into load_json_from_url

6ee08b0

Merge remote-tracking branch 'origin/GSoC' into load_json_from_url

5112f8b

using cache_dir

ed0c714

tests

3e3658b

coverage fix

fc437a1

agriyakhetarpal marked this pull request as draft August 22, 2025 00:02

Merge branch 'develop' into load_json_from_url

00ff44f

agriyakhetarpal changed the title ~~[GSOC 2025] Load model JSON files from URLs~~ [GSoC 2025] Load model JSON files from URLs Aug 27, 2025

medha-14 and others added 2 commits August 31, 2025 15:00

coverage fix

4214681

Merge branch 'develop' into load_json_from_url

71ae0c4

medha-14 marked this pull request as ready for review August 31, 2025 09:32

medha-14 and others added 2 commits September 3, 2025 16:11

url fix

51bf407

Merge branch 'develop' into load_json_from_url

c39d1c0

agriyakhetarpal requested review from santacodes, Saransh-cpp and agriyakhetarpal September 3, 2025 11:56

agriyakhetarpal requested changes Sep 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[GSoC 2025] Load model JSON files from URLs #5137

[GSoC 2025] Load model JSON files from URLs #5137

Uh oh!

medha-14 commented Jul 29, 2025

Uh oh!

medha-14 commented Jul 29, 2025 •

edited

Loading

Uh oh!

Saransh-cpp commented Jul 30, 2025

Uh oh!

agriyakhetarpal Jul 31, 2025

Uh oh!

agriyakhetarpal commented Aug 4, 2025

Uh oh!

codecov bot commented Aug 19, 2025 •

edited

Loading

Uh oh!

agriyakhetarpal Sep 3, 2025

Uh oh!

agriyakhetarpal Sep 3, 2025

Uh oh!

agriyakhetarpal Sep 3, 2025

Uh oh!

agriyakhetarpal Sep 3, 2025

Uh oh!

agriyakhetarpal Sep 3, 2025

Uh oh!

Uh oh!

	def get_data(self, filename: str):
	"""
	Fetches the data file from upstream and stores it in the local cache directory under pybamm directory.

	Parameters
	----------
	filename : str
	Name of the data file to be fetched from the registry.
	Returns
	-------
	pathlib.PurePath
	"""
	self.registry.fetch(filename)
	return pathlib.Path(f"{self.path}/{self.version}/{filename}")

Uh oh!

[GSoC 2025] Load model JSON files from URLs #5137

Are you sure you want to change the base?

[GSoC 2025] Load model JSON files from URLs #5137

Uh oh!

Conversation

medha-14 commented Jul 29, 2025

Description

Type of change

Important checks:

Uh oh!

medha-14 commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Saransh-cpp commented Jul 30, 2025

Uh oh!

agriyakhetarpal Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

agriyakhetarpal commented Aug 4, 2025

Uh oh!

codecov bot commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

agriyakhetarpal Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

agriyakhetarpal Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

agriyakhetarpal Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

agriyakhetarpal Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

agriyakhetarpal Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

medha-14 commented Jul 29, 2025 •

edited

Loading

codecov bot commented Aug 19, 2025 •

edited

Loading