[FR]: Rename hyphens to underscores

[gabrielrussoc]

What is the current behavior?

If you have a package like

my-package/lib.py

and you wrap it like

py_library(
  name = "lib",
  srcs = ["my-package/lib.py"],
)

You can't import it as my-package.lib because - is not allowed by Python. This is not necessarily a rules_py limitation, it's just how Python works.

Describe the feature

At Databricks we have been using some homegrown Python rules for years and a long time ago we have implemented logic to map - to _. This essentially resulted in a lot of Python code ending up inside directories that contain hyphens.

Now that we are trying to use aspect_rules_py, we hit issues where we can't import our code anymore (at least not without hacks) due to hyphens. Renaming is of course an option, but for a large existing codebase, it's proving to be a lot harder than I expected.

Would you be willing to support some opt-in feature to rename such packages? We can discuss the API, perhaps some target-level option like:

py_library(
  name = "lib",
  rename_hyphens = True,
)

I'm happy to contribute, the main motivation here is to understand whether this would be accepted and if yes, what the API should look like

[gabrielrussoc]

gabrielrussoc@7f55e37 is how this could be implemented (likely behind a flag)

[arrdem]

I don't think this is appropriate for aspect_rules_py. This seems like an extremely opinionated feature needed to support your repo and that doesn't have a reasonable general case unless extended to something mtree-like.

My first suggestion would be to entirely avoid the need to create these symlinks by using a custom sys.meta_path finder which searches the foo-bar/ tree before searching the foo_bar tree and dropping that in via a .pth file hook or some other startup time side-effect.

My second suggestion would be to create a custom py_library wrapper rule on your side which applies this mangling as part of generating the runfiles and DefaultInfo at the initial filegroup/library level and making py_library in your repo a macro stack which first applies that input transform and then hands those files plus the generated symlinks off to the rest of the Python machinery unmodified.

As part of the py_venv_binary machinery I'd consider support for something mtree-like which allows users to specify either a custom module mapping {module_name: import_prefix} which could be used to construct a custom loader like the one you want or potentially just a filetree middleware tool again taking cues from mtrees and rules_oci / rules_img which could implement the - mangling you need. But I think those would be fairly heavyweight additions for a very unique use case.

[arrdem]

Closing; feel free to reopen or reach out to me if you care to discuss alternatives or want to advocate for this future further.