Feature Request: Duckdb extensions as pip extras

[JorgeGarciaIrazabal]

Why do you want this feature?
Problem:
In some scenarios, it is not allowed to download binaries at run time in production.

Proposed solution:
In python environments, I think it would make sense to include the extensions as extras where you can install them like python -m pip install duckdb[httpfs] for example.

Was this solution considered and discarded? if so, can you explain why?
If there are not big constrains, we will be happy to help in the implementation of this feature.

Note: This suggestion only applies to python, but I guess similar approaches could be implemented for other languages.

[parikannappan]

Yes this feature would be very useful

currently I am facing the following error.
IOException: IO Error: Extension "C:\Users\002K75744.duckdb\extensions\v0.8.1\windows_amd64\httpfs.duckdb_extension" not found.
Extension "httpfs" is an existing extension.

Install it first using "INSTALL httpfs".

[bodacious-bill]

I also would love to see this feature. I work in an environment that is walled off from the internet. Only recently were we allowed to install anything on pypi through a proxy (before that, there was an internal mirror repo that was maintained for shorter list of packages). Anything outside of that is completely unreachable.

To say the least, it's very unexpected for a python package installed from pypi to require strange automatic installs from external repositories as well. It feels strange, from a security perspective, to normalize the automatic downloading and installing of packages at runtime.

[yinzhs]

+1
after trying to get approval to download from extensions.duckdb.org, or trying to build from source for some platforms e.g. manylinux, or finding the prebuilt/release artifacts in github and S3, was lucked to find some build artifacts for some Linux (e.g. linux_amd64_gcc4) for v1.0.0 1f98600 from the InvokeCI action https://github.com/duckdb/duckdb/actions/runs/9286093161. Find the package you need for your platform at the bottom of the action run page, and you need to log in to github first.
However, the extension of some platforms such as Windows and MacOS are not exposed there. Can they be exposed as well? Can all the artifacts be made available in the duckdb release page? Understand there's a lot taking spaces if not more than they already have.

[ryanhamilton]

I would also need this to enable bundling extensions with QStudio: https://www.timestored.com/qstudio/

[santosh-d3vpl3x]

I have just created a repo to download the extensions from duckdb repo and upload them to pypi. I have also added utility function to load the extension in duckdb.

pip install duckdb-extensions duckdb-extension-spatial

and then in python

from duckdb_extensions import extension_importer
extension_importer.import_extension("spatial")

I really wish we don't have to maintain this for long and publishing to pypi becomes a core feature.

[athensofamerica]

Had the same struggling with FW. This is really helpful! Thank you! It will be nice if duckdb lab make this part of the python SDK core.

[chitralverma]

@santosh-d3vpl3x thanks a lot for this, is it possible to support community extensions with this?