Feature Idea

I want to integrate GDS to the system. This will allow more model inference for GPU-poors (like me).

Not all GPUs support GDS, and I am not sure the equivalent for AMD or Intel GPUs. But we can start with NVIDIA GPUs, and GDS is open source, so devs can try building for their own. And NVIDIA is activly developing this, so we should not be that much worried.

Willing to work on this one.

ref:

• https://docs.nvidia.com/gpudirect-storage/
• https://github.com/NVIDIA/gds-nvidia-fs

Existing Solutions

targeting each node would require extensive changes in all the existing workflows, but we can simply change on the abstract layer, and we are done, we are up and running with new features

Other

Willing to work on this one. Keep me in your prayers.