Libraries caches set to forbidden /.cache for generated non-root users causing vLLM pod failure

[biranofer]

Generated random users (non-root) sometime has no home set (as e,g, in OpenShift), which in such case default to forbidden / , and the libraries of
Triton, XDG, Flushinfer and Hugging pointing their cache there, resulting in failure of the vLLM pod.

The suggested fix is to enable pointing their caches to a safe mount (e,g, under /tmp) by values.yaml parameters and related additions in the deployment.yaml for env, volumes and volumeMounts.

[vMaroon]

Thanks @biranofer - this is settable through:

- name: HF_HOME
              value: {{ .Values.persistence.mountPath }}

The rest are missing if their defaults are not the same.

[biranofer]

Further checking - the only issue is the missing $HOME that defaulting the caches of Triton, XDG, Flushinfer (and seems that also Hugging) to /.cache (or /.triton). Setting HOME in the env to e.g. /tmp solves it.

Setting only HF_HOME (w/o setting HOME) doesn't help, you get:

ERROR 08-11 01:00:25 [core.py:632] File "/usr/local/lib/python3.12/dist-packages/triton/runtime/cache.py", line 69, in init
ERROR 08-11 01:00:25 [core.py:632] os.makedirs(self.cache_dir, exist_ok=True)
ERROR 08-11 01:00:25 [core.py:632] File "", line 215, in makedirs
ERROR 08-11 01:00:25 [core.py:632] File "", line 215, in makedirs
ERROR 08-11 01:00:25 [core.py:632] File "", line 225, in makedirs
ERROR 08-11 01:00:25 [core.py:632] PermissionError: [Errno 13] Permission denied: '/.triton'