-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Open
Open
Copy link
Labels
Description
Is your feature request related to a problem? Please describe.
When loki queriers scale due to an increase in requests for reads, it takes 30s before the querier pods are marked as ready and start taking traffic.
readinessProbe:
failureThreshold: 3
httpGet:
path: /ready
port: http-metrics
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
This causes delays in the ability to scale out to handle the increase in searches.
Describe the solution you'd like
The readiness probe configurations for loki queriers should be configurable to allow fields such as initialDelaySeconds can be customized. Currently the configuration is using a default global configuration:
- https://github.com/grafana/loki/blob/main/production/helm/loki/values.yaml#L68-L73
- https://github.com/grafana/loki/blob/main/production/helm/loki/templates/querier/deployment-querier.yaml#L104-L111
Introduce a new loki.querier.readinessProbe value that would be merged with the default loki.readinessProbe to allow this customization
Describe alternatives you've considered
None
Additional context