[Bug] Cannot Create Dashboards in Other Namespaces than the Grafana-operator Namespace (EKS /k8s / helm)

[NerdySoftPaw]

Describe the bug
When creating dashboards for an external grafana beside the namespace where the Operator is deployed, Dashboard CRDs will be recognized but not deployed to grafana.

Version
Full semver version of the operator being used e.g. v4.10.0, v5.0.0-rc0

To Reproduce

• Install Grafana Operator in EKS Cluster (or other k8s Cluster)
• Deploy Secret and Config for external Grafana
• Deploy 1 GrafanaDashboard to namespace where the operator lives in
• Deploy 1 GrafanaDashboard to another namespace

Expected behavior
The recognized Dashboards in other namespaces should be created

Suspect component/Location where the bug might be occurring
unknown

Config + Logs

External Grafana Config

---
kind: Secret
apiVersion: v1
metadata:
  name: grafana-admin-credentials
stringData:
  GF_SECURITY_ADMIN_USER: ******
  GF_SECURITY_ADMIN_PASSWORD: ******
type: Opaque
---
apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
  name: external-grafana
  labels:
    dashboards: "dev"
spec:
  external:
    url: **** # Grafana URL
    adminPassword:
      name: grafana-admin-credentials
      key: GF_SECURITY_ADMIN_PASSWORD
    adminUser:
      name: grafana-admin-credentials
      key: GF_SECURITY_ADMIN_USER

Dashboard Config

---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
  name: test-dashboard-devops-test-app2
spec:
  resyncPeriod: 30s
  instanceSelector:
    matchLabels:
      dashboards: "dev"
  json: >
    {
      "title": "DevOps Test App Test new",
      "timezone": "browser",
      "refresh": "5s",
      "panels": [],
      "time": {
        "from": "now-6h",
        "to": "now"
      }
    }

Logs from Operator:

Displaying logs from Namespace: monitoring for Pod: grafana-operator-65898bf967-r4crs. Logs from 8.10.2024, 12:43:46
Loading...
2024-10-08T10:43:46Z INFO setup operator running in cluster scoped mode {"version": "v5.14.0"}
2024-10-08T10:43:46Z INFO setup starting manager {"version": "v5.14.0"}
2024-10-08T10:43:46Z INFO controller-runtime.metrics Starting metrics server
2024-10-08T10:43:46Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": "0.0.0.0:9090", "secure": false}
2024-10-08T10:43:46Z INFO starting server {"name": "pprof", "addr": "[::]:8888"}
2024-10-08T10:43:46Z INFO starting server {"name": "health probe", "addr": "[::]:8081"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafanaalertrulegroup", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaAlertRuleGroup", "source": "kind source: *v1beta1.GrafanaAlertRuleGroup"}
2024-10-08T10:43:46Z INFO Starting Controller {"controller": "grafanaalertrulegroup", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaAlertRuleGroup"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafana", "controllerGroup": "grafana.integreatly.org", "controllerKind": "Grafana", "source": "kind source: *v1beta1.Grafana"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafana", "controllerGroup": "grafana.integreatly.org", "controllerKind": "Grafana", "source": "kind source: *v1.Deployment"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafana", "controllerGroup": "grafana.integreatly.org", "controllerKind": "Grafana", "source": "kind source: *v1.ConfigMap"}
2024-10-08T10:43:46Z INFO Starting Controller {"controller": "grafana", "controllerGroup": "grafana.integreatly.org", "controllerKind": "Grafana"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafananotificationpolicy", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaNotificationPolicy", "source": "kind source: *v1beta1.GrafanaNotificationPolicy"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafananotificationpolicy", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaNotificationPolicy", "source": "kind source: *v1beta1.GrafanaContactPoint"}
2024-10-08T10:43:46Z INFO Starting Controller {"controller": "grafananotificationpolicy", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaNotificationPolicy"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafanadatasource", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDatasource", "source": "kind source: *v1beta1.GrafanaDatasource"}
2024-10-08T10:43:46Z INFO Starting Controller {"controller": "grafanadatasource", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDatasource"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "source": "kind source: *v1beta1.GrafanaDashboard"}
2024-10-08T10:43:46Z INFO Starting Controller {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafanafolder", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaFolder", "source": "kind source: *v1beta1.GrafanaFolder"}
2024-10-08T10:43:46Z INFO Starting Controller {"controller": "grafanafolder", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaFolder"}
2024-10-08T10:43:46Z INFO Starting EventSource {"controller": "grafanacontactpoint", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaContactPoint", "source": "kind source: *v1beta1.GrafanaContactPoint"}
2024-10-08T10:43:46Z INFO Starting Controller {"controller": "grafanacontactpoint", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaContactPoint"}
2024-10-08T10:43:47Z INFO Starting workers {"controller": "grafanaalertrulegroup", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaAlertRuleGroup", "worker count": 1}
2024-10-08T10:43:47Z INFO Starting workers {"controller": "grafanacontactpoint", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaContactPoint", "worker count": 1}
2024-10-08T10:43:47Z INFO Starting workers {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "worker count": 1}
2024-10-08T10:43:47Z INFO Starting workers {"controller": "grafananotificationpolicy", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaNotificationPolicy", "worker count": 1}
2024-10-08T10:43:47Z INFO Starting workers {"controller": "grafanafolder", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaFolder", "worker count": 1}
2024-10-08T10:43:47Z INFO Starting workers {"controller": "grafanadatasource", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDatasource", "worker count": 1}
2024-10-08T10:43:47Z INFO Starting workers {"controller": "grafana", "controllerGroup": "grafana.integreatly.org", "controllerKind": "Grafana", "worker count": 1}
2024-10-08T10:43:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "2353b618-2df4-4e42-9922-86bdeb56eafd", "count": 1}
2024-10-08T10:43:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "15b0158f-176f-4add-8a67-34cf3ebc14d9", "count": 1}
2024-10-08T10:43:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-[monitoring](https://2eaa5cb84df266b08562555b1d3fb308.lens.app:64486/api/pods?tabId=5386a50c-e3e0-44a6-80c0-94b7176c8f21&kube-details=%2Fapi%2Fv1%2Fnamespaces%2Fmonitoring&kube-selected=undefined)","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "0418ed3b-4c19-4386-b822-bf2b96fea208", "count": 1}
2024-10-08T10:43:53Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "f0df8a2c-ce3d-4e85-9348-17554e89a2bb", "count": 1}
2024-10-08T10:43:53Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "9b61e3d3-4c5d-4bf5-acda-ca57617fbcaa", "count": 1}
2024-10-08T10:43:53Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "b3fffe22-9d35-49c4-9ba7-faafcea44a79", "count": 1}
2024-10-08T10:43:56Z INFO GrafanaDatasourceReconciler datasources sync complete
2024-10-08T10:43:56Z INFO GrafanaDashboardReconciler dashboard sync complete
2024-10-08T10:43:56Z INFO GrafanaFolderReconciler folder sync complete
2024-10-08T10:44:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "85f970d4-af62-46ff-b93b-9f7821b978e6", "count": 1}
2024-10-08T10:44:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "c9a0f357-25f0-418c-9054-d3b26c87ab94", "count": 1}
2024-10-08T10:44:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "8a7db51d-d9f0-4d40-b19f-a76c4e6f9abd", "count": 1}
2024-10-08T10:44:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "ef2d4b35-0902-4fe7-943e-b9f999becd3d", "count": 1}
2024-10-08T10:44:23Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "20e6aead-bdce-4b0d-8e7c-60ba8e76cb79", "count": 1}
2024-10-08T10:44:23Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "bc86067e-3433-4ad8-92ca-9deec8ab4f46", "count": 1}
2024-10-08T10:44:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "4c95ce93-6db1-4c20-993a-cef5dac2c423", "count": 1}
2024-10-08T10:44:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "93ccd492-8da8-482c-a4ae-d2f03138094c", "count": 1}
2024-10-08T10:44:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "ce8958b6-42e7-4901-9d30-a3a9af28f3cb", "count": 1}
2024-10-08T10:44:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "80ca95e1-21ab-4505-ba1e-e5abc4c5cc17", "count": 1}
2024-10-08T10:44:53Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "bcc9f185-1ed2-410c-9992-245b42b2d1c8", "count": 1}
2024-10-08T10:44:53Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "2936fa42-522f-4267-8f07-2340bbc5ffa4", "count": 1}
2024-10-08T10:45:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "3ed11880-c0ea-4143-8243-a55677f5fa81", "count": 1}
2024-10-08T10:45:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "12041bee-ed73-4c5a-8529-7601e76e7e85", "count": 1}
2024-10-08T10:45:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "b2b18658-4999-4e82-8fe6-82b9bdd84115", "count": 1}
2024-10-08T10:45:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "4a25ba18-6127-4e5f-8a90-8cffd2889b9b", "count": 1}
2024-10-08T10:45:23Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "cdc6d9ac-50a5-4206-8410-5cf13903dcd0", "count": 1}
2024-10-08T10:45:23Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "7f8a6838-941f-4522-a6eb-29bf7a3e6426", "count": 1}
2024-10-08T10:45:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "763dcedf-b31c-4b0b-be08-bad43d80046b", "count": 1}
2024-10-08T10:45:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "0f89731e-35ab-459e-8166-1e47791d1bb2", "count": 1}
2024-10-08T10:45:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "94b81fae-1cf0-409a-b1fd-9e61c483a952", "count": 1}
2024-10-08T10:45:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "9a921cd8-d24a-4e9c-b25b-6b3183e3a278", "count": 1}
2024-10-08T10:45:53Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "52b565bb-12f7-4baf-8d32-de5652a92c63", "count": 1}
2024-10-08T10:45:53Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"sample-dashboard-monitoring","namespace":"monitoring"}, "namespace": "monitoring", "name": "sample-dashboard-monitoring", "reconcileID": "44259db8-7b1a-465f-81a1-db5445324168", "count": 1}
2024-10-08T10:46:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "11bec41e-e459-4b5c-bb25-f3ea3f5bfe7e", "count": 1}
2024-10-08T10:46:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "1505a16a-5cbc-4f03-aabf-71bc6d90902c", "count": 1}
2024-10-08T10:46:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "8556b338-cff3-421f-9995-fefe48efccd5", "count": 1}
2024-10-08T10:46:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "c4558e35-524b-42fb-96d9-bca10c0b7bec", "count": 1}
2024-10-08T10:46:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "ae712ddb-c976-4ac4-92bd-4ac964522dd1", "count": 1}
2024-10-08T10:46:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "5c575a10-2b9a-4286-9bc8-2d2d6c254081", "count": 1}
2024-10-08T10:46:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "285d7cf8-9d3b-4893-9669-92439b141678", "count": 1}
2024-10-08T10:46:47Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "c6dc592f-5ccc-4e54-9f45-ca4db00e7be5", "count": 1}
2024-10-08T10:47:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "ef30d788-29be-435e-a902-7c42a5d2d642", "count": 1}
2024-10-08T10:47:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"test-dashboard-devops-test-app2","namespace":"devops-test-app"}, "namespace": "devops-test-app", "name": "test-dashboard-devops-test-app2", "reconcileID": "687b2a86-e6d6-4307-b090-d23c3b17ddbf", "count": 1}
2024-10-08T10:47:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "e39d1327-d88f-4edb-8424-230e6c94bb68", "count": 1}
2024-10-08T10:47:17Z INFO GrafanaDashboardReconciler found matching Grafana instances for dashboard {"controller": "grafanadashboard", "controllerGroup": "grafana.integreatly.org", "controllerKind": "GrafanaDashboard", "GrafanaDashboard": {"name":"kong-dashboard","namespace":"kong"}, "namespace": "kong", "name": "kong-dashboard", "reconcileID": "ba67567d-8c87-4a04-a267-0eaaf49f0e1d", "count": 1}
Loading...
⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈⎈~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Runtime (please complete the following information):

• OS: k8s Cluster (EKS)
• Grafana Operator Version v5.14.0
• Environment: k8s (EKS)
• Deployment type: Helm
• Other: -

Additional context

[theSuess]

For cross namespace imports, you need to enable them on the Grafana resource:
https://grafana.github.io/grafana-operator/docs/overview/#cross-namespace-grafana-instances

[NerdySoftPaw]

hey @theSuess thanks for the quick answer.

I've added it to the ExternalGrafana Ressource

---
apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
  name: external-grafana
  labels:
    dashboards: "{{env}}"
spec:
  external:
    url: "https://grafana.{{env}}.euca.pacelink.net" # Grafana URL
    adminPassword:
      name: grafana-admin-credentials
      key: GF_SECURITY_ADMIN_PASSWORD
    adminUser:
      name: grafana-admin-credentials
      key: GF_SECURITY_ADMIN_USER
  allowCrossNamespaceImport: true

but nothing changed, the spec extenstion is not visible in the ressource and the other dashboards will now not regognized anymore.

Before this change they were recognized but not deployed, i looked in the helm chart and it looks like it says per default that there is no NameSpaceScoping activated, is it maybe block each other now?

[Baarsgaard]

@NerdySoftPaw allowCrossNamespaceImport: true should be added to the individual manifests that are imported, specifically GrafanaDashboards, GrafanaContactPoints, etc.
I do not think it has any functional use on Grafana instances themselves.
See: https://grafana.github.io/grafana-operator/docs/examples/crossnamespace/readme/

[NerdySoftPaw]

Thanks now it works ^^

[Rohlik]

It is a pity that this cannot be set globally 😞. Imagine that you have one Grafana and many teams contributing with their objects (folders, dashboards, etc.), and each resource needs to have this allowCrossNamespaceImport: true set to work. It's unnecessarily overhead, in my opinion 🤔.
Also, in grafana-operator logs, there are no clues that objects are not synced 🧐. Even the status of the resource doesn't contain any leads:

$ kubectl get grafanadashboards.grafana.integreatly.org  -A
NAMESPACE                  NAME          NO MATCHING INSTANCES   LAST RESYNC   AGE
cosdashboards           poc-sample2                              52s           135m

So, there are situations when teams complain that they don't see their newly added dashboards in Grafana even though there are no visible errors or info on why 🫤.

[Baarsgaard]

I believe there's an old bug report for exactly this problem here: #1164
So it's sadly a known issue.