Skip to content

Monitoring & Alerting

Prometheus

Collecting Prometheus Metrics

If your application exposes Prometheus Metrics, you can use a ServiceMonitor to collect those:

apiVersion: v1
kind: Service
metadata:
  name: radar
spec:
  type: ClusterIP
  ports:
  - port: 80
    name: http
    protocol: TCP
    targetPort: 3001
  - port: 9090
    name: metrics
    protocol: TCP
    targetPort: 9090
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: radar
  namespace: monitoring
  labels:
    release: kube-prometheus-stack # important
spec:
  selector:
    matchLabels:
      app: radar # needs to match the name of your service
  endpoints:
    - port: metrics # needs to match the port of your metrics port
      interval: 30s
      scheme: http

Grafana

How to add a custom dashboard?

Simply drop your dashboard json in under argocd/kube-prometheus-stack/dashboards/ and update the argocd/kube-prometheus-stack/kustomization.yaml to include your file (configMapGenerator[0].files):

Tip

The grafana_dashboard label is required!

The grafana_folder annotation tells Grafana in which directory to put the dashboard

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

configMapGenerator:
  - name: dashboard
    files:
      - ./files/dashboard.json
    options:
      disableNameSuffixHash: true
      labels:
        # https://docs.syseleven.de/metakube-accelerator/building-blocks/observability-monitoring/kube-prometheus-stack#adding-grafana-dashboards
        grafana_dashboard: "1"
      annotations:
        # has been specified in sidecar.dashboards.folderAnnotation in helm-values/kube-prometheus-stack-values.yaml
        grafana_folder: ClearRoute