First, make sure it’s actually not showing up:

  1. Check prometheus-operator logs to verify if there is something abnormal happening
  2. Go to prometheus UI /service-discovery and check if your service is present there. If it is then ServiceMonitor was picked up, the issue is with incorrectly selected port in endpoints field.
  3. If your ServiceMonitor is not registered in /service-discovery then check what is the value of serviceMonitorSelector field in your Prometheus object and adjusting ServiceMonitor labels accordingly. Value of {} selects all serviceMonitors. See bellow for command.
  4. If you still don’t have ServiceMonitor registered, check a field serviceMonitorNamespaceSelector to verify if prometheus should look for serviceMonitor in that namespace. Value of {} selects all namespaces.
  5. If after waiting 15 minutes you still don’t have your ServiceMonitor registered, check RBAC.

You can easily check the values set for the serviceMonitorSelector in your Prometheus Deployment:

kubectl get prometheuses.monitoring.coreos.com --all-namespaces -o jsonpath="{.items[*].spec.serviceMonitorSelector}"

In some case, you might need to add something like:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    ...
    release: prometheus-community

Source