Prometheus operator user guide notes

This blog post is mainly about translation User Guide of Prometheus Operator And take notes

---

from v0.39.0 Start ,Prometheus Operator Need to use Kubernetes v1.16.x And higher .

Concept introduction

• Operator It is a kind of software that can operate other software , It is to transform the operation experience collected by people into software .
• Prometheus Operator It aims to be as simple as possible in Kubernetes Up operation Prometheus, Keep at the same time K8s Native configuration options .

manifest give an example

precondition ： An accessible k8s colony
Related resources ： Introduce additional resources to declare Prometheus and Alertmanager The expected state of the cluster and Prometheus To configure

• Prometheus（ Declaratively describes Prometheus Expected state of deployment ）
• Alertmanager
• ServiceMonitor（ a key , It describes Prometheus Target set monitored .）
  
  Prometheus Resources include a named serviceMonitorSelector Field of , It defines the ServiceMonitor .

About difference namespace The problem of , By default , stay v0.19.0 Before the release ,ServiceMonitors Must be installed with Prometheus Instance in the same namespace . Use Prometheus Operator v0.19.0 And higher , Can pass Prometheus Resources serviceMonitorNamespaceSelector Field in Prometheus Self namespace Alternative ServiceMonitors.

---

First, deploy a simple instance application , stay 8080 Monitor and expose indicators on the port .

• Deployment Deploy stateless applications , Mainly to create 、 to update 、 Roll back

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app # app Name 
spec:
  replicas: 3 #  Copy three 
  selector: #  tag chooser 
    matchLabels: #  The selector selects the matching label 
      app: example-app #  Indicate which 
  ###
  template:
    metadata:
      labels:
        app: example-app 
   ###
    spec:
      containers:
      - name: example-app
        image: fabxc/instrumented_app #  The app Mirror image 
        ports:
        - name: web
          containerPort: 8080 #  The container port is 8080

ServiceMonitor There is a label selector （label selector See the second remark ） To select services and their underlying Endpoint object . Of the sample application Service Object with example-app It's worth it app Label selection Pod. Service The object also specifies that the port of the public index is 8080（ Note that this is not a container port , Above is the container port ）.

• service object

kind: Service #  Above is deployment Here is service 了 
apiVersion: v1
metadata: #  Follow up template The next one is the same 
  name: example-app
  labels:
    app: example-app
spec:
###
  selector:
    app: example-app #  adopt app label , If it's worth example-app Choose this pod
 ###
  ports: # Here starts with the top containers Inside ports Same content , But the port is the actual port 
  - name: web 
    port: 8080

The Service Object by ServiceMonitor Find out , It chooses in the same way . app The value of the tag must be example-app.

• ServiceMonitor

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor #  Now this is a ServiceMonitor
metadata:
  name: example-app
  labels:
    team: frontend
spec:
  selector: #  And deployment and service The selection method in is the same 
    matchLabels:
      app: example-app
  endpoints: # endpoint yes k8s A resource object in the cluster , Stored in etcd in , One for recording service All corresponding pod Access address of 
  - port: web

by Prometheus pod Enable RBAC The rules

If activated RBAC to grant authorization , You must Prometheus and Prometheus Operator establish RBAC The rules . In the example above Prometheus Operator manifest In Chinese, it means Prometheus Operator Created ClusterRole and ClusterRoleBinding. Must be right Prometheus Pod Do the same thing . as follows ：

• by Prometheus Pod establish ClusterRole and ClusterRoleBinding：
  • First create a service Account , be known as prometheus

apiVersion: v1
kind: ServiceAccount
metadata:
  name: prometheus

-  Then create clusterrole, The cluster character 

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus
rules:
- apiGroups: [""]
  resources:
  - nodes
  - nodes/metrics
  - services
  - endpoints
  - pods
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources:
  - configmaps
  verbs: ["get"]
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs: ["get", "list", "watch"]
- nonResourceURLs: ["/metrics"]
  verbs: ["get"]

• Then create binding

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus
roleRef: #  Refer to the role, It's above. clusterrole-prometheus
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: prometheus
subjects: #  The entry is service account 
- kind: ServiceAccount
  name: prometheus
  namespace: default #  Note that there prometheus Of namespace

contain ServiceMonitors

One Prometheus Object defines serviceMonitorSelector To specify which ServiceMonitor Should be included . On top service monitor Of label It specifies team: frontend , Prometheus Objects will pass through this label To select service monitor.

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
 name: prometheus
spec:
 serviceAccountName: prometheus #  above service account
 serviceMonitorSelector: #  Choose which service monitor
   matchLabels:
     team: frontend
 resources:
   requests:
     memory: 400Mi
 enableAdminAPI: false

If you have activated RBAC to grant authorization , Please use support RBAC Of Prometheus detailed list .

This makes frontend team Be able to create new ServiceMonitors And the service , This allows dynamic reconfiguration Prometheus.

contain PodMonitors

Last ,Prometheus Object defines podMonitorSelector To specify what should be included PodMonitor. label It specifies team: frontend , Prometheus Objects will pass through this label To select service monitor.

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus
spec:
  serviceAccountName: prometheus
  podMonitorSelector: #  Now it's changed to podMonitor Of Selector
    matchLabels:
      team: frontend #  The same is frontend team
  resources:
    requests:
      memory: 400Mi
  enableAdminAPI: false

expose Prometheus example 、

To visit Prometheus example , It must be exposed . The following example uses NodePort Type of service to expose instances .

apiVersion: v1
kind: Service
metadata:
  name: prometheus
spec:
  type: NodePort #  Make it accessible outside the cluster 
  ports:
  - name: web
    nodePort: 30900 #  This is the port in the node 
    port: 9090 #  Default port 
    protocol: TCP
    targetPort: web
  selector:
    prometheus: prometheus

After creating this service ,Prometheus Web UI In the port 30900 Upper node IP Under the address You can use . Web UI The target page in now shows instances of the sample application that have been successfully discovered .

expose Prometheus Admin API

Prometheus Admin API Allow access to and deletion of sequences within a specific time range 、 clear tombstones、 Capture snapshots, etc . This is disabled by default API visit , You can use this boolean flag-enableAdminAPI Switch . The following example exposes Admin API：

Warning ： Enable management API Will enable the mutation endpoint 、 Delete data 、 close Prometheus etc. . Be careful to enable this feature , It is recommended that users add additional authentication authorization through the proxy , To ensure that only clients authorized to perform these operations can do so .

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: prometheus
spec:
  serviceAccountName: prometheus
  serviceMonitorSelector: #  still service monitor Selector 
    matchLabels:
      team: frontend
  resources:
    requests:
      memory: 400Mi
  enableAdminAPI: true #