Analysis of applications in Kubernetes using dotnet monitor: Sidecar pattern

🚀 Quality resource sharing 🚀

Learning route guidance (click to unlock)	Knowledge orientation	Crowd positioning
🧡 Python practical wechat ordering applet 🧡	Advanced class	This course is a perfect combination of Python Flash + wechat applet, from project construction to Tencent cloud deployment and launch, to create a full stack food ordering system.
💛 Python quantitative trading practice 💛	beginner	Take you to create an easy to expand, safer and more efficient quantitative trading system

Dotnet monitor can be run as a sidecar in Kubernetes. Sidecar is a container that runs in the same Pod as the application. Using sidecar mode, we can diagnose and monitor the application.

As shown in the following figure, this is our ultimate goal. We can view the indicator information of the application through the visual interface.

application service

Create the dotnetmonitor.yaml file, as shown below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dotnet-monitor-example
spec:
  replicas: 3
  selector:
    matchLabels:
      app: dotnet-monitor-example
  template:
    metadata:
      annotations:
        prometheus.io/scrape: 'true'
        prometheus.io/port: "52325"
      labels:
        app: dotnet-monitor-example
    spec:
      containers:
        - name: server
          image: mcr.microsoft.com/dotnet/core/samples:aspnetapp
          ports:
            - containerPort: 80
          volumeMounts:
            - mountPath: /tmp
              name: tmp
        - name: sidecar
          image: mcr.microsoft.com/dotnet/monitor
          ports:
            - containerPort: 52323
          resources:
            requests:
              cpu: 50m
              memory: 32Mi
            limits:
              cpu: 250m
              memory: 256Mi
          args: ["--no-auth"]
          env:
            - name: DOTNETMONITOR_Urls
              value: "http://+:52323"
          volumeMounts:
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir: {}

Sidecar shares the tmp directory with the application, and maps the directory to a Volume of type emptyDir. Next, create dotnetmonitor-service.yaml to open ports for applications and sidecars.

apiVersion: v1
kind: Service
metadata:
  name: dotnetmonitor
  labels:
    app: dotnetmonitor
spec:
  type: NodePort
  ports:
    - name: sidecar
      protocol: TCP
      port: 52323
      nodePort: 31623
    - name: app
      protocol: TCP
      port: 80
      nodePort: 31624
  selector:
    app: dotnet-monitor-example

Prometheus configuration

Create the prometheus-config.yaml file, manage the configuration files of Prometheus through ConfigMaps, and write the following contents.

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
data:
  prometheus.yaml: |
    global:
      scrape_interval:     2s 
      evaluation_interval: 2s
    scrape_configs:
      - job_name: 'prometheus'
        static_configs:
        - targets: ['localhost:9090']
      - job_name: default/dotnet-monitor-example/0
        honor_timestamps: true
        scrape_interval: 10s
        scrape_timeout: 10s
        metrics_path: /metrics
        scheme: http
        follow_redirects: true
        relabel_configs:
          # Use the value of Label "__meta_kubernetes_pod_container_name"
        - source_labels: [__meta_kubernetes_pod_container_name]
          separator: ;
          # Regular expression, used to match the
          regex: sidecar
          # Value corresponding to the replaced label (target_label) specified by replacement
          replacement: $1
          # Keep is to keep the targets that conform to the regular expression and display them
          action: keep    
        - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
          action: keep
          regex: true
        - source_labels: [__meta_kubernetes_pod_name]
          action: replace
          target_label: pod
        kubernetes_sd_configs:
        - role: endpoints
          follow_redirects: true
          namespaces:
            names:
            - default

In Prometheus, if the static service discovery (static_configs) mode is adopted for registration, the change of HPA (horizontal podautoscaler, Pod level automatic scaling) will make it difficult for the service to register quickly. If the configuration file is frequently changed, the gain is not worth the loss. Therefore, the kubernetes service discovery (kubernetes_sd_configs) mode is selected here. In addition, Prometheus also supports other modes of service discovery.

static_configs: static service discovery
dns_sd_configs: DNS service discovery
file_sd_configs: File Service Discovery
kubernetes_sd_configs: Kubernetes service discovery
gce_sd_configs: GCE service discovery
ec2_sd_configs: EC2 service discovery
openstack_sd_configs: OpenStack service discovery
azure_sd_configs: Azure service discovery

Now, it means that we will keep in Kubernetes__ meta_ Kubernetes_ pod_ container_ If the name value is sidecar, it also needs to meet__ meta_ Kubernetes_ pod_ annotation_ prometheus_ io_ Pod with the scratch attribute of true.

Next, create Prometheus RBAC setup Yaml file. In order to enable Prometheus to access Kubernetes API, we need to authorize Prometheus to access Kubernetes resources through the Role-Based Access Control model in Kubernetes. First, we define the role (ClusterRole) and set the corresponding access permissions; Create an account (ServiceAccount) for Prometheus; Finally, bind the account to the role (ClusterRoleBinding).

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus
rules:
- apiGroups: [""]
  resources:
  - nodes
  - nodes/proxy
  - services
  - endpoints
  - pods
  verbs: ["get", "list", "watch"]
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs: ["get", "list", "watch"]
- nonResourceURLs: ["/metrics"]
  verbs: ["get"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: prometheus
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: prometheus
subjects:
- kind: ServiceAccount
  name: prometheus
  namespace: default

Create the prometheus-deployment.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: prometheus
  name: prometheus
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus  
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      serviceAccountName: prometheus
      containers:
      - name: prometheus
        image: prom/prometheus:latest
        command:
        - "/bin/prometheus"
        args:
        - "--config.file=/etc/prometheus/prometheus.yml"
        ports:
        - containerPort: 9090
          protocol: TCP
        volumeMounts:
        - mountPath: "/etc/prometheus"
          name: prometheus-config
      volumes:
      - name: prometheus-config
        configMap:
          name: prometheus-config

Create the prometheus-service.yaml file.

apiVersion: v1
kind: Service
metadata:
  name: prometheus
  labels:
    name: prometheus
spec:
  type: NodePort
  ports:
  - name: prometheus
    protocol: TCP
    port: 9090
    targetPort: 9090
    nodePort: 32732
  selector:
    app: prometheus

The Prometheus dashboard is shown below