Analysis of applications in Kubernetes using dotnet monitor: Sidecar pattern

🚀 Quality resource sharing 🚀

Learning route guidance (click to unlock)Knowledge orientationCrowd positioning
🧡 Python practical wechat ordering applet 🧡Advanced classThis course is a perfect combination of Python Flash + wechat applet, from project construction to Tencent cloud deployment and launch, to create a full stack food ordering system.
💛 Python quantitative trading practice 💛beginner Take you to create an easy to expand, safer and more efficient quantitative trading system

Dotnet monitor can be run as a sidecar in Kubernetes. Sidecar is a container that runs in the same Pod as the application. Using sidecar mode, we can diagnose and monitor the application.

As shown in the following figure, this is our ultimate goal. We can view the indicator information of the application through the visual interface.

application service

Create the dotnetmonitor.yaml file, as shown below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dotnet-monitor-example
spec:
  replicas: 3
  selector:
    matchLabels:
      app: dotnet-monitor-example
  template:
    metadata:
      annotations:
        prometheus.io/scrape: 'true'
        prometheus.io/port: "52325"
      labels:
        app: dotnet-monitor-example
    spec:
      containers:
        - name: server
          image: mcr.microsoft.com/dotnet/core/samples:aspnetapp
          ports:
            - containerPort: 80
          volumeMounts:
            - mountPath: /tmp
              name: tmp
        - name: sidecar
          image: mcr.microsoft.com/dotnet/monitor
          ports:
            - containerPort: 52323
          resources:
            requests:
              cpu: 50m
              memory: 32Mi
            limits:
              cpu: 250m
              memory: 256Mi
          args: ["--no-auth"]
          env:
            - name: DOTNETMONITOR_Urls
              value: "http://+:52323"
          volumeMounts:
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: tmp
          emptyDir: {}

Sidecar shares the tmp directory with the application, and maps the directory to a Volume of type emptyDir. Next, create dotnetmonitor-service.yaml to open ports for applications and sidecars.

apiVersion: v1
kind: Service
metadata:
  name: dotnetmonitor
  labels:
    app: dotnetmonitor
spec:
  type: NodePort
  ports:
    - name: sidecar
      protocol: TCP
      port: 52323
      nodePort: 31623
    - name: app
      protocol: TCP
      port: 80
      nodePort: 31624
  selector:
    app: dotnet-monitor-example

Prometheus configuration

Create the prometheus-config.yaml file, manage the configuration files of Prometheus through ConfigMaps, and write the following contents.

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
data:
  prometheus.yaml: |
    global:
      scrape_interval:     2s 
      evaluation_interval: 2s
    scrape_configs:
      - job_name: 'prometheus'
        static_configs:
        - targets: ['localhost:9090']
      - job_name: default/dotnet-monitor-example/0
        honor_timestamps: true
        scrape_interval: 10s
        scrape_timeout: 10s
        metrics_path: /metrics
        scheme: http
        follow_redirects: true
        relabel_configs:
          # Use the value of Label "__meta_kubernetes_pod_container_name"
        - source_labels: [__meta_kubernetes_pod_container_name]
          separator: ;
          # Regular expression, used to match the
          regex: sidecar
          # Value corresponding to the replaced label (target_label) specified by replacement
          replacement: $1
          # Keep is to keep the targets that conform to the regular expression and display them
          action: keep    
        - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
          action: keep
          regex: true
        - source_labels: [__meta_kubernetes_pod_name]
          action: replace
          target_label: pod
        kubernetes_sd_configs:
        - role: endpoints
          follow_redirects: true
          namespaces:
            names:
            - default

In Prometheus, if the static service discovery (static_configs) mode is adopted for registration, the change of HPA (horizontal podautoscaler, Pod level automatic scaling) will make it difficult for the service to register quickly. If the configuration file is frequently changed, the gain is not worth the loss. Therefore, the kubernetes service discovery (kubernetes_sd_configs) mode is selected here. In addition, Prometheus also supports other modes of service discovery.

  • static_configs: static service discovery
  • dns_sd_configs: DNS service discovery
  • file_sd_configs: File Service Discovery
  • kubernetes_sd_configs: Kubernetes service discovery
  • gce_sd_configs: GCE service discovery
  • ec2_sd_configs: EC2 service discovery
  • openstack_sd_configs: OpenStack service discovery
  • azure_sd_configs: Azure service discovery

Now, it means that we will keep in Kubernetes__ meta_ Kubernetes_ pod_ container_ If the name value is sidecar, it also needs to meet__ meta_ Kubernetes_ pod_ annotation_ prometheus_ io_ Pod with the scratch attribute of true.

Next, create Prometheus RBAC setup Yaml file. In order to enable Prometheus to access Kubernetes API, we need to authorize Prometheus to access Kubernetes resources through the Role-Based Access Control model in Kubernetes. First, we define the role (ClusterRole) and set the corresponding access permissions; Create an account (ServiceAccount) for Prometheus; Finally, bind the account to the role (ClusterRoleBinding).

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus
rules:
- apiGroups: [""]
  resources:
  - nodes
  - nodes/proxy
  - services
  - endpoints
  - pods
  verbs: ["get", "list", "watch"]
- apiGroups:
  - extensions
  resources:
  - ingresses
  verbs: ["get", "list", "watch"]
- nonResourceURLs: ["/metrics"]
  verbs: ["get"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: prometheus
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: prometheus
subjects:
- kind: ServiceAccount
  name: prometheus
  namespace: default

Create the prometheus-deployment.yaml file.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: prometheus
  name: prometheus
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus  
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      serviceAccountName: prometheus
      containers:
      - name: prometheus
        image: prom/prometheus:latest
        command:
        - "/bin/prometheus"
        args:
        - "--config.file=/etc/prometheus/prometheus.yml"
        ports:
        - containerPort: 9090
          protocol: TCP
        volumeMounts:
        - mountPath: "/etc/prometheus"
          name: prometheus-config
      volumes:
      - name: prometheus-config
        configMap:
          name: prometheus-config

Create the prometheus-service.yaml file.

apiVersion: v1
kind: Service
metadata:
  name: prometheus
  labels:
    name: prometheus
spec:
  type: NodePort
  ports:
  - name: prometheus
    protocol: TCP
    port: 9090
    targetPort: 9090
    nodePort: 32732
  selector:
    app: prometheus

The Prometheus dashboard is shown below

Grafana

The content of Grafana will not be expanded. Of course, you can directly view or use my dashboard file.

https://github.com/hueifeng/dotnet-monitor-on-k8s

reference resources

Deploy Prometheus

https://dotnetos.org/blog/2021-11-22-dotnet-monitor-grafana/

Tags: Python Kubernetes Container computer Cloud Native

Posted by Amal on Tue, 30 Aug 2022 01:45:17 +0930