Spring Cloud learning notes (23)

1:N except for some important core businesses, other ordinary businesses can jump to the unified processing result page through @DefaultProperties(defaultFallback = "")

General and exclusive are separated, which avoids code inflation and reasonably reduces the amount of code

import com.lun.springcloud.service.PaymentHystrixService;
import com.netflix.hystrix.contrib.javanica.annotation.DefaultProperties;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
@Slf4j
@DefaultProperties(defaultFallback = "payment_Global_FallbackMethod")
public class OrderHystirxController {
    @Resource
    private PaymentHystrixService paymentHystrixService;

    @GetMapping("/consumer/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id)
    {
        String result = paymentHystrixService.paymentInfo_OK(id);
        return result;
    }

    @GetMapping("/consumer/payment/hystrix/timeout/{id}")
//    @HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod",commandProperties = {
//            @HystrixProperty(name="execution.isolation.thread.timeoutInMilliseconds",value="1500")
//    })
    @HystrixCommand//Using the global fallback method
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id) {
        //int age = 10/0;
        String result = paymentHystrixService.paymentInfo_TimeOut(id);
        return result;
    }
    public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id)
    {
        return "I am a consumer 80,The other party's payment system is busy. Please try again in 10 seconds or check yourself if you run an error,o(╥﹏╥)o";
    }

    // The following is the global fallback method
    public String payment_Global_FallbackMethod()
    {
        return "Global Exception handling information, please try again later,/(ㄒoㄒ)/~~";
    }
}

57_ The universal service of hystrix is degraded FeignFallback

At present, problem 2 is the separation of unification and customization, and the code is chaotic

When the service is degraded, the client calls the server and the server goes down or shuts down

In this case, the service degradation processing is implemented on the client 80, which has nothing to do with the server 8001. It is only necessary to add an implementation class of service degradation processing to the interface defined by Feign client to realize decoupling

The anomalies we will face in the future

function
overtime
Downtime

Modify cloud consumer feign hystrix order80

According to the existing PaymentHystrixService interface of cloud consumer feign hystrix order80,
Create a new class (AaymentFallbackService) to implement the interface, and handle exceptions for the methods in the interface

The PaymentFallbackService class implements the PaymentHystrixService interface

import org.springframework.stereotype.Component;


@Component
public class PaymentFallbackService implements PaymentHystrixService
{
    @Override
    public String paymentInfo_OK(Integer id)
    {
        return "-----PaymentFallbackService fall back-paymentInfo_OK ,o(╥﹏╥)o";
    }

    @Override
    public String paymentInfo_TimeOut(Integer id)
    {
        return "-----PaymentFallbackService fall back-paymentInfo_TimeOut ,o(╥﹏╥)o";
    }
}

YML

server:
  port: 80

eureka:
  client:
    register-with-eureka: false
    service-url:
      defaultZone: http://eureka7001.com:7001/eureka/

#open
feign:
  hystrix:
    enabled: true

PaymentHystrixService interface

import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

@Component
@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT" ,//
             fallback = PaymentFallbackService.class)//Specify PaymentFallbackService class
public interface PaymentHystrixService
{
    @GetMapping("/payment/hystrix/ok/{id}")
    public String paymentInfo_OK(@PathVariable("id") Integer id);

    @GetMapping("/payment/hystrix/timeout/{id}")
    public String paymentInfo_TimeOut(@PathVariable("id") Integer id);
}

test

A single eureka starts 7001 first

PaymentHystrixMain8001 start

Normal access test- http://localhost/consumer/payment/hystrix/ok/1

Deliberately close microservice 8001

The client calls the prompt itself - at this time, the service provider has been down, but we have done the service degradation process, so that the client will also get the prompt information when the service is unavailable without hanging up and killing the server.

58_ Hytrix's service fusing theory

Circuit breaker, equivalent to fuse.

Overview of circuit breaker mechanism

Circuit breaker mechanism is a microservice link protection mechanism to deal with avalanche effect. When a microservice on the fan out link is unavailable due to an error or the response time is too long, the service will be degraded, which will fuse the call of the microservice on the node and quickly return the wrong response information. When it is detected that the microservice call response of the node is normal, the call link is resumed.

In the Spring Cloud framework, the circuit breaker mechanism is implemented through hystrix. Hystrix will monitor the status of calls between microservices. When the failed calls reach a certain threshold, the default is 20 calls in 5 seconds, and the circuit breaker mechanism will be started. The annotation of circuit breaker mechanism is @HystrixCommand.

Relevant papers of Martin Fowler

59_ The service fusing case of hystrix (Part 1)

Hutool domestic tools

Modify cloud provider hystrix payment8001

import cn.hutool.core.util.IdUtil;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.stereotype.Service;
import org.springframework.web.bind.annotation.PathVariable;

import java.util.concurrent.TimeUnit;

@Service
public class PaymentService{    

    ...
    
    //=====Service fusing
    @HystrixCommand(fallbackMethod = "paymentCircuitBreaker_fallback",commandProperties = {
            @HystrixProperty(name = "circuitBreaker.enabled",value = "true"),// Whether to open the circuit breaker
            @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold",value = "10"),// Number of requests
            @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds",value = "10000"), // Time window period
            @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage",value = "60"),// Trip after the failure rate reaches
    })
    public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
        if(id < 0) {
            throw new RuntimeException("******id Cannot be negative");
        }
        String serialNumber = IdUtil.simpleUUID();

        return Thread.currentThread().getName()+"	"+"Call successful, serial number: " + serialNumber;
    }
    public String paymentCircuitBreaker_fallback(@PathVariable("id") Integer id) {
        return "id Cannot be negative. Please try again later,/(ㄒoㄒ)/~~   id: " +id;
    }

}

The precise way that the circuit opening and closing occurs is as follows:

Assuming the volume across a circuit meets a certain threshold : HystrixCommandProperties.circuitBreakerRequestVolumeThreshold()
And assuming that the error percentage, as defined above exceeds the error percentage defined in : HystrixCommandProperties.circuitBreakerErrorThresholdPercentage()
Then the circuit-breaker transitions from CLOSED to OPEN.
While it is open, it short-circuits all requests made against that circuit-breaker.
After some amount of time (HystrixCommandProperties.circuitBreakerSleepWindowInMilliseconds()), the next request is let through. If it fails, the command stays OPEN for the sleep window. If it succeeds, it transitions to CLOSED and the logic in 1) takes over again.

link

HystrixCommandProperties configuration class

package com.netflix.hystrix;

...

public abstract class HystrixCommandProperties {
    private static final Logger logger = LoggerFactory.getLogger(HystrixCommandProperties.class);

    /* defaults */
    /* package */ static final Integer default_metricsRollingStatisticalWindow = 10000;// default => statisticalWindow: 10000 = 10 seconds (and default of 10 buckets so each bucket is 1 second)
    private static final Integer default_metricsRollingStatisticalWindowBuckets = 10;// default => statisticalWindowBuckets: 10 = 10 buckets in a 10 second window so each bucket is 1 second
    private static final Integer default_circuitBreakerRequestVolumeThreshold = 20;// default => statisticalWindowVolumeThreshold: 20 requests in 10 seconds must occur before statistics matter
    private static final Integer default_circuitBreakerSleepWindowInMilliseconds = 5000;// default => sleepWindow: 5000 = 5 seconds that we will sleep before trying again after tripping the circuit
    private static final Integer default_circuitBreakerErrorThresholdPercentage = 50;// default => errorThresholdPercentage = 50 = if 50%+ of requests in 10 seconds are failures or latent then we will trip the circuit
    private static final Boolean default_circuitBreakerForceOpen = false;// default => forceCircuitOpen = false (we want to allow traffic)
    /* package */ static final Boolean default_circuitBreakerForceClosed = false;// default => ignoreErrors = false 
    private static final Integer default_executionTimeoutInMilliseconds = 1000; // default => executionTimeoutInMilliseconds: 1000 = 1 second
    private static final Boolean default_executionTimeoutEnabled = true;

    ...
}

60_ The case of service fusing of hystrix (Part 2)

@RestController
@Slf4j
public class PaymentController
{
    @Resource
    private PaymentService paymentService;

    ...
    
    //====Service fusing
    @GetMapping("/payment/circuit/{id}")
    public String paymentCircuitBreaker(@PathVariable("id") Integer id)
    {
        String result = paymentService.paymentCircuitBreaker(id);
        log.info("****result: "+result);
        return result;
    }
}

test

Self test cloud provider hystrix payment8001

Correct- http://localhost:8001/payment/circuit/1

Error- http://localhost:8001/payment/circuit/-1

Multiple errors, correct again, but errors have to be displayed

Key test - several errors, and then slowly correct. It is found that the conditions are not met at the beginning, and even the correct access address cannot be carried out

61_ Summary of service fusing of hystrix

Great God conclusion

Relevant papers of Martin Fowler

Fuse type

Fusing on: request not to call the current service again. The internal set clock is generally MTTR (mean time to failure). When it is turned on until the set clock is up, it will enter the semi fusing state.
Fusing off: fusing off will not fuse the service.
Fusing half open: some requests call the current service according to the rules. If the request is successful and meets the rules, it is considered that the current service returns to normal, and the fusing is turned off.

Flow chart of circuit breaker on official website

Official website steps

The precise way that the circuit opening and closing occurs is as follows:

Assuming the volume across a circuit meets a certain threshold : HystrixCommandProperties.circuitBreakerRequestVolumeThreshold()
And assuming that the error percentage, as defined above exceeds the error percentage defined in : HystrixCommandProperties.circuitBreakerErrorThresholdPercentage()
Then the circuit-breaker transitions from CLOSED to OPEN.
While it is open, it short-circuits all requests made against that circuit-breaker.
After some amount of time (HystrixCommandProperties.circuitBreakerSleepWindowInMilliseconds()), the next request is let through. If it fails, the command stays OPEN for the sleep window. If it succeeds, it transitions to CLOSED and the logic in 1) takes over again.

link

When does the circuit breaker start to work

//=====Service fusing
@HystrixCommand(fallbackMethod = "paymentCircuitBreaker_fallback",commandProperties = {
    @HystrixProperty(name = "circuitBreaker.enabled",value = "true"),// Whether to open the circuit breaker
    @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold",value = "10"),// Number of requests
    @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds",value = "10000"), // Time window period
    @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage",value = "60"),// Trip after the failure rate reaches
})
public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
    ...
}

Three important parameters related to the circuit breaker:

Snapshot time window: the circuit breaker needs to count some request and error data to determine whether to open, and the statistical time range is the snapshot time window, which defaults to the last 10 seconds.
Threshold of total requests: within the snapshot time window, the threshold of total requests must be met to be eligible for fusing. The default is 20, which means that in 10 seconds, if the number of calls of the hystrix command is less than 20 times 7, even if all requests timeout or fail for other reasons, the circuit breaker will not open.
Error percentage threshold: when the total number of requests exceeds the threshold within the snapshot time window, such as 30 calls, if there are 15 timeout exceptions in these 30 calls, that is, more than 50% error percentage, under the default setting of 50% threshold, the circuit breaker will be opened at this time.

Conditions for opening or closing the circuit breaker

When the following thresholds are reached, the circuit breaker will open:
- When a certain threshold is met (more than 20 requests in 10 seconds by default)
- When the failure rate reaches a certain level (more than 50% of requests fail within 10 seconds by default)
When enabled, all requests will not be forwarded
After a period of time (5 seconds by default), when the circuit breaker is half open, one of the requests will be forwarded. If successful, the circuit breaker will close. If failed, it will continue to open.

After the circuit breaker is opened

1: When there is another request call, the main logic will not be called, but the degraded fallback will be called directly. Through the circuit breaker, it realizes the effect of automatically finding errors and switching the degraded logic to the main logic to reduce the response delay.

2: How to restore the original main logic?

For this problem, hystrix also realizes the automatic recovery function for us.

When the circuit breaker is opened and the main logic is fused, hystrix will start a sleep time window. During this time window, the degraded logic will temporarily become the main logic. When the sleep time window expires, the circuit breaker will enter the half open state and release a request to the original main logic. If the request returns normally, the circuit breaker will continue to close and the main logic will recover. If there is still a problem with this request, The circuit breaker continues to enter the open state, and the sleep time window is timed again.

All configuration

@HystrixCommand(fallbackMethod = "fallbackMethod", 
                groupKey = "strGroupCommand", 
                commandKey = "strCommand", 
                threadPoolKey = "strThreadPool",
                
                commandProperties = {
                    // Set the isolation policy. THREAD indicates THREAD pool SEMAPHORE: signal pool isolation
                    @HystrixProperty(name = "execution.isolation.strategy", value = "THREAD"),
                    // When the isolation strategy selects signal pool isolation, it is used to set the size of the signal pool (maximum concurrency)
                    @HystrixProperty(name = "execution.isolation.semaphore.maxConcurrentRequests", value = "10"),
                    // Configure the timeout for command execution
                    @HystrixProperty(name = "execution.isolation.thread.timeoutinMilliseconds", value = "10"),
                    // Enable timeout
                    @HystrixProperty(name = "execution.timeout.enabled", value = "true"),
                    // Whether to interrupt when the execution times out
                    @HystrixProperty(name = "execution.isolation.thread.interruptOnTimeout", value = "true"),
                    
                    // Whether the execution is interrupted when it is canceled
                    @HystrixProperty(name = "execution.isolation.thread.interruptOnCancel", value = "true"),
                    // Maximum concurrent number of callback method execution allowed
                    @HystrixProperty(name = "fallback.isolation.semaphore.maxConcurrentRequests", value = "10"),
                    // Whether the service degradation is enabled and whether the callback function is executed
                    @HystrixProperty(name = "fallback.enabled", value = "true"),
                    // Whether the circuit breaker is enabled
                    @HystrixProperty(name = "circuitBreaker.enabled", value = "true"),
                    // This attribute is used to set the minimum number of requests for circuit breaker fusing in the rolling time window. For example, when the default value is 20, if only 19 requests are received within the rolling time window (default 10 seconds), even if these 19 requests fail, the circuit breaker will not open.
                    @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "20"),
                    
                    // This attribute is used to set in the rolling time window, indicating that in the rolling time window, the number of requests exceeds circuitbreaker In the case of requestvolumthreshold, if the percentage of wrong requests exceeds 50, set the circuit breaker to the "open" state, otherwise set it to the "closed" state.
                    @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "50"),
                    // This attribute is used to set the sleep time window after the circuit breaker is opened. After the sleep time window ends, the circuit breaker will be set to the "half open" state and the request command to try to fuse. If it still fails, the circuit breaker will continue to be set to the "open" state, and if it succeeds, it will be set to the "closed" state.
                    @HystrixProperty(name = "circuitBreaker.sleepWindowinMilliseconds", value = "5000"),
                    // Forced opening of circuit breaker
                    @HystrixProperty(name = "circuitBreaker.forceOpen", value = "false"),
                    // Forced closing of circuit breaker
                    @HystrixProperty(name = "circuitBreaker.forceClosed", value = "false"),
                    // Set the rolling time window, which is used for the duration of information to be collected when judging the health of the circuit breaker
                    @HystrixProperty(name = "metrics.rollingStats.timeinMilliseconds", value = "10000"),
                    
                    // This attribute is used to set the number of "buckets" divided when rolling time window statistics indicator information. When collecting indicator information, the circuit breaker will be divided into multiple "buckets" according to the length of the set time window to accumulate each measurement value. Each "bucket" records the collection indicator for a period of time.
                    // For example, it is divided into 10 "buckets" in 10 seconds, so timeinMilliseconds must be divisible by numBuckets. Otherwise, exceptions will be thrown
                    @HystrixProperty(name = "metrics.rollingStats.numBuckets", value = "10"),
                    // This attribute is used to set whether the delay in command execution is tracked and calculated using percentiles. If set to false, all summary statistics will return -1.
                    @HystrixProperty(name = "metrics.rollingPercentile.enabled", value = "false"),
                    // This attribute is used to set the duration of the scrolling window of percentile statistics, in milliseconds.
                    @HystrixProperty(name = "metrics.rollingPercentile.timeInMilliseconds", value = "60000"),
                    // This attribute is used to set the number of "buckets" used in the percentile statistics scrolling window.
                    @HystrixProperty(name = "metrics.rollingPercentile.numBuckets", value = "60000"),
                    // This attribute is used to set the maximum number of executions retained in each bucket during execution. If the number of executions exceeding the set value occurs within the rolling time window,
                    // Rewrite it from the original position. For example, set the value to 100 and scroll the window for 10 seconds. If 500 executions occur in a "bucket" within 10 seconds,
                    // Then only the statistics of the last 100 executions are retained in the "bucket". In addition, increasing the size of this value will increase the consumption of memory and the calculation time required to sort percentiles.
                    @HystrixProperty(name = "metrics.rollingPercentile.bucketSize", value = "100"),
                    
                    // This attribute is used to set the interval waiting time for collecting health snapshots (success of requests, error percentage) that affect the status of the circuit breaker.
                    @HystrixProperty(name = "metrics.healthSnapshot.intervalinMilliseconds", value = "500"),
                    // Whether to enable request cache
                    @HystrixProperty(name = "requestCache.enabled", value = "true"),
                    // Whether the execution and events of the HystrixCommand are printed into the HystrixRequestLog
                    @HystrixProperty(name = "requestLog.enabled", value = "true"),

                },
                threadPoolProperties = {
                    // This parameter is used to set the number of core threads in the command execution thread pool, which is the maximum concurrency of command execution
                    @HystrixProperty(name = "coreSize", value = "10"),
                    // This parameter is used to set the maximum queue size of the thread pool. When set to -1, the thread pool will use the queue implemented by SynchronousQueue, otherwise it will use the queue implemented by LinkedBlockingQueue.
                    @HystrixProperty(name = "maxQueueSize", value = "-1"),
                    // This parameter is used to set the rejection threshold for the queue. With this parameter, requests can be rejected even if the queue does not reach the maximum value.
                    // This parameter is mainly a supplement to the LinkedBlockingQueue queue queue, because the LinkedBlockingQueue queue cannot dynamically modify its object size, and the queue size that rejects requests can be adjusted through this attribute.
                    @HystrixProperty(name = "queueSizeRejectionThreshold", value = "5"),
                }
               )
public String doSomething() {
	...
}

62_ Final summary of hystrix workflow

Service flow restriction - the Sentinel description of alibaba will be explained in the later advanced chapter

Official interpretation

Official website legend

[the external link image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-yx9xouzg-1614712032449)( https://raw.githubusercontent.com/wiki/Netflix/Hystrix/images/hystrix-command-flow-chart.png )]

Step description

Create a HystrixCommand (used when the dependent service returns a single operation result) or HystrixObserableCommand (used when the dependent service returns multiple operation results) object.
Command execution.
HystrixCommand implements the first two execution methods below
1. execute(): execute synchronously, return a single result object from the dependent service or throw an exception when an error occurs.
  2. queue(): asynchronous execution, directly returning a Future object, which contains a single result object to be returned at the end of service execution.
The HystrixObservableCommand implements the latter two execution methods:
1. Observe (): returns an Observable object, which represents multiple systems of operation
  As a result, it is a Hot Observable (whether the "event source" has a "Subscriber" or not, the event will be published after creation, so each "Subscriber" of the Hot Observable may start in the middle of the "event source" and may only see the partial process of the whole operation).
  2. Tobservable(): it will also return an Observable object, which also represents multiple results of the operation, but it returns a Cold Observable (when there is no "Subscriber", the event will not be published, but wait until there is a "Subscriber", so for the subscriber of Cold Observable, it can ensure to see the whole process of the operation from the beginning).
If the request caching function of the current command is enabled, and the command cache hits, the cached results will be immediately returned in the form of Observable objects.
Check whether the circuit breaker is open. If the circuit breaker is open, Hystrix will not execute the command, but transfer to the fallback processing logic (step 8); If the circuit breaker is closed, check whether there are resources available to execute the command (step 5).
Whether the thread pool / request queue semaphore is full. If the command depends on the dedicated thread of the service and the request queue, or the semaphore (when the thread is not used) has been occupied, the Hystrix will not execute the command, but will transfer to the fallback processing logic (step 8).
Hytrix will decide how to request dependent services according to the method we write.
1. HystrixCommand.run(): return a single result or throw an exception.
2. HystrixObservableCommand.construct(): return an Observable object to emit multiple results, or send error notification through onError.
Hystix will report "success", "failure", "rejection", "timeout" and other information to the circuit breaker, and the circuit breaker will maintain a set of counters to count these data. The circuit breaker will use these statistics to decide whether to open the circuit breaker to "fuse / short circuit" a service dependent request.
When the command fails to execute, Hystix will enter fallback to try to fallback. We usually call the wave operation "service degradation". The following conditions can cause service degradation:
1. Step 4: the current command is in the "fuse / short circuit" state, and the breaker is open.
2. Step 5: when the money pool, request queue or semaphore of the current command is full.
3. Step 6: when hytrixobsevablecommand. Construct() or HytrixCommand.run() throws an exception.
When the Hystrix command is successfully executed, it will return the processing result directly or in the form of Observable.

tips: if we do not implement the degradation logic for the command or throw an exception in the degradation processing logic, Hystrix will still return an observable object, but it will not send any result data. Instead, it will notify the command to immediately interrupt the request through the onError method, and send the exception causing the command failure to the caller through the onError method.

63_ Establishment of hystrix graphical Dashboard

summary

In addition to isolating the calls of dependent services, hystrix also provides a quasi real-time call monitoring (Hystrix Dashboard). Hystrix will continuously record the execution information of all requests initiated through hystrix, and display it to users in the form of statistical reports and graphs, including how many requests are executed per second, how many are successful, how many are failed, etc.

Netflix monitors the above indicators through the hystrix metrics event stream project. Spring Cloud also provides the integration of the Hystrix Dashboard, transforming the monitoring content into a visual interface.

Instrument cluster 9001

1. Create a new cloud consumer hystrix dashboard9001

2.POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>LearnCloud</artifactId>
        <groupId>com.lun.springcloud</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-consumer-hystrix-dashboard9001</artifactId>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

3.YML

server:
  port: 9001

4.HystrixDashboardMain9001 + new annotation @EnableHystrixDashboard

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;

@SpringBootApplication
@EnableHystrixDashboard
public class HystrixDashboardMain9001
{
    public static void main(String[] args) {
        SpringApplication.run(HystrixDashboardMain9001.class, args);
    }
}

5. All Provider microservice providing classes (8001/8002/8003) need to monitor dependency configuration

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

6. Start cloud consumer hystrix dashboard9001. The microservice will monitor the microservice 8001 later

Browser input http://localhost:9001/hystrix

64_Hystrix graphical Dashboard monitoring practice

Modify cloud provider hystrix payment8001

Note: the new version of Hystrix needs to specify the monitoring path in the main startup class PaymentHystrixMain8001

import com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
@EnableEurekaClient
@EnableCircuitBreaker
public class PaymentHystrixMain8001
{
    public static void main(String[] args) {
            SpringApplication.run(PaymentHystrixMain8001.class, args);
    }


    /**
     *This configuration is for service monitoring and has nothing to do with service fault tolerance itself. The pit after springcloud upgrade
     *ServletRegistrationBean Because the default path of springboot is not "/hystrix.stream",
     *Just configure the following servlet s in your project
     *Otherwise, Unable to connect to Command Metric Stream 404
     */
    @Bean
    public ServletRegistrationBean getServlet() {
        HystrixMetricsStreamServlet streamServlet = new HystrixMetricsStreamServlet();
        ServletRegistrationBean registrationBean = new ServletRegistrationBean(streamServlet);
        registrationBean.setLoadOnStartup(1);
        registrationBean.addUrlMappings("/hystrix.stream");
        registrationBean.setName("HystrixMetricsStreamServlet");
        return registrationBean;
    }
}

Monitoring test

Start 1 eureka

Start 80019001

Observe the monitoring window

9001 monitoring 8001 - fill in the monitoring address- http://localhost:8001/hystrix.stream To http://localhost:9001/hystrix Input box of the page.

Test address

http://localhost:8001/payment/circuit/1
http://localhost:8001/payment/circuit/-1
Test passed
Visit the correct address first, then the wrong address, and then the correct address. You will find that the circuit breakers shown in the figure are slowly released.

How to see

7 colors

1 turn

Filled circle: there are two meanings. It represents the health degree of the instance through the change of color, and its health degree decreases from green < yellow < orange < red.

In addition to the change of color, the size of the solid circle will also change according to the requested flow of the instance. The larger the flow, the larger the solid circle. Therefore, through the display of the solid circle, we can quickly find fault cases and high pressure cases in a large number of examples.

1 line

Curve: it is used to record the relative change of flow within 2 minutes, through which the rising and falling trend of flow can be observed.

Description of the whole drawing

Description of the whole drawing 2

65_GateWay and Zuul course description

Zuul developers are fighting in the dark, which is a thing of the past

Focus on Gate Way

66_ What is gateway

Previous generation zuul 1.x official website

Gateway official website

summary

A very important component of Cloud family bucket is the gateway, which is the Zuul gateway used in version 1.x;

However, in version 2.x, zuul's upgrade has been skipped. SpringCloud finally developed a gateway to replace zuul, that is, SpringCloud Gateway - in a word, gateway is the replacement of the original zuul version 1.x

Gateway is an API gateway service built on the Spring ecosystem, based on Spring 5, Spring Boot 2, Project Reactor and other technologies.

Gateway aims to provide a simple and effective way to route API s and provide some powerful filter functions, such as fusing, current limiting, Retry, etc.

SpringCloud Gateway is a new project of Spring Cloud. It is a gateway developed based on Spring 5.0+Spring Boot 2.0 and Project Reactor. It aims to provide a simple and effective unified API route management method for microservice architecture.

As a gateway in the Spring Cloud ecosystem, SpringCloud Gateway aims to replace Zuul. In Spring Cloud versions above 2.0, there is no integration of the new version of Zul with the latest performance version above 2.0, and it is still the old version of Zuul 1.x non Reactor mode. In order to improve the performance of the gateway, SpringCloud Gateway is implemented based on the WebFlux framework, and the underlying WebFlux framework uses the high-performance Reactor mode communication framework Netty.

The goal of Spring Cloud Gateway is to provide a unified routing method, and the Filter chain based method provides the basic functions of the gateway, such as security, monitoring / indicators, and flow restriction.

effect

Direction proxy
authentication
flow control
Fusing
Log monitoring
...

Location of gateway in microservice architecture

67_GateWay non blocking asynchronous model

Why did Zuull come out of the Gateway again? Why we choose Gateway

netflix is not very reliable. zuul2.0 has been skipped and has not been released yet.
1. On the one hand, Zuul1.0 has entered the maintenance stage, and the Gateway is developed by the SpringCloud team. It is a trusted product. And many functions of Zuul are not used, and it is very simple and convenient.
2. Gateway is developed based on asynchronous non blocking model, so there is no need to worry about performance. Although Netflix has long released the latest Zuul 2.x, Spring Cloud seems to have no integration plan. And Netflix related components are announced to enter the maintenance period; I don't know the future
3. Considering all aspects, Gateway is an ideal Gateway choice.
SpringCloud Gateway has the following features
1. Based on Spring Framework 5, Project Reactor and Spring Boot 2.0;
2. Dynamic routing: it can match any request attribute;
3. You can specify Predicate and filter for the route;
4. Integrated circuit breaker function of Hystrix;
5. Integrate Spring Cloud service discovery function;
6. Easy to write Predicate and Filter;
7. Request current limiting function;
8. Path rewriting is supported.
The difference between SpringCloud Gateway and Zuul
1. Before the official version of SpringCloud Finchley, the gateway recommended by Spring Cloud was Zuul provided by Netflix.
2. Zuul 1.x is an API Gateway based on blocking I/O.
3. Zuul 1.x uses a blocking architecture based on Servlet 2.5. It does not support any long connections (such as WebSockets). Zuul's design pattern is similar to Nginx. Every time I/ О The operation is to select an execution from the worker thread, and the request thread is blocked until the worker thread completes. However, the difference is that Nginx is implemented in C++ and zuul is implemented in Java, while the JVM itself will have a slow loading for the first time, which makes zuul's performance relatively poor.
4. Zuul 2.x has a more advanced concept. It wants to be non blocking based on Netty and support long connections, but SpringCloud has not been integrated yet. The performance of zuul.X is much better than that of Zuul 1.x. In terms of performance, according to the benchmark provided by the official, the RPS (requests per second) of Spring Cloud Gateway is 1.6 times that of zuul.
5. Spring Cloud Gateway is built on Spring Framework 5, Project Reactor and Spring Boot2, and uses non blocking API s.
6. Spring Cloud Gateway also supports WebSocket, and is closely integrated with spring to have a better development experience

Zuul1.x model

The Zuul version integrated in Springcloud uses the Tomcat container and the traditional Serviet IO processing model.

The lifecycle of servlet? Servlets are lifecycle managed by servlet container s.

When the container starts, construct a servlet object and call servlet init() to initialize;
The container runtime accepts requests, allocates a thread for each request (usually obtains idle threads from the thread pool), and then calls the service);
Call servlet destroy() to destroy the servlet when the container is closed.

Disadvantages of the above mode:

Servlet is a simple network IO model. When a request enters the Servlet container, the Servlet container will bind a thread for it. This model is applicable in scenarios with low concurrency. However, once the concurrency is high (such as Jmeter pressure for ventilation), the number of threads will increase, and the cost of thread resources is expensive (online text switching, large memory consumption), which seriously affects the processing time of requests. In some simple business scenarios, you don't want to allocate a thread to each request. You only need one or several threads to deal with highly concurrent requests. In this business scenario, the servlet model has no advantage.

Therefore, Zuul 1.X is a blocking processing model based on servlet, that is, Spring implements a servlet (DispatcherServlet) that processes all request requests and is blocked by the servlet. Therefore, SpringCloud Zuul cannot get rid of the disadvantages of servlet model.

Gateway model

What is WebFlux? Official documents

Traditional Web frameworks, such as Struts2 and SpringMVC, run on the basis of Servlet APl and Servlet container.

But after Servlet3.1, there is asynchronous non blocking support. WebFlux is a typical non blocking asynchronous framework, and its core is implemented based on the relevant API of Reactor. Compared with the traditional web framework, it can run on containers such as Netty, Undertow and Servlet3.1. Non blocking + functional programming (Spring 5 must let you use Java 8).

Spring WebFlux is a new responsive framework introduced by Spring 5.0. Different from Spring MVC, it does not need to rely on Servlet APl. It is completely asynchronous and non blocking, and implements the responsive flow specification based on Reactor.

Spring Cloud Gateway requires the Netty runtime provided by Spring Boot and Spring Webflux. It does not work in a traditional Servlet Container or when built as a WAR.link

68_Gateway workflow

Three core concepts

Route - route is the basic module for building a gateway. It consists of ID, target URI, a series of assertions and filters. If the assertion is true, it matches the route;
Predicate - the reference is java.util.function Predict, the developer can match all the contents of the HTTP request (such as the request header or request parameters), and route if the request matches the assertion;
Filter - refers to the instance of GatewayFilter in the Spring framework. Using filters, you can modify requests before or after they are routed.

The web request locates the real service node through some matching conditions. And before and after this forwarding process, some fine control is carried out.

predicate is our matching condition; fliter can be understood as an omnipotent interceptor. With these two elements, plus the target uri, you can achieve a specific route

Gateway workflow

Official website summary

Clients make requests to Spring Cloud Gateway. If the Gateway Handler Mapping determines that a request matches a route, it is sent to the Gateway Web Handler. This handler runs the request through a filter chain that is specific to the request. The reason the filters are divided by the dotted line is that filters can run logic both before and after the proxy request is sent. All "pre" filter logic is executed. Then the proxy request is made. After the proxy request is made, the "post" filter logic is run. link

The client sends a request to the Spring Cloud Gateway. Then find the route matching the request in the Gateway Handler Mapping and send it to the gateway web handler.

The Handler then sends the request to our actual service through the specified filter chain to execute the business logic, and then returns.

The reason why the filters are separated by dotted lines is that the filter may execute business logic before ("pre") or after ("post") sending the proxy request.

Filter can do parameter verification, permission verification, traffic monitoring, log output, protocol conversion, etc. in "pre" type filter, it can do response content, response header modification, log output, traffic monitoring, etc. in "post" type filter, it plays a very important role.

Core logic: route forwarding + execution filter chain.

69_Gateway9527 construction

1. Create a new module - cloud gateway gateway 9527

2.POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>LearnCloud</artifactId>
        <groupId>com.lun.springcloud</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-gateway-gateway9527</artifactId>

    <dependencies>
        <!--gateway-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-gateway</artifactId>
        </dependency>
        <!--eureka-client-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <!-- Introduce self defined api General package, which can be used Payment payment Entity -->
        <dependency>
            <groupId>com.lun.springcloud</groupId>
            <artifactId>cloud-api-commons</artifactId>
            <version>${project.version}</version>
        </dependency>
        <!--General basic configuration class-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

3.YML

server:
  port: 9527

spring:
  application:
    name: cloud-gateway

eureka:
  instance:
    hostname: cloud-gateway-service
  client: #The service provider is registered in the eureka service list
    service-url:
      register-with-eureka: true
      fetch-registry: true
      defaultZone: http://eureka7001.com:7001/eureka

4. Business

nothing

5. Main startup class

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;


@SpringBootApplication
@EnableEurekaClient
public class GateWayMain9527
{
    public static void main(String[] args) {
        SpringApplication.run(GateWayMain9527.class, args);
    }
}

6.9527 how to do routing mapping for gateway

Cloud provider payment8001 look at the access address of the controller

At present, we don't want to expose the 8001 port. We hope to set a layer of 9527 outside the 8001

7.YML new gateway configuration

server:
  port: 9527

spring:
  application:
    name: cloud-gateway
#############################Add gateway configuration###########################
  cloud:
    gateway:
      routes:
        - id: payment_routh #payment_route    #The ID of the route, which has no fixed rules but requires uniqueness, is recommended to match the service name
          uri: http://localhost:8001 # the routing address of the service provided after matching
          #uri: lb://Cloud payment service # the routing address of the service provided after matching
          predicates:
            - Path=/payment/get/**         # Assert that the route matches the path

        - id: payment_routh2 #payment_route    #The ID of the route, which has no fixed rules but requires uniqueness, is recommended to match the service name
          uri: http://localhost:8001 # the routing address of the service provided after matching
          #uri: lb://Cloud payment service # the routing address of the service provided after matching
          predicates:
            - Path=/payment/lb/**         # Assert that the route matches the path
####################################################################

eureka:
  instance:
    hostname: cloud-gateway-service
  client: #The service provider is registered in the eureka service list
    service-url:
      register-with-eureka: true
      fetch-registry: true
      defaultZone: http://eureka7001.com:7001/eureka

8. Test

Start 7001
Start 8001 cloud provider payment8001
Start 9527 gateway
Access instructions
- Before adding a gateway- http://localhost:8001/payment/get/1
- After adding gateway- http://localhost:9527/payment/get/1
- Both accesses are successful, and the same result is returned

70_ Two ways for gateway to configure routing

Configure in the configuration file yml, see the previous chapter

Inject the Bean of RouteLocator into the code

Official case- link

RemoteAddressResolver resolver = XForwardedRemoteAddressResolver
    .maxTrustedIndex(1);

...

.route("direct-route",
    r -> r.remoteAddr("10.1.1.1", "10.10.1.1/24")
        .uri("https://downstream1")
.route("proxied-route",
    r -> r.remoteAddr(resolver, "10.10.1.1", "10.10.1.1/24")
        .uri("https://downstream2")
)

Baidu domestic news website, need external network- http://news.baidu.com/guonei

Write one by yourself

Business needs - access the Baidu news website of the external network through the 9527 gateway

code

Cloud gateway gateway 9527 service implementation

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;


@Configuration
public class GateWayConfig
{
    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder routeLocatorBuilder)
    {
        RouteLocatorBuilder.Builder routes = routeLocatorBuilder.routes();

        routes.route("path_route_atguigu",
                r -> r.path("/guonei")
                        .uri("http://news.baidu.com/guonei")).build();

        return routes.build();
    }
}

test

Browser input http://localhost:9527/guonei , return http://news.baidu.com/guonei Same page.

71_GateWay configuration dynamic routing

By default, the Gateway will create a dynamic route with the name of the micro service on the registry as the path to forward according to the list of services registered in the registry, so as to realize the function of dynamic route (without writing an address).

start-up

eureka7001
payment8001/8002

POM

<!--eureka-client-->
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

YML

It should be noted that the protocol of uri is lb, which means that the load balancing function of Gateway is enabled.

lb://serviceName is the load balancing uri that spring cloud gateway automatically creates for us in microservices.

server:
  port: 9527

spring:
  application:
    name: cloud-gateway
#############################Add gateway configuration###########################
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true #Enable the function of dynamically creating routes from the registry, and use the microservice name to route
      routes:
        - id: payment_routh #payment_route    #The ID of the route, which has no fixed rules but requires uniqueness, is recommended to match the service name
          #uri: http://localhost:8001 # the routing address of the service provided after matching
          uri: lb://Cloud payment service # the routing address of the service provided after matching
          predicates:
            - Path=/payment/get/**         # Assert that the route matches the path

        - id: payment_routh2 #payment_route    #The ID of the route, which has no fixed rules but requires uniqueness, is recommended to match the service name
          #uri: http://localhost:8001 # the routing address of the service provided after matching
          uri: lb://Cloud payment service # the routing address of the service provided after matching
          predicates:
            - Path=/payment/lb/**         # Assert that the route matches the path
####################################################################

eureka:
  instance:
    hostname: cloud-gateway-service
  client: #The service provider is registered in the eureka service list
    service-url:
      register-with-eureka: true
      fetch-registry: true
      defaultZone: http://eureka7001.com:7001/eureka

test

Browser input- http://localhost:9527/payment/lb

result

Constantly refresh the page and switch between 8001 and 8002 ports.

72_ Commonly used Predicate of gateway

Official documents

What is route predict factories

Spring Cloud Gateway matches routes as part of the Spring WebFlux HandlerMapping infrastructure. Spring Cloud Gateway includes many built-in route predicate factories. All of these predicates match on different attributes of the HTTP request. You can combine multiple route predicate factories with logical and statements. link

Spring Cloud Gateway takes route matching as part of the Spring WebFlux HandlerMapping infrastructure.

Spring Cloud Gateway includes many built-in route predict factories. All these predictions match the different attributes of the HTTP request. Multiple RoutePredicate factories can be combined.

When Spring Cloud Gateway creates a Route object, it uses RoutePredicateFactory to create a Predicate object, which can be assigned to Route. Spring Cloud Gateway includes many built-in Route predict factories.
All of these predicates match different attributes of the HTTP request. A variety of predicate factories can be combined through logical and.

predicate

Us: ['predket] UK: ['predkt]

v. Assertion; Based on; Make based on; indicate

adj. declarative; Predicate

n. Predicate (sentence element, stating the subject, such as went home in John went home)

Common Route Predicate Factory

The After Route Predicate Factory
The Before Route Predicate Factory
The Between Route Predicate Factory
The Cookie Route Predicate Factory
The Header Route Predicate Factory
The Host Route Predicate Factory
The Method Route Predicate Factory
The Path Route Predicate Factory
The Query Route Predicate Factory
The RemoteAddr Route Predicate Factory
The weight Route Predicate Factory

Discuss several route predict factories

The After Route Predicate Factory

spring:
  cloud:
    gateway:
      routes:
      - id: after_route
        uri: https://example.org
        predicates:
        # Only after this time can it take effect
        - After=2017-01-20T17:42:47.789-07:00[America/Denver]

The timestamp string in the above format can be obtained by the following methods

import java.time.ZonedDateTime;


public class T2
{
    public static void main(String[] args)
    {
        ZonedDateTime zbj = ZonedDateTime.now(); // Default time zone
        System.out.println(zbj);

       //2021-02-22T15:51:37.485+08:00[Asia/Shanghai]
    }
}

The Between Route Predicate Factory

spring:
  cloud:
    gateway:
      routes:
      - id: between_route
        uri: https://example.org
        # Between two points in time
        predicates:
        - Between=2017-01-20T17:42:47.789-07:00[America/Denver], 2017-01-21T17:42:47.789-07:00[America/Denver]

The Cookie Route Predicate Factory

spring:
  cloud:
    gateway:
      routes:
      - id: cookie_route
        uri: https://example.org
        predicates:
        - Cookie=chocolate, ch.p

The cookie route predicate factory takes two parameters, the cookie name and a regular expression.

This predicate matches cookies that have the given name and whose values match the regular expression.

test

# This command is equivalent to sending a get request without cookie s
curl http://localhost:9527/payment/lb

# With cookie s
curl http://localhost:9527/payment/lb --cookie "chocolate=chip"

The Header Route Predicate Factory

spring:
  cloud:
    gateway:
      routes:
      - id: header_route
        uri: https://example.org
        predicates:
        - Header=X-Request-Id, d+

The header route predicate factory takes two parameters, the header name and a regular expression.

This predicate matches with a header that has the given name whose value matches the regular expression.

test

# CURL command with parameters specifying the request header
curl http://localhost:9527/payment/lb -H "X-Request-Id:123"

Others, draw inferences from one instance.

Summary

To put it bluntly, Predicate is to implement a set of matching rules, so that requests can be processed by finding the corresponding Route.

73_ Filter of gateway

Official documents

Route filters allow the modification of the incoming HTTP request or outgoing HTTP response in some manner. Route filters are scoped to a particular route. Spring Cloud Gateway includes many built-in GatewayFilter Factories.

Routing filters can be used to modify incoming HTTP requests and returned HTTP responses. Routing filters can only be used by specifying routes. Spring Cloud Gateway has built-in various routing filters, which are generated by the factory class of GatewayFilter.

Filter of Spring Cloud Gateway:

Life cycle:
- pre
- post
Type (see official documents for details):
- GatewayFilter - there are 31 kinds
- GlobalFilter - there are 10 kinds

Common GatewayFilter: AddRequestParameter GatewayFilter

Customize global GlobalFilter:

Two main interfaces are introduced:

GlobalFilter
Ordered

What to do:

Global logging
Unified gateway authentication
...

Code case:

Add the MyLogGateWayFilter class to the GateWay9527 project:

import lombok.extern.slf4j.Slf4j;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.GlobalFilter;
import org.springframework.core.Ordered;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;

import java.util.Date;

@Component
@Slf4j
public class MyLogGateWayFilter implements GlobalFilter,Ordered
{

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain)
    {
        log.info("***********come in MyLogGateWayFilter:  "+new Date());

        String uname = exchange.getRequest().getQueryParams().getFirst("uname");

        if(uname == null)
        {
            log.info("*******User name is null，Illegal user, o(╥﹏╥)o");
            exchange.getResponse().setStatusCode(HttpStatus.NOT_ACCEPTABLE);
            return exchange.getResponse().setComplete();
        }

        return chain.filter(exchange);
    }

    @Override
    public int getOrder()
    {
        return 0;
    }
}

Test:

Start:

EurekaMain7001
PaymentMain8001
GateWayMain9527
PaymentMain8002

Browser input:

http://localhost:9527/payment/lb -Rhetorical abnormality
http://localhost:9527/payment/lbuname=abc -Normal rhetorical question

74_ Introduction to config distributed configuration center

Configuration problems faced by distributed systems

Microservice means to split the business in a single application into a sub service. The granularity of each service is relatively small, so there will be a large number of services in the system. Because each service needs the necessary configuration information to run, a centralized and dynamic configuration management facility is essential.

SpringCloud provides ConfigServer to solve this problem. Each of our microservices carries an application.yml and manages hundreds of configuration files

What is it?

SpringCloud Config provides centralized external configuration support for microservices in the microservice architecture, and the configuration server provides a centralized external configuration for all environments of different microservice applications.

How do you play?

SpringCloud Config is divided into server and client.

The server is also known as the distributed configuration center. It is an independent micro service application, which is used to connect to the configuration server and provide the client with access interfaces for obtaining configuration information, encryption / decryption information and so on.
The client manages application resources and business-related configuration content through the specified configuration center, and obtains and loads configuration information from the configuration center at startup. The configuration server uses git to store configuration information by default, which is conducive to version management of environment configuration, and can easily manage and access configuration content through git client tools.

What can I do

Centrally manage profiles
Different environments have different configurations, dynamic configuration updates, and deployment by environment, such as dev/test/prod/beta/release
During operation, the configuration is dynamically adjusted. It is no longer necessary to write configuration files on each service deployed machine. The service will uniformly pull and configure its own information from the configuration center
When the configuration changes, the service can sense the change of the configuration and apply the new configuration without restarting
Expose the configuration information in the form of REST interface - post/crul access refresh

Integrated configuration with GitHub

Because SpringCloud Config uses git by default to store configuration files (there are other ways, such as supporting SVN and local files), Git is the most recommended, and the form of http/https access is used.

Official website

https://cloud.spring.io/spring-cloud-static/spring-cloud-config/2.2.1.RELEASE/reference/html/

75_Config configuration master control center setup

Create a new Repository named springcloud config on GitHub with your own account.

Get the newly created git address from the previous step- git@github.com:abc/springcloud-config.git.

Create a new git warehouse and clone on the local hard disk directory.

Working directory is D:SpringCloud2021
git clone git@github.com:abc/springcloud-config.git

A folder named springcloud config will be created in the working directory.

Create three configuration files (for this teaching) in the springcloud config folder, and then upload git add., git commit -m "sth" and other upload operations to the new Repository of springcloud config.

config-dev.yml

config:
info: "master branch,springcloud-config/config-dev.yml version=7"
config-prod.yml

config:
info: "master branch,springcloud-config/config-prod.yml version=1"
config-test.yml

config:
info: "master branch,springcloud-config/config-test.yml version=1"

Create a new Module cloud-config-center-3344, which is the Cloud configuration center Module CloudConfig Center

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>LearnCloud</artifactId>
        <groupId>com.lun.springcloud</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-config-center-3344</artifactId>

    <dependencies>
        <!--Add message bus RabbitMQ support-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bus-amqp</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-config-server</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

YML

server:
  port: 3344

spring:
  application:
    name:  cloud-config-center #Microservice name registered into Eureka server
  cloud:
    config:
      server:
        git:
          uri: git@github.com:zzyybs/springcloud-config.git #git warehouse name on GitHub
        ####search for directory
          search-paths:
            - springcloud-config
      ####Read branch
      label: master

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

Main startup class

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.config.server.EnableConfigServer;

@SpringBootApplication
@EnableConfigServer
public class ConfigCenterMain3344
{
    public static void main(String[] args) {
            SpringApplication.run(ConfigCenterMain3344.class, args);
    }
}

Modify the hosts file under windows and add mapping

127.0.0.1 config-3344.com

Test whether the configuration content can be obtained from GitHub through Config microservice

Start ConfigCenterMain3344
Browser anti question- http://config-3344.com:3344/master/config-dev.yml
Page return result:

config:
info: "master branch,springcloud-config/config-dev.yml version=7"

Configure read rules

Official documents
/{label}/{application}-{profile}.yml (recommended)
- master branch
  - http://config-3344.com:3344/master/config-dev.yml
  - http://config-3344.com:3344/master/config-test.yml
  - http://config-3344.com:3344/master/config-prod.yml
- dev branch
  - http://config-3344.com:3344/dev/config-dev.yml
  - http://config-3344.com:3344/dev/config-test.yml
  - http://config-3344.com:3344/dev/config-prod.yml
/{application}-{profile}.yml
- http://config-3344.com:3344/config-dev.yml
- http://config-3344.com:3344/config-test.yml
- http://config-3344.com:3344/config-prod.yml
- http://config-3344.com:3344/config -Xxxx.yml (nonexistent configuration)
/{application}/{profile}[/{label}]
- http://config-3344.com:3344/config/dev/master
- http://config-3344.com:3344/config/test/master
- http://config-3344.com:3344/config/test/dev
Summary of important configuration details
- /{name}-{profiles}.yml
- /{label}-{name}-{profiles}.yml
- label: Branch
- Name: service name
- profiles: environment (dev/test/prod)

Successfully achieved using SpringCloud Config to obtain configuration information through GitHub

76_Config client configuration and testing

Create cloud-config-client-3355

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>LearnCloud</artifactId>
        <groupId>com.lun.springcloud</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-config-client-3355</artifactId>

    <dependencies>
        <!--Add message bus RabbitMQ support-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bus-amqp</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>


</project>

bootstrap.yml

applicaiton.yml is a user level resource configuration item

bootstrap.yml is system level, with higher priority

Spring Cloud will create a Bootstrap Context as the parent context of the Application Context of spring applications.

During initialization, BootstrapContext is responsible for loading configuration properties from external sources and parsing the configuration. These two contexts share an Environment obtained from the outside.

Bootstrap attributes have high priority. By default, they will not be overwritten by local configuration. Bootstrap context and Application Context have different conventions, so a new bootstrap is added YML file to ensure the separation of bootstrap context and Application Context configuration.

It is critical to change the application.yml file under the Client module to bootstrap.yml, because bootstrap.yml is loaded before application.yml. Bootstrap.yml has higher priority than application.yml.

server:
  port: 3355

spring:
  application:
    name: config-client
  cloud:
    #Config client configuration
    config:
      label: master #Branch name
      name: config #Profile name
      profile: dev #Read the suffix name. The above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml
      uri: http://localhost:3344 # configuration center address k


#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

Modify the config-dev.yml configuration and submit it to GitHub, such as adding a variable age or version number version

Main start

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;


@EnableEurekaClient
@SpringBootApplication
public class ConfigClientMain3355
{
    public static void main(String[] args) {
            SpringApplication.run(ConfigClientMain3355.class, args);
    }
}

Business class

import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.context.config.annotation.RefreshScope;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;


@RestController
@RefreshScope
public class ConfigClientController
{
    @Value("${config.info}")
    private String configInfo;

    @GetMapping("/configInfo")
    public String getConfigInfo()
    {
        return configInfo;
    }
}

test

Start Config configuration center 3344 microservice and self test
- http://config-3344.com:3344/master/config-prod.yml
- http://config-3344.com:3344/master/config-dev.yml
Start 3355 as a Client to prepare for access
- http://localhost:3355/configlnfo

The client 3355 accesses SpringCloud Config3344 successfully. The configuration information can be obtained through GitHub at any time

Dynamic refresh of distributed configuration

Linux operation and maintenance modifies the content of the configuration file on GitHub to make adjustments
Refresh 3344, and the ConfigServer configuration center responds immediately
Refresh 3355 and find no response from ConfigClient client
3355 does not change unless you restart or reload
It is so difficult that every time the O & M modifies the configuration file, the client needs to restart

77_ Manual version of config dynamic refresh

Avoid restarting the client microservice 3355 every time the configuration is updated

Dynamic refresh steps:

Modify 3355 module

POM introduces actor monitoring

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Modify YML and add exposure monitoring port configuration:

# Exposure monitoring endpoint
management:
  endpoints:
    web:
      exposure:
        include: "*"

@RefreshScope business class Controller modification

import org.springframework.cloud.context.config.annotation.RefreshScope;
...

@RestController
@RefreshScope//<-----
public class ConfigClientController
{
...
}

test

At this time, modify the contents of github configuration file - > access 3344 - > access 3355

http://localhost:3355/configInfo

3355 no change, there is still a step to be taken

How

The operation and maintenance personnel need to send a Post request to refresh 3355

curl -X POST "http://localhost:3355/actuator/refresh"

Retest

http://localhost:3355/configInfo

3355 no change.

The client 3355 is successfully refreshed to the latest configuration content, avoiding service restart

Think about what else

Suppose there are multiple microservice clients 3355/3366/3377
Each micro service needs to execute a post request and refresh manually
Can you broadcast it? One notice will take effect everywhere
We want a large-scale automatic refresh, and find a method

78_ What is bus message bus

Part I - deepening and expansion of explanation

In a word, the distributed automatic refresh configuration function.

Spring Cloud Bus can be used with Spring Cloud Config to dynamically refresh the configuration.

What is it?

Spring Cloud Bus can be used with Spring Cloud Config to dynamically refresh the configuration.

Spring Cloud Bus is a framework used to link the nodes of distributed systems with lightweight messaging systems. It integrates the event processing mechanism of Java and the functions of messaging middleware. Spring Cloud Bus currently supports RabbitMQ and Kafka.

What can I do

Spring Cloud Bus can manage and disseminate messages between distributed systems, just like a distributed actuator, which can be used to broadcast state changes, push events, etc., and can also be used as a communication channel between microservices.

Why is it called bus

What is a bus

In microservice architecture systems, lightweight message brokers are usually used to build a common message topic and connect all microservice instances in the system. Since the messages generated in this topic will be monitored and consumed by all instances, it is called a message bus. Each instance on the bus can easily broadcast some messages that need to be known by other instances connected to the subject.

Basic principles

ConfigClient instances listen to the same Topic in MQ (spring cloud bus by default). When a service refreshes data, it will put this information into the Topic, so that other services listening to the same Topic can be notified, and then update their own configuration.

79_ RabbitMQ environment configuration of bus

Install Erlang, download address: http://erlang.org/download/otp_win64_21.3.exe
Install RabbitMQ, download address: https://github.com/rabbitmq/rabbitmq-server/releases/download/v3.8.3/rabbitmq-server-3.8.3.exe
Open cmd and enter the SBIN directory under the RabbitMQ installation directory, such as D:devSoftRabbitMQ Scrverk abbitmq_server-3.7.14sbin
Enter the following command to start the management function

rabbitmq-plugins enable rabbitmq _management

This allows you to add visual plug-ins.

Visit the address to check whether the installation is successful: http://localhost:15672/
Enter the account and password and log in: guest guest

80_ Design idea and type selection of bus dynamic refresh global broadcast

You must have a good RabbitMQ environment first

Demonstrate the broadcast effect, increase the complexity, and then make another 3366 with 3355 as the template

1. Create cloud-config-client-3366

2.POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>LearnCloud</artifactId>
        <groupId>com.lun.springcloud</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-config-client-3366</artifactId>

    <dependencies>
        <!--Add message bus RabbitMQ support-->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-bus-amqp</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

3.YML

server:
  port: 3366

spring:
  application:
    name: config-client
  cloud:
    #Config client configuration
    config:
      label: master #Branch name
      name: config #Profile name
      profile: dev #Read the suffix name. The above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml
      uri: http://localhost:3344 # configuration center address

#rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port accessed by MQ
  rabbitmq:
    host: localhost
    port: 5672
    username: guest
    password: guest

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

# Exposure monitoring endpoint
management:
  endpoints:
    web:
      exposure:
        include: "*"

4. Main startup

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;

@EnableEurekaClient
@SpringBootApplication
public class ConfigClientMain3366
{
    public static void main(String[] args)
    {
        SpringApplication.run(ConfigClientMain3366.class,args);
    }
}

5.controller

import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.context.config.annotation.RefreshScope;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

/**
 */
@RestController
@RefreshScope
public class ConfigClientController
{
    @Value("${server.port}")
    private String serverPort;

    @Value("${config.info}")
    private String configInfo;

    @GetMapping("/configInfo")
    public String configInfo()
    {
        return "serverPort: "+serverPort+"	

 configInfo: "+configInfo;
    }

}

design idea

1. Use the message bus to trigger a client /bus/refresh, and refresh the configuration of all clients

2. Use the message bus to trigger the /bus/refresh endpoint of a server ConfigServer, and refresh the configuration of all clients

The architecture in Figure 2 is obviously more suitable. The reasons for the unsuitability are as follows:

It breaks the single responsibility of microservices, because microservices are business modules, which should not bear the responsibility of configuration refresh.
It destroys the peer-to-peer of micro service nodes.
There are certain limitations. For example, when a microservice is migrated, its network address often changes. At this time, if you want to refresh automatically, you will add more modifications.

81_ Implementation of bus dynamic refresh global broadcast configuration

Add message bus support to cloud-config-center-3344 configuration center server

POM

<!--Add message bus RabbitNQ support-->
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-bus-amap</artifactId>
</dependency>
<dependency>
	<groupId>org-springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

YML

server:
  port: 3344

spring:
  application:
    name:  cloud-config-center #Microservice name registered into Eureka server
  cloud:
    config:
      server:
        git:
          uri: git@github.com:zzyybs/springcloud-config.git #git warehouse name on GitHub
        ####search for directory
          search-paths:
            - springcloud-config
      ####Read branch
      label: master
#rabbitmq related configurations<--------------------------
rabbitmq:
    host: localhost
    port: 5672
    username: guest
    password: guest

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

##rabbitmq related configuration, exposing the endpoint of bus refresh configuration<--------------------------
management:
  endpoints: #Expose the endpoint of the bus refresh configuration
    web:
      exposure:
        include: 'bus-refresh'

Add message bus support to cloud-config-client-3355 client

POM

<!--Add message bus RabbitNQ support-->
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-bus-amap</artifactId>
</dependency>
<dependency>
	<groupId>org-springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

YML

server:
  port: 3355

spring:
  application:
    name: config-client
  cloud:
    #Config client configuration
    config:
      label: master #Branch name
      name: config #Profile name
      profile: dev #Read the suffix name. The above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml
      uri: http://localhost:3344 # configuration center address k

#rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port accessed by MQ<----------------------
  rabbitmq:
    host: localhost
    port: 5672
    username: guest
    password: guest

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

# Exposure monitoring endpoint
management:
  endpoints:
    web:
      exposure:
        include: "*"

Add message bus support to cloud-config-client-3366 client

POM

<!--Add message bus RabbitNQ support-->
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-bus-amap</artifactId>
</dependency>
<dependency>
	<groupId>org-springframework.boot</groupId>
	<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

YML

server:
  port: 3366

spring:
  application:
    name: config-client
  cloud:
    #Config client configuration
    config:
      label: master #Branch name
      name: config #Profile name
      profile: dev #Read the suffix name. The above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml
      uri: http://localhost:3344 # configuration center address

#rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port accessed by MQ<-----------------------
  rabbitmq:
    host: localhost
    port: 5672
    username: guest
    password: guest

#Service registration to eureka address
eureka:
  client:
    service-url:
      defaultZone: http://localhost:7001/eureka

# Exposure monitoring endpoint
management:
  endpoints:
    web:
      exposure:
        include: "*"

test

start-up
- EurekaMain7001
- ConfigcenterMain3344
- ConfigclientMain3355
- ConfigclicntMain3366
Operation and maintenance engineer
- Modify the content of the configuration file on Github and increase the version number
- Send POST request
  - curl -X POST "http://localhost:3344/actuator/bus-refresh"
  - Send once, take effect everywhere
Configuration center
- http://config-3344.com:3344/config-dev.yml
client
- http://localhost:3355/configlnfo
- http://localhost:3366/configInfo
- Get the configuration information and find that it has been refreshed

Once modified, broadcast notice, effective everywhere

82_Bus dynamic refresh fixed-point notification

I don't want to be notified all the time, I just want to be notified at a fixed point

Notice only 3355
No notice 3366

In a simple sentence - specify a specific instance to take effect instead of all

Formula: http://localhost:3344/actuator/bus-refresh/{destination}
/The bus/refresh request is no longer sent to the specific service instance, but to the config server. Specify the service or instance that needs to update the configuration through the destination parameter class

case

Here we take refreshing the config client (application name set in the configuration file) running on port 3355 as an example, only notify 3355, not 3366
curl -X POST "http://localhost:3344/actuator/bus-refresh/config-client:3355

Notification summary

83_ Why is stream introduced

Common MQ (Message Oriented Middleware):

ActiveMQ
RabbitMQ
RocketMQ
Kafka

Is there a new technology that makes us no longer pay attention to the details of specific MQ? We just need to use an adaptive binding method to automatically switch between various MQ. (similar to Hibernate)

What is Cloud Stream? Shield the differences between the underlying message middleware, reduce the switching cost, and unify the programming model of the message.

84_ What is stream and Binder introduction

Official document 1

Official document 2

Cloud Stream Chinese instruction manual

What is Spring Cloud Stream?

Officially, Spring Cloud Stream is a framework for building message driven microservices.

The application interacts with the binder object in Spring Cloud Stream through inputs or outputs.

We configure binding, and the binder object of Spring Cloud Stream is responsible for interacting with the message middleware. Therefore, we only need to figure out how to interact with Spring Cloud Stream to facilitate the use of message driven methods.

By using Spring Integration to connect the message broker middleware to realize message event driven.
Spring Cloud Stream provides personalized automatic configuration implementation for some suppliers' message oriented middleware products, referencing the three core concepts of publish subscribe, consumer group and partition.

Currently, only RabbitMQ and Kafka are supported.

85_ Design idea of stream

Standard MQ

Producers / consumers rely on message media to transmit information content
Messages must go through a specific channel - Message Channel
How are messages in the message channel consumed? Who is responsible for sending and receiving? SubscribableChannel, the sub interface of the message channel MessageChannel, is subscribed by the MessageHandler message processor.

Why use Cloud Stream?

For example, we use RabbitMQ and Kafka. Due to the different architectures of these two message oriented middleware, RabbitMQ has exchange and Kafka has Topic and Partitions.

The differences of these middleware lead to some problems in our actual project development. If we use one of the two message queues, and the business requirements behind it, I want to migrate to the other message queue. At this time, it is undoubtedly a disaster. A lot of things have to be pushed down and redone, because it is coupled with our system, At this time, Spring Cloud Stream provides us with a way to decouple.

Why can Stream unify the underlying differences?

Without the concept of binder, when our SpringBoot application wants to directly interact with the message oriented middleware, due to the different original intentions of the message oriented middleware, there will be great differences in their implementation details. By defining the binder as the intermediate layer, we can perfectly realize the isolation between the application and the message oriented middleware details. By exposing the unified Channel channel to the application, the application does not need to consider a variety of different message oriented middleware implementations.

By defining Binder as the middle layer, the isolation between application and message middleware details is realized.

Binder:

INPUT corresponds to the consumer
OUTPUT corresponds to the producer

The message communication mode in Stream follows the publish subscribe mode

Topic topic broadcast

RabbitMQ is Exchange
In Kakfa, it is Topic

86_ Introduction to common annotations of stream coding

Spring Cloud Stream standard process routine

Binder - very convenient connection middleware, shielding differences.
Channel - channel is an abstraction of Queue. In the message communication system, it is the medium to realize storage and forwarding. The Queue is configured through channel.
Source and Sink - simply understood as the reference object is the Spring Cloud Stream itself. Publishing messages from the Stream is output, and receiving messages is input.

Coding API and common annotations

form

explain

Middleware

Middleware, currently only supports RabbitMQ and Kafka

Binder

Binder is a package between applications and message oriented middleware. At present, binder of Kafka and RabbitMQ are implemented. Binder can easily connect middleware and dynamically change message types (corresponding to topic of Kafka and exchange of RabbitMQ). These can be achieved through configuration files

@Input

The annotation identifies the input channel through which messages received enter the application

@Output

The annotation identifies the output channel through which the published message will leave the application

@StreamListener

Listen to the queue, which is used to receive messages from the consumer's queue

@EnableBinding

Refers to the binding of channel and exchange

Case description

Prepare the RabbitMQ environment( 79_ RabbitMQ environment configuration of bus Mentioned)

Three new sub modules in the project

Cloud stream rabbitmq provider8801, as a producer, sends messages
Cloud stream rabbitmq consumer 8802, as a message receiving module
Cloud stream rabbitmq consumer 8803, as a message receiving module

87_Stream message driven producer

Create a new Module: cloud stream rabbitmq provider8801

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>LearnCloud</artifactId>
        <groupId>com.lun.springcloud</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-stream-rabbitmq-provider8801</artifactId>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-rabbit</artifactId>
        </dependency>
        <!--Basic configuration-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

YML

server:
  port: 8801

spring:
  application:
    name: cloud-stream-provider
  cloud:
      stream:
        binders: # Configure the service information of rabbitmq to be bound here;
          defaultRabbit: # Represents the name of the definition, which is used for binding integration
            type: rabbit # Message component type
            environment: # Set the related environment configuration of rabbitmq
              spring:
                rabbitmq:
                  host: localhost
                  port: 5672
                  username: guest
                  password: guest
        bindings: # Integration of services
          output: # This name is the name of a channel
            destination: studyExchange # Indicates the Exchange name definition to use
            content-type: application/json # Set the message type, json this time, and set "text/plain" for text
            binder: defaultRabbit # Set the specific settings of the message service to be bound

eureka:
  client: # Configuration of Eureka registration by the client
    service-url:
      defaultZone: http://localhost:7001/eureka
  instance:
    lease-renewal-interval-in-seconds: 2 # Set the heartbeat interval (the default is 30 seconds)
    lease-expiration-duration-in-seconds: 5 # If the interval exceeds 5 seconds now (the default is 90 seconds)
    instance-id: send-8801.com  # Display host name in information list
    prefer-ip-address: true     # The access path becomes an IP address

Main startup class StreamMQMain8801

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class StreamMQMain8801 {
    public static void main(String[] args) {
        SpringApplication.run(StreamMQMain8801.class,args);
    }
}

Business class

1. Send message interface

public interface IMessageProvider {
    public String send();
}

2. Send message interface implementation class

import com.lun.springcloud.service.IMessageProvider;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.messaging.Source;
import org.springframework.integration.support.MessageBuilder;
import org.springframework.messaging.MessageChannel;

import javax.annotation.Resource;
import java.util.UUID;


@EnableBinding(Source.class) //Define the push pipeline for messages
public class MessageProviderImpl implements IMessageProvider
{
    @Resource
    private MessageChannel output; // Message sending pipeline

    @Override
    public String send()
    {
        String serial = UUID.randomUUID().toString();
        output.send(MessageBuilder.withPayload(serial).build());
        System.out.println("*****serial: "+serial);
        return null;
    }
}

3.Controller

import com.lun.springcloud.service.IMessageProvider;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import javax.annotation.Resource;

@RestController
public class SendMessageController
{
    @Resource
    private IMessageProvider messageProvider;

    @GetMapping(value = "/sendMessage")
    public String sendMessage() {
        return messageProvider.send();
    }

}

test

Start 7001eureka
Start rabbitmq( 79_ RabbitMQ environment configuration of bus)
- rabbitmq-plugins enable rabbitmq_management
- http://localhost:15672/
Start 8801
Visit- http://localhost:8801/sendMessage
- serial: UUID string will be printed in the background

88_Stream message driven consumers

Create a new Module: cloud stream rabbitmq consumer 8802

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>LearnCloud</artifactId>
        <groupId>com.lun.springcloud</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloud-stream-rabbitmq-consumer8802</artifactId>
    
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-stream-rabbit</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!--Basic configuration-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

YML

server:
  port: 8802

spring:
  application:
    name: cloud-stream-consumer
  cloud:
      stream:
        binders: # Configure the service information of rabbitmq to be bound here;
          defaultRabbit: # Represents the name of the definition, which is used for binding integration
            type: rabbit # Message component type
            environment: # Set the related environment configuration of rabbitmq
              spring:
                rabbitmq:
                  host: localhost
                  port: 5672
                  username: guest
                  password: guest
        bindings: # Integration of services
          input: # This name is the name of a channel
            destination: studyExchange # Indicates the Exchange name definition to use
            content-type: application/json # Set the message type. This time, it is an object json. If it is text, set "text/plain"
            binder: defaultRabbit # Set the specific settings of the message service to be bound

eureka:
  client: # Configuration of Eureka registration by the client
    service-url:
      defaultZone: http://localhost:7001/eureka
  instance:
    lease-renewal-interval-in-seconds: 2 # Set the heartbeat interval (the default is 30 seconds)
    lease-expiration-duration-in-seconds: 5 # If the interval exceeds 5 seconds now (the default is 90 seconds)
    instance-id: receive-8802.com  # Display host name in information list
    prefer-ip-address: true     # The access path becomes an IP address

Main startup class StreamMQMain8802

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class StreamMQMain8802 {
    public static void main(String[] args) {
        SpringApplication.run(StreamMQMain8802.class,args);
    }
}

Business class

import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;
import org.springframework.messaging.Message;
import org.springframework.stereotype.Component;


@Component
@EnableBinding(Sink.class)
public class ReceiveMessageListenerController
{
    @Value("${server.port}")
    private String serverPort;


    @StreamListener(Sink.INPUT)
    public void input(Message<String> message)
    {
        System.out.println("Consumer 1,----->Received message: "+message.getPayload()+"	  port: "+serverPort);
    }
}

test

Start EurekaMain7001
Start streamqmain8801
Start streamqmain8802
8801 send 8802 receive message

89_Stream message repeated consumption

According to 8802, clone a copy of 8803 - cloud stream rabbitmq consumer 8803.

start-up

RabbitMQ
Service registration - 8801
Message production - 8801
Message consumption - 8802
Message consumption - 8802

There are two problems after operation

There is a problem of repeated consumption
Message persistence problem

consumption

http://localhost:8801/sendMessage
At present, 8802/8803 have been received at the same time, and there is a problem of repeated consumption
How to solve: grouping and persistent attribute group (important)

Actual production cases

For example, in the following scenario, when we deploy the order system in a cluster, we will get the order information from RabbitMQ. If an order is obtained by two services at the same time, it will cause data errors. We have to avoid this situation. At this time, we can use the message grouping in the Stream to solve the problem.

Note that multiple consumers in the same group in the Stream are in a competitive relationship, which can ensure that messages will only be consumed once by one of the applications. Different groups can be fully consumed (repeated consumption).

90_Stream's group solves message repeated consumption

principle

If microservice applications are placed in the same group, it can ensure that messages will only be consumed by one of them once.

Different groups can be consumed repeatedly. There will be competition within the same group, and only one of them can be consumed.

8802/8803 become different groups, and the two groups are different

group: A_Group,B_Group

8802 modify YML

spring:
  application:
    name: cloud-stream-provider
  cloud:
      stream:
        binders: # Configure the service information of rabbitmq to be bound here;
          defaultRabbit: # Represents the name of the definition, which is used for binding integration
            type: rabbit # Message component type
            environment: # Set the related environment configuration of rabbitmq
              spring:
                rabbitmq:
                  host: localhost
                  port: 5672
                  username: guest
                  password: guest
        bindings: # Integration of services
          output: # This name is the name of a channel
            destination: studyExchange # Indicates the Exchange name definition to use
            content-type: application/json # Set the message type, json this time, and set "text/plain" for text
            binder: defaultRabbit # Set the specific settings of the message service to be bound
            group: A_Group #< ------------------------------------ key

8803 modify YML (similar to 8802, group: B_Group)

Conclusion: repeated consumption

8802/8803 implements polling packets, with only one consumer at a time. Messages sent by 8801 module can only be received by one of 8802 or 8803, which avoids repeated consumption.

8802/8803 become the same group, and the two groups are the same

group: A_Group

8802 modify YMLgroup: A_Group

8803 modify YMLgroup: A_Group

Conclusion: for multiple instances of microservice in the same group, only one will get it at a time

91_ Message persistence of stream

Through the above, we have solved the problem of repeated consumption, and then look at persistence.

Stop 8802/8803 and remove the group of 8802 group: A_Group, group of 8803 group: A_Group is not removed.

8801 sends 4 messages to RabbitMq first.

Start 8802 first, there is no grouping attribute configuration, and there is no message printed in the background.

Start 8803 again, there is grouping attribute configuration, and messages on MQ are printed in the background. (message persistence embodiment)

92_ What is sleuth

Why does this technology appear? What problems should be solved?

In the microservice framework, a request initiated by the client will be called by multiple different service nodes in the back-end system to produce the final request result. Each previous request will form a complex distributed service call link. High delay or error in any link will cause the final failure of the entire request.

What is it?

https://github.com/spring-cloud/spring-cloud-sleuth
Spring Cloud Sleuth provides a complete solution for service tracking
It provides tracking solutions in distributed systems and supports zipkin

solve

sleuth
English [slu θ] Beauty [slu θ]
n. Detective

93_Sleuth's zipkin setup and installation

1.zipkin

download

SpringCloud does not need to build Zipkin Server by itself since version F, just call jar package
https://dl.bintray.com/openzipkin/maven/io/zipkin/java/zipkin-server/
zipkin-server-2.12.9-exec.jar

Run jar

java -jar zipkin-server-2.12.9-exec.jar

Run console

http://localhost:9411/zipkin/

term

Complete call link

Indicates a request link. A link is uniquely identified by Trace ld, and span identifies the request information initiated. Each span is associated by parent id

A link is uniquely identified by Trace ld, span identifies the request information initiated, and each span is associated by parent id.

The dependencies of the whole link are as follows:

Explanation of terms

Trace: a Span set similar to a tree structure, representing a call link with a unique identifier
Span: refers to the source of the calling link. Generally speaking, span is a request for information

94_Sleuth link monitoring display

2. Service provider

cloud-provider-payment8001

POM

<!--Contains sleuth+zipkin-->
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-zipkin</artifactId>
</dependency>

YML

spring:
  application:
    name: cloud-payment-service

  zipkin: #< --------------------------------- key 
      base-url: http://localhost:9411
  sleuth: #< --------------------------------- key
    sampler:
    #The sampling rate value is between 0 and 1, and 1 means all acquisitions
    probability: 1
    
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource            # Current data source operation type
    driver-class-name: org.gjt.mm.mysql.Driver              # mysql driver package
    url: jdbc:mysql://localhost:3306/db2019?useUnicode=true&characterEncoding=utf-8&useSSL=false
    username: root
    password: 123456

Business PaymentController

@RestController
@Slf4j
public class PaymentController {
    
    ...
    
 	@GetMapping("/payment/zipkin")
    public String paymentZipkin() {
        return "hi ,i'am paymentzipkin server fall back，welcome to here, O(∩_∩)O ha-ha~";
    }    
}

3. Service consumer (caller)

cloue-consumer-order80

POM

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-zipkin</artifactId>
</dependency>

YML

spring:
    application:
        name: cloud-order-service
    zipkin:
      base-url: http://localhost:9411
    sleuth:
      sampler:
        probability: 1

Business class OrderController

    // ====================> zipkin+sleuth
    @GetMapping("/consumer/payment/zipkin")
    public String paymentZipkin()
    {
        String result = restTemplate.getForObject("http://localhost:8001"+"/payment/zipkin/", String.class);
        return result;
    }
}

4. Start eureka7001/8001/80 - 80 successively and call 8001 for several tests

5. Open the browser to access: http://localhost:9411

95_ Introduction to cloud Alibaba

Why does SpringCloud alibaba appear

Spring Cloud Netflix project enters maintenance mode

https://spring.io/blog/2018/12/12/spring-cloud-greenwich-rc1-available-now

What is maintenance mode?

Putting the module in maintenance mode means that the Spring Cloud team will not add new functions to the module.

They will fix block level bug s and security issues, and they will also consider and review small pull request s from the community.

What does SpringCloud alibaba bring

What is it?

Official website

Spring Cloud Alibaba is committed to providing a one-stop solution for microservice development. This project includes the necessary components for developing distributed application microservices, so that developers can easily use these components to develop distributed application services through the Spring Cloud programming model.

Relying on Spring Cloud Alibaba, you only need to add some annotations and a few configurations to connect Spring Cloud applications to Alibaba microservice solutions and quickly build distributed application systems through Alibaba middleware.

Birth: on October 31, 2018, Spring Cloud Alibaba officially entered the official incubator of Spring Cloud and released its first version in Maven central library.

What can I do

Service flow restriction and degradation: by default, it supports the access of WebServlet, WebFlux, OpenFeign, RestTemplate, Spring Cloud Gateway, Zuul, Dubbo and RocketMQ flow restriction and degradation functions. It can modify the flow restriction and degradation rules in real time through the console at runtime, and it also supports the viewing of flow restriction and degradation Metrics monitoring.
Service registration and discovery: it adapts to the Spring Cloud service registration and discovery standard, and integrates the support of Ribbon by default.
Distributed configuration management: it supports the external configuration in the distributed system and automatically refreshes when the configuration is changed.
Message driven capabilities: build message driven capabilities for microservice applications based on Spring Cloud Stream.
Distributed transaction: use the @GlobalTransactional annotation to solve the problem of distributed transaction efficiently and without business intrusion.
Alibaba cloud object storage: Alibaba cloud provides massive, safe, low-cost, and highly reliable cloud storage services. It supports the storage and access of any type of data in any application, at any time and anywhere.
Distributed task scheduling: provide second level, accurate, highly reliable and highly available scheduled (based on Cron expression) task scheduling services. At the same time, it provides distributed task execution models, such as grid tasks. Grid tasks support the even distribution of Hai quantum tasks to all workers (schedulerx clients) for execution.
Alibaba cloud SMS service: a global SMS service with friendly, efficient and intelligent interconnected communication capabilities helps enterprises quickly build customer access channels.

Where to

If you need to use a published version, add the following configuration in dependencyManagement.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-alibaba-dependencies</artifactId>
            <version>2.2.5.RELEASE</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Then add the dependencies you need to use in dependencies.

How do you play?

Sentinel : take the flow as the starting point to protect the stability of services from multiple dimensions, such as flow control, fuse degradation, system load protection, etc.
Nacos : a dynamic service discovery, configuration management and service management platform that is easier to build cloud native applications.
RocketMQ : an open source distributed messaging system, based on highly available distributed cluster technology, provides low latency, highly reliable message publishing and subscription services.
Dubbo : Apache Dubbo is a high-performance Java RPC framework.
Seata : Alibaba open source product, an easy-to-use high-performance distributed transaction solution for microservices.
Alibaba Cloud OSS : Alibaba cloud Object Storage Service (OSS) is a massive, secure, low-cost, and highly reliable cloud storage service provided by Alibaba cloud. You can store and access any type of data in any application, anytime, anywhere.
Alibaba Cloud SchedulerX : a distributed task scheduling product developed by Alibaba middleware team provides second level, accurate, highly reliable and highly available timed (based on Cron expression) task scheduling services.
Alibaba Cloud SMS : SMS services covering the world, friendly, efficient and intelligent interconnected communication capabilities, help enterprises quickly build customer access channels.

Spring Cloud Alibaba learning materials acquisition

Official website
- https://spring.io/projects/spring-cloud-alibaba#overview
english
- https://github.com/alibaba/spring-cloud-alibaba
- https://spring-cloud-alibaba-group.github.io/github-pages/greenwich/spring-cloud-alibaba.html
chinese
- https://github.com/alibaba/spring-cloud-alibaba/blob/master/README-zh.md

96_Nacos introduction and download

Why Nacos

The first four letters are the first two letters of Naming and Configuration respectively, and the last s is Service.

What is it?

A dynamic service discovery, configuration management and service management platform that is easier to build cloud native applications.
Nacos: Dynamic Naming and Configuration Service
Nacos is the combination of registration center + configuration center - > Nacos = Eureka + config + bus

What can I do

Replace Eureka as the service registration center
Replace Config as the service configuration center

Where to

https://github.com/alibaba/nacos/releases
Official website documents

Comparison of registration centers in China

Service registration and discovery framework

CAP model

Console management

Community activity

Eureka

support

Low (2.x version closed source)

Zookeeper

I won't support it

consul

support

high

Nacos

support

high

It is said that Nacos has more than 100000 instances running in Alibaba, and has passed the test of various large-scale traffic such as the double 11.

97_Nacos installation

The local Java8+Maven environment has been OK first
from Official website Download Nacos
Unzip the installation package and directly run startup.cmd under the bin directory
Access directly after the command runs successfully http://localhost:8848/nacos The default account and password are all Nacos
Results page

98_ Service provider registration of Nacos

Official documents

New module - cloudalibaba provider payment9001

POM

Parent POM

<dependencyManagement>
    <dependencies>
        <!--spring cloud alibaba 2.1.0.RELEASE-->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-alibaba-dependencies</artifactId>
            <version>2.1.0.RELEASE</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

POM of this module

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud2020</artifactId>
        <groupId>com.atguigu.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloudalibaba-provider-payment9001</artifactId>

    <dependencies>
        <!--SpringCloud ailibaba nacos -->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
        </dependency>
        <!-- SpringBoot integration Web assembly -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!--Daily general jar Package configuration-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

YML

server:
  port: 9001

spring:
  application:
    name: nacos-payment-provider
  cloud:
    nacos:
      discovery:
        server-addr: localhost:8848 #Configure Nacos address

management:
  endpoints:
    web:
      exposure:
        include: '*'

Main start

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;

@EnableDiscoveryClient
@SpringBootApplication
public class PaymentMain9001 {
    public static void main(String[] args) {
            SpringApplication.run(PaymentMain9001.class, args);
    }
}

Business class

import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class PaymentController {
    @Value("${server.port}")
    private String serverPort;

    @GetMapping(value = "/payment/nacos/{id}")
    public String getPayment(@PathVariable("id") Integer id) {
        return "nacos registry, serverPort: "+ serverPort+"	 id"+id;
    }
}

test

http://localhost:9001/payment/nacos/1
nacos console
nacos service registry + service provider 9001 are all OK

In order to demonstrate the load balancing of nacos in the next chapter, refer to 9001 and create 9002

Create cloudalibaba provider payment9002
9002 other steps you know
Or, if you don't want to create repetitive physical labor, you can use the IDEA function to directly copy the virtual port mapping

99_ Service consumer registration and load of Nacos

New module - cloudalibaba consumer Nacos order83

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>LearnCloud</artifactId>
        <groupId>com.lun.springcloud</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloudalibaba-consumer-nacos-order83</artifactId>

    <dependencies>
        <!--SpringCloud ailibaba nacos -->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
        </dependency>
        <!-- Introduce self defined api General package, which can be used Payment payment Entity -->
        <dependency>
            <groupId>com.lun.springcloud</groupId>
            <artifactId>cloud-api-commons</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>
        <!-- SpringBoot integration Web assembly -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!--Daily general jar Package configuration-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

</project>

Why does nacos support load balancing? Because spring cloud starter Alibaba nacos discovery contains Netflix ribbon package.

YML

server:
  port: 83

spring:
  application:
    name: nacos-order-consumer
  cloud:
    nacos:
      discovery:
        server-addr: localhost:8848

#The name of the micro service that the consumer will visit (the micro service provider that successfully registered into nacos)
service-url:
  nacos-user-service: http://nacos-payment-provider

Main start

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;


@EnableDiscoveryClient
@SpringBootApplication
public class OrderNacosMain83
{
    public static void main(String[] args)
    {
        SpringApplication.run(OrderNacosMain83.class,args);
    }
}

Business class

ApplicationContextConfig

import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;

@Configuration
public class ApplicationContextConfig
{
    @Bean
    @LoadBalanced
    public RestTemplate getRestTemplate()
    {
        return new RestTemplate();
    }
}

OrderNacosController

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;

import javax.annotation.Resource;

@RestController
@Slf4j
public class OrderNacosController {
    
    @Resource
    private RestTemplate restTemplate;

    @Value("${service-url.nacos-user-service}")
    private String serverURL;

    @GetMapping(value = "/consumer/payment/nacos/{id}")
    public String paymentInfo(@PathVariable("id") Long id)
    {
        return restTemplate.getForObject(serverURL+"/payment/nacos/"+id,String.class);
    }

}

test

Launch the nacos console
http://localhost:83/Eonsumer/payment/nacos/13
- 83 visit 9001/9002 and poll the load OK

100_ Comparison and improvement of Nacos service registration center

Nacos panorama

Nacos and CAP

Comparison between Nacos and other registries

Nacos service discovery instance model

Nacos supports the switching between AP and CP modes

C is that all nodes see the same data at the same time; The definition of A is that all requests will receive A response.

When to choose which mode to use

Generally speaking, if you don't need to store service level information and the service instance is registered through Nacos client and can maintain heartbeat reporting, you can choose AP mode. The current mainstream services, such as Spring cloud and Dubbo services, are applicable to the AP mode. The AP mode weakens the consistency for the possibility of services, so only temporary instances can be registered under the AP mode.

If you need to edit or store configuration information at the service level, CP is required, and K8S service and DNS service are applicable to CP mode. In CP mode, persistent instance registration is supported. At this time, the Raft protocol is used as the cluster operation mode. In this mode, the service must be registered before registering the instance. If the service does not exist, an error will be returned.

Switching command:

curl -X PUT '$NACOS_SERVER:8848/nacos/v1/ns/operator/switches?entry=serverMode&value=CP

101_ Service configuration center of Nacos

Basic configuration

cloudalibaba-config-nacos-client3377

POM

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>cloud2020</artifactId>
        <groupId>com.atguigu.springcloud</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>cloudalibaba-config-nacos-client3377</artifactId>

    <dependencies>
        <!--nacos-config-->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
        </dependency>
        <!--nacos-discovery-->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
        </dependency>
        <!--web + actuator-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!--General basic configuration-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

YML

Nacos is the same as springcloud config. During the initialization of the project, it is necessary to pull the configuration from the configuration center first. After pulling the configuration, the normal startup of the project can be guaranteed.

There is a priority order for loading configuration files in springboot, and the priority of bootstrap is higher than that of application

bootstrap

# nacos configuration
server:
  port: 3377

spring:
  application:
    name: nacos-config-client
  cloud:
    nacos:
      discovery:
        server-addr: localhost:8848 #Nacos service registry address
      config:
        server-addr: localhost:8848 #Nacos as the configuration center address
        file-extension: yaml #Specify the configuration of yaml format
        group: DEV_GROUP
        namespace: 7d8f0f5a-6a53-4785-9686-dd460158e5d4


# ${spring.application.name}-${spring.profile.active}.${spring.cloud.nacos.config.file-extension}
# nacos-config-client-dev.yaml

# nacos-config-client-test.yaml   ----> config.info

application

spring:
  profiles:
    active: dev # Represents the development environment
    #active: test # Represents the test environment
    #active: info

Main start

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;


@EnableDiscoveryClient
@SpringBootApplication
public class NacosConfigClientMain3377
{
    public static void main(String[] args) {
        SpringApplication.run(NacosConfigClientMain3377.class, args);
    }
}

Business class

import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.context.config.annotation.RefreshScope;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RefreshScope //Support the dynamic refresh function of Nacos.
public class ConfigClientController
{
    @Value("${config.info}")
    private String configInfo;

    @GetMapping("/config/info")
    public String getConfigInfo() {
        return configInfo;
    }
}

Add configuration information in Nacos

The composition format of the dataid in Nacos and the matching rules with the SpringBoot configuration file

Official documents

Note: the reason why spring.application Name, because it is part of the Nacos configuration management dataId field.

In Nacos Spring Cloud, the complete format of dataId is as follows:

${prefix}-${spring-profile.active}.${file-extension}

Prefix defaults to the value of spring.application.name, which can also be configured through the configuration item spring.cloud.nacos.config.prefix.
spring.profile.active is the profile corresponding to the current environment. For details, please refer to the Spring Boot documentation. Note: when spring.profile When active is empty, the corresponding connector - will also not exist, and the splicing format of datald will become ${prefix}.${file extension}
File exception is the data format of the configuration content, which can be configured through the configuration item spring.Cloud.nacos.config.file-extension. Currently, only properties and yaml types are supported.
The configuration is automatically updated through the Spring Cloud native annotation @RefreshScope.

Final formula:

${spring.application.name)}-${spring.profiles.active}.${spring.cloud.nacos.config.file-extension}

Configuration addition

Nacos interface configuration correspondence - set DataId

Configuration summary

test

Before starting, you need to have the corresponding yaml configuration file under the column of nacos client - Configuration Management - configuration management
Run the main startup class of cloud config Nacos client3377
Call the interface to view the configuration information- http://localhost:3377/config/info

Built in dynamic refresh

Modify the yaml configuration file in Nacos, call the interface to view the configuration again, and you will find that the configuration has been refreshed.

102_ The relationship between namespace grouping and DataID of Nacos

Problem - Multi environment and multi project management

Question 1:

In actual development, usually a system will prepare

dev development environment
Test test environment
prod production environment.

How to ensure that the service can correctly read the configuration file of the corresponding environment on Nacos when the specified environment is started

Question 2:

A large distributed microservice system will have many microservice sub projects, and each microservice project will have a corresponding development environment, testing environment, pre launch environment, formal environment... How to manage these microservice configurations

Graphical management interface of Nacos

Namespace+Group+Data lD? Why is it so designed?

What is 1

Similar to the package name and class name in Java, the outermost namespace can be used to distinguish the deployment environment. Group and DatalD logically distinguish two target objects.

2. Conditions of the three parties

DEFAULT: Namespace=public, Group=DEFAULT_GROUP, the DEFAULT Cluster is DEFAULT

The default Namespace of Nacos is public, which is mainly used to realize isolation.
- For example, we now have three environments: development, testing and production. We can create three namespaces, and different namespaces are isolated.
Group defaults to DEFAULT_GROUP, group can divide different micro services into the same group
A Service is a microservice: a Service can contain multiple clusters. The DEFAULT Cluster of Nacos is DEFAULT, and Cluster is a virtual partition of a specified microservice.
- For example, for disaster recovery, the Service microservices are deployed in Hangzhou computer room and Guangzhou computer room respectively. At this time, the Service microservices in Hangzhou computer room can be given a cluster name (HZ), and the Service microservices in Guangzhou computer room can be given a cluster name (GZ). You can also try to make the microservices in the same computer room call each other to improve performance.
Finally, Instance is the Instance of micro service.

103_Nacos' DataID configuration

Specify spring.profile.active and the DatalD of the configuration file to read different configurations in different environments

Default space + default grouping + create two DatalD, dev and test

Create a new dev configuration DatalD

Create a new test configuration DatalD

You can read configuration files in multiple environments through the spring.profile.active attribute

test

http://localhost:3377/config/info
Load test/dev as you configure

104_Nacos' Group grouping scheme

Environment differentiation through groups - create a new Group

Create a new configuration file DatalD on the nacos graphical interface console

bootstrap+application

Add a group configuration under config. Configurable as DEV_GROUP or TEST GROUP

105_Nacos' Namespace scheme

Create a new Namespace for dev/test

Go back to service management - service list view

Fill in according to the domain name configuration

YML

# nacos configuration
server:
  port: 3377

spring:
  application:
    name: nacos-config-client
  cloud:
    nacos:
      discovery:
        server-addr: localhost:8848 #Nacos service registry address
      config:
        server-addr: localhost:8848 #Nacos as the configuration center address
        file-extension: yaml #Specify the configuration of yaml format
        group: DEV_GROUP
        namespace: 7d8f0f5a-6a53-4785-9686-dd460158e5d4 #< ------------ specify a namespace


# ${spring.application.name}-${spring.profile.active}.${spring.cloud.nacos.config.file-extension}
# nacos-config-client-dev.yaml

# nacos-config-client-test.yaml   ----> config.info

106_Nacos cluster_ Architecture description

Official documents

Official website structure diagram

Cluster deployment architecture diagram

Therefore, when open source, users are recommended to put all the service lists under a vip and then hang them under a domain name

http://ip1:port/openAPI In the direct connection IP mode, the machine needs to modify the IP before it can be used.

http://VIP:port/openAPI Mount the VIP mode, directly connect to the VIP, and hang the real ip of the server below, which is not readable.

http://nacos.com:port/openAPI Domain name + VIP mode, good readability, convenient for changing ip, recommended mode

The above official website translation, the real situation

According to the above, we need mysql database.

Official website description

By default, Nacos uses an embedded database to store data. Therefore, if you start multiple Nacos nodes in the default configuration, there is a consistency problem in the data storage. To solve this problem, Nacos adopts centralized storage to support clustered deployment. Currently, it only supports MySQL storage.

Nacos supports three deployment modes

Stand alone mode - used for testing and stand-alone trial.
Cluster mode - used in production environment to ensure high availability.
Multi cluster mode - used in multi data center scenarios.

Windows

cmd startup.cmd or double-click the startup.cmd file

Stand alone mode supports mysql

Before version 0.7, nacos used embedded database to store data in stand-alone mode, which made it inconvenient to observe the basic situation of data storage. Version 0.7 adds the ability to support mysql data sources. Specific operation steps:

Install database, version requirements: 5.6.5+
Initialize mysq database, database initialization file: Nacos mysql.sql
Modify conf/application Properties file, add support for mysql data source configuration (currently only mysql is supported), and Add url, user name and password of mysql data source.

spring.datasource.platform=mysql

db.num=1
db.url.0=jdbc:mysql://11.162.196.16:3306/nacos_devtest?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=nacos_devtest
db.password=youdontknow

Then start nacos in stand-alone mode, and all the data written by nacos to the embedded database is written to mysql.

107_Nacos persistent switching configuration

Nacos comes with an embedded database derby by default, pom.xml of nacos It can be seen from.

Configuration steps for switching from derby to mysql:

nacos-server-1.1.4 acosconf record, find the nacos-mysql.sql file, and execute the script.
Find application.properties in the directory of nacos-server-1.1.4 acosconf, and add the following configuration (modify the corresponding value as needed).

spring.datasource.platform=mysql

db.num=1
db.url.0=jdbc:mysql://localhost:3306/nacos_devtest?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=root
db.password=1234

Start Nacos, and you can see that it is a new empty recording interface, which used to be recorded into derby.

First of all, I would like to introduce myself. I graduated from Jiaotong University in 13 years. I once worked in a small company, went to large factories such as Huawei OPPO, and joined Alibaba in 18 years, until now. I know that most junior and intermediate Java engineers who want to improve their skills often need to explore and grow by themselves or sign up for classes, but there is a lot of pressure on training institutions to pay nearly 10000 yuan in tuition fees. The self-study efficiency of their own fragmentation is very low and long, and it is easy to encounter the ceiling technology to stop. Therefore, I collected a "full set of learning materials for java development" and gave it to you. The original intention is also very simple. I hope to help friends who want to learn by themselves and don't know where to start, and reduce everyone's burden at the same time. Add the business card below to get a full set of learning materials

Tags: Back-end Front-end Android Interview

Posted by silvio on Sun, 31 Jul 2022 01:40:19 +0930

Programs Wiki

Spring Cloud learning notes (23)

56_ Global service degradation of hystrix DefaultProperties

57_ The universal service of hystrix is degraded FeignFallback

58_ Hytrix's service fusing theory

59_ The service fusing case of hystrix (Part 1)

60_ The case of service fusing of hystrix (Part 2)

61_ Summary of service fusing of hystrix

62_ Final summary of hystrix workflow

63_ Establishment of hystrix graphical Dashboard

64_Hystrix graphical Dashboard monitoring practice

65_GateWay and Zuul course description

66_ What is gateway

67_GateWay non blocking asynchronous model

68_Gateway workflow

69_Gateway9527 construction

70_ Two ways for gateway to configure routing

71_GateWay configuration dynamic routing

72_ Commonly used Predicate of gateway

73_ Filter of gateway

74_ Introduction to config distributed configuration center

75_Config configuration master control center setup

76_Config client configuration and testing

77_ Manual version of config dynamic refresh

78_ What is bus message bus

79_ RabbitMQ environment configuration of bus

80_ Design idea and type selection of bus dynamic refresh global broadcast

81_ Implementation of bus dynamic refresh global broadcast configuration

82_Bus dynamic refresh fixed-point notification

83_ Why is stream introduced

84_ What is stream and Binder introduction

85_ Design idea of stream

86_ Introduction to common annotations of stream coding

87_Stream message driven producer

88_Stream message driven consumers

89_Stream message repeated consumption

90_Stream's group solves message repeated consumption

91_ Message persistence of stream

92_ What is sleuth

93_Sleuth's zipkin setup and installation

94_Sleuth link monitoring display

95_ Introduction to cloud Alibaba

96_Nacos introduction and download

97_Nacos installation

98_ Service provider registration of Nacos

99_ Service consumer registration and load of Nacos

100_ Comparison and improvement of Nacos service registration center

101_ Service configuration center of Nacos

102_ The relationship between namespace grouping and DataID of Nacos

103_Nacos' DataID configuration

104_Nacos' Group grouping scheme

105_Nacos' Namespace scheme

106_Nacos cluster_ Architecture description

107_Nacos persistent switching configuration

Popular Tags