Spring Cloud learning notes (1 / 3)
Spring Cloud learning notes (3 / 3)
-
-
-
56_ Global service degradation of hystrix DefaultProperties
57_ The universal service of hystrix is degraded FeignFallback
58_ Hytrix's service fusing theory
59_ The service fusing case of hystrix (Part 1)
60_ The case of service fusing of hystrix (Part 2)
61_ Summary of service fusing of hystrix
62_ Final summary of hystrix workflow
63_ Establishment of hystrix graphical Dashboard
64_Hystrix graphical Dashboard monitoring practice
65_GateWay and Zuul course description
67_GateWay non blocking asynchronous model
70_ Two ways for gateway to configure routing
71_GateWay configuration dynamic routing
72_ Commonly used Predicate of gateway
74_ Introduction to config distributed configuration center
75_Config configuration master control center setup
76_Config client configuration and testing
77_ Manual version of config dynamic refresh
79_ RabbitMQ environment configuration of bus
80_ Design idea and type selection of bus dynamic refresh global broadcast
81_ Implementation of bus dynamic refresh global broadcast configuration
82_Bus dynamic refresh fixed-point notification
84_ What is stream and Binder introduction
86_ Introduction to common annotations of stream coding
87_Stream message driven producer
88_Stream message driven consumers
89_Stream message repeated consumption
90_Stream's group solves message repeated consumption
91_ Message persistence of stream
93_Sleuth's zipkin setup and installation
94_Sleuth link monitoring display
95_ Introduction to cloud Alibaba
96_Nacos introduction and download
98_ Service provider registration of Nacos
99_ Service consumer registration and load of Nacos
100_ Comparison and improvement of Nacos service registration center
101_ Service configuration center of Nacos
102_ The relationship between namespace grouping and DataID of Nacos
103_Nacos' DataID configuration
104_Nacos' Group grouping scheme
106_Nacos cluster_ Architecture description
107_Nacos persistent switching configuration
-
-
56_ Global service degradation of hystrix DefaultProperties
At present, problem 1: each business method corresponds to a thorough method, and the code expands
resolvent
1: 1. Each method is configured with a service degradation method, which is technically OK, but not smart
1:N except for some important core businesses, other ordinary businesses can jump to the unified processing result page through @DefaultProperties(defaultFallback = "")
General and exclusive are separated, which avoids code inflation and reasonably reduces the amount of code
import com.lun.springcloud.service.PaymentHystrixService; import com.netflix.hystrix.contrib.javanica.annotation.DefaultProperties; import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand; import lombok.extern.slf4j.Slf4j; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RestController; import javax.annotation.Resource; @RestController @Slf4j @DefaultProperties(defaultFallback = "payment_Global_FallbackMethod") public class OrderHystirxController { @Resource private PaymentHystrixService paymentHystrixService; @GetMapping("/consumer/payment/hystrix/ok/{id}") public String paymentInfo_OK(@PathVariable("id") Integer id) { String result = paymentHystrixService.paymentInfo_OK(id); return result; } @GetMapping("/consumer/payment/hystrix/timeout/{id}") // @HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod",commandProperties = { // @HystrixProperty(name="execution.isolation.thread.timeoutInMilliseconds",value="1500") // }) @HystrixCommand//Using the global fallback method public String paymentInfo_TimeOut(@PathVariable("id") Integer id) { //int age = 10/0; String result = paymentHystrixService.paymentInfo_TimeOut(id); return result; } public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) { return "I am a consumer 80,The other party's payment system is busy. Please try again in 10 seconds or check yourself if you run an error,o(╥﹏╥)o"; } // The following is the global fallback method public String payment_Global_FallbackMethod() { return "Global Exception handling information, please try again later,/(ㄒoㄒ)/~~"; } }
57_ The universal service of hystrix is degraded FeignFallback
At present, problem 2 is the separation of unification and customization, and the code is chaotic
When the service is degraded, the client calls the server and the server goes down or shuts down
In this case, the service degradation processing is implemented on the client 80, which has nothing to do with the server 8001. It is only necessary to add an implementation class of service degradation processing to the interface defined by Feign client to realize decoupling
The anomalies we will face in the future
- function
- overtime
- Downtime
Modify cloud consumer feign hystrix order80
According to the existing PaymentHystrixService interface of cloud consumer feign hystrix order80,
Create a new class (AaymentFallbackService) to implement the interface, and handle exceptions for the methods in the interface
The PaymentFallbackService class implements the PaymentHystrixService interface
import org.springframework.stereotype.Component; @Component public class PaymentFallbackService implements PaymentHystrixService { @Override public String paymentInfo_OK(Integer id) { return "-----PaymentFallbackService fall back-paymentInfo_OK ,o(╥﹏╥)o"; } @Override public String paymentInfo_TimeOut(Integer id) { return "-----PaymentFallbackService fall back-paymentInfo_TimeOut ,o(╥﹏╥)o"; } }
YML
server: port: 80 eureka: client: register-with-eureka: false service-url: defaultZone: http://eureka7001.com:7001/eureka/ #open feign: hystrix: enabled: true
PaymentHystrixService interface
import org.springframework.cloud.openfeign.FeignClient; import org.springframework.stereotype.Component; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; @Component @FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT" ,// fallback = PaymentFallbackService.class)//Specify PaymentFallbackService class public interface PaymentHystrixService { @GetMapping("/payment/hystrix/ok/{id}") public String paymentInfo_OK(@PathVariable("id") Integer id); @GetMapping("/payment/hystrix/timeout/{id}") public String paymentInfo_TimeOut(@PathVariable("id") Integer id); }
test
A single eureka starts 7001 first
PaymentHystrixMain8001 start
Normal access test- http://localhost/consumer/payment/hystrix/ok/1
Deliberately close microservice 8001
The client calls the prompt itself - at this time, the service provider has been down, but we have done the service degradation process, so that the client will also get the prompt information when the service is unavailable without hanging up and killing the server.
58_ Hytrix's service fusing theory
Circuit breaker, equivalent to fuse.
Overview of circuit breaker mechanism
Circuit breaker mechanism is a microservice link protection mechanism to deal with avalanche effect. When a microservice on the fan out link is unavailable due to an error or the response time is too long, the service will be degraded, which will fuse the call of the microservice on the node and quickly return the wrong response information. When it is detected that the microservice call response of the node is normal, the call link is resumed.
In the Spring Cloud framework, the circuit breaker mechanism is implemented through hystrix. Hystrix will monitor the status of calls between microservices. When the failed calls reach a certain threshold, the default is 20 calls in 5 seconds, and the circuit breaker mechanism will be started. The annotation of circuit breaker mechanism is @HystrixCommand.
Relevant papers of Martin Fowler
59_ The service fusing case of hystrix (Part 1)
Modify cloud provider hystrix payment8001
import cn.hutool.core.util.IdUtil; import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand; import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty; import org.springframework.stereotype.Service; import org.springframework.web.bind.annotation.PathVariable; import java.util.concurrent.TimeUnit; @Service public class PaymentService{ ... //=====Service fusing @HystrixCommand(fallbackMethod = "paymentCircuitBreaker_fallback",commandProperties = { @HystrixProperty(name = "circuitBreaker.enabled",value = "true"),// Whether to open the circuit breaker @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold",value = "10"),// Number of requests @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds",value = "10000"), // Time window period @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage",value = "60"),// Trip after the failure rate reaches }) public String paymentCircuitBreaker(@PathVariable("id") Integer id) { if(id < 0) { throw new RuntimeException("******id Cannot be negative"); } String serialNumber = IdUtil.simpleUUID(); return Thread.currentThread().getName()+" "+"Call successful, serial number: " + serialNumber; } public String paymentCircuitBreaker_fallback(@PathVariable("id") Integer id) { return "id Cannot be negative. Please try again later,/(ㄒoㄒ)/~~ id: " +id; } }
The precise way that the circuit opening and closing occurs is as follows:
- Assuming the volume across a circuit meets a certain threshold : HystrixCommandProperties.circuitBreakerRequestVolumeThreshold()
- And assuming that the error percentage, as defined above exceeds the error percentage defined in : HystrixCommandProperties.circuitBreakerErrorThresholdPercentage()
- Then the circuit-breaker transitions from CLOSED to OPEN.
- While it is open, it short-circuits all requests made against that circuit-breaker.
- After some amount of time (HystrixCommandProperties.circuitBreakerSleepWindowInMilliseconds()), the next request is let through. If it fails, the command stays OPEN for the sleep window. If it succeeds, it transitions to CLOSED and the logic in 1) takes over again.
HystrixCommandProperties configuration class
package com.netflix.hystrix; ... public abstract class HystrixCommandProperties { private static final Logger logger = LoggerFactory.getLogger(HystrixCommandProperties.class); /* defaults */ /* package */ static final Integer default_metricsRollingStatisticalWindow = 10000;// default => statisticalWindow: 10000 = 10 seconds (and default of 10 buckets so each bucket is 1 second) private static final Integer default_metricsRollingStatisticalWindowBuckets = 10;// default => statisticalWindowBuckets: 10 = 10 buckets in a 10 second window so each bucket is 1 second private static final Integer default_circuitBreakerRequestVolumeThreshold = 20;// default => statisticalWindowVolumeThreshold: 20 requests in 10 seconds must occur before statistics matter private static final Integer default_circuitBreakerSleepWindowInMilliseconds = 5000;// default => sleepWindow: 5000 = 5 seconds that we will sleep before trying again after tripping the circuit private static final Integer default_circuitBreakerErrorThresholdPercentage = 50;// default => errorThresholdPercentage = 50 = if 50%+ of requests in 10 seconds are failures or latent then we will trip the circuit private static final Boolean default_circuitBreakerForceOpen = false;// default => forceCircuitOpen = false (we want to allow traffic) /* package */ static final Boolean default_circuitBreakerForceClosed = false;// default => ignoreErrors = false private static final Integer default_executionTimeoutInMilliseconds = 1000; // default => executionTimeoutInMilliseconds: 1000 = 1 second private static final Boolean default_executionTimeoutEnabled = true; ... }
60_ The case of service fusing of hystrix (Part 2)
@RestController @Slf4j public class PaymentController { @Resource private PaymentService paymentService; ... //====Service fusing @GetMapping("/payment/circuit/{id}") public String paymentCircuitBreaker(@PathVariable("id") Integer id) { String result = paymentService.paymentCircuitBreaker(id); log.info("****result: "+result); return result; } }
test
Self test cloud provider hystrix payment8001
Correct- http://localhost:8001/payment/circuit/1
Error- http://localhost:8001/payment/circuit/-1
Multiple errors, correct again, but errors have to be displayed
Key test - several errors, and then slowly correct. It is found that the conditions are not met at the beginning, and even the correct access address cannot be carried out
61_ Summary of service fusing of hystrix
Great God conclusion
Relevant papers of Martin Fowler
Fuse type
- Fusing on: request not to call the current service again. The internal set clock is generally MTTR (mean time to failure). When it is turned on until the set clock is up, it will enter the semi fusing state.
- Fusing off: fusing off will not fuse the service.
- Fusing half open: some requests call the current service according to the rules. If the request is successful and meets the rules, it is considered that the current service returns to normal, and the fusing is turned off.
Flow chart of circuit breaker on official website
Official website steps
The precise way that the circuit opening and closing occurs is as follows:
- Assuming the volume across a circuit meets a certain threshold : HystrixCommandProperties.circuitBreakerRequestVolumeThreshold()
- And assuming that the error percentage, as defined above exceeds the error percentage defined in : HystrixCommandProperties.circuitBreakerErrorThresholdPercentage()
- Then the circuit-breaker transitions from CLOSED to OPEN.
- While it is open, it short-circuits all requests made against that circuit-breaker.
- After some amount of time (HystrixCommandProperties.circuitBreakerSleepWindowInMilliseconds()), the next request is let through. If it fails, the command stays OPEN for the sleep window. If it succeeds, it transitions to CLOSED and the logic in 1) takes over again.
When does the circuit breaker start to work
//=====Service fusing @HystrixCommand(fallbackMethod = "paymentCircuitBreaker_fallback",commandProperties = { @HystrixProperty(name = "circuitBreaker.enabled",value = "true"),// Whether to open the circuit breaker @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold",value = "10"),// Number of requests @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds",value = "10000"), // Time window period @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage",value = "60"),// Trip after the failure rate reaches }) public String paymentCircuitBreaker(@PathVariable("id") Integer id) { ... }
Three important parameters related to the circuit breaker:
- Snapshot time window: the circuit breaker needs to count some request and error data to determine whether to open, and the statistical time range is the snapshot time window, which defaults to the last 10 seconds.
- Threshold of total requests: within the snapshot time window, the threshold of total requests must be met to be eligible for fusing. The default is 20, which means that in 10 seconds, if the number of calls of the hystrix command is less than 20 times 7, even if all requests timeout or fail for other reasons, the circuit breaker will not open.
- Error percentage threshold: when the total number of requests exceeds the threshold within the snapshot time window, such as 30 calls, if there are 15 timeout exceptions in these 30 calls, that is, more than 50% error percentage, under the default setting of 50% threshold, the circuit breaker will be opened at this time.
Conditions for opening or closing the circuit breaker
-
When the following thresholds are reached, the circuit breaker will open:
- When a certain threshold is met (more than 20 requests in 10 seconds by default)
- When the failure rate reaches a certain level (more than 50% of requests fail within 10 seconds by default)
-
When enabled, all requests will not be forwarded
-
After a period of time (5 seconds by default), when the circuit breaker is half open, one of the requests will be forwarded. If successful, the circuit breaker will close. If failed, it will continue to open.
After the circuit breaker is opened
1: When there is another request call, the main logic will not be called, but the degraded fallback will be called directly. Through the circuit breaker, it realizes the effect of automatically finding errors and switching the degraded logic to the main logic to reduce the response delay.
2: How to restore the original main logic?
For this problem, hystrix also realizes the automatic recovery function for us.
When the circuit breaker is opened and the main logic is fused, hystrix will start a sleep time window. During this time window, the degraded logic will temporarily become the main logic. When the sleep time window expires, the circuit breaker will enter the half open state and release a request to the original main logic. If the request returns normally, the circuit breaker will continue to close and the main logic will recover. If there is still a problem with this request, The circuit breaker continues to enter the open state, and the sleep time window is timed again.
All configuration
@HystrixCommand(fallbackMethod = "fallbackMethod", groupKey = "strGroupCommand", commandKey = "strCommand", threadPoolKey = "strThreadPool", commandProperties = { // Set the isolation policy. THREAD indicates THREAD pool SEMAPHORE: signal pool isolation @HystrixProperty(name = "execution.isolation.strategy", value = "THREAD"), // When the isolation strategy selects signal pool isolation, it is used to set the size of the signal pool (maximum concurrency) @HystrixProperty(name = "execution.isolation.semaphore.maxConcurrentRequests", value = "10"), // Configure the timeout for command execution @HystrixProperty(name = "execution.isolation.thread.timeoutinMilliseconds", value = "10"), // Enable timeout @HystrixProperty(name = "execution.timeout.enabled", value = "true"), // Whether to interrupt when the execution times out @HystrixProperty(name = "execution.isolation.thread.interruptOnTimeout", value = "true"), // Whether the execution is interrupted when it is canceled @HystrixProperty(name = "execution.isolation.thread.interruptOnCancel", value = "true"), // Maximum concurrent number of callback method execution allowed @HystrixProperty(name = "fallback.isolation.semaphore.maxConcurrentRequests", value = "10"), // Whether the service degradation is enabled and whether the callback function is executed @HystrixProperty(name = "fallback.enabled", value = "true"), // Whether the circuit breaker is enabled @HystrixProperty(name = "circuitBreaker.enabled", value = "true"), // This attribute is used to set the minimum number of requests for circuit breaker fusing in the rolling time window. For example, when the default value is 20, if only 19 requests are received within the rolling time window (default 10 seconds), even if these 19 requests fail, the circuit breaker will not open. @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "20"), // This attribute is used to set in the rolling time window, indicating that in the rolling time window, the number of requests exceeds circuitbreaker In the case of requestvolumthreshold, if the percentage of wrong requests exceeds 50, set the circuit breaker to the "open" state, otherwise set it to the "closed" state. @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "50"), // This attribute is used to set the sleep time window after the circuit breaker is opened. After the sleep time window ends, the circuit breaker will be set to the "half open" state and the request command to try to fuse. If it still fails, the circuit breaker will continue to be set to the "open" state, and if it succeeds, it will be set to the "closed" state. @HystrixProperty(name = "circuitBreaker.sleepWindowinMilliseconds", value = "5000"), // Forced opening of circuit breaker @HystrixProperty(name = "circuitBreaker.forceOpen", value = "false"), // Forced closing of circuit breaker @HystrixProperty(name = "circuitBreaker.forceClosed", value = "false"), // Set the rolling time window, which is used for the duration of information to be collected when judging the health of the circuit breaker @HystrixProperty(name = "metrics.rollingStats.timeinMilliseconds", value = "10000"), // This attribute is used to set the number of "buckets" divided when rolling time window statistics indicator information. When collecting indicator information, the circuit breaker will be divided into multiple "buckets" according to the length of the set time window to accumulate each measurement value. Each "bucket" records the collection indicator for a period of time. // For example, it is divided into 10 "buckets" in 10 seconds, so timeinMilliseconds must be divisible by numBuckets. Otherwise, exceptions will be thrown @HystrixProperty(name = "metrics.rollingStats.numBuckets", value = "10"), // This attribute is used to set whether the delay in command execution is tracked and calculated using percentiles. If set to false, all summary statistics will return -1. @HystrixProperty(name = "metrics.rollingPercentile.enabled", value = "false"), // This attribute is used to set the duration of the scrolling window of percentile statistics, in milliseconds. @HystrixProperty(name = "metrics.rollingPercentile.timeInMilliseconds", value = "60000"), // This attribute is used to set the number of "buckets" used in the percentile statistics scrolling window. @HystrixProperty(name = "metrics.rollingPercentile.numBuckets", value = "60000"), // This attribute is used to set the maximum number of executions retained in each bucket during execution. If the number of executions exceeding the set value occurs within the rolling time window, // Rewrite it from the original position. For example, set the value to 100 and scroll the window for 10 seconds. If 500 executions occur in a "bucket" within 10 seconds, // Then only the statistics of the last 100 executions are retained in the "bucket". In addition, increasing the size of this value will increase the consumption of memory and the calculation time required to sort percentiles. @HystrixProperty(name = "metrics.rollingPercentile.bucketSize", value = "100"), // This attribute is used to set the interval waiting time for collecting health snapshots (success of requests, error percentage) that affect the status of the circuit breaker. @HystrixProperty(name = "metrics.healthSnapshot.intervalinMilliseconds", value = "500"), // Whether to enable request cache @HystrixProperty(name = "requestCache.enabled", value = "true"), // Whether the execution and events of the HystrixCommand are printed into the HystrixRequestLog @HystrixProperty(name = "requestLog.enabled", value = "true"), }, threadPoolProperties = { // This parameter is used to set the number of core threads in the command execution thread pool, which is the maximum concurrency of command execution @HystrixProperty(name = "coreSize", value = "10"), // This parameter is used to set the maximum queue size of the thread pool. When set to -1, the thread pool will use the queue implemented by SynchronousQueue, otherwise it will use the queue implemented by LinkedBlockingQueue. @HystrixProperty(name = "maxQueueSize", value = "-1"), // This parameter is used to set the rejection threshold for the queue. With this parameter, requests can be rejected even if the queue does not reach the maximum value. // This parameter is mainly a supplement to the LinkedBlockingQueue queue queue, because the LinkedBlockingQueue queue cannot dynamically modify its object size, and the queue size that rejects requests can be adjusted through this attribute. @HystrixProperty(name = "queueSizeRejectionThreshold", value = "5"), } ) public String doSomething() { ... }
62_ Final summary of hystrix workflow
Service flow restriction - the Sentinel description of alibaba will be explained in the later advanced chapter
Official website legend
[the external link image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-yx9xouzg-1614712032449)( https://raw.githubusercontent.com/wiki/Netflix/Hystrix/images/hystrix-command-flow-chart.png )]
Step description
- Create a HystrixCommand (used when the dependent service returns a single operation result) or HystrixObserableCommand (used when the dependent service returns multiple operation results) object.
- Command execution.
- HystrixCommand implements the first two execution methods below
- execute(): execute synchronously, return a single result object from the dependent service or throw an exception when an error occurs.
2. queue(): asynchronous execution, directly returning a Future object, which contains a single result object to be returned at the end of service execution.
- execute(): execute synchronously, return a single result object from the dependent service or throw an exception when an error occurs.
- The HystrixObservableCommand implements the latter two execution methods:
- Observe (): returns an Observable object, which represents multiple systems of operation
As a result, it is a Hot Observable (whether the "event source" has a "Subscriber" or not, the event will be published after creation, so each "Subscriber" of the Hot Observable may start in the middle of the "event source" and may only see the partial process of the whole operation).
2. Tobservable(): it will also return an Observable object, which also represents multiple results of the operation, but it returns a Cold Observable (when there is no "Subscriber", the event will not be published, but wait until there is a "Subscriber", so for the subscriber of Cold Observable, it can ensure to see the whole process of the operation from the beginning).
- Observe (): returns an Observable object, which represents multiple systems of operation
- If the request caching function of the current command is enabled, and the command cache hits, the cached results will be immediately returned in the form of Observable objects.
- Check whether the circuit breaker is open. If the circuit breaker is open, Hystrix will not execute the command, but transfer to the fallback processing logic (step 8); If the circuit breaker is closed, check whether there are resources available to execute the command (step 5).
- Whether the thread pool / request queue semaphore is full. If the command depends on the dedicated thread of the service and the request queue, or the semaphore (when the thread is not used) has been occupied, the Hystrix will not execute the command, but will transfer to the fallback processing logic (step 8).
- Hytrix will decide how to request dependent services according to the method we write.
- HystrixCommand.run(): return a single result or throw an exception.
- HystrixObservableCommand.construct(): return an Observable object to emit multiple results, or send error notification through onError.
- Hystix will report "success", "failure", "rejection", "timeout" and other information to the circuit breaker, and the circuit breaker will maintain a set of counters to count these data. The circuit breaker will use these statistics to decide whether to open the circuit breaker to "fuse / short circuit" a service dependent request.
- When the command fails to execute, Hystix will enter fallback to try to fallback. We usually call the wave operation "service degradation". The following conditions can cause service degradation:
1. Step 4: the current command is in the "fuse / short circuit" state, and the breaker is open.
2. Step 5: when the money pool, request queue or semaphore of the current command is full.
3. Step 6: when hytrixobsevablecommand. Construct() or HytrixCommand.run() throws an exception. - When the Hystrix command is successfully executed, it will return the processing result directly or in the form of Observable.
tips: if we do not implement the degradation logic for the command or throw an exception in the degradation processing logic, Hystrix will still return an observable object, but it will not send any result data. Instead, it will notify the command to immediately interrupt the request through the onError method, and send the exception causing the command failure to the caller through the onError method.
63_ Establishment of hystrix graphical Dashboard
summary
In addition to isolating the calls of dependent services, hystrix also provides a quasi real-time call monitoring (Hystrix Dashboard). Hystrix will continuously record the execution information of all requests initiated through hystrix, and display it to users in the form of statistical reports and graphs, including how many requests are executed per second, how many are successful, how many are failed, etc.
Netflix monitors the above indicators through the hystrix metrics event stream project. Spring Cloud also provides the integration of the Hystrix Dashboard, transforming the monitoring content into a visual interface.
Instrument cluster 9001
1. Create a new cloud consumer hystrix dashboard9001
2.POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>LearnCloud</artifactId> <groupId>com.lun.springcloud</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-consumer-hystrix-dashboard9001</artifactId> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
3.YML
server: port: 9001
4.HystrixDashboardMain9001 + new annotation @EnableHystrixDashboard
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard; @SpringBootApplication @EnableHystrixDashboard public class HystrixDashboardMain9001 { public static void main(String[] args) { SpringApplication.run(HystrixDashboardMain9001.class, args); } }
5. All Provider microservice providing classes (8001/8002/8003) need to monitor dependency configuration
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
6. Start cloud consumer hystrix dashboard9001. The microservice will monitor the microservice 8001 later
Browser input http://localhost:9001/hystrix
64_Hystrix graphical Dashboard monitoring practice
Modify cloud provider hystrix payment8001
Note: the new version of Hystrix needs to specify the monitoring path in the main startup class PaymentHystrixMain8001
import com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.boot.web.servlet.ServletRegistrationBean; import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker; import org.springframework.cloud.netflix.eureka.EnableEurekaClient; import org.springframework.context.annotation.Bean; @SpringBootApplication @EnableEurekaClient @EnableCircuitBreaker public class PaymentHystrixMain8001 { public static void main(String[] args) { SpringApplication.run(PaymentHystrixMain8001.class, args); } /** *This configuration is for service monitoring and has nothing to do with service fault tolerance itself. The pit after springcloud upgrade *ServletRegistrationBean Because the default path of springboot is not "/hystrix.stream", *Just configure the following servlet s in your project *Otherwise, Unable to connect to Command Metric Stream 404 */ @Bean public ServletRegistrationBean getServlet() { HystrixMetricsStreamServlet streamServlet = new HystrixMetricsStreamServlet(); ServletRegistrationBean registrationBean = new ServletRegistrationBean(streamServlet); registrationBean.setLoadOnStartup(1); registrationBean.addUrlMappings("/hystrix.stream"); registrationBean.setName("HystrixMetricsStreamServlet"); return registrationBean; } }
Monitoring test
Start 1 eureka
Start 80019001
Observe the monitoring window
9001 monitoring 8001 - fill in the monitoring address- http://localhost:8001/hystrix.stream To http://localhost:9001/hystrix Input box of the page.
Test address
-
http://localhost:8001/payment/circuit/1
-
http://localhost:8001/payment/circuit/-1
-
Test passed
-
Visit the correct address first, then the wrong address, and then the correct address. You will find that the circuit breakers shown in the figure are slowly released.
How to see
- 7 colors
- 1 turn
Filled circle: there are two meanings. It represents the health degree of the instance through the change of color, and its health degree decreases from green < yellow < orange < red.
In addition to the change of color, the size of the solid circle will also change according to the requested flow of the instance. The larger the flow, the larger the solid circle. Therefore, through the display of the solid circle, we can quickly find fault cases and high pressure cases in a large number of examples.
- 1 line
Curve: it is used to record the relative change of flow within 2 minutes, through which the rising and falling trend of flow can be observed.
- Description of the whole drawing
- Description of the whole drawing 2
65_GateWay and Zuul course description
Zuul developers are fighting in the dark, which is a thing of the past
Focus on Gate Way
66_ What is gateway
Previous generation zuul 1.x official website
summary
A very important component of Cloud family bucket is the gateway, which is the Zuul gateway used in version 1.x;
However, in version 2.x, zuul's upgrade has been skipped. SpringCloud finally developed a gateway to replace zuul, that is, SpringCloud Gateway - in a word, gateway is the replacement of the original zuul version 1.x
Gateway is an API gateway service built on the Spring ecosystem, based on Spring 5, Spring Boot 2, Project Reactor and other technologies.
Gateway aims to provide a simple and effective way to route API s and provide some powerful filter functions, such as fusing, current limiting, Retry, etc.
SpringCloud Gateway is a new project of Spring Cloud. It is a gateway developed based on Spring 5.0+Spring Boot 2.0 and Project Reactor. It aims to provide a simple and effective unified API route management method for microservice architecture.
As a gateway in the Spring Cloud ecosystem, SpringCloud Gateway aims to replace Zuul. In Spring Cloud versions above 2.0, there is no integration of the new version of Zul with the latest performance version above 2.0, and it is still the old version of Zuul 1.x non Reactor mode. In order to improve the performance of the gateway, SpringCloud Gateway is implemented based on the WebFlux framework, and the underlying WebFlux framework uses the high-performance Reactor mode communication framework Netty.
The goal of Spring Cloud Gateway is to provide a unified routing method, and the Filter chain based method provides the basic functions of the gateway, such as security, monitoring / indicators, and flow restriction.
effect
- Direction proxy
- authentication
- flow control
- Fusing
- Log monitoring
- ...
Location of gateway in microservice architecture
67_GateWay non blocking asynchronous model
Why did Zuull come out of the Gateway again? Why we choose Gateway
-
netflix is not very reliable. zuul2.0 has been skipped and has not been released yet.
- On the one hand, Zuul1.0 has entered the maintenance stage, and the Gateway is developed by the SpringCloud team. It is a trusted product. And many functions of Zuul are not used, and it is very simple and convenient.
- Gateway is developed based on asynchronous non blocking model, so there is no need to worry about performance. Although Netflix has long released the latest Zuul 2.x, Spring Cloud seems to have no integration plan. And Netflix related components are announced to enter the maintenance period; I don't know the future
- Considering all aspects, Gateway is an ideal Gateway choice.
-
SpringCloud Gateway has the following features
- Based on Spring Framework 5, Project Reactor and Spring Boot 2.0;
- Dynamic routing: it can match any request attribute;
- You can specify Predicate and filter for the route;
- Integrated circuit breaker function of Hystrix;
- Integrate Spring Cloud service discovery function;
- Easy to write Predicate and Filter;
- Request current limiting function;
- Path rewriting is supported.
-
The difference between SpringCloud Gateway and Zuul
- Before the official version of SpringCloud Finchley, the gateway recommended by Spring Cloud was Zuul provided by Netflix.
- Zuul 1.x is an API Gateway based on blocking I/O.
- Zuul 1.x uses a blocking architecture based on Servlet 2.5. It does not support any long connections (such as WebSockets). Zuul's design pattern is similar to Nginx. Every time I/ О The operation is to select an execution from the worker thread, and the request thread is blocked until the worker thread completes. However, the difference is that Nginx is implemented in C++ and zuul is implemented in Java, while the JVM itself will have a slow loading for the first time, which makes zuul's performance relatively poor.
- Zuul 2.x has a more advanced concept. It wants to be non blocking based on Netty and support long connections, but SpringCloud has not been integrated yet. The performance of zuul.X is much better than that of Zuul 1.x. In terms of performance, according to the benchmark provided by the official, the RPS (requests per second) of Spring Cloud Gateway is 1.6 times that of zuul.
- Spring Cloud Gateway is built on Spring Framework 5, Project Reactor and Spring Boot2, and uses non blocking API s.
- Spring Cloud Gateway also supports WebSocket, and is closely integrated with spring to have a better development experience
Zuul1.x model
The Zuul version integrated in Springcloud uses the Tomcat container and the traditional Serviet IO processing model.
The lifecycle of servlet? Servlets are lifecycle managed by servlet container s.
- When the container starts, construct a servlet object and call servlet init() to initialize;
- The container runtime accepts requests, allocates a thread for each request (usually obtains idle threads from the thread pool), and then calls the service);
- Call servlet destroy() to destroy the servlet when the container is closed.
Disadvantages of the above mode:
Servlet is a simple network IO model. When a request enters the Servlet container, the Servlet container will bind a thread for it. This model is applicable in scenarios with low concurrency. However, once the concurrency is high (such as Jmeter pressure for ventilation), the number of threads will increase, and the cost of thread resources is expensive (online text switching, large memory consumption), which seriously affects the processing time of requests. In some simple business scenarios, you don't want to allocate a thread to each request. You only need one or several threads to deal with highly concurrent requests. In this business scenario, the servlet model has no advantage.
Therefore, Zuul 1.X is a blocking processing model based on servlet, that is, Spring implements a servlet (DispatcherServlet) that processes all request requests and is blocked by the servlet. Therefore, SpringCloud Zuul cannot get rid of the disadvantages of servlet model.
Gateway model
What is WebFlux? Official documents
Traditional Web frameworks, such as Struts2 and SpringMVC, run on the basis of Servlet APl and Servlet container.
But after Servlet3.1, there is asynchronous non blocking support. WebFlux is a typical non blocking asynchronous framework, and its core is implemented based on the relevant API of Reactor. Compared with the traditional web framework, it can run on containers such as Netty, Undertow and Servlet3.1. Non blocking + functional programming (Spring 5 must let you use Java 8).
Spring WebFlux is a new responsive framework introduced by Spring 5.0. Different from Spring MVC, it does not need to rely on Servlet APl. It is completely asynchronous and non blocking, and implements the responsive flow specification based on Reactor.
Spring Cloud Gateway requires the Netty runtime provided by Spring Boot and Spring Webflux. It does not work in a traditional Servlet Container or when built as a WAR.link
68_Gateway workflow
Three core concepts
- Route - route is the basic module for building a gateway. It consists of ID, target URI, a series of assertions and filters. If the assertion is true, it matches the route;
- Predicate - the reference is java.util.function Predict, the developer can match all the contents of the HTTP request (such as the request header or request parameters), and route if the request matches the assertion;
- Filter - refers to the instance of GatewayFilter in the Spring framework. Using filters, you can modify requests before or after they are routed.
The web request locates the real service node through some matching conditions. And before and after this forwarding process, some fine control is carried out.
predicate is our matching condition; fliter can be understood as an omnipotent interceptor. With these two elements, plus the target uri, you can achieve a specific route
Gateway workflow
Clients make requests to Spring Cloud Gateway. If the Gateway Handler Mapping determines that a request matches a route, it is sent to the Gateway Web Handler. This handler runs the request through a filter chain that is specific to the request. The reason the filters are divided by the dotted line is that filters can run logic both before and after the proxy request is sent. All "pre" filter logic is executed. Then the proxy request is made. After the proxy request is made, the "post" filter logic is run. link
The client sends a request to the Spring Cloud Gateway. Then find the route matching the request in the Gateway Handler Mapping and send it to the gateway web handler.
The Handler then sends the request to our actual service through the specified filter chain to execute the business logic, and then returns.
The reason why the filters are separated by dotted lines is that the filter may execute business logic before ("pre") or after ("post") sending the proxy request.
Filter can do parameter verification, permission verification, traffic monitoring, log output, protocol conversion, etc. in "pre" type filter, it can do response content, response header modification, log output, traffic monitoring, etc. in "post" type filter, it plays a very important role.
Core logic: route forwarding + execution filter chain.
69_Gateway9527 construction
1. Create a new module - cloud gateway gateway 9527
2.POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>LearnCloud</artifactId> <groupId>com.lun.springcloud</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-gateway-gateway9527</artifactId> <dependencies> <!--gateway--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-gateway</artifactId> </dependency> <!--eureka-client--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <!-- Introduce self defined api General package, which can be used Payment payment Entity --> <dependency> <groupId>com.lun.springcloud</groupId> <artifactId>cloud-api-commons</artifactId> <version>${project.version}</version> </dependency> <!--General basic configuration class--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
3.YML
server: port: 9527 spring: application: name: cloud-gateway eureka: instance: hostname: cloud-gateway-service client: #The service provider is registered in the eureka service list service-url: register-with-eureka: true fetch-registry: true defaultZone: http://eureka7001.com:7001/eureka
4. Business
nothing
5. Main startup class
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.netflix.eureka.EnableEurekaClient; @SpringBootApplication @EnableEurekaClient public class GateWayMain9527 { public static void main(String[] args) { SpringApplication.run(GateWayMain9527.class, args); } }
6.9527 how to do routing mapping for gateway
Cloud provider payment8001 look at the access address of the controller
- get
- lb
At present, we don't want to expose the 8001 port. We hope to set a layer of 9527 outside the 8001
7.YML new gateway configuration
server: port: 9527 spring: application: name: cloud-gateway #############################Add gateway configuration########################### cloud: gateway: routes: - id: payment_routh #payment_route #The ID of the route, which has no fixed rules but requires uniqueness, is recommended to match the service name uri: http://localhost:8001 # the routing address of the service provided after matching #uri: lb://Cloud payment service # the routing address of the service provided after matching predicates: - Path=/payment/get/** # Assert that the route matches the path - id: payment_routh2 #payment_route #The ID of the route, which has no fixed rules but requires uniqueness, is recommended to match the service name uri: http://localhost:8001 # the routing address of the service provided after matching #uri: lb://Cloud payment service # the routing address of the service provided after matching predicates: - Path=/payment/lb/** # Assert that the route matches the path #################################################################### eureka: instance: hostname: cloud-gateway-service client: #The service provider is registered in the eureka service list service-url: register-with-eureka: true fetch-registry: true defaultZone: http://eureka7001.com:7001/eureka
8. Test
-
Start 7001
-
Start 8001 cloud provider payment8001
-
Start 9527 gateway
-
Access instructions
- Before adding a gateway- http://localhost:8001/payment/get/1
- After adding gateway- http://localhost:9527/payment/get/1
- Both accesses are successful, and the same result is returned
70_ Two ways for gateway to configure routing
Configure in the configuration file yml, see the previous chapter
Inject the Bean of RouteLocator into the code
Official case- link
RemoteAddressResolver resolver = XForwardedRemoteAddressResolver .maxTrustedIndex(1); ... .route("direct-route", r -> r.remoteAddr("10.1.1.1", "10.10.1.1/24") .uri("https://downstream1") .route("proxied-route", r -> r.remoteAddr(resolver, "10.10.1.1", "10.10.1.1/24") .uri("https://downstream2") )
Baidu domestic news website, need external network- http://news.baidu.com/guonei
Write one by yourself
Business needs - access the Baidu news website of the external network through the 9527 gateway
code
Cloud gateway gateway 9527 service implementation
import org.springframework.cloud.gateway.route.RouteLocator; import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration public class GateWayConfig { @Bean public RouteLocator customRouteLocator(RouteLocatorBuilder routeLocatorBuilder) { RouteLocatorBuilder.Builder routes = routeLocatorBuilder.routes(); routes.route("path_route_atguigu", r -> r.path("/guonei") .uri("http://news.baidu.com/guonei")).build(); return routes.build(); } }
test
Browser input http://localhost:9527/guonei , return http://news.baidu.com/guonei Same page.
71_GateWay configuration dynamic routing
By default, the Gateway will create a dynamic route with the name of the micro service on the registry as the path to forward according to the list of services registered in the registry, so as to realize the function of dynamic route (without writing an address).
start-up
- eureka7001
- payment8001/8002
POM
<!--eureka-client--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency>
YML
It should be noted that the protocol of uri is lb, which means that the load balancing function of Gateway is enabled.
lb://serviceName is the load balancing uri that spring cloud gateway automatically creates for us in microservices.
server: port: 9527 spring: application: name: cloud-gateway #############################Add gateway configuration########################### cloud: gateway: discovery: locator: enabled: true #Enable the function of dynamically creating routes from the registry, and use the microservice name to route routes: - id: payment_routh #payment_route #The ID of the route, which has no fixed rules but requires uniqueness, is recommended to match the service name #uri: http://localhost:8001 # the routing address of the service provided after matching uri: lb://Cloud payment service # the routing address of the service provided after matching predicates: - Path=/payment/get/** # Assert that the route matches the path - id: payment_routh2 #payment_route #The ID of the route, which has no fixed rules but requires uniqueness, is recommended to match the service name #uri: http://localhost:8001 # the routing address of the service provided after matching uri: lb://Cloud payment service # the routing address of the service provided after matching predicates: - Path=/payment/lb/** # Assert that the route matches the path #################################################################### eureka: instance: hostname: cloud-gateway-service client: #The service provider is registered in the eureka service list service-url: register-with-eureka: true fetch-registry: true defaultZone: http://eureka7001.com:7001/eureka
test
Browser input- http://localhost:9527/payment/lb
result
Constantly refresh the page and switch between 8001 and 8002 ports.
72_ Commonly used Predicate of gateway
What is route predict factories
Spring Cloud Gateway matches routes as part of the Spring WebFlux HandlerMapping infrastructure. Spring Cloud Gateway includes many built-in route predicate factories. All of these predicates match on different attributes of the HTTP request. You can combine multiple route predicate factories with logical and statements. link
Spring Cloud Gateway takes route matching as part of the Spring WebFlux HandlerMapping infrastructure.
Spring Cloud Gateway includes many built-in route predict factories. All these predictions match the different attributes of the HTTP request. Multiple RoutePredicate factories can be combined.
When Spring Cloud Gateway creates a Route object, it uses RoutePredicateFactory to create a Predicate object, which can be assigned to Route. Spring Cloud Gateway includes many built-in Route predict factories.
All of these predicates match different attributes of the HTTP request. A variety of predicate factories can be combined through logical and.
predicate
Us: ['predket] UK: ['predkt]
v. Assertion; Based on; Make based on; indicate
adj. declarative; Predicate
n. Predicate (sentence element, stating the subject, such as went home in John went home)
Common Route Predicate Factory
- The After Route Predicate Factory
- The Before Route Predicate Factory
- The Between Route Predicate Factory
- The Cookie Route Predicate Factory
- The Header Route Predicate Factory
- The Host Route Predicate Factory
- The Method Route Predicate Factory
- The Path Route Predicate Factory
- The Query Route Predicate Factory
- The RemoteAddr Route Predicate Factory
- The weight Route Predicate Factory
Discuss several route predict factories
The After Route Predicate Factory
spring: cloud: gateway: routes: - id: after_route uri: https://example.org predicates: # Only after this time can it take effect - After=2017-01-20T17:42:47.789-07:00[America/Denver]
The timestamp string in the above format can be obtained by the following methods
import java.time.ZonedDateTime; public class T2 { public static void main(String[] args) { ZonedDateTime zbj = ZonedDateTime.now(); // Default time zone System.out.println(zbj); //2021-02-22T15:51:37.485+08:00[Asia/Shanghai] } }
The Between Route Predicate Factory
spring: cloud: gateway: routes: - id: between_route uri: https://example.org # Between two points in time predicates: - Between=2017-01-20T17:42:47.789-07:00[America/Denver], 2017-01-21T17:42:47.789-07:00[America/Denver]
The Cookie Route Predicate Factory
spring: cloud: gateway: routes: - id: cookie_route uri: https://example.org predicates: - Cookie=chocolate, ch.p
The cookie route predicate factory takes two parameters, the cookie name and a regular expression.
This predicate matches cookies that have the given name and whose values match the regular expression.
test
# This command is equivalent to sending a get request without cookie s curl http://localhost:9527/payment/lb # With cookie s curl http://localhost:9527/payment/lb --cookie "chocolate=chip"
The Header Route Predicate Factory
spring: cloud: gateway: routes: - id: header_route uri: https://example.org predicates: - Header=X-Request-Id, d+
The header route predicate factory takes two parameters, the header name and a regular expression.
This predicate matches with a header that has the given name whose value matches the regular expression.
test
# CURL command with parameters specifying the request header curl http://localhost:9527/payment/lb -H "X-Request-Id:123"
Others, draw inferences from one instance.
Summary
To put it bluntly, Predicate is to implement a set of matching rules, so that requests can be processed by finding the corresponding Route.
73_ Filter of gateway
Route filters allow the modification of the incoming HTTP request or outgoing HTTP response in some manner. Route filters are scoped to a particular route. Spring Cloud Gateway includes many built-in GatewayFilter Factories.
Routing filters can be used to modify incoming HTTP requests and returned HTTP responses. Routing filters can only be used by specifying routes. Spring Cloud Gateway has built-in various routing filters, which are generated by the factory class of GatewayFilter.
Filter of Spring Cloud Gateway:
-
Life cycle:
- pre
- post
-
Type (see official documents for details):
- GatewayFilter - there are 31 kinds
- GlobalFilter - there are 10 kinds
Common GatewayFilter: AddRequestParameter GatewayFilter
Customize global GlobalFilter:
Two main interfaces are introduced:
- GlobalFilter
- Ordered
What to do:
- Global logging
- Unified gateway authentication
- ...
Code case:
Add the MyLogGateWayFilter class to the GateWay9527 project:
import lombok.extern.slf4j.Slf4j; import org.springframework.cloud.gateway.filter.GatewayFilterChain; import org.springframework.cloud.gateway.filter.GlobalFilter; import org.springframework.core.Ordered; import org.springframework.http.HttpStatus; import org.springframework.stereotype.Component; import org.springframework.web.server.ServerWebExchange; import reactor.core.publisher.Mono; import java.util.Date; @Component @Slf4j public class MyLogGateWayFilter implements GlobalFilter,Ordered { @Override public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) { log.info("***********come in MyLogGateWayFilter: "+new Date()); String uname = exchange.getRequest().getQueryParams().getFirst("uname"); if(uname == null) { log.info("*******User name is null,Illegal user, o(╥﹏╥)o"); exchange.getResponse().setStatusCode(HttpStatus.NOT_ACCEPTABLE); return exchange.getResponse().setComplete(); } return chain.filter(exchange); } @Override public int getOrder() { return 0; } }
Test:
Start:
- EurekaMain7001
- PaymentMain8001
- GateWayMain9527
- PaymentMain8002
Browser input:
- http://localhost:9527/payment/lb -Rhetorical abnormality
- http://localhost:9527/payment/lbuname=abc -Normal rhetorical question
74_ Introduction to config distributed configuration center
Configuration problems faced by distributed systems
Microservice means to split the business in a single application into a sub service. The granularity of each service is relatively small, so there will be a large number of services in the system. Because each service needs the necessary configuration information to run, a centralized and dynamic configuration management facility is essential.
SpringCloud provides ConfigServer to solve this problem. Each of our microservices carries an application.yml and manages hundreds of configuration files
What is it?
SpringCloud Config provides centralized external configuration support for microservices in the microservice architecture, and the configuration server provides a centralized external configuration for all environments of different microservice applications.
How do you play?
SpringCloud Config is divided into server and client.
-
The server is also known as the distributed configuration center. It is an independent micro service application, which is used to connect to the configuration server and provide the client with access interfaces for obtaining configuration information, encryption / decryption information and so on.
-
The client manages application resources and business-related configuration content through the specified configuration center, and obtains and loads configuration information from the configuration center at startup. The configuration server uses git to store configuration information by default, which is conducive to version management of environment configuration, and can easily manage and access configuration content through git client tools.
What can I do
- Centrally manage profiles
- Different environments have different configurations, dynamic configuration updates, and deployment by environment, such as dev/test/prod/beta/release
- During operation, the configuration is dynamically adjusted. It is no longer necessary to write configuration files on each service deployed machine. The service will uniformly pull and configure its own information from the configuration center
- When the configuration changes, the service can sense the change of the configuration and apply the new configuration without restarting
- Expose the configuration information in the form of REST interface - post/crul access refresh
Integrated configuration with GitHub
Because SpringCloud Config uses git by default to store configuration files (there are other ways, such as supporting SVN and local files), Git is the most recommended, and the form of http/https access is used.
Official website
https://cloud.spring.io/spring-cloud-static/spring-cloud-config/2.2.1.RELEASE/reference/html/
75_Config configuration master control center setup
Create a new Repository named springcloud config on GitHub with your own account.
Get the newly created git address from the previous step- git@github.com:abc/springcloud-config.git.
Create a new git warehouse and clone on the local hard disk directory.
-
Working directory is D:SpringCloud2021
-
git clone git@github.com:abc/springcloud-config.git
A folder named springcloud config will be created in the working directory.
Create three configuration files (for this teaching) in the springcloud config folder, and then upload git add., git commit -m "sth" and other upload operations to the new Repository of springcloud config.
-
config-dev.yml
config:
info: "master branch,springcloud-config/config-dev.yml version=7" -
config-prod.yml
config:
info: "master branch,springcloud-config/config-prod.yml version=1" -
config-test.yml
config:
info: "master branch,springcloud-config/config-test.yml version=1"
Create a new Module cloud-config-center-3344, which is the Cloud configuration center Module CloudConfig Center
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>LearnCloud</artifactId> <groupId>com.lun.springcloud</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-config-center-3344</artifactId> <dependencies> <!--Add message bus RabbitMQ support--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-bus-amqp</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-config-server</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
YML
server: port: 3344 spring: application: name: cloud-config-center #Microservice name registered into Eureka server cloud: config: server: git: uri: git@github.com:zzyybs/springcloud-config.git #git warehouse name on GitHub ####search for directory search-paths: - springcloud-config ####Read branch label: master #Service registration to eureka address eureka: client: service-url: defaultZone: http://localhost:7001/eureka
Main startup class
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.config.server.EnableConfigServer; @SpringBootApplication @EnableConfigServer public class ConfigCenterMain3344 { public static void main(String[] args) { SpringApplication.run(ConfigCenterMain3344.class, args); } }
Modify the hosts file under windows and add mapping
127.0.0.1 config-3344.com
Test whether the configuration content can be obtained from GitHub through Config microservice
-
Start ConfigCenterMain3344
-
Browser anti question- http://config-3344.com:3344/master/config-dev.yml
-
Page return result:
config:
info: "master branch,springcloud-config/config-dev.yml version=7"
Configure read rules
-
/{label}/{application}-{profile}.yml (recommended)
- master branch
- http://config-3344.com:3344/master/config-dev.yml
- http://config-3344.com:3344/master/config-test.yml
- http://config-3344.com:3344/master/config-prod.yml
- dev branch
- http://config-3344.com:3344/dev/config-dev.yml
- http://config-3344.com:3344/dev/config-test.yml
- http://config-3344.com:3344/dev/config-prod.yml
- master branch
-
/{application}-{profile}.yml
- http://config-3344.com:3344/config-dev.yml
- http://config-3344.com:3344/config-test.yml
- http://config-3344.com:3344/config-prod.yml
- http://config-3344.com:3344/config -Xxxx.yml (nonexistent configuration)
-
/{application}/{profile}[/{label}]
- http://config-3344.com:3344/config/dev/master
- http://config-3344.com:3344/config/test/master
- http://config-3344.com:3344/config/test/dev
-
Summary of important configuration details
- /{name}-{profiles}.yml
- /{label}-{name}-{profiles}.yml
- label: Branch
- Name: service name
- profiles: environment (dev/test/prod)
Successfully achieved using SpringCloud Config to obtain configuration information through GitHub
76_Config client configuration and testing
Create cloud-config-client-3355
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>LearnCloud</artifactId> <groupId>com.lun.springcloud</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-config-client-3355</artifactId> <dependencies> <!--Add message bus RabbitMQ support--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-bus-amqp</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-config</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
bootstrap.yml
applicaiton.yml is a user level resource configuration item
bootstrap.yml is system level, with higher priority
Spring Cloud will create a Bootstrap Context as the parent context of the Application Context of spring applications.
During initialization, BootstrapContext is responsible for loading configuration properties from external sources and parsing the configuration. These two contexts share an Environment obtained from the outside.
Bootstrap attributes have high priority. By default, they will not be overwritten by local configuration. Bootstrap context and Application Context have different conventions, so a new bootstrap is added YML file to ensure the separation of bootstrap context and Application Context configuration.
It is critical to change the application.yml file under the Client module to bootstrap.yml, because bootstrap.yml is loaded before application.yml. Bootstrap.yml has higher priority than application.yml.
server: port: 3355 spring: application: name: config-client cloud: #Config client configuration config: label: master #Branch name name: config #Profile name profile: dev #Read the suffix name. The above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml uri: http://localhost:3344 # configuration center address k #Service registration to eureka address eureka: client: service-url: defaultZone: http://localhost:7001/eureka
Modify the config-dev.yml configuration and submit it to GitHub, such as adding a variable age or version number version
Main start
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.netflix.eureka.EnableEurekaClient; @EnableEurekaClient @SpringBootApplication public class ConfigClientMain3355 { public static void main(String[] args) { SpringApplication.run(ConfigClientMain3355.class, args); } }
Business class
import org.springframework.beans.factory.annotation.Value; import org.springframework.cloud.context.config.annotation.RefreshScope; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; @RestController @RefreshScope public class ConfigClientController { @Value("${config.info}") private String configInfo; @GetMapping("/configInfo") public String getConfigInfo() { return configInfo; } }
test
-
Start Config configuration center 3344 microservice and self test
- http://config-3344.com:3344/master/config-prod.yml
- http://config-3344.com:3344/master/config-dev.yml
-
Start 3355 as a Client to prepare for access
- http://localhost:3355/configlnfo
The client 3355 accesses SpringCloud Config3344 successfully. The configuration information can be obtained through GitHub at any time
Dynamic refresh of distributed configuration
- Linux operation and maintenance modifies the content of the configuration file on GitHub to make adjustments
- Refresh 3344, and the ConfigServer configuration center responds immediately
- Refresh 3355 and find no response from ConfigClient client
- 3355 does not change unless you restart or reload
- It is so difficult that every time the O & M modifies the configuration file, the client needs to restart
77_ Manual version of config dynamic refresh
Avoid restarting the client microservice 3355 every time the configuration is updated
Dynamic refresh steps:
Modify 3355 module
POM introduces actor monitoring
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
Modify YML and add exposure monitoring port configuration:
# Exposure monitoring endpoint management: endpoints: web: exposure: include: "*"
@RefreshScope business class Controller modification
import org.springframework.cloud.context.config.annotation.RefreshScope; ... @RestController @RefreshScope//<----- public class ConfigClientController { ... }
test
At this time, modify the contents of github configuration file - > access 3344 - > access 3355
http://localhost:3355/configInfo
3355 no change, there is still a step to be taken
How
The operation and maintenance personnel need to send a Post request to refresh 3355
curl -X POST "http://localhost:3355/actuator/refresh"
Retest
http://localhost:3355/configInfo
3355 no change.
The client 3355 is successfully refreshed to the latest configuration content, avoiding service restart
Think about what else
- Suppose there are multiple microservice clients 3355/3366/3377
- Each micro service needs to execute a post request and refresh manually
- Can you broadcast it? One notice will take effect everywhere
- We want a large-scale automatic refresh, and find a method
78_ What is bus message bus
Part I - deepening and expansion of explanation
In a word, the distributed automatic refresh configuration function.
Spring Cloud Bus can be used with Spring Cloud Config to dynamically refresh the configuration.
What is it?
Spring Cloud Bus can be used with Spring Cloud Config to dynamically refresh the configuration.
Spring Cloud Bus is a framework used to link the nodes of distributed systems with lightweight messaging systems. It integrates the event processing mechanism of Java and the functions of messaging middleware. Spring Cloud Bus currently supports RabbitMQ and Kafka.
What can I do
Spring Cloud Bus can manage and disseminate messages between distributed systems, just like a distributed actuator, which can be used to broadcast state changes, push events, etc., and can also be used as a communication channel between microservices.
Why is it called bus
What is a bus
In microservice architecture systems, lightweight message brokers are usually used to build a common message topic and connect all microservice instances in the system. Since the messages generated in this topic will be monitored and consumed by all instances, it is called a message bus. Each instance on the bus can easily broadcast some messages that need to be known by other instances connected to the subject.
Basic principles
ConfigClient instances listen to the same Topic in MQ (spring cloud bus by default). When a service refreshes data, it will put this information into the Topic, so that other services listening to the same Topic can be notified, and then update their own configuration.
79_ RabbitMQ environment configuration of bus
-
Install Erlang, download address: http://erlang.org/download/otp_win64_21.3.exe
-
Install RabbitMQ, download address: https://github.com/rabbitmq/rabbitmq-server/releases/download/v3.8.3/rabbitmq-server-3.8.3.exe
-
Open cmd and enter the SBIN directory under the RabbitMQ installation directory, such as D:devSoftRabbitMQ Scrverk abbitmq_server-3.7.14sbin
-
Enter the following command to start the management function
rabbitmq-plugins enable rabbitmq _management
This allows you to add visual plug-ins.
-
Visit the address to check whether the installation is successful: http://localhost:15672/
-
Enter the account and password and log in: guest guest
80_ Design idea and type selection of bus dynamic refresh global broadcast
You must have a good RabbitMQ environment first
Demonstrate the broadcast effect, increase the complexity, and then make another 3366 with 3355 as the template
1. Create cloud-config-client-3366
2.POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>LearnCloud</artifactId> <groupId>com.lun.springcloud</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-config-client-3366</artifactId> <dependencies> <!--Add message bus RabbitMQ support--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-bus-amqp</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-config</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
3.YML
server: port: 3366 spring: application: name: config-client cloud: #Config client configuration config: label: master #Branch name name: config #Profile name profile: dev #Read the suffix name. The above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml uri: http://localhost:3344 # configuration center address #rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port accessed by MQ rabbitmq: host: localhost port: 5672 username: guest password: guest #Service registration to eureka address eureka: client: service-url: defaultZone: http://localhost:7001/eureka # Exposure monitoring endpoint management: endpoints: web: exposure: include: "*"
4. Main startup
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.netflix.eureka.EnableEurekaClient; @EnableEurekaClient @SpringBootApplication public class ConfigClientMain3366 { public static void main(String[] args) { SpringApplication.run(ConfigClientMain3366.class,args); } }
5.controller
import org.springframework.beans.factory.annotation.Value; import org.springframework.cloud.context.config.annotation.RefreshScope; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; /** */ @RestController @RefreshScope public class ConfigClientController { @Value("${server.port}") private String serverPort; @Value("${config.info}") private String configInfo; @GetMapping("/configInfo") public String configInfo() { return "serverPort: "+serverPort+" configInfo: "+configInfo; } }
design idea
1. Use the message bus to trigger a client /bus/refresh, and refresh the configuration of all clients
2. Use the message bus to trigger the /bus/refresh endpoint of a server ConfigServer, and refresh the configuration of all clients
The architecture in Figure 2 is obviously more suitable. The reasons for the unsuitability are as follows:
-
It breaks the single responsibility of microservices, because microservices are business modules, which should not bear the responsibility of configuration refresh.
-
It destroys the peer-to-peer of micro service nodes.
-
There are certain limitations. For example, when a microservice is migrated, its network address often changes. At this time, if you want to refresh automatically, you will add more modifications.
81_ Implementation of bus dynamic refresh global broadcast configuration
Add message bus support to cloud-config-center-3344 configuration center server
POM
<!--Add message bus RabbitNQ support--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-bus-amap</artifactId> </dependency> <dependency> <groupId>org-springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
YML
server: port: 3344 spring: application: name: cloud-config-center #Microservice name registered into Eureka server cloud: config: server: git: uri: git@github.com:zzyybs/springcloud-config.git #git warehouse name on GitHub ####search for directory search-paths: - springcloud-config ####Read branch label: master #rabbitmq related configurations<-------------------------- rabbitmq: host: localhost port: 5672 username: guest password: guest #Service registration to eureka address eureka: client: service-url: defaultZone: http://localhost:7001/eureka ##rabbitmq related configuration, exposing the endpoint of bus refresh configuration<-------------------------- management: endpoints: #Expose the endpoint of the bus refresh configuration web: exposure: include: 'bus-refresh'
Add message bus support to cloud-config-client-3355 client
POM
<!--Add message bus RabbitNQ support--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-bus-amap</artifactId> </dependency> <dependency> <groupId>org-springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
YML
server: port: 3355 spring: application: name: config-client cloud: #Config client configuration config: label: master #Branch name name: config #Profile name profile: dev #Read the suffix name. The above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml uri: http://localhost:3344 # configuration center address k #rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port accessed by MQ<---------------------- rabbitmq: host: localhost port: 5672 username: guest password: guest #Service registration to eureka address eureka: client: service-url: defaultZone: http://localhost:7001/eureka # Exposure monitoring endpoint management: endpoints: web: exposure: include: "*"
Add message bus support to cloud-config-client-3366 client
POM
<!--Add message bus RabbitNQ support--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-bus-amap</artifactId> </dependency> <dependency> <groupId>org-springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
YML
server: port: 3366 spring: application: name: config-client cloud: #Config client configuration config: label: master #Branch name name: config #Profile name profile: dev #Read the suffix name. The above three combinations: the configuration file of config-dev.yml on the master branch is read http://config-3344.com:3344/master/config-dev.yml uri: http://localhost:3344 # configuration center address #rabbitmq related configuration 15672 is the port of the Web management interface; 5672 is the port accessed by MQ<----------------------- rabbitmq: host: localhost port: 5672 username: guest password: guest #Service registration to eureka address eureka: client: service-url: defaultZone: http://localhost:7001/eureka # Exposure monitoring endpoint management: endpoints: web: exposure: include: "*"
test
-
start-up
- EurekaMain7001
- ConfigcenterMain3344
- ConfigclientMain3355
- ConfigclicntMain3366
-
Operation and maintenance engineer
- Modify the content of the configuration file on Github and increase the version number
- Send POST request
- curl -X POST "http://localhost:3344/actuator/bus-refresh"
- Send once, take effect everywhere
-
Configuration center
- http://config-3344.com:3344/config-dev.yml
-
client
- http://localhost:3355/configlnfo
- http://localhost:3366/configInfo
- Get the configuration information and find that it has been refreshed
Once modified, broadcast notice, effective everywhere
82_Bus dynamic refresh fixed-point notification
I don't want to be notified all the time, I just want to be notified at a fixed point
- Notice only 3355
- No notice 3366
In a simple sentence - specify a specific instance to take effect instead of all
-
Formula: http://localhost:3344/actuator/bus-refresh/{destination}
-
/The bus/refresh request is no longer sent to the specific service instance, but to the config server. Specify the service or instance that needs to update the configuration through the destination parameter class
case
- Here we take refreshing the config client (application name set in the configuration file) running on port 3355 as an example, only notify 3355, not 3366
- curl -X POST "http://localhost:3344/actuator/bus-refresh/config-client:3355
Notification summary
83_ Why is stream introduced
Common MQ (Message Oriented Middleware):
- ActiveMQ
- RabbitMQ
- RocketMQ
- Kafka
Is there a new technology that makes us no longer pay attention to the details of specific MQ? We just need to use an adaptive binding method to automatically switch between various MQ. (similar to Hibernate)
What is Cloud Stream? Shield the differences between the underlying message middleware, reduce the switching cost, and unify the programming model of the message.
84_ What is stream and Binder introduction
Cloud Stream Chinese instruction manual
What is Spring Cloud Stream?
Officially, Spring Cloud Stream is a framework for building message driven microservices.
The application interacts with the binder object in Spring Cloud Stream through inputs or outputs.
We configure binding, and the binder object of Spring Cloud Stream is responsible for interacting with the message middleware. Therefore, we only need to figure out how to interact with Spring Cloud Stream to facilitate the use of message driven methods.
By using Spring Integration to connect the message broker middleware to realize message event driven.
Spring Cloud Stream provides personalized automatic configuration implementation for some suppliers' message oriented middleware products, referencing the three core concepts of publish subscribe, consumer group and partition.
Currently, only RabbitMQ and Kafka are supported.
85_ Design idea of stream
Standard MQ
- Producers / consumers rely on message media to transmit information content
- Messages must go through a specific channel - Message Channel
- How are messages in the message channel consumed? Who is responsible for sending and receiving? SubscribableChannel, the sub interface of the message channel MessageChannel, is subscribed by the MessageHandler message processor.
Why use Cloud Stream?
For example, we use RabbitMQ and Kafka. Due to the different architectures of these two message oriented middleware, RabbitMQ has exchange and Kafka has Topic and Partitions.
The differences of these middleware lead to some problems in our actual project development. If we use one of the two message queues, and the business requirements behind it, I want to migrate to the other message queue. At this time, it is undoubtedly a disaster. A lot of things have to be pushed down and redone, because it is coupled with our system, At this time, Spring Cloud Stream provides us with a way to decouple.
Why can Stream unify the underlying differences?
Without the concept of binder, when our SpringBoot application wants to directly interact with the message oriented middleware, due to the different original intentions of the message oriented middleware, there will be great differences in their implementation details. By defining the binder as the intermediate layer, we can perfectly realize the isolation between the application and the message oriented middleware details. By exposing the unified Channel channel to the application, the application does not need to consider a variety of different message oriented middleware implementations.
By defining Binder as the middle layer, the isolation between application and message middleware details is realized.
Binder:
-
INPUT corresponds to the consumer
-
OUTPUT corresponds to the producer
The message communication mode in Stream follows the publish subscribe mode
Topic topic broadcast
- RabbitMQ is Exchange
- In Kakfa, it is Topic
86_ Introduction to common annotations of stream coding
Spring Cloud Stream standard process routine
-
Binder - very convenient connection middleware, shielding differences.
-
Channel - channel is an abstraction of Queue. In the message communication system, it is the medium to realize storage and forwarding. The Queue is configured through channel.
-
Source and Sink - simply understood as the reference object is the Spring Cloud Stream itself. Publishing messages from the Stream is output, and receiving messages is input.
Coding API and common annotations
form
explain
Middleware
Middleware, currently only supports RabbitMQ and Kafka
Binder
Binder is a package between applications and message oriented middleware. At present, binder of Kafka and RabbitMQ are implemented. Binder can easily connect middleware and dynamically change message types (corresponding to topic of Kafka and exchange of RabbitMQ). These can be achieved through configuration files
@Input
The annotation identifies the input channel through which messages received enter the application
@Output
The annotation identifies the output channel through which the published message will leave the application
@StreamListener
Listen to the queue, which is used to receive messages from the consumer's queue
@EnableBinding
Refers to the binding of channel and exchange
Case description
Prepare the RabbitMQ environment( 79_ RabbitMQ environment configuration of bus Mentioned)
Three new sub modules in the project
- Cloud stream rabbitmq provider8801, as a producer, sends messages
- Cloud stream rabbitmq consumer 8802, as a message receiving module
- Cloud stream rabbitmq consumer 8803, as a message receiving module
87_Stream message driven producer
Create a new Module: cloud stream rabbitmq provider8801
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>LearnCloud</artifactId> <groupId>com.lun.springcloud</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-stream-rabbitmq-provider8801</artifactId> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-stream-rabbit</artifactId> </dependency> <!--Basic configuration--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
YML
server: port: 8801 spring: application: name: cloud-stream-provider cloud: stream: binders: # Configure the service information of rabbitmq to be bound here; defaultRabbit: # Represents the name of the definition, which is used for binding integration type: rabbit # Message component type environment: # Set the related environment configuration of rabbitmq spring: rabbitmq: host: localhost port: 5672 username: guest password: guest bindings: # Integration of services output: # This name is the name of a channel destination: studyExchange # Indicates the Exchange name definition to use content-type: application/json # Set the message type, json this time, and set "text/plain" for text binder: defaultRabbit # Set the specific settings of the message service to be bound eureka: client: # Configuration of Eureka registration by the client service-url: defaultZone: http://localhost:7001/eureka instance: lease-renewal-interval-in-seconds: 2 # Set the heartbeat interval (the default is 30 seconds) lease-expiration-duration-in-seconds: 5 # If the interval exceeds 5 seconds now (the default is 90 seconds) instance-id: send-8801.com # Display host name in information list prefer-ip-address: true # The access path becomes an IP address
Main startup class StreamMQMain8801
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication public class StreamMQMain8801 { public static void main(String[] args) { SpringApplication.run(StreamMQMain8801.class,args); } }
Business class
1. Send message interface
public interface IMessageProvider { public String send(); }
2. Send message interface implementation class
import com.lun.springcloud.service.IMessageProvider; import org.springframework.cloud.stream.annotation.EnableBinding; import org.springframework.cloud.stream.messaging.Source; import org.springframework.integration.support.MessageBuilder; import org.springframework.messaging.MessageChannel; import javax.annotation.Resource; import java.util.UUID; @EnableBinding(Source.class) //Define the push pipeline for messages public class MessageProviderImpl implements IMessageProvider { @Resource private MessageChannel output; // Message sending pipeline @Override public String send() { String serial = UUID.randomUUID().toString(); output.send(MessageBuilder.withPayload(serial).build()); System.out.println("*****serial: "+serial); return null; } }
3.Controller
import com.lun.springcloud.service.IMessageProvider; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; import javax.annotation.Resource; @RestController public class SendMessageController { @Resource private IMessageProvider messageProvider; @GetMapping(value = "/sendMessage") public String sendMessage() { return messageProvider.send(); } }
test
- Start 7001eureka
- Start rabbitmq( 79_ RabbitMQ environment configuration of bus)
- rabbitmq-plugins enable rabbitmq_management
- http://localhost:15672/
- Start 8801
- Visit- http://localhost:8801/sendMessage
- serial: UUID string will be printed in the background
88_Stream message driven consumers
Create a new Module: cloud stream rabbitmq consumer 8802
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>LearnCloud</artifactId> <groupId>com.lun.springcloud</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloud-stream-rabbitmq-consumer8802</artifactId> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-stream-rabbit</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <!--Basic configuration--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
YML
server: port: 8802 spring: application: name: cloud-stream-consumer cloud: stream: binders: # Configure the service information of rabbitmq to be bound here; defaultRabbit: # Represents the name of the definition, which is used for binding integration type: rabbit # Message component type environment: # Set the related environment configuration of rabbitmq spring: rabbitmq: host: localhost port: 5672 username: guest password: guest bindings: # Integration of services input: # This name is the name of a channel destination: studyExchange # Indicates the Exchange name definition to use content-type: application/json # Set the message type. This time, it is an object json. If it is text, set "text/plain" binder: defaultRabbit # Set the specific settings of the message service to be bound eureka: client: # Configuration of Eureka registration by the client service-url: defaultZone: http://localhost:7001/eureka instance: lease-renewal-interval-in-seconds: 2 # Set the heartbeat interval (the default is 30 seconds) lease-expiration-duration-in-seconds: 5 # If the interval exceeds 5 seconds now (the default is 90 seconds) instance-id: receive-8802.com # Display host name in information list prefer-ip-address: true # The access path becomes an IP address
Main startup class StreamMQMain8802
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication public class StreamMQMain8802 { public static void main(String[] args) { SpringApplication.run(StreamMQMain8802.class,args); } }
Business class
import org.springframework.beans.factory.annotation.Value; import org.springframework.cloud.stream.annotation.EnableBinding; import org.springframework.cloud.stream.annotation.StreamListener; import org.springframework.cloud.stream.messaging.Sink; import org.springframework.messaging.Message; import org.springframework.stereotype.Component; @Component @EnableBinding(Sink.class) public class ReceiveMessageListenerController { @Value("${server.port}") private String serverPort; @StreamListener(Sink.INPUT) public void input(Message<String> message) { System.out.println("Consumer 1,----->Received message: "+message.getPayload()+" port: "+serverPort); } }
test
-
Start EurekaMain7001
-
Start streamqmain8801
-
Start streamqmain8802
-
8801 send 8802 receive message
89_Stream message repeated consumption
According to 8802, clone a copy of 8803 - cloud stream rabbitmq consumer 8803.
start-up
- RabbitMQ
- Service registration - 8801
- Message production - 8801
- Message consumption - 8802
- Message consumption - 8802
There are two problems after operation
- There is a problem of repeated consumption
- Message persistence problem
consumption
- http://localhost:8801/sendMessage
- At present, 8802/8803 have been received at the same time, and there is a problem of repeated consumption
- How to solve: grouping and persistent attribute group (important)
Actual production cases
For example, in the following scenario, when we deploy the order system in a cluster, we will get the order information from RabbitMQ. If an order is obtained by two services at the same time, it will cause data errors. We have to avoid this situation. At this time, we can use the message grouping in the Stream to solve the problem.
Note that multiple consumers in the same group in the Stream are in a competitive relationship, which can ensure that messages will only be consumed once by one of the applications. Different groups can be fully consumed (repeated consumption).
90_Stream's group solves message repeated consumption
principle
If microservice applications are placed in the same group, it can ensure that messages will only be consumed by one of them once.
Different groups can be consumed repeatedly. There will be competition within the same group, and only one of them can be consumed.
8802/8803 become different groups, and the two groups are different
group: A_Group,B_Group
8802 modify YML
spring: application: name: cloud-stream-provider cloud: stream: binders: # Configure the service information of rabbitmq to be bound here; defaultRabbit: # Represents the name of the definition, which is used for binding integration type: rabbit # Message component type environment: # Set the related environment configuration of rabbitmq spring: rabbitmq: host: localhost port: 5672 username: guest password: guest bindings: # Integration of services output: # This name is the name of a channel destination: studyExchange # Indicates the Exchange name definition to use content-type: application/json # Set the message type, json this time, and set "text/plain" for text binder: defaultRabbit # Set the specific settings of the message service to be bound group: A_Group #< ------------------------------------ key
8803 modify YML (similar to 8802, group: B_Group)
Conclusion: repeated consumption
8802/8803 implements polling packets, with only one consumer at a time. Messages sent by 8801 module can only be received by one of 8802 or 8803, which avoids repeated consumption.
8802/8803 become the same group, and the two groups are the same
group: A_Group
8802 modify YMLgroup: A_Group
8803 modify YMLgroup: A_Group
Conclusion: for multiple instances of microservice in the same group, only one will get it at a time
91_ Message persistence of stream
Through the above, we have solved the problem of repeated consumption, and then look at persistence.
Stop 8802/8803 and remove the group of 8802 group: A_Group, group of 8803 group: A_Group is not removed.
8801 sends 4 messages to RabbitMq first.
Start 8802 first, there is no grouping attribute configuration, and there is no message printed in the background.
Start 8803 again, there is grouping attribute configuration, and messages on MQ are printed in the background. (message persistence embodiment)
92_ What is sleuth
Why does this technology appear? What problems should be solved?
In the microservice framework, a request initiated by the client will be called by multiple different service nodes in the back-end system to produce the final request result. Each previous request will form a complex distributed service call link. High delay or error in any link will cause the final failure of the entire request.
What is it?
- https://github.com/spring-cloud/spring-cloud-sleuth
- Spring Cloud Sleuth provides a complete solution for service tracking
- It provides tracking solutions in distributed systems and supports zipkin
solve
sleuth
English [slu θ] Beauty [slu θ]
n. Detective
93_Sleuth's zipkin setup and installation
1.zipkin
download
- SpringCloud does not need to build Zipkin Server by itself since version F, just call jar package
- https://dl.bintray.com/openzipkin/maven/io/zipkin/java/zipkin-server/
- zipkin-server-2.12.9-exec.jar
Run jar
java -jar zipkin-server-2.12.9-exec.jar
Run console
http://localhost:9411/zipkin/
term
Complete call link
Indicates a request link. A link is uniquely identified by Trace ld, and span identifies the request information initiated. Each span is associated by parent id
A link is uniquely identified by Trace ld, span identifies the request information initiated, and each span is associated by parent id.
The dependencies of the whole link are as follows:
Explanation of terms
- Trace: a Span set similar to a tree structure, representing a call link with a unique identifier
- Span: refers to the source of the calling link. Generally speaking, span is a request for information
94_Sleuth link monitoring display
2. Service provider
cloud-provider-payment8001
POM
<!--Contains sleuth+zipkin--> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-zipkin</artifactId> </dependency>
YML
spring: application: name: cloud-payment-service zipkin: #< --------------------------------- key base-url: http://localhost:9411 sleuth: #< --------------------------------- key sampler: #The sampling rate value is between 0 and 1, and 1 means all acquisitions probability: 1 datasource: type: com.alibaba.druid.pool.DruidDataSource # Current data source operation type driver-class-name: org.gjt.mm.mysql.Driver # mysql driver package url: jdbc:mysql://localhost:3306/db2019?useUnicode=true&characterEncoding=utf-8&useSSL=false username: root password: 123456
Business PaymentController
@RestController @Slf4j public class PaymentController { ... @GetMapping("/payment/zipkin") public String paymentZipkin() { return "hi ,i'am paymentzipkin server fall back,welcome to here, O(∩_∩)O ha-ha~"; } }
3. Service consumer (caller)
cloue-consumer-order80
POM
<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-zipkin</artifactId> </dependency>
YML
spring: application: name: cloud-order-service zipkin: base-url: http://localhost:9411 sleuth: sampler: probability: 1
Business class OrderController
// ====================> zipkin+sleuth @GetMapping("/consumer/payment/zipkin") public String paymentZipkin() { String result = restTemplate.getForObject("http://localhost:8001"+"/payment/zipkin/", String.class); return result; } }
4. Start eureka7001/8001/80 - 80 successively and call 8001 for several tests
5. Open the browser to access: http://localhost:9411
95_ Introduction to cloud Alibaba
Why does SpringCloud alibaba appear
Spring Cloud Netflix project enters maintenance mode
https://spring.io/blog/2018/12/12/spring-cloud-greenwich-rc1-available-now
What is maintenance mode?
Putting the module in maintenance mode means that the Spring Cloud team will not add new functions to the module.
They will fix block level bug s and security issues, and they will also consider and review small pull request s from the community.
What does SpringCloud alibaba bring
What is it?
Spring Cloud Alibaba is committed to providing a one-stop solution for microservice development. This project includes the necessary components for developing distributed application microservices, so that developers can easily use these components to develop distributed application services through the Spring Cloud programming model.
Relying on Spring Cloud Alibaba, you only need to add some annotations and a few configurations to connect Spring Cloud applications to Alibaba microservice solutions and quickly build distributed application systems through Alibaba middleware.
Birth: on October 31, 2018, Spring Cloud Alibaba officially entered the official incubator of Spring Cloud and released its first version in Maven central library.
What can I do
- Service flow restriction and degradation: by default, it supports the access of WebServlet, WebFlux, OpenFeign, RestTemplate, Spring Cloud Gateway, Zuul, Dubbo and RocketMQ flow restriction and degradation functions. It can modify the flow restriction and degradation rules in real time through the console at runtime, and it also supports the viewing of flow restriction and degradation Metrics monitoring.
- Service registration and discovery: it adapts to the Spring Cloud service registration and discovery standard, and integrates the support of Ribbon by default.
- Distributed configuration management: it supports the external configuration in the distributed system and automatically refreshes when the configuration is changed.
- Message driven capabilities: build message driven capabilities for microservice applications based on Spring Cloud Stream.
- Distributed transaction: use the @GlobalTransactional annotation to solve the problem of distributed transaction efficiently and without business intrusion.
- Alibaba cloud object storage: Alibaba cloud provides massive, safe, low-cost, and highly reliable cloud storage services. It supports the storage and access of any type of data in any application, at any time and anywhere.
- Distributed task scheduling: provide second level, accurate, highly reliable and highly available scheduled (based on Cron expression) task scheduling services. At the same time, it provides distributed task execution models, such as grid tasks. Grid tasks support the even distribution of Hai quantum tasks to all workers (schedulerx clients) for execution.
- Alibaba cloud SMS service: a global SMS service with friendly, efficient and intelligent interconnected communication capabilities helps enterprises quickly build customer access channels.
Where to
If you need to use a published version, add the following configuration in dependencyManagement.
<dependencyManagement> <dependencies> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-alibaba-dependencies</artifactId> <version>2.2.5.RELEASE</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>
Then add the dependencies you need to use in dependencies.
How do you play?
- Sentinel : take the flow as the starting point to protect the stability of services from multiple dimensions, such as flow control, fuse degradation, system load protection, etc.
- Nacos : a dynamic service discovery, configuration management and service management platform that is easier to build cloud native applications.
- RocketMQ : an open source distributed messaging system, based on highly available distributed cluster technology, provides low latency, highly reliable message publishing and subscription services.
- Dubbo : Apache Dubbo is a high-performance Java RPC framework.
- Seata : Alibaba open source product, an easy-to-use high-performance distributed transaction solution for microservices.
- Alibaba Cloud OSS : Alibaba cloud Object Storage Service (OSS) is a massive, secure, low-cost, and highly reliable cloud storage service provided by Alibaba cloud. You can store and access any type of data in any application, anytime, anywhere.
- Alibaba Cloud SchedulerX : a distributed task scheduling product developed by Alibaba middleware team provides second level, accurate, highly reliable and highly available timed (based on Cron expression) task scheduling services.
- Alibaba Cloud SMS : SMS services covering the world, friendly, efficient and intelligent interconnected communication capabilities, help enterprises quickly build customer access channels.
Spring Cloud Alibaba learning materials acquisition
-
Official website
- https://spring.io/projects/spring-cloud-alibaba#overview
-
english
- https://github.com/alibaba/spring-cloud-alibaba
- https://spring-cloud-alibaba-group.github.io/github-pages/greenwich/spring-cloud-alibaba.html
-
chinese
- https://github.com/alibaba/spring-cloud-alibaba/blob/master/README-zh.md
96_Nacos introduction and download
Why Nacos
- The first four letters are the first two letters of Naming and Configuration respectively, and the last s is Service.
What is it?
- A dynamic service discovery, configuration management and service management platform that is easier to build cloud native applications.
- Nacos: Dynamic Naming and Configuration Service
- Nacos is the combination of registration center + configuration center - > Nacos = Eureka + config + bus
What can I do
- Replace Eureka as the service registration center
- Replace Config as the service configuration center
Where to
- https://github.com/alibaba/nacos/releases
- Official website documents
Comparison of registration centers in China
Service registration and discovery framework
CAP model
Console management
Community activity
Eureka
AP
support
Low (2.x version closed source)
Zookeeper
CP
I won't support it
in
consul
CP
support
high
Nacos
AP
support
high
It is said that Nacos has more than 100000 instances running in Alibaba, and has passed the test of various large-scale traffic such as the double 11.
97_Nacos installation
- The local Java8+Maven environment has been OK first
- from Official website Download Nacos
- Unzip the installation package and directly run startup.cmd under the bin directory
- Access directly after the command runs successfully http://localhost:8848/nacos The default account and password are all Nacos
- Results page
98_ Service provider registration of Nacos
New module - cloudalibaba provider payment9001
POM
Parent POM
<dependencyManagement> <dependencies> <!--spring cloud alibaba 2.1.0.RELEASE--> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-alibaba-dependencies</artifactId> <version>2.1.0.RELEASE</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>
POM of this module
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>cloud2020</artifactId> <groupId>com.atguigu.springcloud</groupId> <version>1.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloudalibaba-provider-payment9001</artifactId> <dependencies> <!--SpringCloud ailibaba nacos --> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId> </dependency> <!-- SpringBoot integration Web assembly --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <!--Daily general jar Package configuration--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
YML
server: port: 9001 spring: application: name: nacos-payment-provider cloud: nacos: discovery: server-addr: localhost:8848 #Configure Nacos address management: endpoints: web: exposure: include: '*'
Main start
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.client.discovery.EnableDiscoveryClient; @EnableDiscoveryClient @SpringBootApplication public class PaymentMain9001 { public static void main(String[] args) { SpringApplication.run(PaymentMain9001.class, args); } }
Business class
import org.springframework.beans.factory.annotation.Value; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RestController; @RestController public class PaymentController { @Value("${server.port}") private String serverPort; @GetMapping(value = "/payment/nacos/{id}") public String getPayment(@PathVariable("id") Integer id) { return "nacos registry, serverPort: "+ serverPort+" id"+id; } }
test
- http://localhost:9001/payment/nacos/1
- nacos console
- nacos service registry + service provider 9001 are all OK
In order to demonstrate the load balancing of nacos in the next chapter, refer to 9001 and create 9002
- Create cloudalibaba provider payment9002
- 9002 other steps you know
- Or, if you don't want to create repetitive physical labor, you can use the IDEA function to directly copy the virtual port mapping
99_ Service consumer registration and load of Nacos
New module - cloudalibaba consumer Nacos order83
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>LearnCloud</artifactId> <groupId>com.lun.springcloud</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloudalibaba-consumer-nacos-order83</artifactId> <dependencies> <!--SpringCloud ailibaba nacos --> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId> </dependency> <!-- Introduce self defined api General package, which can be used Payment payment Entity --> <dependency> <groupId>com.lun.springcloud</groupId> <artifactId>cloud-api-commons</artifactId> <version>1.0.0-SNAPSHOT</version> </dependency> <!-- SpringBoot integration Web assembly --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <!--Daily general jar Package configuration--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
Why does nacos support load balancing? Because spring cloud starter Alibaba nacos discovery contains Netflix ribbon package.
YML
server: port: 83 spring: application: name: nacos-order-consumer cloud: nacos: discovery: server-addr: localhost:8848 #The name of the micro service that the consumer will visit (the micro service provider that successfully registered into nacos) service-url: nacos-user-service: http://nacos-payment-provider
Main start
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.client.discovery.EnableDiscoveryClient; @EnableDiscoveryClient @SpringBootApplication public class OrderNacosMain83 { public static void main(String[] args) { SpringApplication.run(OrderNacosMain83.class,args); } }
Business class
ApplicationContextConfig
import org.springframework.cloud.client.loadbalancer.LoadBalanced; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.web.client.RestTemplate; @Configuration public class ApplicationContextConfig { @Bean @LoadBalanced public RestTemplate getRestTemplate() { return new RestTemplate(); } }
OrderNacosController
import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Value; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.RestController; import org.springframework.web.client.RestTemplate; import javax.annotation.Resource; @RestController @Slf4j public class OrderNacosController { @Resource private RestTemplate restTemplate; @Value("${service-url.nacos-user-service}") private String serverURL; @GetMapping(value = "/consumer/payment/nacos/{id}") public String paymentInfo(@PathVariable("id") Long id) { return restTemplate.getForObject(serverURL+"/payment/nacos/"+id,String.class); } }
test
- Launch the nacos console
- http://localhost:83/Eonsumer/payment/nacos/13
- 83 visit 9001/9002 and poll the load OK
100_ Comparison and improvement of Nacos service registration center
Nacos panorama
Nacos and CAP
Comparison between Nacos and other registries
Nacos service discovery instance model
Nacos supports the switching between AP and CP modes
C is that all nodes see the same data at the same time; The definition of A is that all requests will receive A response.
When to choose which mode to use
Generally speaking, if you don't need to store service level information and the service instance is registered through Nacos client and can maintain heartbeat reporting, you can choose AP mode. The current mainstream services, such as Spring cloud and Dubbo services, are applicable to the AP mode. The AP mode weakens the consistency for the possibility of services, so only temporary instances can be registered under the AP mode.
If you need to edit or store configuration information at the service level, CP is required, and K8S service and DNS service are applicable to CP mode. In CP mode, persistent instance registration is supported. At this time, the Raft protocol is used as the cluster operation mode. In this mode, the service must be registered before registering the instance. If the service does not exist, an error will be returned.
Switching command:
curl -X PUT '$NACOS_SERVER:8848/nacos/v1/ns/operator/switches?entry=serverMode&value=CP
101_ Service configuration center of Nacos
Basic configuration
cloudalibaba-config-nacos-client3377
POM
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <parent> <artifactId>cloud2020</artifactId> <groupId>com.atguigu.springcloud</groupId> <version>1.0-SNAPSHOT</version> </parent> <modelVersion>4.0.0</modelVersion> <artifactId>cloudalibaba-config-nacos-client3377</artifactId> <dependencies> <!--nacos-config--> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId> </dependency> <!--nacos-discovery--> <dependency> <groupId>com.alibaba.cloud</groupId> <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId> </dependency> <!--web + actuator--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <!--General basic configuration--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-devtools</artifactId> <scope>runtime</scope> <optional>true</optional> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> </dependencies> </project>
YML
Nacos is the same as springcloud config. During the initialization of the project, it is necessary to pull the configuration from the configuration center first. After pulling the configuration, the normal startup of the project can be guaranteed.
There is a priority order for loading configuration files in springboot, and the priority of bootstrap is higher than that of application
bootstrap
# nacos configuration server: port: 3377 spring: application: name: nacos-config-client cloud: nacos: discovery: server-addr: localhost:8848 #Nacos service registry address config: server-addr: localhost:8848 #Nacos as the configuration center address file-extension: yaml #Specify the configuration of yaml format group: DEV_GROUP namespace: 7d8f0f5a-6a53-4785-9686-dd460158e5d4 # ${spring.application.name}-${spring.profile.active}.${spring.cloud.nacos.config.file-extension} # nacos-config-client-dev.yaml # nacos-config-client-test.yaml ----> config.info
application
spring: profiles: active: dev # Represents the development environment #active: test # Represents the test environment #active: info
Main start
import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.client.discovery.EnableDiscoveryClient; @EnableDiscoveryClient @SpringBootApplication public class NacosConfigClientMain3377 { public static void main(String[] args) { SpringApplication.run(NacosConfigClientMain3377.class, args); } }
Business class
import org.springframework.beans.factory.annotation.Value; import org.springframework.cloud.context.config.annotation.RefreshScope; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RestController; @RestController @RefreshScope //Support the dynamic refresh function of Nacos. public class ConfigClientController { @Value("${config.info}") private String configInfo; @GetMapping("/config/info") public String getConfigInfo() { return configInfo; } }
Add configuration information in Nacos
The composition format of the dataid in Nacos and the matching rules with the SpringBoot configuration file
Note: the reason why spring.application Name, because it is part of the Nacos configuration management dataId field.
In Nacos Spring Cloud, the complete format of dataId is as follows:
${prefix}-${spring-profile.active}.${file-extension}
- Prefix defaults to the value of spring.application.name, which can also be configured through the configuration item spring.cloud.nacos.config.prefix.
- spring.profile.active is the profile corresponding to the current environment. For details, please refer to the Spring Boot documentation. Note: when spring.profile When active is empty, the corresponding connector - will also not exist, and the splicing format of datald will become ${prefix}.${file extension}
- File exception is the data format of the configuration content, which can be configured through the configuration item spring.Cloud.nacos.config.file-extension. Currently, only properties and yaml types are supported.
- The configuration is automatically updated through the Spring Cloud native annotation @RefreshScope.
Final formula:
${spring.application.name)}-${spring.profiles.active}.${spring.cloud.nacos.config.file-extension}
Configuration addition
Nacos interface configuration correspondence - set DataId
Configuration summary
test
- Before starting, you need to have the corresponding yaml configuration file under the column of nacos client - Configuration Management - configuration management
- Run the main startup class of cloud config Nacos client3377
- Call the interface to view the configuration information- http://localhost:3377/config/info
Built in dynamic refresh
Modify the yaml configuration file in Nacos, call the interface to view the configuration again, and you will find that the configuration has been refreshed.
102_ The relationship between namespace grouping and DataID of Nacos
Problem - Multi environment and multi project management
Question 1:
In actual development, usually a system will prepare
- dev development environment
- Test test environment
- prod production environment.
How to ensure that the service can correctly read the configuration file of the corresponding environment on Nacos when the specified environment is started
Question 2:
A large distributed microservice system will have many microservice sub projects, and each microservice project will have a corresponding development environment, testing environment, pre launch environment, formal environment... How to manage these microservice configurations
Graphical management interface of Nacos
Namespace+Group+Data lD? Why is it so designed?
What is 1
Similar to the package name and class name in Java, the outermost namespace can be used to distinguish the deployment environment. Group and DatalD logically distinguish two target objects.
2. Conditions of the three parties
DEFAULT: Namespace=public, Group=DEFAULT_GROUP, the DEFAULT Cluster is DEFAULT
- The default Namespace of Nacos is public, which is mainly used to realize isolation.
- For example, we now have three environments: development, testing and production. We can create three namespaces, and different namespaces are isolated.
- Group defaults to DEFAULT_GROUP, group can divide different micro services into the same group
- A Service is a microservice: a Service can contain multiple clusters. The DEFAULT Cluster of Nacos is DEFAULT, and Cluster is a virtual partition of a specified microservice.
- For example, for disaster recovery, the Service microservices are deployed in Hangzhou computer room and Guangzhou computer room respectively. At this time, the Service microservices in Hangzhou computer room can be given a cluster name (HZ), and the Service microservices in Guangzhou computer room can be given a cluster name (GZ). You can also try to make the microservices in the same computer room call each other to improve performance.
- Finally, Instance is the Instance of micro service.
103_Nacos' DataID configuration
Specify spring.profile.active and the DatalD of the configuration file to read different configurations in different environments
Default space + default grouping + create two DatalD, dev and test
- Create a new dev configuration DatalD
- Create a new test configuration DatalD
You can read configuration files in multiple environments through the spring.profile.active attribute
test
- http://localhost:3377/config/info
- Load test/dev as you configure
104_Nacos' Group grouping scheme
Environment differentiation through groups - create a new Group
Create a new configuration file DatalD on the nacos graphical interface console
bootstrap+application
Add a group configuration under config. Configurable as DEV_GROUP or TEST GROUP
105_Nacos' Namespace scheme
Create a new Namespace for dev/test
Go back to service management - service list view
Fill in according to the domain name configuration
YML
# nacos configuration server: port: 3377 spring: application: name: nacos-config-client cloud: nacos: discovery: server-addr: localhost:8848 #Nacos service registry address config: server-addr: localhost:8848 #Nacos as the configuration center address file-extension: yaml #Specify the configuration of yaml format group: DEV_GROUP namespace: 7d8f0f5a-6a53-4785-9686-dd460158e5d4 #< ------------ specify a namespace # ${spring.application.name}-${spring.profile.active}.${spring.cloud.nacos.config.file-extension} # nacos-config-client-dev.yaml # nacos-config-client-test.yaml ----> config.info
106_Nacos cluster_ Architecture description
Official website structure diagram
Cluster deployment architecture diagram
Therefore, when open source, users are recommended to put all the service lists under a vip and then hang them under a domain name
http://ip1:port/openAPI In the direct connection IP mode, the machine needs to modify the IP before it can be used.
http://VIP:port/openAPI Mount the VIP mode, directly connect to the VIP, and hang the real ip of the server below, which is not readable.
http://nacos.com:port/openAPI Domain name + VIP mode, good readability, convenient for changing ip, recommended mode
The above official website translation, the real situation
According to the above, we need mysql database.
By default, Nacos uses an embedded database to store data. Therefore, if you start multiple Nacos nodes in the default configuration, there is a consistency problem in the data storage. To solve this problem, Nacos adopts centralized storage to support clustered deployment. Currently, it only supports MySQL storage.
Nacos supports three deployment modes
- Stand alone mode - used for testing and stand-alone trial.
- Cluster mode - used in production environment to ensure high availability.
- Multi cluster mode - used in multi data center scenarios.
Windows
cmd startup.cmd or double-click the startup.cmd file
Stand alone mode supports mysql
Before version 0.7, nacos used embedded database to store data in stand-alone mode, which made it inconvenient to observe the basic situation of data storage. Version 0.7 adds the ability to support mysql data sources. Specific operation steps:
-
Install database, version requirements: 5.6.5+
-
Initialize mysq database, database initialization file: Nacos mysql.sql
-
Modify conf/application Properties file, add support for mysql data source configuration (currently only mysql is supported), and Add url, user name and password of mysql data source.
spring.datasource.platform=mysql
db.num=1
db.url.0=jdbc:mysql://11.162.196.16:3306/nacos_devtest?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=nacos_devtest
db.password=youdontknow
Then start nacos in stand-alone mode, and all the data written by nacos to the embedded database is written to mysql.
107_Nacos persistent switching configuration
Nacos comes with an embedded database derby by default, pom.xml of nacos It can be seen from.
Configuration steps for switching from derby to mysql:
-
nacos-server-1.1.4 acosconf record, find the nacos-mysql.sql file, and execute the script.
-
Find application.properties in the directory of nacos-server-1.1.4 acosconf, and add the following configuration (modify the corresponding value as needed).
spring.datasource.platform=mysql
db.num=1
db.url.0=jdbc:mysql://localhost:3306/nacos_devtest?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=root
db.password=1234
Start Nacos, and you can see that it is a new empty recording interface, which used to be recorded into derby.
First of all, I would like to introduce myself. I graduated from Jiaotong University in 13 years. I once worked in a small company, went to large factories such as Huawei OPPO, and joined Alibaba in 18 years, until now. I know that most junior and intermediate Java engineers who want to improve their skills often need to explore and grow by themselves or sign up for classes, but there is a lot of pressure on training institutions to pay nearly 10000 yuan in tuition fees. The self-study efficiency of their own fragmentation is very low and long, and it is easy to encounter the ceiling technology to stop. Therefore, I collected a "full set of learning materials for java development" and gave it to you. The original intention is also very simple. I hope to help friends who want to learn by themselves and don't know where to start, and reduce everyone's burden at the same time. Add the business card below to get a full set of learning materials