Nacos cluster and persistent configuration

1, Official website description

https://nacos.io/zh-cn/docs/cluster-mode-quick-start.html

Real situation

By default, Nacos uses embedded database to store data. Therefore, if multiple Nacos nodes under the default configuration are started, there is a consistency problem in the data storage.
In order to solve this problem, Nacos adopts centralized storage to support cluster deployment. At present, it only supports MySQL storage.
https://nacos.io/zh-cn/docs/deployment.html

spring.datasource.platform=mysql

db.num=1
db.url.0=jdbc:mysql://11.162.196.16:3306/nacos_devtest?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=nacos_devtest
db.password=youdontknow

2, Nacos persistence configuration explanation

1. Nacos comes with embedded database derby by default

https://github.com/alibaba/nacos/blob/develop/config/pom.xml

2. Configuration steps for switching from derby to mysql

Find Nacos MySQL in the directory of nacos-server-1.1.4\nacos\conf SQL script, executed in your own database

/******************************************/
/*   Full database name = nacos_config   */
/*   Table name = config_info   */
/******************************************/
CREATE TABLE `config_info` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
  `data_id` varchar(255) NOT NULL COMMENT 'data_id',
  `group_id` varchar(255) DEFAULT NULL,
  `content` longtext NOT NULL COMMENT 'content',
  `md5` varchar(32) DEFAULT NULL COMMENT 'md5',
  `gmt_create` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Creation time',
  `gmt_modified` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Modification time',
  `src_user` text COMMENT 'source user',
  `src_ip` varchar(20) DEFAULT NULL COMMENT 'source ip',
  `app_name` varchar(128) DEFAULT NULL,
  `tenant_id` varchar(128) DEFAULT '' COMMENT 'Tenant field',
  `c_desc` varchar(256) DEFAULT NULL,
  `c_use` varchar(64) DEFAULT NULL,
  `effect` varchar(64) DEFAULT NULL,
  `type` varchar(64) DEFAULT NULL,
  `c_schema` text,
  PRIMARY KEY (`id`),
  UNIQUE KEY `uk_configinfo_datagrouptenant` (`data_id`,`group_id`,`tenant_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='config_info';

/******************************************/
/*   Full database name = nacos_config   */
/*   Table name = config_info_aggr   */
/******************************************/
CREATE TABLE `config_info_aggr` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
  `data_id` varchar(255) NOT NULL COMMENT 'data_id',
  `group_id` varchar(255) NOT NULL COMMENT 'group_id',
  `datum_id` varchar(255) NOT NULL COMMENT 'datum_id',
  `content` longtext NOT NULL COMMENT 'content',
  `gmt_modified` datetime NOT NULL COMMENT 'Modification time',
  `app_name` varchar(128) DEFAULT NULL,
  `tenant_id` varchar(128) DEFAULT '' COMMENT 'Tenant field',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uk_configinfoaggr_datagrouptenantdatum` (`data_id`,`group_id`,`tenant_id`,`datum_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Add tenant field';


/******************************************/
/*   Full database name = nacos_config   */
/*   Table name = config_info_beta   */
/******************************************/
CREATE TABLE `config_info_beta` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
  `data_id` varchar(255) NOT NULL COMMENT 'data_id',
  `group_id` varchar(128) NOT NULL COMMENT 'group_id',
  `app_name` varchar(128) DEFAULT NULL COMMENT 'app_name',
  `content` longtext NOT NULL COMMENT 'content',
  `beta_ips` varchar(1024) DEFAULT NULL COMMENT 'betaIps',
  `md5` varchar(32) DEFAULT NULL COMMENT 'md5',
  `gmt_create` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Creation time',
  `gmt_modified` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Modification time',
  `src_user` text COMMENT 'source user',
  `src_ip` varchar(20) DEFAULT NULL COMMENT 'source ip',
  `tenant_id` varchar(128) DEFAULT '' COMMENT 'Tenant field',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uk_configinfobeta_datagrouptenant` (`data_id`,`group_id`,`tenant_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='config_info_beta';

/******************************************/
/*   Full database name = nacos_config   */
/*   Table name = config_info_tag   */
/******************************************/
CREATE TABLE `config_info_tag` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
  `data_id` varchar(255) NOT NULL COMMENT 'data_id',
  `group_id` varchar(128) NOT NULL COMMENT 'group_id',
  `tenant_id` varchar(128) DEFAULT '' COMMENT 'tenant_id',
  `tag_id` varchar(128) NOT NULL COMMENT 'tag_id',
  `app_name` varchar(128) DEFAULT NULL COMMENT 'app_name',
  `content` longtext NOT NULL COMMENT 'content',
  `md5` varchar(32) DEFAULT NULL COMMENT 'md5',
  `gmt_create` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Creation time',
  `gmt_modified` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Modification time',
  `src_user` text COMMENT 'source user',
  `src_ip` varchar(20) DEFAULT NULL COMMENT 'source ip',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uk_configinfotag_datagrouptenanttag` (`data_id`,`group_id`,`tenant_id`,`tag_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='config_info_tag';

/******************************************/
/*   Full database name = nacos_config   */
/*   Table name = config_tags_relation   */
/******************************************/
CREATE TABLE `config_tags_relation` (
  `id` bigint(20) NOT NULL COMMENT 'id',
  `tag_name` varchar(128) NOT NULL COMMENT 'tag_name',
  `tag_type` varchar(64) DEFAULT NULL COMMENT 'tag_type',
  `data_id` varchar(255) NOT NULL COMMENT 'data_id',
  `group_id` varchar(128) NOT NULL COMMENT 'group_id',
  `tenant_id` varchar(128) DEFAULT '' COMMENT 'tenant_id',
  `nid` bigint(20) NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (`nid`),
  UNIQUE KEY `uk_configtagrelation_configidtag` (`id`,`tag_name`,`tag_type`),
  KEY `idx_tenant_id` (`tenant_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='config_tag_relation';

/******************************************/
/*   Full database name = nacos_config   */
/*   Table name = group_capacity   */
/******************************************/
CREATE TABLE `group_capacity` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Primary key ID',
  `group_id` varchar(128) NOT NULL DEFAULT '' COMMENT 'Group ID，An empty character indicates the entire cluster',
  `quota` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Quota, 0 means the default value is used',
  `usage` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Usage',
  `max_size` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'The maximum size of a single configuration, in bytes. 0 means the default value is used',
  `max_aggr_count` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'The maximum number of aggregate sub configurations, 0 means the default value is used',
  `max_aggr_size` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'The upper limit of the sub configuration size of a single aggregate data, in bytes. 0 means the default value is used',
  `max_history_count` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Maximum change history quantity',
  `gmt_create` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Creation time',
  `gmt_modified` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Modification time',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uk_group_id` (`group_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Cluster, each Group Capacity information table';

/******************************************/
/*   Full database name = nacos_config   */
/*   Table name = his_config_info   */
/******************************************/
CREATE TABLE `his_config_info` (
  `id` bigint(64) unsigned NOT NULL,
  `nid` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `data_id` varchar(255) NOT NULL,
  `group_id` varchar(128) NOT NULL,
  `app_name` varchar(128) DEFAULT NULL COMMENT 'app_name',
  `content` longtext NOT NULL,
  `md5` varchar(32) DEFAULT NULL,
  `gmt_create` datetime NOT NULL DEFAULT '2010-05-05 00:00:00',
  `gmt_modified` datetime NOT NULL DEFAULT '2010-05-05 00:00:00',
  `src_user` text,
  `src_ip` varchar(20) DEFAULT NULL,
  `op_type` char(10) DEFAULT NULL,
  `tenant_id` varchar(128) DEFAULT '' COMMENT 'Tenant field',
  PRIMARY KEY (`nid`),
  KEY `idx_gmt_create` (`gmt_create`),
  KEY `idx_gmt_modified` (`gmt_modified`),
  KEY `idx_did` (`data_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Multi tenant transformation';


/******************************************/
/*   Full database name = nacos_config   */
/*   Table name = tenant_capacity   */
/******************************************/
CREATE TABLE `tenant_capacity` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Primary key ID',
  `tenant_id` varchar(128) NOT NULL DEFAULT '' COMMENT 'Tenant ID',
  `quota` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Quota, 0 means the default value is used',
  `usage` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Usage',
  `max_size` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'The maximum size of a single configuration, in bytes. 0 means the default value is used',
  `max_aggr_count` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Maximum number of aggregate sub configurations',
  `max_aggr_size` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'The upper limit of the sub configuration size of a single aggregate data, in bytes. 0 means the default value is used',
  `max_history_count` int(10) unsigned NOT NULL DEFAULT '0' COMMENT 'Maximum change history quantity',
  `gmt_create` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Creation time',
  `gmt_modified` datetime NOT NULL DEFAULT '2010-05-05 00:00:00' COMMENT 'Modification time',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uk_tenant_id` (`tenant_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='Tenant capacity information table';


CREATE TABLE `tenant_info` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'id',
  `kp` varchar(128) NOT NULL COMMENT 'kp',
  `tenant_id` varchar(128) default '' COMMENT 'tenant_id',
  `tenant_name` varchar(128) default '' COMMENT 'tenant_name',
  `tenant_desc` varchar(256) DEFAULT NULL COMMENT 'tenant_desc',
  `create_source` varchar(32) DEFAULT NULL COMMENT 'create_source',
  `gmt_create` bigint(20) NOT NULL COMMENT 'Creation time',
  `gmt_modified` bigint(20) NOT NULL COMMENT 'Modification time',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uk_tenant_info_kptenantid` (`kp`,`tenant_id`),
  KEY `idx_tenant_id` (`tenant_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='tenant_info';

CREATE TABLE users (
    username varchar(50) NOT NULL PRIMARY KEY,
    password varchar(500) NOT NULL,
    enabled boolean NOT NULL
);

CREATE TABLE roles (
    username varchar(50) NOT NULL,
    role varchar(50) NOT NULL
);

INSERT INTO users (username, password, enabled) VALUES ('nacos', '$2a$10$EuWPZHzz32dJN7jexM34MOeYirDdFAZm2kuWj7VEOJhhZkDrxfvUu', TRUE);

INSERT INTO roles (username, role) VALUES ('nacos', 'ROLE_ADMIN');

Find application. In the directory of nacos-server-1.1.4\nacos\conf Properties file, add the content to be configured

spring.datasource.platform=mysql
 
db.num=1
db.url.0=jdbc:mysql://localhost:3306/nacos_config?characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true
db.user=root
db.password=123456

3, Nacos+MySQL production environment configuration for Linux

It is estimated that 1 nginx+3 nacos registries + 1 mysql

Download linux version from Nacos: https://github.com/alibaba/nacos/releases/tag/1.1.4 , unzip and install

Cluster configuration steps (key points)

1. mysql database configuration on Linux server

Refer to the above and execute the SQL script

2,application.properties configuration

Refer to the above and add the configuration. If you need remote access to the database, remember to authorize

#mysql authorized remote access
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;
flush privileges;

3. Cluster configuration of nacos on Linux server conf

Sort out the different service port numbers of three nacos machines

Copy out the cluster conf

The modifications are as follows. This IP cannot be written to 127.0.0.1. It must be an IP recognized by the Linux command hostname -i

4. Edit Nacos startup script startup SH, so that it can accept different boot ends

/There is startup in the nacos/bin directory sh. Usually, the startup of the stand-alone version is/ startup.sh is enough. However, for cluster startup, we hope to start different nacos instances by passing different port numbers similar to the shell commands of other software.

Command:/ startup.sh -p 3333 indicates to start the nacos server instance with port number 3333 and the cluster in the previous step Conf configuration is consistent.

The amendments are as follows:

The implementation method is as follows:

5. The configuration of Nginx is used as the load balancer

Modify the configuration file of nginx conf

upstream cluster{                                                        
    server 127.0.0.1:3333;
    server 127.0.0.1:4444;
    server 127.0.0.1:5555;
}

server{                 
    listen 1111;
    server_name localhost;
    location /{
         proxy_pass http://cluster;
                                                        
    }
....ellipsis
}

Start as specified

6. Testing

Up to now, the construction of 1 Nginx+3 nacos registries + 1 mysql has been completed

Test access to nacos through nginx: https: / / write your own virtual machine ip:1111/nacos/#/login
Create a new configuration test: insert a record in mysql of linux server
Test the microservice cloudalibaba provider payment9002 and start to register into the nacos cluster

7. Summary

Posted by jingcleovil on Thu, 14 Apr 2022 01:57:25 +0930

Programs Wiki