rsync backup case

Client requirements
1. The client prepares the directory of backup in advance. The directory rules are as follows: / backup/nfs_172.16.1.31_2018-09-02
2. The client packs backup locally (system configuration file, application configuration, etc.) and copies it to / backup/nfs_172.16.1.31_2018-09-02
3. The client finally pushes the backed up data to the backup server
4. The client executes the script at 1 a.m. every day
5. The client keeps the data of the last 7 days locally to avoid wasting disk space

Server requirements
1. The server deploys rsync to receive the backup data pushed by the client
2. The server needs to check whether the data pushed by the client is complete every day
3. The server needs to notify the administrator of the verification results every day
4. The server only keeps the backup data for 6 months, and all the rest are deleted

Note: the backup directory of all servers must be / backup

1. The client puts the files to be backed up into the specified directory / backup/nfs_172.16.1.31_2018-09-02
2. The client uses the rsync command to push NFS once a day at 1 a.m_ 172.16.1.31_ 2018-09-05
3. The client can keep the data of the last 7 days

1, Host planning

host	IP	identity
backup	172.16.1.41	rsync server
nfs	172.16.1.31	rsync client
web01	172.161.7	rsync client

2, Study needs

customer demand

1.The backup directory prepared by the client in advance. The directory rules are as follows::**/backup/nfs_172.16.1.31_2018-09-02** 

#The client needs to prepare a directory [backup] to store backup files. Under this directory, a directory composed of [hostname _IP _time] is automatically generated

2.The client packs the backup locally(System configuration file, application configuration, etc)Copy to**/backup/nfs_172.16.1.31_2018-09-02**

#The system configuration file and application configuration are uniformly defined here as [/ etc/passwd] [/ etc/fstab], packaged with * * tar * * command and copied to the directory just created

3.The client finally pushes the backed up data to the backup server

#rsync push to backup server [backup]

4.The client executes the script at 1 a.m. every day

#Just schedule the task

5.The client keeps the data of the last 7 days locally, Avoid wasting disk space

#Delete the contents of the previous seven days

Server requirements

1.Server deployment rsync，It is used to receive backup data pushed by the client

#Deploy rsync

2.The server needs to check whether the data pushed by the client is complete every day

#md5sum can be used for verification

3.The server needs to notify the administrator of the verification results every day

#Deploy mail system

4.The server only retains backup data for 6 months,Delete the rest

#Delete the content of the previous 180 days

3, Client preparation

[on web01 and nfs]

Take nfs as an example, web01 can copy the script

Batch modification time. And execute the script to generate the data file

[root@nfs scripts]# for i in {1..30};do date -s 2021/2/$i;sh /scripts/client_backup.sh;done

Client script

1,establish/backup catalogue
[root@nfs ~]# mkdir /backup

2,Create script storage directory
[root@nfs ~]# mkdir /scripts

3,Create a script in the script directory

[root@nfs scripts]# vim client_backup.sh

#!/bin/bash
#Date:2021-2-8
#Author:zxm
#Warning: the command uses an absolute path

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin

#Define variables
Hostname=$(hostname)
IP=$(ifconfig eth1 | awk 'NR==2 {print $2}')
Date=$(date +%F)
SRC=/backup
DEST=${Hostname}_${IP}_${Date}

#1. Create directory

#mkdir -p $SRC/$DEST

#There is no error in this way. If you need to back up twice on the same day,
#Then the action of creating a directory has been operated twice, and a judgment is made here
#If this folder exists, you don't have to create it 

#[- d $SRC/$DEST] judge whether the directory exists and | realize the function of logical or [see extended note content]
[ -d $SRC/$DEST ] || mkdir -p $SRC/$DEST

#2. Backup files

#Here, we can also judge whether there are compressed files at one time. If the files to be backed up need to be backed up twice a day, and
#If the content is changed, you cannot judge because the added content cannot be backed up

#In the tar command, c is to create, z is to compress gz format, f is to force, and P is to solve this error = = > tar: removing leading ` / 'from member names

[ -f $SRC/$DEST/sys.tar.gz ] || /usr/bin/tar czfP $SRC/$DEST/sys.tar.gz /etc/fstab /etc/passwd && \
[ -f $SRC/$DEST/other.tar.gz ] || /usr/bin/tar czfP $SRC/$DEST/other.tar.gz  /scripts/client_backup.sh  

#3. Marking with md5sum

#Similarly, if the script is executed only once a day and the content is not increased, there is no need to judge whether the value of md5sum has changed. Here are multiple tests on the same day to prevent the value from changing
[ -f $SRC/$DEST/flag_$Date ] || /usr/bin/md5sum $SRC/$DEST/*.tar.gz > $SRC/$DEST/flag_$Date

#4. Push backup files to backup server
export RSYNC_PASSWORD=123456
rsync -avz $SRC/$DEST rsync_backup@172.16.1.41::backup

#5. Keep the local data of the last 7 days
find $SRC/ -type d -mtime +7|xargs rm -rf

Expand knowledge

&&operator
 format
command1 && command2

&&The command on the left (command 1) returns true(That is, after 0 is returned and successfully executed,&&The command on the right (Command 2) can be executed; In other words, "if this command is executed successfully&&Then execute this command ".

	1,Use between commands && Connect to realize the function of logic and.

	2,Only in && The command on the left returns true (command returns value) $? == 0)，&& The command on the right will be executed.

	3,As long as one command returns false (command returns value) $? == 1)，Later commands will not be executed.
=======================================================================================================================
||operator
 format
command1 || command2

||Then and&&contrary. If||Command on the left( command1)If the execution is not successful, execute||Command on the right( command2)；Or in other words, "if this command fails||Then execute this command.

	1,Use between commands || Connect and realize the function of logical or.

	2,Only in || The command on the left returns false (command return value) $? == 1)，|| The command on the right will be executed. This and c The logic or syntax functions in the language are the same, that is, to realize short-circuit logic or operation.

	3,As long as one command returns true (command returns value) $? == 0)，Later commands will not be executed.
	
-----------------------------------------------------------------------------------------------------------------------	
command mtime ,xargs use

mtime Chang Yu find Use it together, and there are other commands[ atime,ctime,amin,cmin,mmin]

Usage: find . {-atime/-ctime/-mtime/-amin/-cmin/-mmin} [-/+]num

Parameter analysis:

      1.First parameter“."，Represents the current directory. For other directories, you can enter absolute directory and relative directory location;

      2.The second parameter is divided into two parts, preceded by letters a,c,m Operation type, followed by time Is the date, min Minutes (note that only time,min As a unit);

      3.The third parameter is quantity, in which no sign indicates that it meets the quantity-Indicates that after the quantity is met, with+Indicates that the quantity meets the previous.

Specific explanation:
=======================================================================================================================
	atmin: Visit time( access time)，It refers to the time when the file was last read. It can be used touch Change the command to the current time;

     -atime<24 Hours> Search for files or directories that have been accessed at a specified time, in 24 hours.

     For example, the current time is May 18, 2016 14:10:00，Query may 18, 2016 00:00:00 By May 18, 2016 23:59:59 Accessed files.

      time Represents the date, and the time unit is day，The query statement is:

            find . -atime 0      (-amin (similarly)
=======================================================================================================================
	ctime: Change time( change time)，It refers to the file itself (permission, group and location)......)The last time to be changed, the change action can make chmod,chgrp,mv wait;

   -ctime<24 Hours> Find the file or directory that was changed at the specified time, in 24 hours.

     For example, the current time is May 18, 2016 14:10:00，Query may 18, 2016 00:00:00 By May 18, 2016 23:59:59 A modified file. The query statement is:

       find . -ctime 0
=======================================================================================================================	
	mtime: Modification time( modify time)，It refers to the time when the file content was last modified. The modification action can make echo Redirection vi wait;

     -mtime<24 Hours> Find files or directories that have been changed at a specified time, in 24 hours.

     For example, the current time is May 18, 2016 14:10:00，Query may 18, 2016 00:00:00 By May 18, 2016 23:59:59 A file whose contents have been modified. The query statement is:

            find . -mtime 0
            
-----------------------------------------------------------------------------------------------------------------------            
xargs

4, Server preparation

Install rsync

1.install rsync(Backup servers are installed)
[root@backup ~]# yum install -y rsync

    [root@backup ~]# rpm -qc rsync      #Find the configuration file
    /etc/rsyncd.conf    	 #Master profile
    /etc/sysconfig/rsyncd    #option

2.Server configuration
[root@backup ~]# vim /etc/rsyncd.conf
uid = rsync
gid = rsync
port = 873
fake super = yes
use chroot = no
max connections = 200
timeout = 600
ignore errors
read only = false
list = false
auth users = rsync_backup
secrets file = /etc/rsync.passwd
log file = /var/log/rsyncd.log

[backup]
comment = welcome to oldboyedu backup!
path = /backup


3.Create user on server
[root@backup ~]# useradd rsync -s /sbin/nologin -M

4.Server creates password file
[root@backup ~]# vim /etc/rsync.passwd
rsync_backup:123456

[root@backup ~]# echo "rsync_backup:123456" > /etc/rsync.passwd

#to grant authorization
[root@backup ~]# chmod 600 /etc/rsync.passwd

5.The server creates a real file directory
[root@backup ~]# mkdir /backup
[root@backup ~]# chown -R rsync.rsync /backup/

6.Server start service
[root@backup ~]# systemctl start rsyncd
[root@backup ~]# systemctl  enable rsyncd

#Verify startup
[root@backup ~]# netstat -lntp
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN      18673/rsync 
tcp6       0      0 :::873                  :::*                    LISTEN      18673/rsync   
       
[root@backup ~]# ps -ef | grep rsync
root      18673      1  0 17:01 ?        00:00:00 /usr/bin/rsync --daemon --no-detach
root      18680   7850  0 17:02 pts/0    00:00:00 grep --color=auto rsync

Configure mail server

1.Configure mail
[root@backup ~]# yum install mailx -y
[root@backup ~]# vim /etc/mail.rc
set from=123@qq.com
set smtp=smtps://smtp.qq.com:465
set smtp-auth-user=123@qq.com
set smtp-auth-password=xxxxxx  #Generate authorization code
set smtp-auth=login
set ssl-verify=ignore
set nss-config-dir=/etc/pki/nssdb/

2.Verify whether the mail can be sent successfully
[root@backup ~]# echo  "test"  | mail  -s "hello"  123@hotmail.com

Server script

1,establish/backup catalogue
[root@nfs ~]# mkdir /backup

2,Create script storage directory
[root@nfs ~]# mkdir /scripts

3,Write server script
[root@backup scripts]# vim server_backup.sh
#!/bin/bash
#Date:2021-2-8
#Author:zxm
#Warning: the command uses an absolute path

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin

#Define variables
Date=$(date +%F)
SRC=/backup

#1. Use md5 for verification and save the verification results
md5sum -c $SRC/*_$DATE/flag_$DATE > $SRC/Data_check_$DATE

#2. Send the saved result file to the administrator
mail -s "Rsync Backup $DATE" zxmabc@hotmail.com < $SRC/Data_check_$DATE

#3. Keep the data of the last 180 days
find $SRC/ -type d -mtime +180|xargs rm -rf

Finally, conduct the overall test:

​ 1. Delete the entire directory of the client
​ 2. Delete all contents under server / backup
​ 3. Write scheduled task test [at 1:00 a.m. on the modified client and 5:00 a.m. on the server]

#client
[root@nfs ~]# rm -rvf /backup/*

[root@nfs ~]# crontab -e
* 1 * * * sh /scripts/client_backup.sh

#Server
[root@backup ~]# rm -rvf /backup/*

[root@backup ~]# crontab -e
* 5 * * * sh /scripts/server_backup.sh

How to extend backup to multiple servers:

[root@web01 ~]# rsync -avz root@172.16.1.31:/server  /
[root@web01 ~]# sh /server/scripts/client_push_data.sh