New and old cluster migration of Minio data

mc Command provides a convenient management ~/.mc/config.json How to configure the host information in the file , You can also manually modify this configuration file with a text editor

1、 download

wget https://dl.min.io/client/mc/release/linux-amd64/mc -P /usr/local/bin/
chmod +x /usr/local/bin/mc

2、 add to minio Server configuration { The target service And The source service Need to be }

mc config host add ${
    MINIO_NAME} http://${
    HOST_IP}:${
    Port} ${
    UserName} ${
    PASSWD}
#  Refer to the following :
mc config host add minio_AZ http://192.168.181.28:9024 admin 11111111
mc config host add minio_UAT http://192.168.180.37:39000 admin [email protected]

3、 see minio Server configuration

mc config host ls

4、 minio Data synchronization between

mc mirror --overwrite ${
    Source} ${
    Dest}
#  for example ：
mc mirror --overwrite minio_AZ/az-flow minio_UAT/az-flow

5、 Other commands

mc Detailed command

MinIO Client (mc) by ls,cat,cp,mirror,diff,find etc. UNIX The command provides an alternative . It supports file systems and is compatible with Amazon S3 Cloud storage services for （AWS Signature v2 and v4）.

Copyls List files and folders .
mb Create a bucket or folder .
cat Show file and object content .
pipe Will a STDIN Redirect to an object or file or STDOUT.
share Generate... For sharing URL.
cp Copy files and objects .
mirror Mirror buckets and folders .
find Find files based on parameters .
diff Compare the differences between two folders or buckets .
rm Delete files and objects .
events Management object notification .
watch Listen for file and Object Events .
policy Manage access policies .
session by cp Command management saved session .
config management mc The configuration file .
update Check for software updates .
version Output version information .

Get cluster information

#  Execute the command to test whether it can be connected , If there is a file or at least one  Bucket, Will be listed in the echo ：
mc ls minio_UAT

#  Execute the command to obtain the cluster data size , If there is a file or at least one exists  Bucket, The total size of all files in the current cluster will be printed on the last line ：
mc du minio_UAT

Torture （ Mirror image ）

# Use the command to  Bucket  Torture ,mc  It is impossible to copy the whole cluster , A single command can only be used one by one  Bucket  To operate ：
mc mirror $SrcCluster/$srcBucket $DestCluster
# for example ：
mc mirror clusterA/bucketa clusterB
mc mirror  The command may not specify  Dest  Of  Bucket, If  $DestCluster  There is no corresponding name  Bucket, It will be created automatically during copying .

# To realize copying without stopping the machine , You can use the following command ：
mc mirror -w $srcCluster/Bucket $destCluster
#-w  Parameters can make  mirror  Command continuously monitors a directory . because  mirror  You can only copy the list Bucket, If you use  Shell  The script goes on for  When polling the cluster for copying , once Bucket  Too much , This command may cause a large number of monitoring threads , Increase the load .

Check whether it is completely finished

# Use the command to check if there are incomplete files transferred , If there are incomplete files , Will be listed in the echo ：
mc ls --incomplete minio_UAT

# If necessary , You can delete incomplete files in a bucket ：
mc rm --incomplete $hostName/$bucket

Some instructions

There is no need to worry about data fragmentation , Even if the number of nodes is not equal .

Because it accesses and obtains data directly from the cluster portal , Not at all ” disk to disk “ How to move , It doesn't matter whether a disk on a node stores erasure codes or fragmented data , Because the data read from the cluster address is complete , This is why it is necessary to obtain the cluster address .

Bucket stay MinIO Shall be deemed to be ” Mount Directory “ (mount), instead of “ Disk directory ” (mkdir).

mc cp The command needs to have the same name in the target cluster Bucket,mc mirror Unwanted , meanwhile mc mirror The hierarchy information of the directory will be saved automatically ,mc cp Need to specify additional --recursive Parameters , Please choose according to your needs .

mc mirror The cluster of to colony 、 colony to Local usage is not in Official documents Listed in , Probably belong to hack usage , Please pay attention to the possible risks .