Command line interface of alluxio

One 、 Command line interface —— Common user commands fs

Command line interface - Alluxio v2.6.0 Documentation

Alluxio The command line interface provides users with basic file system operations , You can use the following command to get all the subcommands ：

$ ./bin/alluxio fs
Usage: alluxio fs [generic options]
       [cat <path>]
       [checkConsistency [-r] <Alluxio path>]
       ...

For using Alluxio URI（ Such as ls, mkdir） parametric fs Sons command Come on , Parameter should either be complete Alluxio URI alluxio://<master-hostname>:<master-port>/<path>, Or the header information is omitted /<path>, To use the conf/allluxio-site.properties The default host name and port set in .

1.copyFromLocal Copy local files to Alluxio

copyFromLocal The command copies files from the local file system to Alluxio in , If you run this command on a machine that has Alluxio worker, Then the data will be stored here worker On , otherwise , The data will be randomly copied to a run Alluxio worker On the remote node of . If the target specified by this command is a folder , Then this folder and all its contents will be recursively copied to Alluxio in .

Use examples ： Use copyFromLocal The command can quickly copy data to alluxio In the system for subsequent processing ：

(base) [[email protected] alluxio-2.6.0]# ./bin/alluxio fs copyFromLocal /root/datatest /
Copied file:///root/datatest to /

2.copyToLocal Copy Alluxio File to local

copyToLocal The order will Alluxio Copy the files in to the local file system , If the target specified by this command is a folder , Then the folder and all its contents will be copied recursively .

Use examples ： Use copyToLocal The command can quickly download the output data for subsequent research or debugging ：

(base) [[email protected] alluxio-2.6.0]# alluxio fs copyToLocal /t00.txt t00.txt
Copied /t00.txt to file:///root/alluxio-2.6.0/t00.txt
(base) [[email protected] alluxio-2.6.0]# wc -l t00.txt

3.rm Delete

(base) [[email protected] alluxio-2.6.0]# ./bin/alluxio fs rm '/test.txt'
/test.txt has been removed
(base) [[email protected] alluxio-2.6.0]# ./bin/alluxio fs rm -R /default_tests_files
/default_tests_files has been removed
#-R  Parameters   For recursively deleting directories 


# In the path here / Generation refers to alluxio The root of the mount 
#Alluxio The root directory of must be a mount point 

4.checkConsistency contrast Alluxio With the underlying storage （ Synchronization problem ）

checkConsistency The command will compare Alluxio And the metadata of the underlying storage system under a given path . If the path is a directory , Then all contents in the directory will be compared . This command returns a list of all inconsistent files and directories , The system administrator decides whether to adjust these inconsistent data . for fear of Alluxio Inconsistent with the metadata of the underlying storage system , Your system should try to pass Alluxio To modify files and directories , Avoid direct access to the underlying storage system for modification .

If used -r Options , that checkConsistency The command will repair inconsistent files or directories , If inconsistent files or folders exist only on the underlying storage system , Then the corresponding metadata will be loaded into Alluxio in . If the metadata and specific data of the inconsistent file already exist Alluxio in , that Alluxio Specific data will be deleted , And reload the metadata of the file .

Be careful ： This command needs to request the of the directory subtree to be checked Read the lock , This means that you cannot write or update the files or directories of the directory subtree before the command is completed .

for example ,checkConsistency Command can be used to periodically check namespace integrity ：

#  List inconsistent files or directories 
$ ./bin/alluxio fs checkConsistency /
#
#  Fix inconsistent files or directories 
$ ./bin/alluxio fs checkConsistency -r /

problem ： Store directly at the bottom HDFS After adding a file in ,Alluxio No inconsistency detected in consistency detection ,Alluxio Out of sync with underlying storage

Solution ：

1） On demand synchronization , Forced to refresh

alluxio fs ls -R -Dalluxio.user.file.metadata.sync.interval=0 /xxxx
# Use the command to refresh a directory directly 
# System default alluxio.user.file.metadata.sync.interval=-1, Is to turn off automatic refresh 

2） Active synchronization

# Specify the directory to enable active synchronization 
$ alluxio fs startSync /XXX

# Turn off active synchronization 
$ alluxio fs stopSync /XX

problem ： I won't support it hdfs

Active Syncing is not supported on this UFS type: hdfs

5. Metadata synchronization function

By default ,* Alluxio expect All changes to the underlying file system are adopt Alluxio To carry out *. such Alluxio Just scan each UFS Catalog once , Thus in UFS Significantly improve performance when metadata operations are slow . When Appear in the Alluxio Outside right UFS In the case of changes , You need to use the metadata synchronization function to synchronize the two namespaces .

When Alluxio scanning UFS Directory and load its subdirectories Metadata when , It will create a copy of the metadata , So that in the future there is no need to start from UFS load . A cached copy of the metadata will be based on alluxio.user.file.metadata.sync.interval Configured by client properties Interval refresh . This attribute applies to client operations . for example , If the customer executes a command based on the configuration with the interval set to one minute , If the last refresh was a minute ago , The relevant metadata will be based on UFS Refresh . Set value to 0 Indicates that real-time metadata synchronization will be performed for each operation , And the default value -1 Indicates that metadata will not be resynchronized after initial loading .

Low interval value send Alluxio The client can quickly discover the right UFS External modification of , However, due to the result that UFS The number of times increased , So Reduce performance At a cost .

Metadata synchronization preserves each UFS Fingerprint records of documents , In order to Alluxio You can update the file when it changes . Fingerprint records include information such as file size and last modification time . If in UFS The file was modified in ,Alluxio The change will be detected by fingerprint , Release existing files Metadata , Then reload the metadata of the update file . If in UFS Added or deleted files in ,Alluxio The update will also refresh the metadata in its namespace accordingly .

1） Regular metadata synchronization

If UFS Update at scheduled intervals , It can be updated Manual trigger sync command . Run the following command to set the synchronization interval to 0：（ That is, forced refresh synchronization ）

$ ./bin/alluxio fs ls -R -Dalluxio.user.file.metadata.sync.interval=0 /path/to/sync

Centralized configuration

For the use of... From frequent updates UFS Cluster job of data , It is inconvenient to specify a synchronization interval for each client . If in master The synchronization interval is set in the configuration , All requests will be processed at the default synchronization interval .

stay master Point on alluxio-site.properties Set in ：

alluxio.user.file.metadata.sync.interval=1m

Be careful , need Restart master Node to enable New configuration .

2）HDFS Metadata active synchronization

stay 2.0 Version of the , A new feature has been introduced , Used in UFS by HDFS Keep Alluxio Space and UFS Synchronization between . This function is called active synchronization , Can be monitored HDFS Events and with master The upper and background tasks are scheduled in the UFS and Alluxio Synchronize metadata between namespaces . Since the active synchronization function depends on HDFS event , So only if UFS HDFS Version higher than 2.6.1 when , This feature is only available . You may need to change in the configuration file alluxio.underfs.version Value . About supported Hdfs List of versions , Please refer to HDFS Underlying storage .

To enable active synchronization on a directory , Run the following Alluxio command .

$ ./bin/alluxio fs startSync /syncdir

You can change it alluxio.master.ufs.active.sync.interval Option to control the active synchronization interval , The default value is 30 second .

To stop using active synchronization on a directory , Run the following Alluxio command .

$ ./bin/alluxio fs stopSync /syncdir

Be careful ： Release startSync when , A full scan of the synchronization point is scheduled . If the Alluxio Run as superuser ,stopSync All incomplete scans will be interrupted . If running as another user ,stopSync Will wait for the full scan to complete before executing .

You can use the following command to check which directories are currently actively synchronized .

$ ./bin/alluxio fs getSyncPathList

problem ：alluxio.exception.AlluxioException: Active Syncing is not supported on this UFS type: hdfs

6.alluxio And the data loading and releasing of the underlying storage load、free、persist

• free: Releasing data means from Alluxio Delete data from cache , Not from the bottom UFS Delete data from . After the release operation , Data is still available to users , But yes Alluxio Attempt to access the file after releasing it The performance of the client may be degraded .

• load: Loading data means removing it from UFS Copied to the Alluxio In cache . If Alluxio Use Memory based storage , After loading, the user may see I/O Performance improvement .

• persist: Persistent data means that Alluxio Data that may or may not have been modified in storage is written back to UFS. By writing data back to UFS, Can guarantee that if Alluxio When a node fails, the data is recoverable .

• TTL(Time to Live):TTL Property to set the lifetime of files and directories , With Remove data from when it exceeds its lifetime Alluxio Delete from space . It can also be configured TTL To delete data stored in UFS The corresponding data in .

1） Upload files to alluxio

（ adopt Alluxio Upload files ）

alluxio fs copyFromLocal anaconda-ks.cfg /

2） alluxio File is solidified to HDFS

（ Write to underlying storage HDFS in ）

alluxio fs persist /anaconda-ks.cfg

hadoop fs -ls /alluxio

persist The order will Alluxio Data in is persisted to the underlying file system . This command is an operation on data , Therefore, the execution time depends on the size of the file . After persistence , This file is backed up in the underlying file system , So the file is in Alluxio In the case that the data block in is eliminated or even lost , Still have access to .

Use examples ： After filtering out files containing useful data from a series of temporary files , You can use persist Command to persist it .

3） Release alluxio The file of

（free alluxio Documents in , Files on the underlying storage are not deleted , Just release Alluxio The storage ）

alluxio fs free /anaconda-ks.cfg

free Command request Alluxio master Transfer all data blocks of a file from Alluxio worker Middle elimination , If the command parameter is a folder , Then it will recursively act on its sub files and sub folders . This request is not guaranteed to have an immediate effect , Because the data block of the file may be being read .free The command is being master It will return immediately after receiving . Note that this command does not delete any data in the underlying file system , It will only affect the storage in Alluxio Data in . in addition , This operation will not affect the metadata , This means that if you run ls command , The file will still be displayed .

4） Reload HDFS File to alluxio file

alluxio fs load /anaconda-ks.cfg

load Command loads the data in the underlying file system into Alluxio in . If the machine running this command is running a Alluxio worker, Then the data will be moved to this worker On , otherwise , The data will be randomly moved to a worker On . If the file already exists in Alluxio in , Set up --local Options , And there are local worker, Then the data will be moved to this worker On . Otherwise, the command will not do anything . If the target of this command is a folder , Then its sub files and sub folders will be loaded recursively .

Use examples ： Use load Command can obtain data for data analysis .

5） Delete alluxio The file of

（ adopt Alluxio To delete files from the underlying storage system , The files of the underlying storage system will be deleted ）

alluxio fs rm /anaconda-ks.cfg

6. Set file expiration time

for example , want 3 Minutes later delete by runTests Created files :

$ ./bin/alluxio runTests -Dalluxio.user.file.create.ttl=3m \
  -Dalluxio.user.file.create.ttl.action=DELETE

Alluxio The client can be configured as long as Alluxio Namespace when you add a new file TTL attribute . When the intended user is using the file temporarily , passive TTL It is useful to , But it is inflexible , Because all requests from the same client will inherit same TTL attribute .

passive TTL Configure... With the following options :

• alluxio.user.file.create.ttl- stay Alluxio Set on the file in TTL The duration of the . By default , Not set TTL The duration of the .
• alluxio.user.file.create.ttl.action- For file settings TTL Actions after expiration stay Alluxio in . Be careful ： By default , This action “DELETE”, It will cause the file to be permanently deleted .

TTL It is not used by default , Enable only if the customer has strict data access mode .

for example , want 3 Minutes later delete by runTests Created files :

$ ./bin/alluxio runTests -Dalluxio.user.file.create.ttl=3m \
  -Dalluxio.user.file.create.ttl.action=DELETE

For this example , Make sure alluxio.master.ttl.checker.interval Is set to short interval , For example, one minute , In order to master Can quickly identify expired documents .

1）setTtl

setTtl Command to set a file or folder ttl Time , The unit is millisecond . If the current time is greater than the file creation time and ttl The sum of time , The action parameter will indicate the action to be performed .delete operation （ Default ） Will also delete Alluxio And files in the underlying file system , and free The operation will only delete Alluxio Documents in .

Use examples ： The administrator knows that some files will be useless after a period of time , You can use the delete Operation of the setTtl Command to clean up files ; If you just want to Alluxio Free up more space , You can use the free Operation of the setTtl Command to clean up Alluxio The contents of the document in .

# After 1 day, delete the file in Alluxio and UFS
$ ./bin/alluxio fs setTtl /data/good-for-one-day 86400000
# After 1 day, free the file from Alluxio
$ ./bin/alluxio fs setTtl --action free /data/good-for-one-day 86400000

example ：

# Time without units , The default is milliseconds 
#Sets a new TTL value for the file at path, performing an action, delete(default)/free after TTL expiry. The TTL to set can be in one of the unit: ms, millisecond, s, second, m, min, minute, h, hour, d, day, default to ms

(base) [[email protected] alluxio-2.6.0]# alluxio fs setTtl --action free /t00.txt 1m
TTL of path '/t00.txt' was successfully set to 60000 milliseconds, with expiry action set to FREE
(base) [[email protected] alluxio-2.6.0]# alluxio fs setTtl --action free /t00.txt 1000sec
TTL of path '/t00.txt' was successfully set to 1000000 milliseconds, with expiry action set to FREE
(base) [[email protected] alluxio-2.6.0]# alluxio fs setTtl --action free /t00.txt 1h
TTL of path '/t00.txt' was successfully set to 3600000 milliseconds, with expiry action set to FREE
(base) [[email protected] alluxio-2.6.0]# alluxio fs setTtl --action free /t00.txt 1d
TTL of path '/t00.txt' was successfully set to 86400000 milliseconds, with expiry action set to FREE

2）unsetTtl

unsetTtl Command deletion Alluxio Of a file in TTL. This command only works on metadata , Will not be culled or deleted Alluxio Data blocks in . Of the document TTL Value can be determined by setTtl Command reset .

Use examples ： In some special cases , When a file that was originally automatically managed needs to be manually managed , have access to unsetTtl command .

$ ./bin/alluxio fs unsetTtl /data/yesterday/data-not-yet-analyzed

7. Check Alluxio Cache capacity and usage

Alluxio shell command fsadmin report Provide a brief summary of available space And other useful information . The output example is as follows :

$ ./bin/alluxio fsadmin report
Alluxio cluster summary:
    Master Address: localhost/127.0.0.1:19998
    Web Port: 19999
    Rpc Port: 19998
    Started: 09-28-2018 12:52:09:486
    Uptime: 0 day(s), 0 hour(s), 0 minute(s), and 26 second(s)
    Version: 2.0.0
    Safe Mode: true
    Zookeeper Enabled: false
    Live Workers: 1
    Lost Workers: 0
    Total Capacity: 10.67GB
        Tier: MEM  Size: 10.67GB
    Used Capacity: 0B
        Tier: MEM  Size: 0B
    Free Capacity: 10.67GB

Alluxio shell It also allows the user to check Alluxio How much space is available and in use in the cache .

get Alluxio The cache always runs in bytes :

$ ./bin/alluxio fs getUsedBytes

get Alluxio Total cache capacity in bytes

$ ./bin/alluxio fs getCapacityBytes

8.checksum Calculation md5 value

checksum Command outputs a Alluxio Of documents md5 value .

for example ,checksum It can be used to verify Alluxio Whether the file contents in the file match those stored in the underlying file system or the local file system ：

$ ./bin/alluxio fs checksum /LICENSE
md5sum: bf0513403ff54711966f39b058e059a3
md5 LICENSE
MD5 (LICENSE) = bf0513403ff54711966f39b058e059a3

MD5 value , To put it simply , It's a document “ The fingerprint ”. If the original file is modified , that MD5 The value will change accordingly , So we can MD5 Change to determine whether the file has been changed

9.chgrp Change the file's group

chgrp Orders can change Alluxio The group to which the file or folder in ,Alluxio Support POSIX Standard file permissions , Group in POSIX The file permission model is an authorization entity , The file owner or super user can execute this command to change the group of a file or folder .

add -R Option can recursively change the group of sub files and sub folders in the folder .

Use examples ： Use chgrp Command can quickly modify the group of a file ：

$ ./bin/alluxio fs chgrp alluxio-group-new /input/file1

10.chmod Change file access rights

chmod Command to change Alluxio Access rights to files or folders in , At present, octal mode is supported ： The three octal digits correspond to the file owner 、 Permissions of the group and other users . The following is the corresponding table of numbers and permissions ：

Number	Permission	rwx
7	read, write and execute	rwx
6	read and write	rw-
5	read and execute	r-x
4	read only	r–
3	write and execute	-wx
2	write only	-w-
1	execute only	–x
0	none	—

add -R Option can recursively change the permissions of sub files and sub folders in a folder .

Use examples ： Use chmod Command can quickly modify the permissions of a file ：

$ ./bin/alluxio fs chmod 755 /input/file1

11.chown Change file owner

chown The command is used to modify Alluxio The owner of the file or folder in the , For safety reasons , Only super users can change the owner of a file .

add -R Option can recursively change the owner of sub files and sub folders in a folder .

Use examples ： Use chown Command can quickly change the owner of a file .

$ ./bin/alluxio fs chown alluxio-user /input/file1
$ ./bin/alluxio fs chown alluxio-user:alluxio-group /input/file2

12.cat see Alluxio Documents in

(base) [[email protected] alluxio-2.6.0]# ./bin/alluxio fs cat /datatest
Hello World!
hi
hello
hi
hello
world

13.cp

cp Command copy Alluxio A file or directory in a file system , It can also be on the local file system and Alluxio File systems copy each other .

filescheme Represents the local file system ,alluxioscheme Or not scheme Express Alluxio file system .

If used -R Options , And the source path is a directory ,cp Copy the entire subtree under the source path to the target path .

for example ,cp You can copy files between underlying file systems .

$ ./bin/alluxio fs cp /hdfs/file1 /s3/

14.count

count Command output Alluxio The total number of files and folders in which all names match a given prefix , And their total size , This command recursively processes the contents of the folder . When users have predefined naming conventions for files ,count Orders are very useful .

Use examples ： If files are named after their creation date , Use count Command can get any date 、 Number of all files for month and year and their total size ：

$ ./bin/alluxio fs count /data/2014

15.du Output file size

du Command outputs the size of a file , If the specified destination is a folder , This command outputs the total size of all sub files under the folder and the contents in the sub folder .

Use examples ： If Alluxio Space is overused , Use du The command can detect which folders take up most of the space ：

# Shows the size information of all the files in root directory
$ ./bin/alluxio fs du /
File Size     In Alluxio       Path
1337          0 (0%)           /alluxio-site.properties
4352          4352 (100%)      /testFolder/NOTICE
26847         0 (0%)           /testDir/LICENSE
2970          2970 (100%)      /testDir/README.md

# Shows the in memory size information
$ ./bin/alluxio fs du --memory /
File Size     In Alluxio       In Memory        Path
1337          0 (0%)           0 (0%)           /alluxio-site.properties
4352          4352 (100%)      4352 (100%)      /testFolder/NOTICE
26847         0 (0%)           0 (0%)           /testDir/LICENSE
2970          2970 (100%)      2970 (100%)      /testDir/README.md

# Shows the aggregate size information in human-readable format
$ ./bin/alluxio fs du -h -s /
File Size     In Alluxio       In Memory        Path
34.67KB       7.15KB (20%)     7.15KB (20%)     /

# Can be used to detect which folders are taking up the most space
$ ./bin/alluxio fs du -h -s /\\*
File Size     In Alluxio       Path
1337B         0B (0%)          /alluxio-site.properties
29.12KB       2970B (9%)       /testDir
4352B         4352B (100%)     /testFolder

16.ls

ls The command lists all sub files, sub folders and file sizes under a folder 、 Last modified time and memory status of the file . Use... For a file ls The command only displays information about the file . ls The command also loads the metadata of any file or subdirectory under the directory from the underlying storage system to Alluxio Namespace , If Alluxio If you don't have this metadata . ls Command to query the file or directory matching the given path in the underlying file system , And then stay Alluxio Create a mirror file of this file in . Metadata only , Such as file name and size , Will load in this way and No data transfer occurs .

Options ：

• -d Option to list directories as normal files . for example ,ls -d / Display the properties of the root directory .
• -f Options Force load Metadata of subdirectories in the directory . By default , Only when the directory is first listed , Metadata will be loaded .
• -h Option to display the file size readable .
• -p Option lists all fixed files .
• -R Option can recursively list all sub files and sub folders under the input path , List all subtrees starting from the input path .
• --sort Sort the results according to the given options . Possible value ：size|creationTime|inMemoryPercentage|lastModificationTime|path
• -r Reverse the sort order .

Use examples ： Use ls Command can browse the file system .

$ ./bin/alluxio fs mount /s3/data s3://data-bucket/
# Loads metadata for all immediate children of /s3/data and lists them.
$ ./bin/alluxio fs ls /s3/data/
#
# Forces loading metadata.
$ aws s3 cp /tmp/somedata s3://data-bucket/somedata
$ ./bin/alluxio fs ls -f /s3/data
#
# Files are not removed from Alluxio if they are removed from the UFS (s3 here) only.
$ aws s3 rm s3://data-bucket/somedata
$ ./bin/alluxio fs ls -f /s3/data

17.test

test Command to test the properties of the path , If Attribute is true , return 0, Otherwise return to 1. have access to -d Option to test whether the path is a directory , Use -f Option to test whether the path is a file , Use -e Option to test whether the path exists , Use -z Option to test whether the file length is 0, Use -s Option to test whether the path is empty ,

Options ：

• -d Option to test whether the path is a directory .
• -e Option to test whether the path exists .
• -f Option to test whether the path is a file .
• -s Option to test whether the path is empty .
• -z Option to test whether the file length is 0.

Use examples :

(base) [[email protected] alluxio-2.6.0]# alluxio fs test -d /datatest
(base) [[email protected] alluxio-2.6.0]# echo $?
1
(base) [[email protected] alluxio-2.6.0]# alluxio fs test -d /data
(base) [[email protected] alluxio-2.6.0]# echo $?
0

18.pin

pin Command to Alluxio Mark files or folders in the . This command only operates on metadata , Will not cause any data to be loaded into Alluxio in . If A file in Alluxio Marked in , No data block of the file will be from Alluxio worker Be eliminated from . If there are too many locked files ,Alluxio worker There will be a small amount of storage space left , As a result, other files cannot be cached .

Use examples ： If the administrator is very clear about the job running process , Then you can use pin Command manual performance improvement .

$ ./bin/alluxio fs pin /data/today

19.unpin

unpin The order will Alluxio Unmark files or folders in . This command only works on metadata , No data blocks will be eliminated or deleted . Once the file is unlocked ,Alluxio worker The data block of the file can be eliminated .

Use examples ： When the administrator knows that the data access mode has changed , have access to unpin command .

$ ./bin/alluxio fs unpin /data/yesterday/join-table

20.stat

stat The command outputs the main information of a file or folder to the console , This is mainly to allow users to debug their systems . Generally speaking , stay Web UI It is much easier to understand the file information on the .

You can specify -f <arg> To display information in a specified format ：

• “%N”: file name ;
• “%z”: file size (bytes);
• “%u”: File owner ;
• “%g”: Group name of the owner ;
• “%y” or “%Y”: Edit time , %y shows ‘yyyy-MM-dd HH:mm:ss’ (the UTC date), %Y For since January 1, 1970 UTC Milliseconds since ;
• “%b”: Number of data blocks allocated to the file

for example , Use stat Command can get the location of the data block of a file , This is very useful in obtaining data locality in computing tasks .

# Displays file's stat
$ ./bin/alluxio fs stat /data/2015/logs-1.txt
#
# Displays directory's stat
$ ./bin/alluxio fs stat /data/2015
#
# Displays the size of file
$ ./bin/alluxio fs stat -f %z /data/2015/logs-1.txt

Operation list

operation	grammar	describe
cat	cat “path”	take Alluxio The contents of a file in are printed in the console
checkConsistency	checkConsistency “path”	Check Alluxio Consistency with the metadata of the underlying storage system
checksum	checksum “path”	Calculate the of a file md5 Check code
chgrp	chgrp “group” “path”	modify Alluxio The group to which the file or folder in
chmod	chmod “permission” “path”	modify Alluxio Access rights to files or folders in
chown	chown “owner” “path”	modify Alluxio The owner of the file or folder in the
copyFromLocal	copyFromLocal “source path” “remote path”	take “source path” Copy files from the specified local file system to Alluxio in "remote path" Specified path If "remote path" The command will fail if it already exists
copyToLocal	copyToLocal “remote path” “local path”	take "remote path" designated Alluxio Copy the files in to the local file system
count	count “path”	Output "path" The total number of files and folders in which all names match a given prefix
cp	cp “src” “dst”	stay Alluxio Copy a file or directory from a file system
du	du “path”	Output the size of a specified file or folder
fileInfo	fileInfo “path”	Output the data block information of the specified file
free	free “path”	take Alluxio Remove files or folders from , If the file or folder exists in the underlying storage , Then you can still visit there
getCapacityBytes	getCapacityBytes	obtain Alluxio The capacity of the file system
getfacl	getfacl “path”
getUsedBytes	getUsedBytes	obtain Alluxio Number of bytes used by the file system
help	help “cmd”	Print help information for a given command , If no command is given , Print help information for all supported commands
leader	leader	Print current Alluxio leader master Node hostname
load	load “path”	Load the files or directories of the underlying file system into Alluxio in
loadMetadata	loadMetadata “path”	Load the metadata of the file or directory of the underlying file system into Alluxio in
location	location “path”	Output the host containing a file data
ls	ls “path”	List the information of all direct files and directories under the given path , For example, size
masterInfo	masterInfo	Print Alluxio master Fault tolerance related information , for example leader The address of 、 all master Address list and configured Zookeeper Address
mkdir	mkdir “path1” … “pathn”	Create a folder under the given path , And the required parent folder , Multiple paths use spaces or tab Separate , If any of these paths already exist , The command failed
mount	mount “path” “uri”	The underlying file system "uri" The path is mounted to Alluxio In namespace "path" Under the path ,"path" The path cannot exist in advance and is generated by this command . No data or metadata is loaded from the underlying file system . When the mount is complete , Operations on the mount path will also act on the mount point of the underlying file system .
mv	mv “source” “destination”	take "source" Move the specified file or folder to "destination" Specified new path , If "destination" The command already exists... Failed .
persist	persist “path1” … “pathn”	Will only exist in Alluxio Files or folders in are persisted to the underlying file system
pin	pin “path”	Lock the given file into the content to prevent culling . If it's a catalog , Recursion works on its sub files and newly created files
report	report “path”	towards master Report that a file has been lost
rm	rm “path”	Delete a file , If the input path is a directory, the command fails
setfacl	setfacl “newACL” “path”
setTtl	setTtl “path” “time”	Set a file's TTL Time , Unit millisecond , Be careful , The default action is DELETE, Will remove the file from Alluxio Namespace and underlying storage
stat	stat “path”	Displays information about the specified path of files and directories
tail	tail “path”	The last... Of the specified file 1KB Output content to console
test	test “path”	Test the properties of the path , If the attribute is correct , return 0, Otherwise return to 1
touch	touch “path”	Create an empty file in the specified path
unmount	unmount “path”	Uninstall and mount in Alluxio in "path" Specify the underlying file path on the path ,Alluxio All objects in the mount point will be deleted , But the underlying file system will keep it .
unpin	unpin “path”	Unlocks a file so that it can be deleted , If it is a directory, the recursive function
unsetTtl	unsetTtl “path”	Delete the file's ttl value

Two 、 Command line interface —— Administrator order fsadmin

Administrator command line interface - Alluxio v2.6.0 Documentation

Alluxio The administrator command line interface of provides management for administrators Alluxio File system operations . You can call the following command line to get all the subcommands ：

$ ./bin/alluxio fsadmin
Usage: alluxio fsadmin [generic options]
       [report]
       [ufs --mode <noAccess/readOnly/readWrite> <ufsPath>]
       ...

With UFS URI parametric fsadmin ufs Sons command , The parameter should be something like hdfs://<name-service>/ Such roots UFS URI, instead of hdfs://<name-service>/<folder>.

1.backup Backup

backup Command to create Alluxio Backup of metadata

1） Back up to the default backup folder （ from alluxio.master.backup.directory） To configure

./bin/alluxio fsadmin backup
Successfully backed up journal to hdfs://mycluster/opt/alluxio/backups/alluxio-backup-2018-5-29-1527644810.gz

2） Backup to next In storage Specific directory .

./bin/alluxio fsadmin backup /alluxio/special_backups
Successfully backed up journal to hdfs://mycluster/opt/alluxio/backups/alluxio-backup-2018-5-29-1527644810.gz

example ：

(base) [[email protected] alluxio-2.6.0]# alluxio fsadmin backup /root/backup-alluxio
Backup Host        : clu00                                                 
Backup URI         : hdfs://clu00:9090/root/backup-alluxio/alluxio-backup-2021-10-14-1634196191864.gz
Backup Entry Count : 14368

[[email protected] ~]$ hdfs dfs -ls /root/backup-alluxio
Found 2 items
-rw-r--r--   3 root supergroup     288461 2021-10-14 15:23 /root/backup-alluxio/alluxio-backup-2021-10-14-1634196191864.gz
-rw-r--r--   3 root supergroup          0 2021-10-14 15:23 /root/backup-alluxio/alluxio-backup-2021-10-14-1634196191864.gz.complete

3） Backup to the primary host Local file system Specific directories for .

./bin/alluxio fsadmin backup /opt/alluxio/backups/ --local
Successfully backed up journal to file:///opt/alluxio/backups/alluxio-backup-2018-5-29-1527644810.gz on master Master2

example ：

(base) [[email protected] alluxio-2.6.0]# alluxio fsadmin backup /root/backup-alluxio-new --local
Backup Host        : clu00                                                 
Backup URI         : file:///root/backup-alluxio/alluxio-backup-2021-10-14-1634196431506.gz
Backup Entry Count : 14368
(base) [[email protected] alluxio-2.6.0]# ls /root/backup-alluxio/
alluxio-backup-2021-10-14-1634196431506.gz
alluxio-backup-2021-10-14-1634196431506.gz.complete

Restore metadata from backup files

# Restore metadata from backup files 
$ ./alluxio-start.sh -i /opt/alluxio-backup-2019-11-13-1573637112922.gz masters

Executing the following command on all master nodes and logging to /usr/local/git/alluxio/logs/task.log: /usr/local/git/alluxio/bin/alluxio-stop.sh master
Waiting for tasks to finish...
All tasks finished
Executing the following command on all master nodes and logging to /usr/local/git/alluxio/logs/task.log: /usr/local/git/alluxio/bin/alluxio-start.sh -i /opt/alluxio-backup-2019-11-13-1573637112922.gz master
Waiting for tasks to finish...
All tasks finished

example ：

(base) [[email protected] bin]# alluxio-start.sh -i /root/backup-alluxio/alluxio-backup-2021-10-14-1634196431506.gz masters
Executing the following command on all master nodes and logging to /root/alluxio-2.6.0/logs/task.log: /root/alluxio-2.6.0/bin/alluxio-stop.sh master
Waiting for tasks to finish...
All tasks finished
Executing the following command on all master nodes and logging to /root/alluxio-2.6.0/logs/task.log: /root/alluxio-2.6.0/bin/alluxio-start.sh -i /root/backup-alluxio/alluxio-backup-2021-10-14-1634196431506.gz -a master
Waiting for tasks to finish...
All tasks finished
-----------------------------------------
Starting to monitor all Alluxio masters.
-----------------------------------------
--- [ OK ] The master service @ clu00 is in a healthy state.

2.doctor Display running errors and warnings

doctor Command display Alluxio Mistakes and warnings .

# shows server-side configuration errors and warnings
$ ./bin/alluxio fsadmin doctor configuration

example ：

(base) [[email protected] alluxio-2.6.0]# alluxio fsadmin doctor --help
Failed to parse args for doctor: Unrecognized option: --help
Usage: doctor [category]
Show Alluxio errors and warnings.
Where [category] is an optional argument. If no arguments are passed in, all categories of errors/warnings will be printed out.
[category] can be one of the following:
    configuration    server-side configuration errors/warnings
    storage          worker lost storage warnings

(base) [[email protected] alluxio-2.6.0]# alluxio fsadmin doctor configuration
No server-side configuration errors or warnings.
(base) [[email protected] alluxio-2.6.0]# alluxio fsadmin doctor storage
All worker storage paths are in working state.

3.report Report cluster information

report The order provides Alluxio Running cluster information .

# Report cluster summary
$ ./bin/alluxio fsadmin report

# Report worker capacity information
$ ./bin/alluxio fsadmin report capacity

# Report runtime configuration information 
$ ./bin/alluxio fsadmin report configuration

# Report metrics information
$ ./bin/alluxio fsadmin report metrics

# Report under file system information
$ ./bin/alluxio fsadmin report ufs

Use -h Options for more information .

4.ufs Update the properties of the underlying mount

ufs The command provides options to update the properties of the mounted underlying storage .mode Option can be used to set the underlying storage to maintenance mode . At present, some operations may be limited .

for example , An underlying storage can be set to readOnly Mode to disable write operations . Alluxio No writes will be attempted to the underlying storage .

$ ./bin/alluxio fsadmin ufs --mode readOnly hdfs://ns

fsadmin ufs Order to accept a UFS URI As a parameter . This parameter needs to be a UFS URI The root of the , similar hdfs://<name-service>/, Instead of hdfs://<name-service>/<folder>.

Operation list

operation	grammar	describe
backup	backup [directory] [–local]	Back up all Alluxio metadata to the backup directory configured on master.
doctor	doctor [category]	Show Alluxio errors and warnings.
report	report [category] [category args]	Report Alluxio running cluster information.
ufs	ufs --mode <noAccess/readOnly/readWrite> “ufsPath”	Update attributes for a mounted under storage system.