6. Regular expression of shell

One , Regular expressions

• Regular expressions , Also known as regular expression .（ English ：Regular Expression）, In code it is often abbreviated as regex、regexp or RE）, A concept of computer science .

Linux There are two kinds of regular expressions commonly used in ：

1. Basic regular expressions ：BRE
2. Extended regular expression : ERE

1.1 Regular expression definition

1. Regular expressions are often used for retrieval 、 Replace those that match a pattern ( The rules ) The text of

2. Regular expressions , It is usually used to judge , Used to check whether a string satisfies a certain format

3. Regular expressions are composed of ordinary characters and characters

4. Ordinary characters include upper and lower case letters , Numbers , Punctuation and some other symbols

5. Metacharacters are special characters with special meaning in regular expressions , It can be used to specify its leading characters （ The character before the metacharacter ） The occurrence pattern in the target object

• There is more than one regular expression , and LINUX Different programs in may use different regular expressions , Such as ：
  Tools ：grep sed awk egrep

1.2 Common metacharacters in basic regular expressions

• Supported tools : grep、egrep、sed、awk

• Special characters ：

\： Escape character , Used to cancel the meaning of special symbols , example : !、\n、$ etc.

^： Match the beginning of the line ,^ Is the beginning of the matching string ^tux Match with tux Beginning line

$： Match the end of the line , be yes horse with word operator strand Of junction tail t u x Is the end of the matching string tux Is the end of the matching string tux Match with tux The line at the end

.： Match break \r,\n Any single character other than ,awk Then you can ab. matching abc or bad, Unmatched abcd or abde, Only single characters can be matched …

[list]： matching list A character in the list

 example :
go[ola]d
[abc]
[a-z]
[a-z0-9]

[^list]： Any match is not in list A character in the list

 example :
[ ^a-z ]
[ ^0-9 ]
[ ^A-Z0-9]

• Match the front face expression 0 Times or more example ：goo*d、go.*d

The asterisk matches the front sub expression 0 Times or more example ：goo*d、go.*d
{n} : Match the previous subexpression n Time , example :go{2}d、'[O-9]{2}‘ Match two numbers
{n,}: Match the preceding subexpression no less than n Time , example : go{2,}d、’ [0-9]{2,}‘ Match two or more digits
{n,m}﹔ Match the previous subexpression n To m Time , example : go{2,3)d、’[0-9]{2,3}' Match two to three digits
notes : egrep、awk Use {n}、{n, }、{n, m} When the match “{}“ There is no need to add ”\”

• Locator

^： Matches where the input string starts

$： Matches the position of the end of the input string

• Nonprinting characters

\n Match a line break

\r Match a carriage return

\t Match a tab

1.3 Extended regular expression metacharacter

1. In general, it's enough to use basic regular expressions , But sometimes in order to simplify the whole instruction , Need to use A wider range of extended regular expressions . for example , Use the basic regular expression to query the blank line in the file and the first line is “#”

2. Outside the line （ It is usually used to view the effective configuration file ）, perform “grep -v‘^KaTeX parse error: Expected group after '^' at position 21: ….txt | grep -v‘^̲#’” That is to say . Here you need to use tubes …|^#’test.txt”, among , The pipe symbol in single quotation marks indicates or （or）

3. Besides ,grep Command only supports basic regular expressions , If you use extended regular expressions , Need to use egrep or awk command .awk command , Here we use it directly egrep command .egrep Command and grep The usage of commands is basically similar .egrep Command is a search file acquisition mode , Use this command to search for any string and symbol in the file , You can also search for a string of one or more files , A prompt can be a single character 、 A string 、 A word or sentence .
The same type as the underlying regular expression , The extended regular expression also contains multiple metacharacters , Common extended regular expressions The metacharacters of expression mainly include the following

• Support tools （egrep awk）

(+) Repeat one or more previous characters

 example ：
“egrep -n 'wo+d' test.txt” command , You can query "wood" "woood" "woooooood" Etc 

？ Zero or the previous character of one

 example 
“egrep -n 'bes?t' test.txt” command , You can query “bet”“best” These two strings 

() lookup “ Group ” character string

 example ：
“egrep -n 't(a	e)st’ test.txt”. You can query "tast" perhaps "test" character string 

()+ Identify multiple repeating groups

 example ：
“egrep -n ‘A(xyz)+C’ test.txt”. The command is at the beginning of the query "A" It ends with "C", More than one in the middle "xyz" character string 

| Use or （or） How to find multiple characters

 example ：
“egrep -n 'of|is|on' test.txt” Command to query "of" perhaps "if" perhaps "on" character string 

Example ： perform “egrep -n ‘of|is|on’ test.txt” Command to query "of" perhaps "if" perhaps "on" character string

Two ,grep command

Format ：

grep [ Options ] Find files Target file

2.1 Common options

-a	take binary Document to text Searching data by file
-E	Turn on extended regular expressions
-o	Only the string matched by the pattern
-c	Calculate find ‘ Search string ’ The number of times
-i	Ignore case differences , So case is the same
-n	Output line number
-v	Reverse selection , That is to say, no ‘ Search string ’ Content line ！
-w	Exactly match , It means that what I input matches what
–color=auto ：	You can add color to the key words you find ！

example ：

[[email protected] ky20]# grep -cw ens33 ifcfg-ens33 ## see file , Accurate matching display ens33 Several times 
2

[[email protected] ky20]# grep -in ens33 ifcfg-ens33 ## Check the file, ignoring the difference in case , Output line number 
12:NAME=ens33
14:DEVICE=ens33

[[email protected] ky20]# cat ifcfg-ens33 |grep -v '^$\|#$' ## see file , And put the blank line and # No. filter out 

[[email protected] ky20]# ifconfig ens33 | grep -o "[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+"|head -1
192.168.113.126
## hold ifconfig ens33 The first one in IP To filter out  （|head -1）
## Only the string matched by the pattern （grep -o ）
## Meaning of splicing （[0-9]\+\.）
##\+ Indicates matching the previous string 1 Or more , Because there is a special meaning to escape with escape character 
##\. Dot also has special meaning , Escape character escape the normal point after , That is, it has been divided 

2.2grep Case series cooperating with metacharacter operation

（1） Find specific characters

1. Finding specific characters is very simple , If you execute the following command, you can test.txt Find specific characters in the file “the” The position of . among “-n” Indicates display line number 、“-i” Indicates case insensitive . After the execution of the command , Matching characters , The font color changes to red （ All in this chapter are shown in bold instead of ）

 [[email protected] ky20]# grep -n 'the' test.txt

2. If reverse selection , If the search does not contain “the” Lines of characters , You have to go through grep Ordered “-v” Option implementation , And cooperate with “-n” Use together to display line numbers

 [[email protected] ky20]# grep -vn 'the' test.txt

（2） Using brackets “[]” To find set characters

1. Want to find “shirt” And “short” When these two strings , You can find that both strings contain “sh” And “rt”. At this time, execute the following command to find “shirt” And “short” These two strings , among “[]” No matter how many characters there are , All represent only one character , in other words “[io]” Represents a match “i” perhaps “o”

 [[email protected] ky20]# grep -n 'sh[io]rt' test.txt

2. To find a single character that contains duplicates “oo” when , Just execute the following command

 [[email protected] ky20]# grep -n 'oo' test.txt

3. Find out if “oo” The front is not “w” String , Only through the reverse selection of set characters “[^]” To achieve that goal . For example, to perform “grep -n‘[^w]oo’test.txt” The command indicates in test.txt Find... In the text “oo” The front is not “w” String

 [[email protected] ky20]# grep -n '[^w]oo' test.txt

4. In the execution result of the above command, it is found that “woood” And “wooooood” It also meets the matching rules , Both include “w”. In fact, through the implementation results, we can see , Characters that meet the matching criteria are displayed in bold , From the above results, we can know , “#woood #” Bold in shows “ooo”, and “oo” Ahead “o” It meets the matching rules . Empathy “#woooooood #” It also meets the matching rules .
If not “oo” Lowercase before , have access to “grep -n‘[^a-z]oo’test.txt” Command implementation , among

“a-z” For lowercase letters , Capital letters pass “A-Z” Express

 [[email protected] ky20]# grep -n '[^a-z]oo' test.txt

5. To find a row that contains numbers, you can use “grep -n ‘[0-9]’test.txt” Command to implement

[[email protected] ky20]# grep -n '[0-9]' test.txt

（3） Find the beginning of a line “^” And end of line characters “$”

1. The base regular expression contains two positioning metacharacters ：“^”（ Head of line ） And “$”（ At the end of the line ）. In the example above , Inquire about “the” There are many strings that contain “the” The line of , If you want to query with “the” The line with the string at the beginning of the line , You can use the “^” Metacharacters

[[email protected] ky20]# grep -n '^the' test.txt

2. Query lines that start with lowercase letters can be through “^{[a-z]” Rules to filter , For lines that start with uppercase letters, use “}[A-Z]” The rules , If the query does not start with a letter, use “^[a-zA-Z]” The rules

 Query lines that start with lowercase letters can be through “^[a-z]” Rules to filter , For lines that start with uppercase letters, use “^[A-Z]” The rules , If the query does not start with a letter, use “^[^a-zA-Z]” The rules 

[[email protected] ky20]# grep -n '^[a-z]' test.txtbash
[[email protected] ky20]# grep -n '^[A-Z]' test.txt
[[email protected] ky20]# grep -n '^[^a-zA-Z]' test.txt

3.“^” The symbol is in the metacharacter set “[]” The functions inside and outside the symbol are different , stay “[]” The symbol indicates reverse selection , stay “[]” Outside the symbol stands for positioning the beginning of the line . conversely , If you want to find a line that ends with a particular character, you can use “$” Locator . for example , Execute the following command to implement the query with decimal point （.） The line at the end . Because of the decimal point （.） It's also a metacharacter in regular expressions , So here we need to use escape characters “\” Convert characters with special meaning into ordinary characters

[[email protected] ky20]# grep -n '\.$' test.txt

4. When querying blank rows , perform “grep -n‘^$’test.txt” Command is enough

[[email protected] ky20]# grep -n '^$' test.txt

（4） Find any character “.” And repeating characters “*”

1. In regular expressions, the decimal point （.） It's also a metacharacter , Represents any character . For example, execute the following command to find “ With w Begin with d Is the number of characters in the middle of the end （ How many . Express ）“, Four characters in total , With w start d ending

[[email protected] ky20]# grep -n 'w..d' test.txt

2.“wood” character string “w…d” Matching rules . If you want to query oo、ooo、ooooo Other information , Asterisk is required （） Metacharacters . But it should be noted that ,“” Represents the repetition of zero or more preceding single characters . “o*” Indicates zero （ Is an empty character ） Or greater than or equal to one “o” The characters of , Because empty characters are allowed , So execute “grep
-n ‘o*’ test.txt” The command prints everything in the text . If it is “oo*”, Is the first o There must be , the second o Zero or more o, Therefore, the o、oo、ooo、ooo, All the data are up to the standard . Empathy , If the query contains at least two o String above , execute “grep -n ‘ooo*’ test.txt” Command is enough

[[email protected] ky20]# grep -n 'ooo*' test.txt

3. Query to w start d ending , At least one in between o String , Execute the following command

[[email protected] ky20]# grep -n 'woo*d' test.txt

4. Execute the following command to query w start d ending , The middle character can have a string of optional characters

[[email protected] ky20]# grep -n 'w.*d' test.txt

5. Execute the following command to query the line of any number

[[email protected] ky20]# grep -n '[0-9][0-9]*' test.txt 

（5） Find range of consecutive characters “{}”

Used “.” And “*” To set zero to an infinite number of repeating characters , If you want to limit a range of repeated strings, how to achieve it ？ for example , Find three to five o Continuous characters of , At this time, you need to use the limited range of characters in the basic regular expression “{}”. because “{}” stay Shell Has special significance in , So it's using “{}” Character time , Escape character required “\”, take “{}” Character to normal .“{}” The use of characters is as follows

1. Check two o The characters of

[[email protected] ky20]# grep -n 'o\{2\}' test.txt

2. Query to w Begin with d ending , The middle contains 2～5 individual o String

[[email protected] ky20]# grep -n 'wo\{2,5\}d' test.txt

3. Query to w Begin with d ending , The middle contains 2 Or 2 More than o String

[[email protected] ky20]# grep -n 'wo\{2,\}d' test.txt

summary

• Through the above simple examples , You can see that the metacharacters of common basic regular expressions mainly include the following ：

1.^ Matches the start of the input string . Unless used in a bracket expression , Indicates that the character set... Is not included . To match “^” Character itself , Please use “^”

2.$ Matches the end of the input string . If set RegExp Object's Multiline attribute , be “KaTeX parse error: Undefined control sequence: \n at position 6: ” Also match ‘\̲n̲’ or ‘\r’. To match “” Character itself , Please use “$”. Matching elimination “\r\n” Any single character other than

3.\ The backslash , Also called escape character , Remove the special meaning of the following metacharacter or wildcard

4.“ star *” Match previous subexpression zero or more times . To match the star character , Please use “ star *”

5.[] Character set . Matches any of the contained characters . for example ,“[abc]” Can match “plain” Medium “a”

6.[^] Assignment character set . Matches an arbitrary character that is not included . for example ,“[^abc]” Can match “plain” Any letter in

7.[n1-n2] character in range . Match any character in the specified range . for example ,“[a-z]” Can match “a” To “z” Any lowercase character in the range .
Be careful ： Hyphens only （-） Inside character group , And when it appears between two characters , To represent the range of characters ; If it appears at the beginning of a character group , Only the hyphen itself

8.{n} n Is a non negative integer , Matched definite n Time . for example ,“o{2}” Can't match “Bob” Medium “o”, But it matches “food” Medium “oo”

9.{n,} n Is a non negative integer , Match at least n Time . for example ,“o{2,}” Can't match “Bob” Medium “o”, But it can match. “foooood” All in o.“o{1,}” Equivalent to “o+”.“o{0,}” Is equivalent to “o*”

10.{n,m} m and n All non negative integers , among n<=m, Least match n Times and at most m Time
Any character in the range . for example ,“[a-z]” Can match “a” To “z” Any lowercase character in the range .
Be careful ： Hyphens only （-） Inside character group , And when it appears between two characters , To represent the range of characters ; If it appears at the beginning of a character group , Only the hyphen itself

8.{n} n Is a non negative integer , Matched definite n Time . for example ,“o{2}” Can't match “Bob” Medium “o”, But it matches “food” Medium “oo”

9.{n,} n Is a non negative integer , Match at least n Time . for example ,“o{2,}” Can't match “Bob” Medium “o”, But it can match. “foooood” All in o.“o{1,}” Equivalent to “o+”.“o{0,}” Is equivalent to “o*”

10.{n,m} m and n All non negative integers , among n<=m, Least match n Times and at most m Time