Detailed explanation of MySQL fuzzy query

Single column fuzzy query

Table structure used

CREATE TABLE `user` (
  `id` int(11) NOT NULL AUTO_INCREMENT COMMENT ' Primary key ',
  `name` varchar(32) DEFAULT NULL COMMENT ' full name ',
  `city` varchar(32) DEFAULT NULL COMMENT ' City ',
  `age` int(11) DEFAULT NULL COMMENT ' Age ',
  PRIMARY KEY (`id`),
  KEY `idx_name_city` (`name`,`city`),
  KEY `idx_city` (`city`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=14 DEFAULT CHARSET=utf8;

insert into user(name, city, age) values("ZhaoDa", "BeiJing", 20),("QianEr", "ShangHai", 21),("SunSan", "GuanZhou", 22), ("LiSi", "ShenZhen", 24), ("ZhouWu", "NingBo", 25),  ("WuLiu", "HangZhou", 26), ("ZhengQi", "NanNing", 27), ("WangBa", "YinChuan", 28), ("LiSi", "TianJin", 29), ("ZhangSan", "NanJing", 30), ("CuiShi", "ZhengZhou", 65),  ("LiSi", "KunMing", 29), ("LiSi", "ZhengZhou", 30);

  Execute the following three items SQL

explain select * from user where city like '%h'; 

  

explain select * from user where city like 'h%'; 

 

explain select * from user where city like '%h%';

  From the perspective of implementation results, there is only the second one sql The index of is not invalidated ,

Conclusion ： If it's just tail blur matching , The index will not fail . If it's a fuzzy head match , Index failure .

Single column fuzzy query optimization

For the following SQL

explain select * from user where city like '%h%'

From the implementation plan, we can see that  type=ALL,Extra=Using where  It's all scanned , Didn't make use of ICP characteristic . Secondary index idx_city(city) The inner part contains the primary key id Of , Equivalent to (id,city) Composite index of , Try to take advantage of the overlay index feature

explain select id from user where city like '%h%'; 

  From the implementation plan, we can see that ,type=index,Extra=Using where; Using index, Index full scan , But all the data needed can be found in the index column , There is no need to return the form . Take advantage of this feature , Will be original SQL Statement to get the primary key first id, And then through id Associate with the original table , Analyze its implementation plan .

explain select u.* from user u , (select id from user where city like '%h%') t where u.id = t.id;

  From the execution plan , The index is gone idx_city(city), No need to return to the table to access data , Execution time from 60ms Reduced to 40ms,type = index The explanation does not use ICP characteristic , But it can be used  Using where; Using index  This index scan does not return to the table way to reduce resource overhead to improve performance .

Federated index fuzzy query

Push... Under index conditions ICP

ICP Introduce

MySQL 5.6 Start supporting ICP（Index Condition Pushdown）, I won't support it ICP Before , When doing index queries , First, look up the data according to the index , And then based on where Conditions to filter , Scanning a lot of unnecessary data , Added database IO operation .

In support of ICP after ,MySQL At the same time of retrieving the index data , To judge whether it is possible to where filter , take where Part of the filtering operation is placed in the storage engine layer to filter out unnecessary data in advance , It reduces the amount of unnecessary data being scanned IO expenses .

Under certain queries , Can reduce the Server Read data from storage engine layer by layer , So as to provide the overall performance of the database .

Official documents ：MySQL :: MySQL 8.0 Reference Manual :: 8.2.1.6 Index Condition Pushdown Optimization

ICP It has the following characteristics

 

principle

In order to understand ICP How it works , Let's first understand that it is not used ICP Under the circumstances ,MySQL How to query ：

• The storage engine reads index records ;

• According to the primary key value in the index , Locate and read the complete row record ;

• The storage engine gives the records to Server Layer to detect whether the record meets WHERE Conditions .

Use ICP Under the circumstances , The query process is as follows ：

• Read index records （ Not a complete line record ）;

• Judge WHERE Whether the condition part can be checked with the columns in the index , Condition not satisfied , Then process the next row of index records ;

• Conditions met , Use the primary key in the index to locate and read the complete row record （ It's the so-called back table ）;

• The storage engine gives the records to Server layer ,Server The layer detects whether the record meets WHERE The rest of the condition .

practice

set optimizer_switch = 'index_condition_pushdown=off';
explain select * from user where name="LiSi" and city like "%Z%" and age > 25;

Without index push down , According to the joint index “ The leftmost match ” principle , Only name Columns can use indexes ,city Columns are then fuzzy matched , You can't use indexes , The execution process is as follows ：

 

Use index to push down

set optimizer_switch = 'index_condition_pushdown=on';
explain select * from user where name="LiSi" and city like "%Z%" and age > 25;

• The storage engine is based on (name, city) Joint index , find name='LiSi' The record of , common 4 strip ;

• Because the union index contains city Column , The storage engine presses... Directly in the federated index city like "%Z%" To filter , After filtering, there is 2 Bar record ;

• According to the filtered records id value , Scan the table one by one , Remove the complete row record from the clustered index , And return these records to Server layer ;

• Server According to WHERE Other conditions of the statement age > 25, Filter the row records again , In the end, only ("LiSi", "ZhengZhou", 30) This record .

 

Yes InnoDB For storage engines , Index push down is only applicable to secondary indexes （ Also called secondary index ）, The purpose of index push down is to reduce the number of table returns , That is to reduce IO operation . about InnoDB Of Cluster index Come on , The complete row record has been loaded into the cache , There is no point in pushing down the index .