What do Niu B programmers pay attention to when "creating an index"?

Have to say , How to create an index is one of the skills that we developers must master . When designing system data sheets , You may be able to adapt to specific business needs , Add a normal index or a unique index to a corresponding table field ; It may also be based on Leftmost prefix principle 、 Index push down feature and overlay index , Combine columns into a joint index to use .

   When my colleagues asked me about my experience in creating indexes , As a veteran programmer , I suggest trying to make every SQL Medium where、group by、order by Conditions can maximize the use of indexes . Of course , stay Write more and read less and read more and write less It is used in different scenarios in different ways . We're making sure SQL While implementing efficiency , Also pay attention to the cost of maintaining the index files in the database , Take time to deal with common and annoying situations such as ： Fuzzy query 、 Large text retrieval 、 Super paging etc. .

One 、 Identify the advantages and disadvantages of indexing

   Enemy and know yourself , you can fight a hundred battles with no danger of defeat . Want to use the index correctly , First, we need to know the characteristics of the index and its advantages and disadvantages .

1-1、 advantage

• Indexing greatly reduces the amount of data that the server needs to scan （ Data pages ）

• Indexes can help the server avoid sorting and temporary tables

• Index can be random I/O Into order I/O

1-2、 shortcoming

• Although the index greatly improves the query speed , At the same time, it will reduce the speed of updating the table , Such as on the table INSERT、UPDATE and DELETE. Because when updating tables ,MySQL Not only to save data , still more Save index file .

• Index files that take up disk space . Generally, this problem is not serious , But if you create multiple composite indexes on a large table , And with a large amount of data insertion , Index file size will also expand rapidly .

• If a data column contains many duplicate contents , Indexing it doesn't have much practical effect .

• For very small tables , In most cases, a simple full table scan is more efficient ;

• Just index the most frequently queried and sorted data columns .（MySQL The maximum number of indexes in the same data table in the is 16 individual ）

   The purpose of indexing is to improve query efficiency , Just like we borrow books in the library ： You need to locate the classification area first → bookshelf → book → chapter → the number of pages . A library can be seen as a database , If all the data is misplaced , I believe that one day you will not find the one you want 《 Sunflower treasure dian 》. The perspective-taking , In fact, the server is also very tired , Be nice to it ~

   In fact, the essence of indexing is to filter out the final desired results by constantly narrowing the range of data you want to obtain , And turn random events into sequential events , in other words , With this indexing mechanism , We can efficiently lock certain data at the same time , You can also quickly locate the scope and sort the work .

   In general, the read / write ratio in the application system will be in 10:1 ~ 15:1 Even higher , And the insert operation and the update delete operation （ We become DML operation ） There are few performance problems , Mostly just in transaction processing . In the production environment , We encounter more performance problems in some complex queries SQL in . therefore , The index optimization of query statements is obviously the top priority .

   When it comes to indexes , We must understand its data structure and its storage and query methods . take MysQL Come on ,InnoDB、MyISAM、Memory Each storage engine is different .

Binary sort tree → Binary balance tree → B-Tree（B Trees ） → B+Tree（B+ Trees ）

   about MySQL Most commonly used InnoDB engine , The data structure is B+Tree, choose B+ Trees have gone through a long process of evolution （ Above ）.

   It should be noted that ,B+Tree Its characteristic is N Fork tree + Orderly storage .B+ The leaf nodes of the tree establish chain pointers in order , Strengthened the interval visit , therefore B+ Tree to index has natural advantages over range query and sorting .

Two 、 What should I pay attention to when creating an index in development （ Wise remark of an experienced person ）

In the example of this article, we construct a simple LOL Hero information table , as follows ：

# The official account is 【 Code farmer Programming Advanced notes 】
mysql> select * from t_lol;
+----+--------------+--------------+-------+
| id | hero_title   | hero_name    | price |
+----+--------------+--------------+-------+
|  1 |  The shadow of the blade      |  Tyrone          |  6300 |
|  2 |  Swift scout      |  Timo          |  6300 |
|  3 |  Bright girl      |  Laches        |  1350 |
|  4 |  Wind up demon      |  Oriana      |  6300 |
|  5 |  The fist of the highest      |  Li Qing          |  6300 |
|  6 |  Limitless swordsman      |  easy            |   450 |
|  7 |  The strong wind sword      |  The rope          |  6300 |
|  8 |  Female gun          |  Good luck          |  1350 |
+----+--------------+--------------+-------+
8 rows in set (0.00 sec)

2-1、 Try to construct overlay indexes

For example, you created hero_name,price The index of idx_name_price(hero_name,price), Use this pose when querying data ：

SELECT * from t_lol where hero_name = ' The rope ' and price = 6300;

   Because there are only hero_name、price And primary key columns , After hitting the index ,select * What about the other fields of the ？ The database must also go back to the clustered index to find other column data through the primary key , This is the back table , This is the one you carry ： To use less select * Why , He will make SQL Missing the use of overlay indexes .

   We go through EXPLAIN Check the SQL The implementation of , I found that although the index was used , But the coverage index is not reached , A table return occurred . When the amount of data is large , It may take more than ten times as long to write back the table as to cover the index .

mysql> EXPLAIN SELECT * from t_lol where hero_name = ' The rope ' and price = 6300;
+----+-------------+-------+------------+------+--------------------------+----------------+---------+-------------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys            | key            | key_len | ref         | rows | filtered | Extra |
+----+-------------+-------+------------+------+--------------------------+----------------+---------+-------------+------+----------+-------+
|  1 | SIMPLE      | t_lol | NULL       | ref  | idx_price,idx_name_price | idx_name_price | 136     | const,const |    1 |   100.00 | NULL  |
+----+-------------+-------+------------+------+--------------------------+----------------+---------+-------------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)

And if you only check select hero_name,price Two , Or add the primary key id This column , Can be used to overwrite the index without going back to the table . namely key=idx_name_price;Extra=Using index;

mysql> EXPLAIN SELECT hero_name,price from t_lol where hero_name = ' The rope ' and price = 6300;
+----+-------------+-------+------------+------+--------------------------+----------------+---------+-------------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys            | key            | key_len | ref         | rows | filtered | Extra       |
+----+-------------+-------+------------+------+--------------------------+----------------+---------+-------------+------+----------+-------------+
|  1 | SIMPLE      | t_lol | NULL       | ref  | idx_price,idx_name_price | idx_name_price | 136     | const,const |    1 |   100.00 | Using index |
+----+-------------+-------+------------+------+--------------------------+----------------+---------+-------------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)

2-2、 Create reusable indexes

   Or this one t_lol surface , If you add a high frequency interface , Through the price （price） Query hero nickname （hero_title）, So what we created idx_name_price(hero_name,price) Can the index still be used ？

mysql> explain select * from t_lol where price =6300;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | t_lol | NULL       | ALL  | idx_price     | NULL | NULL    | NULL |    8 |    62.50 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec

  key=NULL;Extra=Using where; Obviously, the index is not used idx_name_price(hero_name,price), Because in MySQL The index in implements the leftmost prefix principle . This leftmost prefix can be the leftmost of the union index X A field , It can also be the leftmost of a string index Y Characters .

Leftmost prefix principle

  B+ The node storage index order of the tree is from left to right （ Explain , This left to right is just a logical one-way order , Not the left and right .. Don't be stubborn ）, In the process of matching, it is necessary to match from left to right ;

   Usually when we're building a federated index , That is to index multiple fields , I believe that the students who have established the index will find out , Whether it's Oracle still MySQL Will let us choose the order of the index , For example, we want to be in a,b,c Create a union index on three fields , We can choose the priority we want ,a、b、c, Or is it b、a、c Or is it c、a、b Equal order . Why does the database let us choose the order of the fields ？ Isn't it all a union index of three fields ？ This leads to the leftmost prefix principle of database index .

   In our development, we often encounter that this field has built a joint index , however SQL Query the field without using the index . Such as the index abc_index:(a,b,c) yes a,b,c Joint index of three fields , The following sql The index cannot be hit at execution time abc_index Of ;

select * from table where c = '1';

select * from table where b ='1' and c ='2'

The following three situations are indexed :

select * from table where a = '1';

select * from table where a = '1' and b = '2';

select * from table where a = '1' and b = '2'  and c='3';

From the above two examples, we can see whether you are broad or not ？

   Yes , Indexes abc_index:(a,b,c), Only in (a)、(a,b)、(a,b,c) Three types of queries use . In fact, there is a little ambiguity here , Actually (a,c) I'll go, too , But just go a Field index , Not going c Field .

   In addition, there is a special case , There will only be a And b Go to the index ,c Not going .

select * from table where a = '1' and b > '2'  and c='3'

   Like the one above sql sentence , stay a、b After going through the index ,c It's out of order , therefore c You can't go to the index , The optimizer thinks it's not as good as a full table scan c Fields come fast .

   Left most prefix ： seeing the name of a thing one thinks of its function , It's the top left priority , In the example above, we created a_b_c Multi column index , It's equivalent to creating (a) Single index ,(a,b) Composite index and (a,b,c) Composite index .

   therefore , When creating a multi-column index , According to business needs ,where The most frequently used column in the clause is at the far left .

   After we understand the leftmost prefix principle, we find that , It is simply impossible to maximize the use of indexes for every request , You can't just add an index to an interface ？

mysql> select * from t_lol;
+----+--------------+--------------+-------+
| id | hero_title   | hero_name    | price |
+----+--------------+--------------+-------+
|  1 |  The shadow of the blade      |  Tyrone          |  6300 |
|  2 |  Swift scout      |  Timo          |  6300 |
|  3 |  Bright girl      |  Laches        |  1350 |
|  4 |  Wind up demon      |  Oriana      |  6300 |
|  5 |  The fist of the highest      |  Li Qing          |  6300 |
|  6 |  Limitless swordsman      |  easy            |   450 
|  7 |  The strong wind sword      |  The rope          |  6300 |
|  8 |  Female gun          |  Good luck          |  1350 |
+----+--------------+--------------+-------+
8 rows in set (0.00 sec)

Back to the question we mentioned above , If there is a high frequency interface ： Through the price （price） Query hero nickname （hero_title）, Then I'm going to build a new one index(price) Indexes ？

   In fact, this raises a question , When building a federated index , How to arrange the field order of the index ？  That is, the reusability of indexes .

   Because it can support the leftmost prefix , So when it's already there idx_name_price(hero_name,price) After this joint index , In general, you don't need to be alone in hero_name Index on . But check it alone price The union index cannot be used when , Then if you want to use this index, you can also pass price Column query requirements . What do I do ？ As you think , Modify index column order .

   therefore , The first principle is , If by adjusting the order , One index can be maintained at least , So this order is often the one that needs to be prioritized .

   So you should know , In the question at the beginning of this paragraph , We need to create both for high frequency requests (price,hero_name) This joint index , And support it with this index according to price Inquire about hero_title The needs of . Then we just need to change the joint index order to idx_name_price(price,hero_name) that will do .

2-3、 More indexes is not better

   Obviously , We explained the shortcomings of the index mentioned earlier in the article , Index is a double-edged sword , While improving query efficiency, it also needs to use a large number of resources in the database to maintain it . Larger and larger index files 、 Slower and slower DML Operations are consequences to be considered .

   Therefore, we need to create the index according to the needs of the actual scenario , Read more and write less or read less and write more ？ The need to create an index of data volume ？ The hard injury of the index ？ etc. .

   A classmate asked me when there was little data （ Dozens of ？） Create index and do not create index Query efficiency and maintenance cost How much difference will there be ？

   I didn't know how to answer for the moment .. As an old programmer , I suggest you take a long view , Don't spend too much time on this problem . It can only be said that , If there is a business, it will use , It is recommended to create indexes according to the specifications for creating indexes during development , It will always be useful in the future . With less data, the cost of index maintenance can be ignored , Just don't leave a hole .

2-4、 Some heartwarming suggestions for using indexes

1、 The index will not contain null Columns of values

   As long as the column contains null Values will not be included in the index , Only one column in a composite index contains null value , So this column is invalid for this composite index . Therefore, we suggest not to let the default value of the field be null.

2、 Use short index

   Index series Columns , If possible, you should specify a prefix length . for example , If there is a char(255) The column of , If in front of 10 Or 20 Within a character , Multiple values are unique , So don't index the entire column . Short index can not only improve query speed but also save disk space and I/O operation .

3、 Index column sort

   The query uses only one index , So if where If index has been used in clause , that order by The columns in will not use indexes . So the database default sorting can meet the requirements of the case do not use sorting operations ; Try not to include sorting of multiple columns , If you need to create a composite index for these columns .

4、like Statement operation

   In general, it is not recommended to use like operation , If necessary , How to use it is also a problem .like % Chen % Index will not be used and like Chen % You can use index .

5、 Don't operate on Columns

   This will cause the index to fail and perform a full table scan , for example

SELECT * FROM table_name WHERE YEAR(column_name)<2017;

6、 Don't use not in and <> Such non set operations

   This is not a supported query condition , No index .

summary

   Before we actually operate the index , It is suggested that according to the actual needs , Combined with search engine indexing features , First, design the index type and structure of each table , Try to avoid changing while writing . It is very troublesome to modify the index after the data volume increases dramatically , It takes a long time to modify , And the table will be locked during modification . by the way , Never modify the index of the online library at will , Don't ask me why ..

Excellent article recommendation

SQL Will know ( The principle of indexing )

Do you know how database indexing works ？

MySQL Several scenarios of index failure

The illustration MySQL Indexes —— B-Tree、B+Tree

MySQL Of order by How to avoid " Miss index "