72 results and development suggestions of the latest on-site production system optimization

Last week, I optimized the database of an important business system for a customer in Nanjing , What can be implemented is implemented immediately , The performance comparison before and after optimization is very obvious , The system is the most serious IO The problem of overload has been basically solved ： Optimize the physical reading of the previous day 48 100 million times , After optimization is 15 100 million times , The effect is just , Business processing efficiency has been significantly improved , The lifetime of the disk will also be greatly improved . These are just some of the optimization recommendations .

If the developer can change SQL And business logic optimization suggestions are completed , One day physics reading is less than 5 A hundred million times should be no problem . If you don't know that optimization can bring such a huge improvement , Maybe the business department will consider spending millions on flash memory storage , In fact, today's normal storage is more than enough for business requirements .

The current performance improvement is mainly achieved by adjusting the index , Created 11 An index . At the same time, the database parameters are adjusted , hold optimizer_index_cost_adj Parameters from 20 Change back to the default 100; And then pass hint+sql profile, Give Way sql Choose an efficient execution plan . These things DBA It can be done . however , If SQL Writing is not good , You can only change the code , This cycle may be longer . Here are some of them ：

1、 part sql No binding variables are used , The number of hard parsing per second is nearly 200. This is a very high value （ Hard resolution is less than 10 Time / Second system , Basically OK）. The rational use of bound variables is OLTP One of the most basic requirements of the system , However, there are still many systems that do not use bound variables , It can only be said that the database development level of many domestic development teams is still very elementary . Some companies will require DBA Want to have OCP、OCM Wait for qualification to take the post , But very tolerant of developers , As long as the function is realized, everything will be fine , If after some simple training , This low-level problem of not using bound variables should not occur . The hard parsing of another set of database of this customer in a certain period of time is close to 1000 Time / second ,CPU Close usage 100%, When DBA Tell the developer that this is because bound variables are not used , The developers actually said that there had been no similar problems before , This is the typical ignorant man fearless .

A helpless solution to not using bound variables is to change the database parameters , take cursor_sharing There is a default exact Change to force. This change is not recommended , It is likely to cause other performance problems , There's a lot of bug Related to it .

2、 This is a very low-level and widespread writing problem , Do... On a date field to_char：where to_char(starttime,'yyyy-mm-dd'）='20200110'. This starttime Field , A single field index already exists , One more to_char(starttime,'yyyymmdd') Function index . But the top one sql How to write it , If you don't rewrite , You can only create one more to_char(starttime,'yyyy-mm-dd') Function index of . If the developer writes to_char(starttime,'yyyy/mm/dd') Of sql Well ？ Is it necessary to create another functional index ？ such sql The existence of , It indicates that the development team does not know what is the development specification and SQL to examine .

If you're a developer , Both of the above expressions hit you （ Maybe some developers still don't know what the concept of binding variable writing is , It is suggested to Baidu immediately ）, It can only be said that your database development level still has great room for progress .

3、select xxseq.nextval from (select 1 from all_objects where rownum<=10) ; This sql With the help of all_objects View , Generate 10 individual sequence Sequence value . The function of this method is no problem , however sql Frequent execution , A lot of all_objects Call of view , Take these that would have been negligible for resource consumption sql, Turned into top cpu SQL. It is suggested to change it to select xxseq.nextval from dual connect by level<=10;

4、select ...... from xxtab where name like '% This is the full value of the contents of a field %'; SQL（ Simplified edition ） The tables involved are relatively large , Consumed a lot of IO resources . Randomly select the contents between the two percent signs to the table and query by equivalent value , You can find , It shows that the two percent signs can be removed , meanwhile like You can change to =. If there are only thousands or tens of thousands of records in the table at the initial stage of business launch , This sql The resources consumed are also very small , But over time , The data volume of the table reaches tens of millions or hundreds of millions , You are such a SQL Can take most of IO Resources are consumed . Those that use two percent sign fuzzy queries as the main filter criteria , Think more about it from a business perspective , Try to avoid using on large tables .

5、 I don't know why , Draw a dog after a cat （ Not a tiger ） Pagination of （xxtab This watch is very big ,sql Full table scan , Consumed a lot of IO resources ）：

SELECT * FROM

( SELECT A.*, ROWNUM RN

FROM ( SELECT * FROM xxtab) A

) WHERE RN >= 1

and rownum<= 3000 and ((type = 'typename') and (state = 0));

I don't know where I learned this pagination , The wrong way out. .type and state The index on the two fields doesn't work at all , Only the execution plan of full table scan can be used . The normal way of writing should be ：

SELECT A.*, ROWNUM RN

FROM

(SELECT * FROM xxtab

where rownum<= 3000 and ((type = 'typename') and (state = 0))

) A; In this way, large tables can be indexed ,IO Consumption will also be greatly reduced .

6、select ... from xxtab where (:b1 is null or col1=:b1) and (:b2 is null or col2=:b2) ; This kind of writing , It may come from some search requirements , Two input fields , Whether or not query criteria are entered , Can use the above sql. The writing is simple , But the efficiency is much worse ： No matter b1、b2 Whether there is input ,col1 and col2 Even if there is an index on the field , You can only select full table scanning . This can require an input condition in business , that sql According to different input , Generate different content , such as b1 There's input ,b2 No input ：select .... from xxtab where col1=b1; Both have input , Corresponding sql yes select ... from xxtab where col1=:b1 and col2=:b2; In this way, the index can be used efficiently .

7、 use rownum Paging write access ：

SELECT *

FROM ( SELECT t.*, ROWNUM RN

FROM ( select t1.*,ROWNUM

from ( select *

from p

left join c on c.objectid = p.objectid

left join o on o.objectid = p.objectid

left join e on e.objectid = o.objectid

) t1

) t WHERE ROWNUM < 1500000

) WHERE RN >= 1200000;

This page sql The framework of is completely correct , Full marks . But the usage scenario is seriously inappropriate ：p surface 7700 Ten thousand records （o The watch is also big ）,4 surface left join There is no predicate condition , The main query result set should also be at least 7700 ten thousand , Every time 30 ten thousand （1500000-1200000） strip , To execute 250 many times , All execution plans are full table scans , Need two big watches to do 250 Multiple full table scans . If these sql It's serial execution , Then there is another logical problem , The data of these tables are dynamic , use rownum Paging this way , It will definitely cause missing or repeated fetching . Actually this sql The best way to do this is without paging , Finish at one time . Only one table scan is required , And there will be no problem of missing or repeated fetching .

summary ：

SQL Optimization is important for most systems , Can bring very big performance improvement . Many problems cannot be solved by replacing advanced hardware , For example, a large table full table scan , Existing hardware condition execution time 10 minute , Execution time after hardware replacement 1 minute . And through optimization SQL, Build an appropriate index , Without replacing the hardware , Maybe not 10 You can get results in milliseconds .

Allied sql There may still be a lot of problems in writing , This time, only a few of them were selected top sql Share .SQL The writing is flexible , Good performance SQL They all conform to the specifications , Poor performance SQL It will expose your development level .