Kunlundb query optimization (I)

front said

Previously, I introduced KunlunDB Architecture and technology of , Some friends should be curious about the process of query optimization , This chapter introduces the specific process of query optimization , An example will be given later to demonstrate .

Of course, the follow-up will also be on KunlunDB Other technical aspects of （ For example, data fragmentation ） Make an example demonstration , In order to facilitate everyone to understand KunlunDB.

Query optimization process

KunlunDB It is a distributed database system that separates computing and storage , When a query SQL Send to KunlunDB Any computing node （CN） when ,KunlunDB Syntax parser （Parser） First, the original query text will be parsed and some simple legitimacy verification , After that, the query will be logically optimized ： Such as query rewriting , Area pruning , Column cut , Predicate pushdown, etc .

KunlunDB In the process of logic optimization, the strategy of maximum push down will be adopted .

Calculating push down can not only avoid CN and DN The data network interaction between can also make full use of the ability of multi slice concurrent execution and various functions DN resources , Speed up queries .

The optimized operators are divided into two categories ：

• Operators that can be pushed down ：RemoteScan Push the operator to the corresponding data node for execution , After execution, pull the corresponding data to the calculation node for subsequent processing . Operators that support push down include ： Filter conditions , Such as WHERE or HAVING Conditions in . Aggregation operator , Such as COUNT,GROUP BY etc. , The aggregation calculation will be carried out in two stages . Sort operator , Such as ORDER BY,JOIN And subquery .Project, Projection operation .Distinct Weight removal .

• Some operators that cannot be pushed down ： Such as span shard Of join, You need to pull data from the data node to the calculation node for calculation , The optimizer will choose the best way to execute , Such as choosing the appropriate parallelism strategy

   

The global execution process is as follows ：

The optimization process is as follows ：

 

The maximum push down strategy is as follows ：

review ： For maximum performance , When defining partition keys, we should fully consider the execution of business SQL The scene of the statement , To minimize cross node data operations .

The project is open source

【GitHub：】
https://github.com/zettadb

【Gitee：】
https://gitee.com/zettadb

-END