[learning notes] tidb learning notes (III)

This article is about 《 Geek time 》-《TiDb Introduction to minimalism 》 Learning notes of . Portal ：https://time.geekbang.org/opencourse/videointro/100089601

---

TiDB Of HTAP The way

HTAP(Hybrid Transaction and Analytical Process, Mixed transactions and analytical processing )

Support at the same time OLTP and OLAP, Support real-time analysis .

OLTP( Online transactions ): Focus on high concurrency , Low latency

OLAP( Online analysis ): More focus on throughput

1. Distributed database provides... In large data scale HTAP The basis of

2. TiDB-serer Calculation method under maximum program and Hash/Join Key operators provide the basis for AP Ability

TiDB It can be compared to a large one Mysql, Earliest TiDB It is to solve the problem of dividing databases and tables in online business , Due to the following characteristics ：

1. Mass storage allows multiple data sources to converge , Real time data synchronization

2. Support standards sql, Multi table association can quickly generate results

3. Multi business module with the same name , Support task dimension query after sub table aggregation

4.TiDB Maximum push down mechanism , And parallel hash join Equal operator , Decisive TiDb Advantages in table Association

These features are very suitable for some businesses in the data center , Is accidentally applied to the data center , Provides some basic AP Ability

3. With the help of ecology , Give Way spark Run in the Tikv On

however TiDB Our initial orientation is to OLTP The system of , in the light of OLAP, It's easy to cause OOM, So we introduced spark, Repackage as Ti-spark, It eases the problem of computer power in data . but spark Only low concurrency heavyweight queries can be provided , Small and medium-sized AP Queries can lead to high resource consumption .

4. Row and column mixing engine , The columnar engine provides real-time write capability

Now OLTP And TIspark With the same set of underlying storage TiKv,OLTP and OLAP It is difficult to isolate resources at the software level

Physical isolation is the best resource isolation

List natural pairs OLAP Inquiry friendly , Columnar storage is friendly to batch writing , Not friendly to real-time updates . Learn from it LSM Thought , Introduced on the columnar engine delta tree Methods , Finally, a columnar engine supporting quasi real-time update is implemented :Ti-flash（ be used for OLAP Copy of data ）.

5. The row and column engine takes raft-base replication, It solves the problem of data synchronization efficiency

How to synchronize replicas to the columnar engine ？

Ti-Flash With Raft Learner The way to access Multi-Raft Group , Transfer data asynchronously , Yes Tikv Create a very small burden , When data is synchronized to Ti-flash, Will be disassembled from row format to column format .

6.TiDB-servert Unified technical services

7.Mpp Solve the expansion of computing nodes and parallel computing

OLAP In the scene of , Large table associations often occur , In the previous architecture join It can't be pushed down , Introduced MPP Computing framework

TiDB Key technology innovation

1. The automatic slicing technology is the foundation of the finer dimensional elasticity

2. Elastic slices form a dynamic system

3. multi-raft Make the replication group more discrete

4. be based on multi-raft Realize linear extension of writing

5. be based on multi-raft Realize cross IDC Single table multi node write

6. Decentralized distributed transactions

7. Local Read and Geo-partition

8. Larger data capacity TP and AP The fusion

9. Unification of data services

TiDB Typical application scenarios and user cases

1. OLTP Scale

2. Real-Time HTAP