Kunlun distributed database sequence function and its implementation mechanism

Kunlun distributed database （ Kunlun or Kunlun database for short ） The calculation node of is derived from PostgreSQL, So I inherited PostgreSQL Of Sequence function , This paper introduces the design of Kunlun distributed database Sequence Function and usage of 、 Use cases and implementations .

Usage and use cases

Kunlun database Sequence And MySQL Of autoincrement（ On the column ） comparison , Its functions are more powerful and flexible . Mainly reflected in the following aspects ：

1.  Kunlun Sequence And tables are many to many , and MySQL The autoincrement and table of are 1 Yes 1 Relationship

say concretely , Each table of Kunlun database can have any number of sequence Columns use the same or different sequence Generate sequence values ;

And each of these sequence Can be used by any number of columns of any number of tables to generate ID value .

and MySQL Each table can have at most one auto incrementing column, and this auto incrementing column can only be used by this table （ This is bullshit , But for the sake of content symmetry, I still need to mention ）.

2.  Can be adjusted at any time sequence The initial value of the , Maximum , step , Range and other attributes, and then continue to use , then sequence A new sequence value will be generated according to the new attribute .

3.  Don't rely on Index , After clearing the table, the sequence value does not wrap around .

4.  Multiple computing nodes in Kunlun database cluster use the same server directly or indirectly sequence Can produce globally unique sequence values .

Let's take an example , First create a table t1,t1 The primary key column of serial Type indicates that it uses an implicitly created sequence To generate field values , So you can insert without specifying the field value .

create table t1(a serial primary key, b int);

Then create sequence seq_b, Ready to use seq_b To generate field values . You can optionally specify sequence Properties of , Use default values without specifying .

create sequence seq_b;

First execute this statement to insert 9 That's ok , Explicit call seq_b Generate field values .

insert into t1(b) values(nextval('seq_b'));

You can see t1 The data are as follows ：

select*from t1;

Then create the table t2, its b and c All columns use seq_b Generate default field values , And its primary key column is also applicable to implicit sequence To generate field values .

create table t2(a serial primary key, b int default nextval('seq_b'), c int default nextval('seq_b'));

because t2 All fields have default values , So insert the table with the following statement t2 3 That's ok .

insert into t2 default values;

see t2 The data of , You can see each line b and c The field is to use seq_b The field values generated in turn , And from seq_b Last generated 9 Then start generating sequence values .

select*from t2;

Last , You can also use select nextval('seq_b'); Such a statement to directly generate sequence values .

modify sequence Metadata and others

have access to ALTER SEQUENCE Statement to modify sequence Properties of , You can also use ALTER TABLE ... ALTER COLUMN ... SET seqoptions Statement to modify the implicit sequence Properties of .

You can also use the above alter table sentence restart One sequence. And you can use lastval() Function to obtain sequence Last returned value .

sequence Realization

Kunlun database sequence The implementation inherits PostgreSQL The original sequence Mechanism .

In order to make sequence Data has disaster tolerance capability and can be used by any number of computing nodes at the same time , therefore sequence The numerical data related to the distribution of sequence values is stored in the storage node mysql.sequences In the table , Each line corresponds to one sequence.

One sequence In which storage cluster is the metadata stored , Is to create sequence Is dynamically allocated by the computing node .

sequence Other metadata of is stored in the computing node , You can use the following statement to view sequence Metadata in the compute node ：

select t2.relname, t2.oid, seqstart, seqincrement, seqmax, seqmin, seqcache, seqcycle from pg_sequence t1, pg_class t2 where t1.seqrelid = t2.oid;

You can see t1 and t2 The implicit of the primary key column sequence Namely t1_a_seq and t2_a_seq, And explicitly created seq_b , these sequence The storage clusters of the numerical metadata are 1,2,1.

meanwhile , You can see sequence The basic metadata of is also stored in pg_class Metadata table , Its unique properties are stored in pg_sequence In the table .

Respectively connected to No 1 and 2 Of shard Check this out 3 individual sequence In these two storage clusters mysql.sequence Numerical metadata in the table , You can see the following information ：

When using a for the first time sequence Or when the reserved value range is exhausted , A computing node CN Will pass through its cluster_log_applier Process to this sequence In the same storage cluster reserve (curval, cur_val + max(10, seqcache)) The field value of this range , then CN Use this reserve The scope for this sequence Distribute sequence values , Until it runs out again .

such , Even if multiple compute nodes use the same sequence To distribute sequence values , It can still maintain high performance and keep the sequence values distributed by all computing nodes unique .

Conclusion

Popular speaking , If both reading and writing to the database are operated in the same database server , Business system performance will be degraded . 

In order to improve the performance of business system , Optimize the user experience , You can reduce the load of the master database by master-slave replication . 

And if the primary database goes down , It can quickly switch the business system to the slave database , Avoid data loss .

The project is open source

【GitHub：】

https://github.com/zettadb

【Gitee：】

https://gitee.com/zettadb

THE END