Tidb and MySQL modify system variables / common statements (kill the process in process)

--  Query all global variables 
SHOW GLOBAL VARIABLES;
--  Query all session variables 
SHOW SESSION VARIABLES;

--  Specify query global variables 
SELECT @@GLOBAL.tidb_retry_limit;
--  Specify query session variables 
SELECT @@SESSION.tidb_retry_limit;
--  Query session variables first , Then query the global variables 
SELECT @@tidb_retry_limit;

--  Modify global variables 
SET GLOBAL 
--  Modify session variables 
SET SESSION 

Question 1 、（ modify sql_mode）

notes ： When synchronizing data, the related database sql_mode Be consistent or problems will arise
eg：

drop table if exists t;
create table t(a bigint, b bigint, c bigint);
insert into t values(1, 2, 1), (1, 2, 2), (1, 3, 1), (1, 3, 2);
select distinct a, b from t order by c;

Report errors ：ERROR 1055 (42000): Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'b' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by

stay MySQL in ,ORDER BY The expression must satisfy at least one of the following conditions , otherwise DISTINCT and ORDER BY Queries will be rejected for non-compliance ：

• The expression is equivalent to SELECT One of the list .
• All columns referenced by the expression and belonging to the query selection table are SELECT Elements of the list .

terms of settlement ：
see ：

SELECT @@GLOBAL.sql_mode 
ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

Modify to remove ONLY_FULL_GROUP_BY

set @@sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION';

Question two 、（ The transaction automatically retries and brings exceptions ）

• 1、tidb_disable_txn_auto_retry This variable is used to set whether to disable automatic retry of explicit transactions , Set to on when , Will not automatically retry , If there is a transaction conflict, you need to retry at the application layer . The default value is ：on/1;
  If the value of this variable is set to off/0,TiDB The transaction will be retried automatically , This will result in fewer errors when the transaction is committed . It should be noted that , This may cause data updates to be lost .

select @@GLOBAL.tidb_disable_txn_auto_retry;
-- SET GLOBAL tidb_disable_txn_auto_retry = 0;

• 2、tidb_retry_limit This variable is used to set the maximum number of retries , That is, a retrieable error is encountered in the execution of a transaction ( For example, transaction conflicts 、 Slow transaction commit or table structure change ) when , This transaction can be re executed , The value of this variable indicates the maximum number of retries . The default value is =10

select @@GLOBAL.tidb_retry_limit;
SET GLOBAL tidb_retry_limit = 20

• 3、tidb_backoff_weight Transaction retry time .TiDB towards PD take TSO Base timeout for , The default value is ：2;(TiDB towards PD take TSO The base timeout for is 15 second ), When tidb_backoff_weight = 2 Time base time 15 * 2 be equal to 30 second .

select @@GLOBAL.tidb_backoff_weight;
-- SET GLOBAL tidb_backoff_weight = 1

Question 3 、（ Set pessimistic lock tidb_txn_mode）

tidb_txn_mode
Scope ：SESSION（ since TiDB 3.0.4 Support from GLOBAL）
The default value is ：””

• This variable is used to set the transaction mode , The default is optimistic lock mode .TiDB 3.0 Pessimistic lock mode is added （ experimental ）. take tidb_txn_mode Set to ‘pessimistic’ after , This session All explicit transactions executed （ It's not autocommit The business of ） Will enter the pessimistic business mode .
• since TiDB 3.0.4 rise , This variable also supports GLOBAL Scope , Used to set the global transaction mode . When setting the global transaction mode , Created only after the modification takes effect session Will be affected .

SESSION Effect the currently modified tidb-server

select @@GLOBAL.tidb_txn_mode
select @@SESSION.tidb_txn_mode

Set to pessimistic lock

set @@global.tidb_txn_mode = 'pessimistic';
-- SET SESSION tidb_txn_mode = ''

Please refer to this article for pessimistic locking
The test script

Question 4 、

INFORMATION_SCHEMA.SLOW_QUERY, Slow query log query The statement is too long , Can't show full SQL sentence

[[email protected] tidb-ansible]$ vim /home/tidb/tidb-ansible/conf/tidb.yml
......
log:
  # Log level: debug, info, warn, error, fatal.
  # level: "info"

  # Log format, one of json, text, console.
  # format: "text"

  # Disable automatic timestamps in output
  # disable-timestamp: false

  # Queries with execution time greater than this value will be logged. (Milliseconds)
  # slow-threshold: 300

  # Queries with internal result greater than this value will be logged.
  # expensive-threshold: 10000

  # Maximum query length recorded in log.
  # query-log-max-len: 2048
  query-log-max-len: 20480
......
[[email protected] tidb-ansible]$
[[email protected] tidb-ansible]$
#  After rolling update , The subsequent collected statements can display all 
[[email protected] tidb-ansible]$ ansible-playbook rolling_update.yml

Congrats! All goes well. :-)
[[email protected] tidb-ansible]$

Question five 、（ Start large transaction ）

1、tidb_batch_insert( Turn on insert Large transactions )
Scope ：SESSION

The default value is ：0

This variable is used to set whether the inserted data is automatically segmented . Only in autocommit Effective when turned on . When inserting large amounts of data , You can set it to 1, In this way, the inserted data will be automatically divided into multiple batch, Every batch Insert using a single transaction . This usage destroys the atomicity and isolation of transactions , When using this feature , The user needs to ensure that there is no other operation on the table being processed , And when an error is reported , Timely manual intervention is required , Check the consistency and integrity of the data . therefore , It is not recommended for use in a production environment .

select @@SESSION.tidb_batch_insert
SET SESSION tidb_batch_insert = '1'

2、tidb_batch_delete( Turn on delete Large transactions )
Scope ：SESSION

The default value is ：0

This variable is used to set whether to automatically segment the data to be deleted . Only in autocommit Turn on , And it is deleted from a single table SQL Effective when . About deleting a single table SQL The definition of , See DELETE Syntax. When deleting large amounts of data , You can set it to 1, In this way, the data to be deleted will be automatically divided into multiple batch, Every batch Use a separate transaction to delete . This usage destroys the atomicity and isolation of transactions , When using this feature , The user needs to ensure that there is no other operation on the table being processed , And when an error is reported , Timely manual intervention is required , Check the consistency and integrity of the data . therefore , It is not recommended for use in a production environment .

Configure the whole SQL Memory usage threshold

Official parameter reference
Official book location
Modify one as much as possible about tidb_mem_* All change

SELECT @@tidb_mem_quota_query
 Set to 1G
set @@tidb_mem_quota_query=1073741824

Statement maximum execution time , The unit is millisecond . The default value is (0) Means unlimited .

select @@GLOBAL.max_execution_time;

---

tidb4.0

When changing the configuration file, refer to the configuration parameters of the corresponding version

tidb To configure

tiup cluster display test-cluster
tiup cluster edit-config test-cluster
tiup cluster reload test-cluster -N 172.168.192.33:4000

1、TiDB The maximum number of statements allowed for a single transaction .

4.0 Official reference

• stmt-count-limit
  TiDB The maximum number of statements allowed for a single transaction .
  The default value is ：5000
  In a transaction , exceed stmt-count-limit There is no... After the statement rollback perhaps commit,TiDB Will return statement count 5001 exceeds the transaction limitation, autocommit = false error . This restriction applies only to Retrieable ** Optimistic business ** Enter into force , If Use pessimistic transactions perhaps Closed transaction retry , The number of statements in a transaction will not be subject to this limit .

The solution refers to the problem 2、3

2、 adjustment TiDB Single transaction size limit

Report errors ：
Transaction is too large, size: 104857600;

You can adjust the following parameters

• txn-total-size-limit
  TiDB Single transaction size limit
  The default value is ：104857600 (Byte)/100M
  In a single transaction , all key-value The total size of the record cannot exceed this limit . The maximum value of this configuration item does not exceed 10737418240（ Express 10GB）. Be careful , If used with Kafka For downstream consumers binlog, Such as ：arbiter colony , The value of this configuration item cannot exceed 1073741824（ Express 1GB）, Because this is Kafka The maximum limit for processing a single message , Beyond that limit Kafka Will be an error .

3、 The maximum time a single transaction can hold a lock max-txn-ttl

Report errors ：
TTL manager has timed out, pessimistic locks may expire, please commit or rollback this transaction

• max-txn-ttl
  The maximum time a single transaction can hold a lock , Beyond that time , The lock of this transaction may be cleared by other transactions , The transaction cannot be successfully committed .
  The default value is ：600000
  Company ： millisecond
  Transactions beyond this time can only be committed or rolled back , Submission may not be successful .

---

Optimization parameter suggestions ：

Join Operator optimization

• tidb_distsql_scan_concurrency
  Scope ：SESSION | GLOBAL
  The default value is ：15
  This variable is used to set scan Concurrency of operations .
  AP Class applications are suitable for larger values ,TP Class applications are suitable for smaller values . about AP Class application , The maximum value is recommended not to exceed all TiKV Node CPU Check the number .

-- Set up GLOBAL Scope 
mysql> set @@global.tidb_distsql_scan_concurrency=30;

• tidb_index_lookup_size
  Scope ：SESSION | GLOBAL
  The default value is ：20000
  meaning ： This variable is used to set index lookup Operation of the batch size ,AP Class applications are suitable for larger values ,TP Class applications are suitable for smaller values ,SQL Specific examples of setting are as follows ：

-- Set up GLOBAL Scope 
mysql> set @@global.tidb_index_lookup_size=40000;

• tidb_index_lookup_concurrency
  Scope ：SESSION | GLOBAL
  The default value is ：4
  meaning ： This variable is used to set index lookup Concurrency of operations ,AP Class applications are suitable for larger values ,TP Class applications are suitable for smaller values ,SQL Specific examples of setting are as follows

-- Set up GLOBAL Scope 
mysql> set @@global.tidb_index_lookup_concurrency=8;

• tidb_index_lookup_join_concurrency
  Scope ：SESSION | GLOBAL
  The default value is ：4
  meaning ： This variable is used to set index lookup join The concurrency of the algorithm ,SQL Specific examples of setting are as follows ：

mysql> set @@global.tidb_index_lookup_join_concurrency=8;

• tidb_hash_join_concurrency
  Scope ：SESSION | GLOBAL
  The default value is ：5
  meaning ： This variable is used to set hash join The concurrency of the algorithm ,SQL Specific examples of setting are as follows ：

mysql> set @@global.tidb_hash_join_concurrency=10;

• tidb_index_serial_scan_concurrency
  Scope ：SESSION | GLOBAL
  The default value is ：1
  meaning ： This variable is used to set the order scan Concurrency of operations ,AP Class applications are suitable for larger values ,TP Class applications are suitable for smaller values ,SQL Specific examples of setting are as follows ：

mysql> set @@global.tidb_index_serial_scan_concurrency=4;

Kill process Process in

Example

mysql> SHOW PROCESSLIST;
+------+------+-----------+------+---------+------+-------+------------------+
| Id   | User | Host      | db   | Command | Time | State | Info             |
+------+------+-----------+------+---------+------+-------+------------------+
|    1 | root | 127.0.0.1 | test | Query   |    0 | 2     | SHOW PROCESSLIST |
|    2 | root | 127.0.0.1 |      | Sleep   |    4 | 2     |                  |
+------+------+-----------+------+---------+------+-------+------------------+
2 rows in set (0.00 sec)

mysql> KILL TIDB 2;
Query OK, 0 rows affected (0.00 sec)

According to design , This statement is the same as by default MySQL Are not compatible . This helps prevent errors in TiDB Termination of connection on the server , Because there are usually multiple TiDB The server is placed behind the load balancer .
mysql Usage method

KILL [CONNECTION | QUERY] thread_id

---

TiDB How to use the window function in MySQL 8.0 Almost the same , Details can be found at MySQL Window function . Because the window function will use some reserved keywords , May result in the normal execution of SQL Statement is upgrading TiDB The syntax cannot be parsed after , Now you can put tidb_enable_window_function Set to 0, The default value for this parameter is 1.