Black horse PostgreSQL, why is it black in the end

hello ！ Hello everyone , I am a 【IT bond 】, Jianghu people jeames007,10 year DBA Work experience
A highly motivated 【 Bloggers in big data field 】！
China DBA union (ACDU) member , Currently engaged in DBA And program programming ,B Station and Tencent classroom lecturer , Live broadcast volume breaking 10W
Good at mainstream data Oracle、MySQL、PG Operations and development , Backup recovery , Installation migration , performance optimization 、 Fault emergency treatment, etc .
If there is a pair of 【 database 】 Interested in 【 Cutie 】, Welcome to your attention 【IT bond 】
️️️ Thank you, big and small ！️️️

️1.PostgreSQL Introduce

PostgreSQL Touting itself as the world's most advanced open source database .
PostgreSQL Some of our fans say it works with Oracle Comparable ,
And it's not that expensive and haughty customer service .
It has a long history , Initially 1985 Developed at the University of California, Berkeley
After a 15 More than years of active development and continuous improvement ,
PostgreSQL Under reliability 、 stability 、 Data consistency has won a relatively high reputation in the industry .

2022 Annual database ranking , Compared with previous years 2 As calm as the month . There is no change in the top ten ranking compared with last month ,PG Ranked fourth

PostgreSQL What are the advantages of database ？

PostgreSQL Database is a powerful open source database ,
 It supports rich data types （ Such as JSON and JSONB type 、 An array type ） And custom types .

PostgreSQL The database provides rich interfaces , It's easy to extend its functions ,
 If you can GiST Implement your own index type under the framework , Support use C Language to write custom functions 、 trigger ,
 It also supports writing custom functions in popular programming languages .

PostgreSQL Database has the following advantages ：

PostgreSQL Database is the most powerful open source database at present ,
 It is closest to the industrial standard SQL92 Query language of ,
 At least it has SQL:2011 What is required in the standard 179 Of the main functions 160 term 
（ notes ： At present, no database management system can fully realize SQL:2011 All the main functions in the standard ）.

 Stable and reliable ：PostgreSQL It is the only open source database that can achieve zero data loss .
 At present, it is reported that some banks at home and abroad use PostgreSQL database .

 Open source saves money ： PostgreSQL The database is open source 、 Free of charge ,
 And it uses classes BSD agreement , There are basically no restrictions on use and secondary development .

 There is broad support for ：PostgreSQL  The database supports a large number of mainstream development languages ,
 Include C、C++、Perl、Python、Java、Tcl as well as PHP etc. .

PostgreSQL The community is active ：PostgreSQL Basically every 3 A patch version will be released next month ,
 This means known Bug It will be repaired soon , The requirements of application scenarios will also be responded in time .

️2.PostgreSQL be relative to MySQL The advantages of

stay SQL The implementation of the standard is better than MySQL perfect , And the function realization is more rigorous ;
The function support of stored procedure is better than MySQL good , Ability to cache execution plans locally ;
Table connection support is more complete , The optimizer is more complete , There are many types of indexes supported , Complex query ability is strong ;
PG The main meter is stored by stacking meters ,MySQL Organize tables by index , Can support more than MySQL More data .
PG The primary and secondary replication of is a physical replication , be relative to MySQL be based on binlog Logical replication of , Data consistency is more reliable , Better replication , It also has less impact on the performance of the host .
MySQL Storage engine plug-in mechanism , There is a problem of complex lock mechanism affecting concurrency , and PG non-existent .

️3.PG、Oracle and MySQL contrast

️4.PG Learning Guide

1.  Recommended reading 《PostgreSQL The nine story pagoda of learning 》：
https://mp.weixin.qq.com/s/i7b6FvY3PYC2JENCgiVxjQ
2. Can learn and read independently PG Basic documents of application management ,
 This part is mainly about China PG Branch training certification PGCA Excerpts from the course ,
 Related links ：http://www.postgresqlchina.com/tecdoc
3.PG The main site of learning 
PG The international community ：https://www.postgresql.org/
1) PG Summary ：https://www.postgresql.org/about/
2) PG Online help documentation （ English version , many PG edition ）：
https://www.postgresql.org/docs/
3） Can also pass PG Consult and learn Chinese manuals , Access address ：
http://www.postgres.cn/docs/10/;
http://www.postgres.cn/docs/11/
4) Installation media download address ：
https://www.postgresql.org/download/
 There are mainly binary 、 There are two ways to compile and install the source code , Binary installation media correspond to different operating systems .
 There is also an exception based on PG Installation of product release version ,
 The installation media and installation methods can be obtained through the company's official website of the product ,
 For example, Ali POLARDB、 Yaxin ANTDB、 tencent TBase、 Huawei GaussDB、 Hangao HGDB etc. .
5) Online learning resources 
https://www.postgresql.org/docs/online-resources/
 Contains rich tutorials 、 Hands on practice resources 

️5.PostgreSQL Tools

PostgreSQL Tools can be broadly divided into the following categories ：

Backup recovery tools
Monitoring tools
Logic and trigger based replication tools
Multi master replication tool
High availability and failover tools
Pool connection tool
Table partitioning tool
Migration tools

5.1 Backup recovery tools

1. Barman
Barman (Backup and Recovery Manager- Backup recovery manager ) 
 It's a use. Python The realization of language PostgreSQL Disaster recovery management tools ,
 It is run by the second quadrant company (2ndQuadrant) Open source and maintain . It allows us to perform remote backups in business critical environments ,
 Provide effective data assurance for database administrators in the recovery phase .
Barman The best features include backing up metadata 、 Incremental backup 、
 Retention policy 、 Remote reply 、WAL File archive compression and backup .

2. EDB BART
EDB BART(Backup and Recovery Tool - Backup recovery tools ) It's enterprise PostgreSQL Key components of a data management strategy .
BART For large-scale deployment PostgreSQL The service provides the implementation of retention policy and point in time recovery .
BART 2.0 Version provides block level incremental backups .

3. PgBackRest
pgBackRest The main purpose of the tool is to make a simple and reliable backup and recovery tool ,
 To be able to seamlessly access to large-scale databases and workloads .pgBackRest Abandon other traditional backup tools tar and rsync The routine ,
 Its backup functions are implemented from within the software , And use client protocol to interact with remote server . Removed right tar and rsync Dependence ,
 Make it better able to meet the backup challenges for specific databases . Client side remote protocols are more flexible ,
 The protocol can restrict the connection type as required to ensure that the backup process is more secure .

5.2、 Monitoring tools

1. PoWA
PoWA(PostgreSQL Workload Analyzer) yes PostgreSQL Workload analysis tools for ,
 It collects performance data and provides real-time display of icons and images , To help us monitor and tune PostgreSQL The server .
 It and Oracle AWR perhaps SQL Server MDW It's like .

2. PgCluu
pgCluu It's a PostgreSQL Performance monitoring and auditing tools for .
 It shows you from... In the form of a view PostgreSQL All statistics collected by the database cluster .
 It can display a completed database cluster information and system utilization information .

3. Pgwatch2
Pgwatch2 It's surveillance PostgreSQL One of the easiest to use database tools .
 It's based on Grafana And for PostgreSQL The database provides out of the box monitoring functions .
 Because it has been integrated into the container , So we don't have to worry about dependencies and complex installation steps ,
 The monitoring can be set up in a few minutes , Everything has been configured in advance .
 We only need to configure the database connection to the monitoring to run the normal monitoring operation .

5.3、 Logic and trigger based replication tools

1. pgLogical
pglogical Is to use PostgreSQL A logical replication tool implemented in the form of an extension .
 Perfect integration , Do not use any triggers or external programs . This plug-in acts as a substitute for physical replication ,
 Adopt publishing for selective replication / Subscription model , Is an effective way to replicate data .

2. Slony-I
Slony-I yes PostgreSQL Implementation of one master multi-slave replication system , Support cascading replication .
 Development Slony-I The main purpose of is to realize master-slave replication ,
 The replication system includes all the features and capabilities required by a large database system to properly configure the slave system .

Slony-I It is mainly designed for data center and backup site scenarios ,
 This scenario usually requires all nodes to be available .

3. Bucardo
Bucardo It's a PostgreSQL Asynchronous replication system , Allow to configure multi master and multi slave operations .
 It is http://Backcountry.com The company's Jon Jensen and Greg Sabino Developed .

5.4、 Multi master replication tool

BDR
Postgres-BDR(Bi-Directional Replication for PostgreSQL)
 It is the first open source in the world PostgreSQL Multi master replication system , The purpose is to strengthen the production environment .
 From the second quadrant (2ndQuadrant) The company opens source and maintains ,BDR Specially designed for geographically distributed cluster environments ,
 Use the funny asynchronous logical replication method , Support from the 2 A to 48 More than nodes are distributed in different regions .

5.5、 High availability and failover tools

1. Repmgr
repmgr It's an open source 、 be used for PostgreSQL A tool for server cluster replication management and failover .
 It expands PostgreSQL The built-in hot-standby Ability ,
 You can set up a hot backup server 、 Monitor replication 、 Perform management tasks ( Fail over 、 Manual switching, etc ).
repmgr It's the second quadrant ( 2ndQuadrant) company-developed .

2. PAF
PAF(PostgreSQL Automatic Failover- Automatic fault transfer tool ) yes OCF Resource agents contribute to PostgreSQL Of ,
 Its initial purpose is to Pacemaker Management and PostgreSQL Make clear the rules , Make things simple 、 Documented and effective .
 If your PostgreSQL The cluster has enabled internal flow replication ,
PAF Exposed to the Pacemaker At present every PostgreSQL Status of the instance node ：
 Which is the Lord , Which is from , Which has stopped , Which is chasing the copy status, etc . If the primary node fails ,
Pacemaker By default, restore the failed primary node first . If it fails, it cannot be recovered ,
PAF Will select the best one from the nodes ( Closest to the failed master node data ) Promote to a new master .

3. Patroni
Patroni It's a template , It USES Python To provide you with a customized , Highly available solutions , by 
 Maximum availability , Its configuration information is stored in places like ZooKeeper, etcd perhaps Consul in .
 If DBAs,DevOps Engineers or SRE Looking for a way to quickly deploy high availability in the data center PostgreSQL programme ,
 Or other uses ,Patroni  Can help .

4. Stolon
Stolon It's a cloud native Of PostgreSQL High availability management tools .
 The reason it is cloud native What's important is that it can be used inside the container PostgreSQL Provide high availability (Kubernetes  Integrate ),
 And it supports other kinds of infrastructure ( such as ：cloud IaaS, Old style infrastructure, etc )

5.6、Connection Pooling Tools

1. PgBouncer
PgBouncer yes Skype The R & D personnel of 2007 Connection pooling tool developed in .
 For many years after that , The project has been improved by many developers , But no matter how it changes ,
 It reduces PostgreSQL The role of connection costs has not changed .
PgBouncer allow PostgreSQL A database operation provides client access that is greater than the maximum number of connections it can provide .
 It essentially tracks every client connection , Then based on the configuration information ,
 Create some client connections and serve client access based on the first in first service principle .

2. PgPool-II
pgpool-II It is also a connection pool , We usually call it pgpool.
 It is another popular connection broker , It predates PgBouncer It will be released in about a year (2006 Released in the second half of ).
pgpool The scope of use of is very relevant , The functions that can be provided include ： Query based replication , Connection pool function , Load balancing , Parallel query, etc ,
pgpool An important feature of is connection pooling . If we have two PostgreSQL The server , We want to use virtual IP,
 In this way, the client will not feel the impact of the master database switch . occasionally , To move between servers IP Address ,
 First of all, you need to transfer the data from the primary database server IP remove , And rebuild it on the other one , This will break the active link ,
 Cause temporary service unavailability . Use pgpool The server can be cached until another server is promoted ,
pgpool Handles failover internally , From the perspective of application and client , It seems that the database has never been offline .

5.7、 Table partitioning tool

1. Pg_Partman
pg_partman yes PostgreSQL An extension of , Used to create and manage time-based or sequence based table partitions .
 It also supports multi-level sub partitions . Child tables and triggers are managed by the extension itself . Tables that already have data can easily add fine-grained partitions .
 The optional retention policy can automatically delete partitions that are no longer needed .
 Background work process (BGW) It can automatically run partition maintenance scheduled tasks , It doesn't have to depend on linux cron And other programs are maintained from the outside .

2. pg_Pathman
pg_pathman yes PostgreSQL Pro The company's open source extension , It can provide optimized partition solutions for large distributed databases . Use pg_pathman Can partition large databases without downtime , Speed up partition table query , Dynamically manage and add partitions ,
 Add an external table to the partition , Operate joint partitions, etc .

5.8、 Migration tools

Ora2pg
Ora2Pg It is used to put Oracle or MySQL Database migration to PostgreSQL Free tools .
 It can be connected to Oracle database , Then automatically scan and export the table structure or data on the source side ,
 Turn into PostgreSQL database SQL Script .Ora2Pg Can be viewed as Oracle Reverse engine of database ,
 Used for large enterprise database migration or Oracle Copy data to PostgreSQL Database and other scenarios .
 It's easy to use , No need for any Oracle Database background , All you need to do is build and Oracle Database connection .

Everybody likes it 、 Collection 、 Focus on 、 Comments WeChat official account