Flink checkpoint and savepoint

savepoint and checkpoint All are flink Powerful features for fault tolerance , It can be saved automatically or manually job Operating state

Difference between them

checkpoint：

Application timing trigger , User save status , Will expire , Enable when the internal application fails to restart , But manually cancel when , Will delete the previous checkpoint

savepoint：

The user manually sets the top , The equivalent of a state backup , Can be in bin/flink cancel xx Called when , It is generally used to modify the parallelism , Program upgrade, etc .

If you want to restart after the program stops , When the data is consistent , Highly recommended cancel Do it on time savepoint, Of course, if the program does not involve state, Don't have to

Trigger savepoint

Trigger yes 2 Ways of planting ：

1） Direct trigger ===>bin/flinksavepointjobId [targetDirectory] [-yid yarnAppId] in the light of yarn Mode needs to be specified -yid Parameters

2）cancel/stop Trigger when ===> bin/flinkcancel/stop-s [targetDirectory] jobId [-yid yarnAppId]. in the light of yarn Mode needs to be specified -yid Parameters

Checkpoint Reading

stay Flink In Distributed Computing , Fault tolerance is often a concern ,Flink Itself through Checkpoint Operation for snapshot processing , Guarantee job The stability of execution and Failover The reliability of the .checkpoint No interruptions stream、data-flow.

stay Checkpoint There is a key element in ：barriers

• barrier take stream Medium record Divide the different checkpoint paragraph ,barrier Not more than record, Strictly behind the data
• Every barrier Will carry the corresponding ID, To distinguish the corresponding snapshot snapshot（checkpoint）
• Different checkpoint Of barrier Can be in stream Medium parallel transmission , This means that snapshot operations can be performed in parallel at the same time