当前位置:网站首页>Flink partition policy

Flink partition policy

2022-06-25 11:39:00 Direction_ Wind

  1. GlobalPartitioner The data will be distributed to the first instance of the downstream operator for processing
  2. RebalancePartitioner Count It is reported that Be followed Ring hair Deliver to Next Swimming each A reality In the example Into the Office The reason is .
  3. RescalePartitioner This kind of partition will be based on the parallelism of upstream and downstream operators , Loop output to every instance of the downstream operator .
  4. BroadcastPartitioner The broadcast partition will output the upstream data to each instance of the downstream operator . optimum close For big data sets and small data sets Jion Scene .
  5. ForwardPartitioner Used to output records to downstream local operator instances . It requires upstream and downstream count Son and That's ok degree One sample . To put it simply , ForwardPartitioner Console printing for data .( It's also chain Conditions for operators )
  6. KeyGroupStreamPartitioner Hash Comparator . Will press the data Key Of Hash Value output to downstream operator instance .
  7. CustomPartitionerWrapper User defined partition . It needs to be implemented by users themselves Partitioner Interface , To define your own partition logic

RescalePartitioner It's a little hard to understand , Suppose the upstream parallelism is 2, The number is A and B. The downstream parallelism is 4, The number is 1,2,3,4. that A Then send the data loop to 1 and 2,B Then send the data loop to 3 and 4. Suppose the upstream parallelism is 4, The number is A,B,C,D. Downstream parallel Line degree , The number is 1,2. that A and B Send the data to 1,C and D Send the data to 2.

原网站

版权声明
本文为[Direction_ Wind]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/02/202202200537152797.html