It supports running in kubernetes, adds multiple connectors, and seatunnel version 2.1.2 is officially released!

stay Apache SeaTunnel(Incubating) 2.1.1 About a month after the release , The community has accepted nearly 100 teams or individuals from all over the world PR, To bring you 2.1.2 edition . This version improves stability , And in the function 、 file 、 Examples are optimized .

This article will introduce Apache SeaTunnel (Incubating) 2.1.2 Version update .

• Release Note:

  https://github.com/apache/incubator-seatunnel/blob/2.1.2/release-note.md

• Download address ：https://seatunnel.apache.org/download

01 Main function update

add to Webhook and Http2 Kind of connector , The main enhancement Http Relevant data processing capacity .

Special thanks tmljob Contributions of students .

01 Webhook

Users can schedule tasks through this connector , Scheduling of events , Various practical functions such as data push , It only needs to be provided at the output Http The support of service capability is enough .

For details, see https://seatunnel.apache.org/docs/2.1.2/connector/source/Webhook

02 Http

Support reading Http Interface data , Provide passage for upstream Http Complete data transfer to SeaTunnel The ability to do further processing ,Http It is a general standard interface , In this way, various services can be accessed . Use as follows ：

Http {   url = "http://date.jsontest.com/"   result_table_name= "response_body"  }

FlinkSQL Module added Kafka and ElasticSearch The connector , Now? SeaTunnel It can be used SQL To read the data written to these data sources .

Transfrom Aspect added UUID and Replace Support for , It is more flexible to simply process the data . It also supports the addition of custom functions , Help users implement various custom business logic .

03 Support in Kubernetes Up operation SeaTunnel

Kubernetes It has become a necessary component in the cloud native era ,SeaTunnel It is natural to provide corresponding support .

Official fit SeaTunnel Running on the Kubernetes, See tutorial

https://seatunnel.apache.org/docs/2.1.2/start/kubernetes

02 Specific updates

01 [Connector]

• add to Spark webhook connector Support for

• Optimize Connector Of Jar Package structure

• add to Spark Replace transform Components

• add to Spark Uuid transform Components

• Flink Of JDBC source Added right Oracle adapter

• add to Flink HTTP connector Support for

• add to Flink Register custom function function

• Flink SQL Module add Kafka and ElasticSearch connector Support for

02 [Core]

• add to Flink application The operating mode supports

• Support Flink Dynamic addition of configuration

03 [Bug Fix]

• Repair Clickhouse Sink Component part type conversion problem

• In some cases of repair Spark The first time you run a script, it will fail

• Repair some cases in Spark on yarn cluster Mode failed to get configuration file problem

• Repair Spark extraJavaOptions  Questions that cannot be empty

• Repair Spark standalone cluster Unable to extract internal files in mode

• Repair Clickhouse Sink Unable to properly handle the multi node configuration problem

• Repair Flink SQL Configuration parsing error

• Repair Flink JDBC Mysql The problem of incomplete type matching

• Fix variable in Flink Problems that cannot be set in mode

• Repair SeaTunnel stay Flink Problems that cannot be checked by configuration in mode

04 Optimize

• upgrade Jackson Version to 12.6

• Add deployment SeaTunnel To Kubernetes Wizard for

• Adjust some generic type code

• add to Flink SQL e2e modular

• Flink JDBC connector  add to pre sql and post sql characteristic

• Use @AutoService Generate SPI file

• Flink FakeSourceStream Support mock data

• Supported by Flink JDBC connector Read Hive data

• ClickhouseFile Support ReplicatedMergeTree engine

• Support Hive sink preservation ORC Format data

• Support Spark Redis sink Custom expiration time

• add to Spark JDBC Transaction isolation level configuration

• use Jackson Replace... In the code Fastjson

03 thank

Thanks to the following students who participated in the contribution ( by GitHub ID, In no particular order ), It is the dedication and efforts of everyone , So we can quickly launch this version , We also welcome more students to participate in Apache SeaTunnel(Incubating)  Community contribution .

v-wx-v, GezimSejdiu, zhongjiajie, CalvinKirs, ruanwenjun, tmljob, Hisoka-X, 1996fanrui, wuchunfu, legendtkl, mans2singh, whb-bigdata, xpleaf, wuzhenhua01, chang-wd, quanzhian, taokelu, gleiyu, chenhu, dijiekstra, tobezhou33, LingangJiang, mosence, asdf2014, waywtdcc, Emor-nj, dik111, forecasted