Flink format series (1) -json

Flink Of json Format support read and write json Formatted data , Just specify 'format' = 'json', Let's say kafka For example .

CREATE TABLE teacher (
  name STRING,
  age INT,
  title ROW<id INT, name STRING, level INT>,
  addr STRING
) WITH (
 'connector' = 'kafka',
 'topic' = 'test_teacher_info',
 'properties.bootstrap.servers' = 'localhost:9092',
 'properties.group.id' = 'testGroup',
 'format' = 'json',
 'json.fail-on-missing-field' = 'false',
 'json.ignore-parse-errors' = 'true'
)

Description of key configurations ：

json.fail-on-missing-field： Is in the json Whether an error is reported when a field is missing

json.ignore-parse-errors： In parsing json Whether an error is reported in case of failure

Generally, there is no guarantee json Format , So the above two configurations are more important .

Because there is ROW and ARRAY、MAP Data type of , So you can parse complex nested json Format , adopt xx.xx To access internally nested json Field . For example, the above one can pass select title.name To visit title This one json Inside name value . For example, the data is ：

{"name":"jack", "age":44, "title":{"id":1, "name":" associate professor ", "level": 3}, "addr":"xxxx1"}

{"name":"rose", "age":47, "title":{"id":1, "name":" associate professor ", "level": 2}, "addr":"xxxx2"}

Note here title Nesting of json The format of , Don't write  "title":\"{\"id\":1, \"name\":\" associate professor \", \"level\": 3}\" This format , Otherwise, it will report a mistake , Cannot be TextNode Turn into ObjectNode.

Some instructions for use ：

1）Json Each of the {} All need to use Row By type
2）Json Each of the [] All need to use Arrary By type
3） The subscript of an array is from 1 Not at the beginning 0
4）SQL When the field name in is a keyword , It needs to be decorated with backquotes for example ：`timestamp`
5）select The field type and order in the statement must be consistent with the field type and order of the result table

json The type and flink sql Type correspondence