当前位置:网站首页>[Flink] aggregation operator
[Flink] aggregation operator
2022-07-25 03:27:00 【No bug is the biggest bug】
One 、 Entity class
@Data
public class Event {
public String user;
public String url;
public Long timestamp;
public Event(String user, String url, Long timestamp) {
this.user = user;
this.url = url;
this.timestamp = timestamp;
}
}Two 、 Custom data sources simulate streaming data
public class ClickSource implements SourceFunction<Event> {
// Declare a flag class
private Boolean running = true;
@Override
public void run(SourceContext<Event> ctx) throws Exception {
// Randomly generated data
Random random = new Random();
// Define the data set selected by the field
String[] users = {"Mary","Alice","Bob","Cary"};
String[] urls = {"./home","./cart","./fav","./prod?id=100","./prod:id=10"};
// Loop data
while (running){
String user = users[random.nextInt(users.length)];
String url = urls[random.nextInt(user.length())];
Long time = System.currentTimeMillis();
ctx.collect(new Event(user,url,time));
Thread.sleep(2000);
}
}
@Override
public void cancel() {
running = false;
}
}3、 ... and 、 Task code
public static void main(String[] args) throws Exception {
// Create an execution environment
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// Set parallelism
env.setParallelism(1);
// Read data source
DataStreamSource<Event> dataStream = env.addSource(new ClickSource());
// Press key groups to perform aggregate query Extract the latest access data of the current user max Only take the maximum value of the current specified field maxBy Round the maximum value of a piece of data
SingleOutputStreamOperator<Event> max = dataStream.keyBy(new KeySelector<Event, String>() {
// Group here by user name
@Override
public String getKey(Event event) throws Exception {
return event.user;
}
}).max("timestamp");
SingleOutputStreamOperator<Event> maxBy = dataStream.keyBy(new KeySelector<Event, String>() {
// Group here by user name
@Override
public String getKey(Event event) throws Exception {
return event.user;
}
}).maxBy("timestamp");
// Print
max.print("max");
maxBy.print("maxBy");
// Start
env.execute();
}边栏推荐
- C language writes a circular advertising lantern or changes it to a confession system
- Force deduction problem 238. product of arrays other than itself
- Question D: pruning shrubs
- Query the information of students whose grades are above 80
- Dc-2-range practice
- Innobackupex parameter description
- Riotboard development board series notes (VII) -- the use of framebuffer
- Solution: owner's smart site supervision cloud platform
- How does Jupiter notebook change themes and font sizes?
- Introduction to Apache Doris grafana monitoring indicators
猜你喜欢

Swagger key configuration items

Force deduction brush question 26. Delete duplicates in the ordered array

Machine learning exercise 8 - anomaly detection and recommendation system (collaborative filtering)

Task02 | EDA initial experience

kettle_ Configure database connection_ report errors

Experiment 4 CTF practice

Use reflection to convert RDD to dataframe

Analysis of browser working principle

Solution: owner's smart site supervision cloud platform

P100 MSSQL database penetration test of secondary vocational network security skills competition
随机推荐
Experience sharing of system architecture designers in preparing for the exam: how to prepare for the exam effectively
mysql_ User table_ Field meaning
How does Jupiter notebook change themes and font sizes?
Secondary vocational network security skills competition P100 vulnerability detection
Unified return data format
Database transactions (often asked)
List type to string type
Message queue (MQ)
mysql_ Backup restore_ Specify table_ Backup table_ Restore table_ innobackup
Idea configuration
Use of CCleaner
New features commonly used in ES6
The difference between abstract classes and interfaces
C language writes a circular advertising lantern or changes it to a confession system
Secondary vocational network security skills competition P100 web penetration test
Moveit2 - 7. Scenario planning ROS API
Chrome debugging skills
10. 509 Certificate (structure + principle)
Network security - information hiding - use steganography to prevent sensitive data from being stolen
Analysis of browser working principle