当前位置:网站首页>Still have 1 requests outstanding when connection from slaveX/X.X.X.X:33202 is closed
Still have 1 requests outstanding when connection from slaveX/X.X.X.X:33202 is closed
2022-07-23 21:41:00 【Gegwu MMQ!!】
ERROR TransportResponseHandler:
Still have 1 requests outstanding when connection from is closed
I added the following parameters
spark.driver.memory=4g
spark.hadoop.mapreduce.input.fileinputformat.split.maxsize=134217728
spark.dynamicAllocation.initialExecutors=50
spark.yarn.executor.memoryOverhead=2048
spark.dynamicAllocation.maxExecutors=1000
spark.dynamicAllocation.tasksPerExecutorSlot=1
The running program is actually logically simple , Just from hive The amount of data read in the table is very large , almost 60+G,
And you need to put some hive Table read to dirver Node , Used to get each executor Mapping values of some data on ,
therefore driver The set resources are large . Exception information thrown at runtime , Check the reason from the Internet. It's probably the server
The number of concurrent connections exceeds its capacity , The server will connect some of these Down fall , That is to say, it is running spark The program , Too many application resources are executed concurrently .
java.util.concurrent.RejectedExecutionException: Task [email protected] rejected from java.util.concurrent.ThreadPoolExecutor
spark Log as shown above . There are other logs ,
18/06/11 18:40:43 ERROR TransportResponseHandler:
边栏推荐
- It's good to change jobs for a while, and it's good to change jobs all the time?
- googletest
- Summary of database stress testing methods
- Why cluster chat server introduces load balancer
- Qt桌面白板工具其一(解决曲线不平滑的问题——贝塞尔曲线)
- Scala programming (elementary)
- 集群聊天服务器:数据库表的设计
- Leaderboard design in game server
- 2022-7-23 12点 程序爱生活 小时线顶背离出现,保持下跌趋势,等待反弹信号出现。
- What is Kai Niu? Excuse me, is it safe to open a stock account by mobile phone?
猜你喜欢

集群聊天服务器:如何解决跨服务器通信问题 | redis发布-订阅

Cluster chat server: cluster and distributed theory

Qt桌面白板工具其一(解决曲线不平滑的问题——贝塞尔曲线)

High numbers | calculation of double integral 4 | high numbers | handwritten notes

手机测试相关基础知识

Broadcast (broadcast)

05_ UE4 advanced_ Material UV scaling
![[attack and defense world web] difficulty four-star 12 point advanced question: confusion1](/img/ee/3ce5cc3004f9113bbf70e0c52ff4d9.png)
[attack and defense world web] difficulty four-star 12 point advanced question: confusion1

TCP half connection queue and full connection queue (the most complete in History)

Basic principle of synchronized lock
随机推荐
集群聊天服务器为什么引入负载均衡器
High numbers | calculation of double integral 3 | high numbers | handwritten notes
uniapp使用canvas写环形进度条
(note) learning rate setting of optimizer Adam
Proof of green Tao theorem (1): preparation, notation and Gowers norm
Scala Programming (Junior)
scala编程(初级)
Be a professional software craftsman
Why cluster chat server introduces load balancer
Postgraduate entrance examination | advanced mathematics Chapter4 indefinite integral
Chapter 2 回归
Openlayers instances advanced mapbox vector tiles advanced mapbox vector maps
寻找消失的类名
googletest
05_ UE4 advanced_ Material UV scaling
Yushu A1 robot dog gesture control
2022.7.22 js对象
Day109.尚医通:集成Nacos、医院列表、下拉列表查询、医院上线功能、医院详情查询
One of QT desktop whiteboard tools (to solve the problem of unsmooth curve -- Bezier curve)
First acquaintance with JS (programming suitable for beginners)