当前位置:网站首页>Still have 1 requests outstanding when connection from slaveX/X.X.X.X:33202 is closed
Still have 1 requests outstanding when connection from slaveX/X.X.X.X:33202 is closed
2022-07-23 21:41:00 【Gegwu MMQ!!】
ERROR TransportResponseHandler:
Still have 1 requests outstanding when connection from is closed
I added the following parameters
spark.driver.memory=4g
spark.hadoop.mapreduce.input.fileinputformat.split.maxsize=134217728
spark.dynamicAllocation.initialExecutors=50
spark.yarn.executor.memoryOverhead=2048
spark.dynamicAllocation.maxExecutors=1000
spark.dynamicAllocation.tasksPerExecutorSlot=1
The running program is actually logically simple , Just from hive The amount of data read in the table is very large , almost 60+G,
And you need to put some hive Table read to dirver Node , Used to get each executor Mapping values of some data on ,
therefore driver The set resources are large . Exception information thrown at runtime , Check the reason from the Internet. It's probably the server
The number of concurrent connections exceeds its capacity , The server will connect some of these Down fall , That is to say, it is running spark The program , Too many application resources are executed concurrently .
java.util.concurrent.RejectedExecutionException: Task [email protected] rejected from java.util.concurrent.ThreadPoolExecutor
spark Log as shown above . There are other logs ,
18/06/11 18:40:43 ERROR TransportResponseHandler:
边栏推荐
- High numbers | calculation of triple integral 1 | high numbers | handwritten notes
- Basic syntax of MySQL DDL and DML and DQL
- Union and union all of Hana SQL
- Protocol buffers 的问题和滥用
- 启牛是什么?请问一下手机开户股票开户安全吗?
- Cluster chat server: Design of database table
- 给定一个以数字组成的数组,实现输出id为数字,并且从小到大排序的name
- High numbers | calculation of double integral 2 | high numbers | handwritten notes
- MySql的DDL和DML和DQL的基本语法
- uniapp使用canvas写环形进度条
猜你喜欢

欧氏聚类(API)及其单木分割

Yushu A1 robot dog gesture control

Postgraduate entrance examination | advanced mathematics Chapter4 indefinite integral

Construction and application progress of ten billion level knowledge map of meituan brain

prime_ series_ level-1

Cmake learning

-2021 sorting and sharing of the latest required papers related to comparative learning

Synchronized同步锁的基本原理

Chapter1 data cleaning

集群聊天服务器:Model数据层的框架设计和数据库代码的封装
随机推荐
博客总排名为918
Jianzhi offer II 115. reconstruction sequence: topological sorting construction problem
集群聊天服務器:數據庫錶的設計
scala編程(初級)
Broadcast (broadcast)
High numbers | calculation of double integral 4 | high numbers | handwritten notes
[isprint function determines whether characters can be output]
(note) learning rate setting of optimizer Adam
【arxiv】第一次上传论文小记
Failed to introspect class feignclientfactorybean exception troubleshooting
启牛是什么?请问一下手机开户股票开户安全吗?
The total ranking of blogs is 918
DBSCAN点云聚类
集群聊天服务器:网络模块ChatServer
TCP half connection queue and full connection queue (the most complete in History)
Synchro esp32c3 Hardware Configuration Information serial port Print Output
Construction and application progress of ten billion level knowledge map of meituan brain
Principle and implementation of hash table, unordered set and mapping
Green Tao theorem (4): energy increment method
Customer exit variable in query