Limit Parameter Adjustment

This article describes how to adjust parameters when resources such as memory and CPU are insufficient.

An error is reported during task execution: error code 12001, the error message queue has insufficient CPU resources, you can adjust the number of Spark executors

Click on the management console—parameter configuration—IDE—Spark—display advanced settings—woker engine resource settings (2)—adjust the concurrent number of executors

Or set the global setting to adjust the upper limit of queue resource usage.

An error is reported during task execution: error code 12003, the number of error message queue instances exceeds the limit

Click Management Console—Parameter Configuration—Global Settings—Queue Resources—Maximum Number of Yarn Queue Instances [wds.linkis.rm.yarnqueue.instance.max]

An error is reported during task execution: error code 12004, error message upper limit of global driver memory usage, lower driver memory can be set

Click Management Console—Parameter Configuration—Global Settings—Queue Resources—Maximum Number of Yarn Queue Memory 队列内存使用上限[wds.linkis.rm.yarnqueue.memory.max]

An error is reported during task execution: error code 12005, the error message exceeds the upper limit of the number of CPUs in the global drive, and idle engines can be cleaned up

Click Management Console - Parameter Configuration - Global Settings - Queue Resources - Upper Limit of The Number of Engine Cores[wds.linkis.rm.client.core.max]

Or solve it by cleaning idle engines.

An error is reported during task execution: error code 12006, the error message exceeds the maximum concurrency limit of the engine, and idle engines can be cleaned up

Click Management Console—Parameter Configuration—Global Settings—Queue Resources—Maximum concurrent number of each engine globally [wds.linkis.rm.instance]

Spark engine

Hive engine

Other engine settings are similar to Spark and Hive engines.