Troubleshooting

IllegalConfigurationException

If you see an IllegalConfigurationException thrown from TaskExecutorProcessUtils, it usually indicatesthat there is either an invalid configuration value (e.g. negative memory size, fraction that is greater than 1, etc.)or configuration conflicts. Check the documentation chapters related to the memory componentsmentioned in the exception message.

OutOfMemoryError: Java heap space

The exception usually indicates that the JVM heap is too small. You can try to increase the JVM heap sizeby increasing total memory or task heap memory.

Note You can also increase the framework heap memory but this optionis advanced and should only be changed if you are sure that the Flink framework itself needs more memory.

OutOfMemoryError: Direct buffer memory

The exception usually indicates that the JVM direct memory limit is too small or that there is a direct memory leak.Check whether user code or other external dependencies use the JVM direct memory and that it is properly accounted for.You can try to increase its limit by adjusting direct off-heap memory.See also how to configure off-heap memory andthe JVM arguments which Flink sets.

OutOfMemoryError: Metaspace

The exception usually indicates that JVM metaspace limit is configured too small.You can try to increase the JVM metaspace option.

IOException: Insufficient number of network buffers

The exception usually indicates that the size of the configured network memoryis not big enough. You can try to increase the network memory by adjusting the following options:

Container Memory Exceeded

If a task executor container tries to allocate memory beyond its requested size (Yarn, Mesos or Kubernetes),this usually indicates that Flink has not reserved enough native memory. You can observe this either by using an externalmonitoring system or from the error messages when a container gets killed by the deployment environment.

If RocksDBStateBackend is used and the memory controlling is disabled,you can try to increase the managed memory.

Alternatively, you can increase the JVM overhead.See also how to configure memory for containers.