Reporting troubleshooting

Having trouble? Here are solutions to common problems you might encounter while using Reporting.

System dependencies

Reporting launches a “headless” web browser called Chromium on the Kibana server. It is a custom build made by Elastic of an open source project, and it is intended to have minimal dependencies on OS libraries. However, the Kibana server OS might still require additional dependencies to run the Chromium executable.

Make sure Kibana server OS has the appropriate packages installed for the distribution.

If you are using CentOS/RHEL systems, install the following packages:

  • ipa-gothic-fonts
  • xorg-x11-fonts-100dpi
  • xorg-x11-fonts-75dpi
  • xorg-x11-utils
  • xorg-x11-fonts-cyrillic
  • xorg-x11-fonts-Type1
  • xorg-x11-fonts-misc
  • fontconfig
  • freetype

If you are using Ubuntu/Debian systems, install the following packages:

  • fonts-liberation
  • libfontconfig1

If the system is missing dependencies, then Reporting will fail in a non-deterministic way. Kibana runs a self-test at server startup, and if it encounters errors, logs them in the Console. Unfortunately, the error message does not include information about why Chromium failed to run. The most common error message is Error: connect ECONNREFUSED, which indicates that Kibana could not connect to the Chromium process.

To troubleshoot the problem, start the Kibana server with environment variables that tell Chromium to print verbose logs. See the Puppeteer debug method for more information.

Text rendered incorrectly in generated reports

If a report label is rendered as an empty rectangle, no system fonts are available. Install at least one font package on the system.

If the report is missing certain Chinese, Japanese or Korean characters, ensure that a system font with those characters is installed.

Missing data in PDF report of data table visualization

There is currently a known limitation with the Data Table visualization that only the first page of data rows, which are the only data visible on the screen, are shown in PDF reports.

File permissions

Ensure that the headless_shell binary located in your Kibana data directory is owned by the user who is running Kibana, that the user has the execute permission, and if applicable, that the filesystem is mounted with the exec option.

The Chromium binary is located in the Kibana installation directory as data/headless_shell-OS_TYPE/headless_shell. The full path is logged the first time Kibana starts when verbose logging is enabled.

Error messages

Whenever possible, a Reporting error message tries to be as self-explanatory as possible. Here are some error messages you might encounter, along with the solution.

Max attempts reached

There are two primary causes of this error:

  • You’re creating a PDF of a visualization or dashboard that spans a large amount of data and Kibana is hitting the xpack.reporting.queue.timeout
  • Kibana is hosted behind a reverse-proxy, and the Kibana server settings are not configured correctly

Create a Markdown visualization and then create a PDF report. If this succeeds, increase the xpack.reporting.queue.timeout setting. If the PDF report fails with “Max attempts reached,” check your Kibana server settings.

You must install nss for Reporting to work

Reporting using the Chromium browser relies on the Network Security Service libraries (NSS). Install the appropriate nss package for your distribution.

Unable to use Chromium sandbox

Chromium uses sandboxing techniques that are built on top of operating system primitives. The Linux sandbox depends on user namespaces, which were introduced with the 3.8 Linux kernel. However, many distributions don’t have user namespaces enabled by default, or they require the CAP_SYS_ADMIN capability.

Elastic recommends that you research the feasibility of enabling unprivileged user namespaces before disabling the sandbox. An exception is if you are running Kibana in Docker because the container runs in a user namespace with the built-in seccomp/bpf filters.

Verbose logs

Kibana server logs have a lot of useful information for troubleshooting and understanding how things work. If you’re having any issues at all, the full logs from Reporting will be the first place to look. In kibana.yml:

  1. logging.verbose: true

For more information about logging, see Kibana configuration settings.

Puppeteer debug logs

The Chromium browser that Kibana launches on the server is driven by a NodeJS library for Chromium called Puppeteer. The Puppeteer library has its own command-line method to generate its own debug logs, which can sometimes be helpful, particularly to figure out if a problem is caused by Kibana or Chromium. See more at debugging tips.

Using Puppeteer’s debug method when launching Kibana would look like:

  1. env DEBUG="puppeteer:*" ./bin/kibana

The internal DevTools protocol traffic will be logged via the debug module under the puppeteer namespace.

The Puppeteer logs are very verbose and could possibly contain sensitive information. Handle the generated output with care.

System requirements

In Elastic Cloud, the Kibana instances that most configurations provide by default is for 1GB of RAM for the instance. That is enough for Kibana Reporting when the visualization or dashboard is relatively simple, such as a single pie chart or a dashboard with a few visualizations. However, certain visualization types incur more load than others. For example, a TSVB panel has a lot of network requests to render.

If the Kibana instance doesn’t have enough memory to run the report, the report fails with an error such as Error: Page crashed! In this case, try increasing the memory for the Kibana instance to 2GB.

Most Popular