system.trace_log

Contains stack traces collected by the sampling query profiler.

ClickHouse creates this table when the trace_log server configuration section is set. Also the query_profiler_real_time_period_ns and query_profiler_cpu_time_period_ns settings should be set.

To analyze logs, use the addressToLine, addressToSymbol and demangle introspection functions.

Columns:

  • event_date (Date) — Date of sampling moment.

  • event_time (DateTime) — Timestamp of the sampling moment.

  • event_time_microseconds (DateTime64) — Timestamp of the sampling moment with microseconds precision.

  • timestamp_ns (UInt64) — Timestamp of the sampling moment in nanoseconds.

  • revision (UInt32) — ClickHouse server build revision.

    When connecting to the server by clickhouse-client, you see the string similar to Connected to ClickHouse server version 19.18.1 revision 54429.. This field contains the revision, but not the version of a server.

  • trace_type (Enum8) — Trace type:

    • Real represents collecting stack traces by wall-clock time.
    • CPU represents collecting stack traces by CPU time.
    • Memory represents collecting allocations and deallocations when memory allocation exceeds the subsequent watermark.
    • MemorySample represents collecting random allocations and deallocations.

  • thread_number (UInt32) — Thread identifier.

  • query_id (String) — Query identifier that can be used to get details about a query that was running from the query_log system table.

  • trace (Array(UInt64)) — Stack trace at the moment of sampling. Each element is a virtual memory address inside ClickHouse server process.

Example

  1. SELECT * FROM system.trace_log LIMIT 1 \G
  1. Row 1:
  2. ──────
  3. event_date:              2020-09-10
  4. event_time:              2020-09-10 11:23:09
  5. event_time_microseconds: 2020-09-10 11:23:09.872924
  6. timestamp_ns:            1599762189872924510
  7. revision:                54440
  8. trace_type:              Memory
  9. thread_id:               564963
  10. query_id:
  11. trace:                   [371912858,371912789,371798468,371799717,371801313,371790250,624462773,566365041,566440261,566445834,566460071,566459914,566459842,566459580,566459469,566459389,566459341,566455774,371993941,371988245,372158848,372187428,372187309,372187093,372185478,140222123165193,140222122205443]
  12. size:                    5244400