- Configuration Examples
- Online Event Log Builder
- Emulating Other HTTP Server Formats
- Hourly Rotated Squid Proxy Logs
- Summarizing Number of Requests and Total Bytes Sent Every 10 Seconds
- Dual Output to Compact Binary Logs and ASCII Pipes
- Filtering Events to ASCII Pipe for Alerting
- Configuring ASCII Pipe Buffer Size
- Summarizing Origin Responses by Hour
Configuration Examples
This section provides examples for a wide range of logging scenarios. While not exhaustive of all possibilities (Traffic Server logging is quite flexible), these entries should hopefully get most administrators headed onto the right path.
Unless otherwise noted, the example configurations here are to be applied in logging.yaml
.
Online Event Log Builder
If you need any assistance building your event log, you can try out our online log builder. This is a work in progress, so any comments, critique or suggestions are most welcome.
Emulating Other HTTP Server Formats
Netscape Common
The following figure shows a sample log entry in a Netscape Common log file.
The numbered sections correspond to the following log fields in Traffic Server:
No. | Field | Description |
---|---|---|
1 | chi | The IP address of the client’s host machine. |
2 | – | This hyphen ( |
3 | caun | The authenticated client username. A hyphen ( |
4 | cqtn | The date and time of the client request, enclosed in brackets. |
5 | cqtx | The request line, enclosed in quotes. |
6 | pssc | The proxy response status code (HTTP reply code). |
7 | pscl | The length of the Traffic Server response to the client in bytes. |
To recreate this as a log format in logging.yaml
you would define the following format object:
formats:
- name: common
format: '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl>'
Netscape Extended
The following figure shows a sample log entry in a Netscape Extended log file.
In addition to fields 1-7 from the Netscape Common log format, the Extended format adds the following fields:
No. | Field | Description |
---|---|---|
8 | sssc | The origin server response status code. |
9 | sshl | The server response transfer length; the body length in the origin server response to Traffic Server, in bytes. |
10 | cqcl | The client request transfer length; the body length in the client request to Traffic Server, in bytes. |
11 | pqcl | The proxy request transfer length; the body length in the Traffic Server request to the origin server. |
12 | cqhl | The client request header length; the header length in the client request to Traffic Server. |
13 | pshl | The proxy response header length; the header length in the Traffic Server response to the client. |
14 | pqhl | The proxy request header length; the header length in Traffic Server request to the origin server. |
15 | sshl | The server response header length; the header length in the origin server response to Traffic Server. |
16 | tts | The time Traffic Server spent processing the client request; the number of seconds between the time that the client established the connection with Traffic Server and the time that Traffic Server sent the last byte of the response back to the client. |
To recreate this as a log format in logging.yaml
you would define the following format object:
formats:
- name: extended
format: '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl> %<sssc> %<sscl> %<cqcl> %<pqcl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts>'
Netscape Extended-2
The following figure shows a sample log entry in a Netscape Extended2 log file.
In addition to fields 1-16 from the Netscape Common and Netscape Extended log formats above, the Extended-2 format adds the following fields:
No. | Field | Description |
---|---|---|
17 | phr | The proxy hierarchy route; the route Traffic Server used to retrieve the object. |
18 | cfsc | The client finish status code: |
19 | pfsc | The proxy finish status code: |
20 | crc | The cache result code; how the Traffic Server cache responded to the request: |
To recreate this as a log format in logging.yaml
you would define the following format object:
formats:
- name: extended2
format: '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl> %<sssc> %<sscl> %<cqcl> %<pqcl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts> %<phr> %<cfsc> %<pfsc> %<crc>'
Squid
The following figure shows a sample log entry in a Squid log file.
The numbered sections correspond to the following log fields in Traffic Server:
No. | Field | Description |
---|---|---|
1 | cqtq | The client request timestamp in Squid format. The time of the client request in seconds since January 1, 1970 UTC (with millisecond resolution). |
2 | ttms | The time Traffic Server spent processing the client request. The number of milliseconds between the time the client established the connection with Traffic Server and the time Traffic Server sent the last byte of the response back to the client. |
3 | chi | The IP address of the client’s host machine. |
4 | crc/pssc | The cache result code; how the cache responded to the request: |
5 | psql | The length of the Traffic Server response to the client in bytes, including headers and content. |
6 | cqhm | The client request method: |
7 | cauc | The client request canonical URL; blanks and other characters that might not be parsed by log analysis tools are replaced by escape sequences. The escape sequence is a percentage sign followed by the ASCII code number of the replaced character in hex. |
8 | caun | The username of the authenticated client. A hyphen ( |
9 | phr/shn | The proxy hierarchy route. The route Traffic Server used to retrieve the object. |
10 | psct | The proxy response content type. The object content type taken from the Traffic Server response header. |
To recreate this as a log format in logging.yaml
you would define the following format object:
formats:
- name: squid
format: '%<cqtq> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc> %<caun> %<phr>/%<shn> %<psct>'
Hourly Rotated Squid Proxy Logs
The following example demonstrates the creation of a Squid-compatible log format, which is then applied to a log object containing an hourly rotation policy.
formats:
- name: squid
format: '%<cqtq> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc> %<caun> %<phr>/%<shn> %<psct>'
logs:
- mode: ascii
format: squid
filename: squid
rolling_enabled: time
rolling_interval_sec: 3600
rolling_offset_hr: 0
Summarizing Number of Requests and Total Bytes Sent Every 10 Seconds
The following example format generates one entry every 10 seconds. Each entry contains the timestamp of the last entry of the interval, a count of the number of entries seen within that 10-second interval, and the sum of all bytes sent to clients:
formats:
- name: mysummary
format: '%<LAST(cqts)>:%<COUNT(*)>:%<SUM(psql)>'
interval: 10
Dual Output to Compact Binary Logs and ASCII Pipes
This example demonstrates logging the same event data to multiple locations, in a hypothetical scenario where we may wish to keep a compact form of our logs available for archival purposes, while performing live log analysis on a stream of the event data.
ourformat = format {
Format = '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl>'
}
log.binary {
Format = ourformat,
Filename = 'archived_events'
}
log.pipe {
Format = ourformat,
Filename = 'streaming_log'
}
Filtering Events to ASCII Pipe for Alerting
This example illustrates a situation in which our Traffic Server cache contains canary objects, which upon their access we want an external alerting system to fire off all sorts of alarms. To accomplish this, we demonstrate the use of a filter object that matches events against these particular canaries and emits log data for them to a UNIX pipe that the alerting software can constantly read from.
formats:
- name: canaryformat
format: '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl>'
filters:
- name: canaryfilter
accept: cqup MATCH "/nightmare/scenario/dont/touch"
logs:
- mode: ascii_pipe
format: canaryformat
filters:
- canaryfilter
filename: alerting_canaries
Configuring ASCII Pipe Buffer Size
This example mirrors the one above but also sets a `pipe_buffer_size`
of 1024 * 1024 for the pipe. This can be set on a per-pipe basis but is only available on Linux (later than 2.6.35). If this field is not set, the pipe buffer will default to the OS default size.
logs:
- mode: ascii_pipe
format: canaryformat
filters:
- canaryfilter
filename: alerting_canaries
pipe_buffer_size: 1048576
Summarizing Origin Responses by Hour
This example demonstrates a simple use of aggregation operators to produce an hourly event line reporting on the total number of requests made to origin servers (where we assume that any cache result code without the string HIT
in it signals origin access), as well as the average time it took to fulfill the request to clients during that hour.
logging:
formats:
- name: originrepformat
format: '%<FIRST(cqtq)> %<COUNT(*)> %<AVG(ttms)>'
interval: 3600
filters:
- name: originfilter
action: reject
condition: crc CONTAINS "HIT"
logs:
- mode: ascii
format: originrepformat
filters:
- originfilter
filename: origin_access_summary