Quotas

Quotas allow you to limit resource usage over a period of time or track the use of resources.
Quotas are set up in the user config, which is usually ‘users.xml’.

The system also has a feature for limiting the complexity of a single query. See the section Restrictions on query complexity.

In contrast to query complexity restrictions, quotas:

  • Place restrictions on a set of queries that can be run over a period of time, instead of limiting a single query.
  • Account for resources spent on all remote servers for distributed query processing.

Let’s look at the section of the ‘users.xml’ file that defines quotas.

  1. <!-- Quotas -->
  2. <quotas>
  3.     <!-- Quota name. -->
  4.     <default>
  5.         <!-- Restrictions for a time period. You can set many intervals with different restrictions. -->
  6.         <interval>
  7.             <!-- Length of the interval. -->
  8.             <duration>3600</duration>
  10.             <!-- Unlimited. Just collect data for the specified time interval. -->
  11.             <queries>0</queries>
  12.             <errors>0</errors>
  13.             <result_rows>0</result_rows>
  14.             <read_rows>0</read_rows>
  15.             <execution_time>0</execution_time>
  16.         </interval>
  17.     </default>

By default, the quota tracks resource consumption for each hour, without limiting usage.
The resource consumption calculated for each interval is output to the server log after each request.

  1. <statbox>
  2.     <!-- Restrictions for a time period. You can set many intervals with different restrictions. -->
  3.     <interval>
  4.         <!-- Length of the interval. -->
  5.         <duration>3600</duration>
  7.         <queries>1000</queries>
  8.         <errors>100</errors>
  9.         <result_rows>1000000000</result_rows>
  10.         <read_rows>100000000000</read_rows>
  11.         <execution_time>900</execution_time>
  12.     </interval>
  14.     <interval>
  15.         <duration>86400</duration>
  17.         <queries>10000</queries>
  18.         <errors>1000</errors>
  19.         <result_rows>5000000000</result_rows>
  20.         <read_rows>500000000000</read_rows>
  21.         <execution_time>7200</execution_time>
  22.     </interval>
  23. </statbox>

For the ‘statbox’ quota, restrictions are set for every hour and for every 24 hours (86,400 seconds). The time interval is counted, starting from an implementation-defined fixed moment in time. In other words, the 24-hour interval doesn’t necessarily begin at midnight.

When the interval ends, all collected values are cleared. For the next hour, the quota calculation starts over.

Here are the amounts that can be restricted:

queries – The total number of requests.

errors – The number of queries that threw an exception.

result_rows – The total number of rows given as a result.

read_rows – The total number of source rows read from tables for running the query on all remote servers.

execution_time – The total query execution time, in seconds (wall time).

If the limit is exceeded for at least one time interval, an exception is thrown with a text about which restriction was exceeded, for which interval, and when the new interval begins (when queries can be sent again).

Quotas can use the “quota key” feature to report on resources for multiple keys independently. Here is an example of this:

  1. <!-- For the global reports designer. -->
  2. <web_global>
  3.     <!-- keyed – The quota_key "key" is passed in the query parameter,
  4.             and the quota is tracked separately for each key value.
  5.         For example, you can pass a Yandex.Metrica username as the key,
  6.             so the quota will be counted separately for each username.
  7.         Using keys makes sense only if quota_key is transmitted by the program, not by a user.
  9.         You can also write <keyed_by_ip />, so the IP address is used as the quota key.
  10.         (But keep in mind that users can change the IPv6 address fairly easily.)
  11.     -->
  12.     <keyed />

The quota is assigned to users in the ‘users’ section of the config. See the section “Access rights”.

For distributed query processing, the accumulated amounts are stored on the requestor server. So if the user goes to another server, the quota there will “start over”.

When the server is restarted, quotas are reset.

Original article