ua-restriction

Description

The ua-restriction Plugin allows you to restrict access to a Route or Service based on the User-Agent header with an allowlist and a denylist.

A common scenario is to set crawler rules. User-Agent is the identity of the client when sending requests to the server, and the user can whitelist or blacklist some crawler request headers in the ua-restriction Plugin.

Attributes

NameTypeRequiredDefaultValid valuesDescription
bypass_missingbooleanFalsefalseWhen set to true, bypasses the check when the User-Agent header is missing.
allowlistarray[string]FalseList of allowed User-Agent headers.
denylistarray[string]FalseList of denied User-Agent headers.
messagestringFalse“Not allowed”[1, 1024]Message with the reason for denial to be added to the response.
ua-restriction - 图1note

Both allowlist and denylist can be used on their own. If they are used together, the allowlist matches before the denylist.

Enabling the Plugin

You can enable the Plugin on a Route or a Service as shown below:

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
  2. {
  3.     "uri": "/index.html",
  4.     "upstream": {
  5.         "type": "roundrobin",
  6.         "nodes": {
  7.             "127.0.0.1:1980": 1
  8.         }
  9.     },
  10.     "plugins": {
  11.         "ua-restriction": {
  12.              "bypass_missing": true,
  13.              "allowlist": [
  14.                  "my-bot1",
  15.                  "(Baiduspider)/(\\d+)\\.(\\d+)"
  16.              ],
  17.              "denylist": [
  18.                  "my-bot2",
  19.                  "(Twitterspider)/(\\d+)\\.(\\d+)"
  20.              ]
  21.         }
  22.     }
  23. }'

You can also configure the Plugin to respond with a custom rejection message:

  1. "plugins": {
  2.     "ua-restriction": {
  3.         "denylist": [
  4.             "my-bot2",
  5.             "(Twitterspider)/(\\d+)\\.(\\d+)"
  6.         ],
  7.         "message": "Do you want to do something bad?"
  8.     }
  9. }

Example usage

After you have configured the Plugin as shown above, you can make a normal request which will get accepted:

  1. curl http://127.0.0.1:9080/index.html -i
  1. HTTP/1.1 200 OK
  2. ...

Now if the User-Agent header is in the denylist i.e the bot User-Agent:

  1. curl http://127.0.0.1:9080/index.html --header 'User-Agent: Twitterspider/2.0'
  1. HTTP/1.1 403 Forbidden
  2. ...
  3. {"message":"Not allowed"}

Disable Plugin

To disable the ua-restriction Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect.

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
  2. {
  3.     "uri": "/index.html",
  4.     "plugins": {},
  5.     "upstream": {
  6.         "type": "roundrobin",
  7.         "nodes": {
  8.             "127.0.0.1:1980": 1
  9.         }
  10.     }
  11. }'