Rate Limiting

How it works

Rate limiting is implemented on the router app’s nginx configuration. We limit the number of requests per unique IP address to 2 per second for POST requests and 50 per second for all other methods. A user exceeding the limit is returned a static error page with a 429 (Too Many Requests) status code.

Rate limiting being turned on depends on the environment variables present when supervisor compiles the templates and launches nginx using the (digitalmarketplace-router/nginx.sh bash script).

In the case of rate limiting the DM_RATE_LIMITING_ENABLED environment variable is parsed and supplied to the templates as rate_limiting_enabled here.

The www template (which is put in /etc/nginx/sites_enabled) and the main nginx.conf ( which ends up in /etc/nginx) both contain rate limiting dependent sections.

nginx.conf

In the main Nginx config we:

  • Change the return code for rate limited requests from the default 503 Service Unavailable to 429 Too Many Requests
  • Create a new custom variable to signify if the request is a POST or not $post_remote_addr
  • Create 2 zones/ buckets for requests
    • One which catches everything that has a non-blank $binary_remote_addr (everything)
    • One which catches everything that has a non-blank custom $post_remote_addr (only POST requests)

www

In www we:

  • Statically serve too_many_requests.html on port 10823
  • State that we want to apply our rate limiting buckets to incoming requests to the server on port 80 (the application)
    • Set burst allowances for these requests
  • Define a ‘location’ where every request is proxied to localhost:10823 as a GET
  • Tell Nginx that requests that trigger a 429 should be passed to this location

Burst

Nginx rate limiting works as a ‘leaky bucket algorithm’ or ‘FIFO’ pipeline.

We set the rate limit to 50 requests per second with 40 burst.

This means that:

  • the fastest a user can be served 2 requests is 0.02 (1/50) seconds apart.
  • if requests arrive quicker than that they are put in a queue.
  • if the queue size exceeds 40 (for that ip) they are served 429s

Deactivating rate limiting

You might want to turn off rate limiting for load/ stress testing or for debugging purposes. You can achieve this in 2 ways but both involve changing the DM_RATE_LIMITING_ENABLED variable to something that is not "enabled" and restaging the router.

The first, and preferred method, would be to update the variable in the appropriate manifest variable file.

https://github.com/alphagov/digitalmarketplace-aws/blob/master/vars/preview.yml#L10 https://github.com/alphagov/digitalmarketplace-aws/blob/master/vars/staging.yml#L10 https://github.com/alphagov/digitalmarketplace-aws/blob/master/vars/production.yml#L10

The second (which will be overwritten to enabled if a release is triggered on the router) is to use the Cloud Foundry Command Line Interface to update the variable and restage the router:

cf target -s preview
cf env router DM_RATE_LIMITING_ENABLED "disabled"
cf restage router