Rate-limit and mitigate WebSockets DDoS attacks with Cloudflare API
Published on June 8, 2024   •   6 min read

Rate-limit and mitigate WebSockets DDoS attacks with Cloudflare API

Jarek CeborskiJarek Ceborski

In this post, we will dive into protecting your WebSocket server from attacks. I will share my learnings from mitigating attacks on Webhook.cool, a free online webhook tester that is #3 on Google with lots of traffic and is open to use for everyone without a sign-up.

WebSockets allow for real-time communication between a server and clients, improving the usability and UX of many web applications. However, they can be a vector for attacks that are not easy to spot. Imagine this: server logs look clean, request numbers are normal, but CPU usage is through the roof. What’s happening? You could be the target of a DDoS attack focused on WebSockets!

Understanding WebSocket Attacks

Firewalls and DDoS protections are usually meant for typical HTTP web traffic and may not be good at stopping WebSocket threats. One common WebSocket attack happens when an attacker connects to your server and sends hundreds of thousands of messages per second, often just "ping" messages. These messages do not even appear in server logs, however they can flood the server to the point of making it unresponsive!

Mitigation Strategy

The approach we are going to take involves tracking the number of WebSocket messages sent by each connected client per second. If a client exceeds a limit, we will destroy the connection and use the Cloudflare API to dynamically block the client's IP address by adding it to a list of blocked IPs in the firewall (WAF). To make it work you must first transfer your domain to Cloudflare. Alternatively, you can block these IPs in your app and skip Cloudflare, but it has a downside of still consuming too many resources. If the attacker tries to reconnect hundreds of thousands of times per second, it can still bring your server down.

Tech Stack

Our implementation is based on the following technologies:

  • Cloudflare API. While it doesn’t have WebSocket-specific features, we can leverage its firewall via API to dynamically block IP addresses as soon as we discover malicious behaviour from them.
  • Fastify.js and @fastify/websocket: Fastify is a fast and low-overhead server framework, and @fastify/websocket adds robust WebSocket support. The code example is generic enough to be used in any server framework.
  • node-cache: This simple in-memory caching module will serve as our rate limiter, tracking the number of requests per IP. While node-cache is effective for our needs, you could easily switch to Redis if you need persistent storage, as it offers a similar API.

Setting up Cloudflare Firewall (WAF)

We will use Cloudflare Lists API and specifically the Update a list endpoint to manage the list of blocked IP addresses. As a prerequisite you will first need to go to Cloudflare dashboard and create this list manually:

  1. Go to Manage accountConfigurationsLists
  2. Create new list with Type of IP
  3. Get your Cloudflare Account ID and this list ID to use later in code, click Edit next to the list. Both IDs is are in the URL address: https://dash.cloudflare.com/<account-id>/configurations/lists/<list-id>

Then, create a WAF rule to block connections from that list:

  1. Select your website on the dashboard, then:
  2. Go to SecurityWAF, then Add rule
  3. Set the new rule to If incoming requests match + IP source address + operator: is on list and pick your list from the dropdown
  4. Click Deploy

Implementation

Now the fun part – let's dive into the code! We will create messagesCounter and blocklistedIPs caches with their respective TTLs and attach event handlers to update the list on Cloudflare whenever an IP address is added or removed (expired via TTL).

TypeScript
import NodeCache from 'node-cache'

// Count messages per IP, with TTL of 1 seconds
const messagesCounter = new NodeCache({ stdTTL: 1})

// Blocklist with a TTL of 24 hours
const blocklistedIPs = new NodeCache({ stdTTL: 24 * 60 * 60 })

// On 'set' and 'expire' events update the list on Cloudflare
blocklistedIPs.on('set', () => updateBlockListOnCloudflare())
blocklistedIPs.on('expired', () => updateBlockListOnCloudflare())

Then we create wsHandler a Fastify request handler for incoming WebSocket requests and add rate limiting to it. After 100 messages in 1 second or less, we will block the IP address.

wsHandler.ts
TypeScript / wsHandler.ts
import type { FastifyRequest } from 'fastify'
import type { SocketStream } from '@fastify/websocket'

// Handler for /ws route
const wsHandler = async (connection: SocketStream, request: FastifyRequest) => {
	// Get IP of the incomming request passed by Cloudflare
  const requestIp = request.headers['cf-connecting-ip'] as string

  // Don't connect blocklisted IPs
  // Useful when blocking via Cloudflare list fails or not yet in effect
  if (blocklistedIPs.has(requestIp)) {
    connection.destroy()
    return
  }

  // Handler for websocket 'message' event
  connection.socket.on('message', (message) => {
    // Update messages counter
    let messages = (messagesCounter.get(requestIp) as number) || 0
    messages++
    messagesCounter.set(requestIp, messages)

    // Check if rate limit exceeded
    if (messages > 100) {
      blocklistedIPs.set(requestIp, true)
      connection.destroy()
      return
    }

    // Process WebSocket message here...
  })
}

Use request handler in Fastify:

server.ts
TypeScript / server.ts
...
fastify.get(`/ws`, { websocket: true }, wsHandler)
...

Looks clean so far, but there is one catch! According to Cloudflare, IPv6 addresses must not be larger than /64 in CIDR notation. This essentially means that we can't block a single IPv6 address, but rather an entire subnet (consequently, a single IPv6 entry on the block list can impact thousands or more actual users).

In this case need a few utilities because we will handle IPv4 and IPv6 separately.

utils.ts
TypeScript / utils.ts
const ipv4Regex = /^(25[0-5]|2[0-4]\d|1?\d{1,2})(\.(25[0-5]|2[0-4]\d|1?\d{1,2})){3}$/
const ipv6Regex =
  /^(([0-9a-fA-F]{1,4}:){7}([0-9a-fA-F]{1,4}|:)|(([0-9a-fA-F]{1,4}:){1,6}:)|(([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2})|(([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3})|(([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4})|(([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5})|([0-9a-fA-F]{1,4}:)((:[0-9a-fA-F]{1,4}){1,6})|(:((:[0-9a-fA-F]{1,4}){1,7}|:)))(%.+)?$/

function isValidIPv4Address(ip: string) {
  return ipv4Regex.test(ip)
}

function isValidIPv6Address(ip: string): boolean {
  return ipv6Regex.test(ip)
}

function convertIPv6ToSlash64(ipv6: string): string {
  const blocks = ipv6.split(':')
  const prefix = blocks.slice(0, 4).join(':')
  const result = `${prefix}::/64`
  return result
}

Now we need to update the rate limiter to format the IPv6 address correctly.

wsHandler.ts
TypeScript / wsHandler.ts
    if (messages > 100) {
+     if (isValidIPv4Address(requestIp)) {
        blocklistedIPs.set(requestIp, true)
+     }
+     if (isValidIPv6Address(requestIp)) {
+       // Respect Cloudflare policy: "IPv6 addresses must not be larger than /64"
+       // Convert actual IPv6 e.g. 2804:4b0:1356:4f00:9b3e:aa1d:ec4a:d57a
+       // to a format acceptable Cloudflare, so: 2804:4b0:1356:4f00::/64
+       const ipv6slash64 = convertIPv6ToSlash64(requestIp)
+       blocklistedIPs.set(ipv6slash64, true)
+     }
      connection.destroy()
      return
    }

Finally, the actual function to call the Cloudflare API and update the block list. It replaces the entire list with a new one because node-cache doesn't persist the list across server restarts. Therefore, adding and removing each individual IP address would cause a mess.

TypeScript
async function updateBlockListOnCloudflare() {
  const cloudflareAccountId = '...'
  const cloudflareApiToken = '...'
  const listId = '...'

  try {
    let url = `https://api.cloudflare.com/client/v4/accounts/${cloudflareAccountId}/rules/lists/${listId}/items`

    const body = blocklistedIPs.keys().map((ip) => ({ ip }))

    let options = {
      method: 'PUT',
      headers: {
        'Content-Type': 'application/json',
        Authorization: `Bearer ${cloudflareApiToken}`,
      },
      body: JSON.stringify(body),
    }

    fetch(url, options)
      .then((res) => res.json())
      .then((json) => console.log(json))
      .catch((err) => console.error('error:' + err))
  } catch (err) {
    console.error('Failed to push IP list to CF', err)
  }
}

Conclusion

By implementing the above strategy, you can significantly mitigate the risk of WebSocket DDoS attacks on your server. Combining Fastify, node-cache, and Cloudflare's firewall offers a robust defense mechanism against malicious WebSocket traffic.

If possible, set up monitors on your server that will send you notifications when it’s down or when CPU or memory usage is above a threshold. This will allow you to react quickly in case of attacks.

Check out LocalCan for testing WebSockets locally or by using Public URL for your local environment – both methods allow for WSS (Secure WebSockets). This not only saves time but also reduces potential errors when running WebSockets on production!