Note di Matteo

Vercel ha annunciato di aver implementato il request collapsing per impedire valanghe di richieste contemporanee in caso di cache miss.

Raramente si conoscono i dettagli implementativi di una CDN, quindi è interessante leggere l'approccio basato su lock distribuiti:

Request collapsing works through a two-level distributed locking system.

At the node level, each CDN node maintains an in-memory lock. When multiple requests for the same uncached path arrive at that node, the lock ensures that only one proceeds. Others wait until the cache fills. This prevents a single node from firing multiple function invocations for the same path.

At the regional level, every region enforces its own lock across all nodes. After acquiring the node lock, a request tries to get the regional lock. Only one request per region can hold both locks simultaneously, meaning only one function invocation per region can regenerate a page.

This design makes collapsing scalable. Without the node-level grouping, hundreds of concurrent requests could all compete for the regional lock simultaneously. This would create a thundering herd problem where the lock coordination itself becomes a bottleneck.

TIL il firewall di Cloudflare era inizialmente fatto con nginx + PHP (!):

[...] the first version of FL was implemented based on the NGINX webserver, with product logic implemented in PHP. After 3 years, the system became too complex to manage effectively, and too slow to respond, and an almost complete rewrite of the running system was performed.

E poi fino al 2024 nginx (openresty) con scripting Lua:

From this point on, FL was implemented using NGINX, the OpenResty framework, and LuaJIT. While this was great for a long time, over the last few years it started to show its age. We had to spend increasing amounts of time fixing or working around obscure bugs in LuaJIT. The highly dynamic and unstructured nature of our Lua code, which was a blessing when first trying to implement logic quickly, became a source of errors and delay when trying to integrate large amounts of complex product logic.

Da quest'anno è in rollout la nuova versione scritta in Rust.