Tools: Cloudflare Workers performance: an experiment with Astro and worldwide latencies

Why use Cloudflare Workers? ## When not to use it ## What will we build? ## Benchmarking latencies worldwide ## Static assets ## Stateless function ## Hot KV read ## Cold KV read ## KV writes ## SSR Page with KV cold reads ## Observations ## A word of caution ## Single-Page-Applications vs Server-Side-Rendering ## Sources Cloudflare Workers let you host pages and run code without managing servers. Unlike traditional servers placed in a single or a few locations, the deployed static assets and code are mirrored around the globe in the data centers shown as blue dots below. Naturally, this offers better latencies, scalability and robustness. Their developer platform also extends beyond “Workers” (the compute part) and include storage, databases, queues, AI and lots of other developer tooling. The whole with a generous free tier and reasonable pricing beyond that. Why am I writing this? I find it fairly good, had a good experience with it, and that’s why I will present it here. This article is not sponsored in any way. I just think it’s somehow a responsibility of developers to communicate about the tools they use in order to keep their ecosystem lively. I’ve seen too much good stuff getting abandoned because there was no “buzz”. The benefits of using Cloudflare Workers is: Great latencies worldwide Unlimited scalability No servers to take care of Further tooling for data, files, AI, etc. GitHub pull requests preview URLs Free tier good enough for most hobby projects Like every tool, it has use cases for which it shines and others it is not suited for. This is important to grasp and understanding the underlying technology helps tremendously. Basically, in loads your whole app bundled as a script and evaluates it on the fly. It’s fast and works wonderfully if your API and used frameworks are slim and minimalistic. However, it would be ill-advised in following use cases: Large complex apps The cost of evaluating your API / SSR script will grow as your app grows. The larger it becomes, the more inefficient its invocation as a whole will become. There are also some limits how large your “script” can be. Although it has been raised multiple times in the past, the fact that this is extremely inefficient will always remain. Thus, be careful when picking dependencies/frameworks since they can quickly bloat your codebase. Heavy resource consumption Due to its nature, it is not suited to compute stuff requiring large amounts of CPU/RAM/time like statistic models or scientific computation. Large caches are problematic too. Waiting for long-running async server-side requests is OK though, the execution is suspended in-between and do not count towards execution time. Long-lived connections That’s also problematic. You should rather use polling than keeping connections open. In other words: “The slimmer, the better!” It’s kind of difficult to say what’s small enough and when it becomes too large. This is rather suited for small self-contained microservices of modest size. Even debugging using breakpoint might turn out challenging. For such larger applications, traditional server deployments would be more suited. A “Quote of the Day” Web application. The purpose is not to build something big, but rather a simple proof-of-concept. The quotes will be stored in a KV store and fetched Client-side. That way, we can measure how fast the whole works and if it lives up to the expectations. The default version of https://quoted.day is available in two flavours: https://quoted.day/spa: a static page, fetching the quote text/author asynchronously https://quoted.day/ssr: Server-Side-Rendering, rendering the page with the quote on the server I swapped which one is the default from time to time to perform experiments. Performance (latency) may vary depending where you are located and whether what you fetch is “hot” or “cold”. Before we delve into details on how to build such an app, let’s take a look at the performance we can expect. Unlike the internal Cloudflare latency measures, measured “inside” the worker and therefore quite optimistic, we will look at the “real” external latency thanks to the great tool https://www.openstatus.dev/play/checker . Thanks to that, we can obtain a pretty good idea of the overall latencies that can be observed all over the world. Note however that Australia, Asia and Africa may have rather erratic latencies that “jump” sometimes. We will also benchmark multiple things separately: Also, every case will get “two passes”, to hopefully fill caches on the way, and only record the second one. This was obtained by fetch the main page at https://quoted.day/spa his is obtained by fetching the endpoint https://quoted.day/api/time which simply returns the current time. This is obtained by fetching a fixed quote from the KV store using the endpoint https://quoted.day/api/quote/123 This is obtained by fetching a random quote from the KV store using the endpoint https://quoted.day/api/quote Note that each call will cache the result for a day at the edge location, resulting in possibly turning cold reads into hot reads as traffic increases. This is obtained by fetching quoted.day/api/bump-counter which creates a temporary KV pair with an expiration time of 10 minutes. It kind of emulates the concept of initiating a “session”. Lastly, in this test, we combine the reading a random quote (that usually results in a cold KV read) and renders it server-side in a page. In is interesting to see how you can infer how the KV works just by watching the numbers. It appears the KV store is not actively replicated, but rather KV pairs are copied “on-demand” at remote locations. When cached (by default 1 minute), subsequent reads are fast. The latencies of such “hot” KV pairs are pretty good overall. No complains here. How long the pair remains cached there can also be configured using the cacheTtl parameter during the KV get request. However, the downside of increasing that value is that this cached copy do not reflect changes / updates triggered from other locations during that time. Unsurprisingly, cold reads have worse latencies. The other thing you can infer from the numbers is that there seem to be an “origin location”, and cold reads latencies increase proportionally according to the distance to this location. Therefore, pay attention “where” you create the KV store, as it impacts all future latencies around the globe. Note that workers KV might change in the future, this is merely an observation of its state right now. While read operations are OK, the write operations are rather disappointing right now. I expected it to have great latencies too, writing to the “edge” and letting the propagation take place asynchronously, but it is the opposite. Writes appear to communicates with the “origin” storage. The time it takes to set a value gets higher the further away you are from where you created the KV store. This is kind of bad news, because setting/updating values is a pretty common operation, for example to authenticate users. Dear Cloudflare team, I hope you improve that part in the future. If you develop your webapp, publish it and take a look at it, you will probably not even notice the bad latencies. You will face the optimal latencies with the origin KV store being near you. However, someone at the other end of the planet will have an uglier experience. If that person has a handful of cache misses or writes, the response time might quickly climb into a few seconds before the response arrives. That is not how I would expect a “distributed” KV store to behave. Let us be clear, right now this behaves more like a centralized KV store with on-demand cached copies at the edge. Quite ironically, it basically feels more like a traditional single-location database right now (+caches). While latencies of a single cache miss or a single write is not dramatic, it can quickly pile up with multiple calls and especially write-heavy webapps risk facing increased “sluggishness” depending on their location. Here as well, being “minimalistic” regarding KV calls should be taken to heart during the conception of the webapp using workers. Lastly, there was one more setting available in the Worker: “Default Placement” vs “Smart Placement”. I tried both but I did not see noticeable changes within the latencies. I think it’s due to the fact that there is a single KV store call and that it takes time and traffic to gather telemetry and adjust the placement of workers. It might be great, but for this experiment, it had no effect at all. Here as well, one is not universally better or worse than the other and the answer which one to use is “it depends”. Besides strong differences regarding frameworks and overall architecture, it also has practical fundamental differences for the end user. It’s also fascinating to see history repeating itself, where the internet first started with server rendered pages, than single-page-application with data fetching took over and a resurgence of SSR, just like in the past, just with new tech stacks. SSR is actually the easiest one to explain: you fetch all the required data server side, put everything in a template and return the resulting page to the end user. It takes a bit of time and processing power server-side, is not cachable, but the client gets a “finished” page. The SPA does the opposite. Although the HTML/CSS/JS is static and cached (hence quickly fetched), the resources are typically much larger due to all the client-side javascript libs needed. Then starts the heavy lifting, where data is fetched and the page rendered, typically while showing a loading spinner. As a result, the total time to render the page is longer. However, interacting with the SPA is typically smoother afterwards, because interactions just exchange data with the server and make local changes to the page. In contrast, SSR means navigating and loading a new page. Hence, the choice whether SPA or SSR is more suited depends on how “interactive” the page/app should be. As a rule of thumb, if it’s more like a static “web page”, go for SSR, if it’s more like an interactive “web app”, go for SPA. Lastly, the nice thing about Astro, picked here as illustrative example, is that the whole spectrum is possible: static pages, SPA and SSR. The source code of this experiment is here: https://github.com/dagnelies/quoted-day If you have a Github and a Cloudflare Account, you can also fork & deploy by clicking here: If the button doesn’t work, here it is as link instead: https://deploy.workers.cloudflare.com/?url=https://github.com/dagnelies/quoted-day It will fork the GitHub repository and deploy it on an internal URL so that you can preview it. Afterwards, you can edit the code and it will auto-deploy it, etc. Note that the example references a KV store that is mine. So you will have to create your own KV store named and swap the QUOTES KV id in the wrangler.json file with yours. You will also have to initially fill it with quotes if you want to reproduce the example. Luckily, there are scripts in the package.json to do just that. Everything beyond this point would deserve a tutorial on its own. This was merely the result of an experiment, how the latencies hold up and some insights on the platform. Enjoy! Templates let you quickly answer FAQs or store snippets for re-use. Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse - Great latencies worldwide - Unlimited scalability - No servers to take care of - Further tooling for data, files, AI, etc. - GitHub pull requests preview URLs - Free tier good enough for most hobby projects - Large complex apps The cost of evaluating your API / SSR script will grow as your app grows. The larger it becomes, the more inefficient its invocation as a whole will become. There are also some limits how large your “script” can be. Although it has been raised multiple times in the past, the fact that this is extremely inefficient will always remain. Thus, be careful when picking dependencies/frameworks since they can quickly bloat your codebase. - Heavy resource consumption Due to its nature, it is not suited to compute stuff requiring large amounts of CPU/RAM/time like statistic models or scientific computation. Large caches are problematic too. Waiting for long-running async server-side requests is OK though, the execution is suspended in-between and do not count towards execution time. - Long-lived connections That’s also problematic. You should rather use polling than keeping connections open. - https://quoted.day/spa: a static page, fetching the quote text/author asynchronously - https://quoted.day/ssr: Server-Side-Rendering, rendering the page with the quote on the server - Static assets - Stateless functions - Hot KV read - Cold KV read