Of That

Brandt Redd on Education, Technology, Energy, and Trust

12 June 2022

Web Scale Authentication


Today I’m presenting to the BYU IT & Cybersecurity program on the topic of Web Scale Authentication. This post is a resource for those attending and a condensed summary for everyone else. It’s a departure from my typical focus on learning technology and goes deep into web architecture.

Click here for the slide deck.

Click here to access the source code to the authentication performance simulator.

Nine years ago, I wrote on this blog that enterprise scale is not web scale and I outlined seven principles that should be applied to web scale design. In this presentation I focus on handling authentication and state.

Despite the name, most web applications are developed to enterprise scale, anticipating tens to hundreds of thousands of users. To scale up enterprise applications you get bigger servers. But there's a limit to how big servers can get. Web scale applications have millions or billions of users. Think Google, Ancestry, Reddit, Facebook, etc. To scale up a web scale application you add more servers. Which you can keep doing for a long time. But even if you don't anticipate web-scale demand, there are important benefits to web-scale development.

Most web development frameworks have a way to store data associated with a web session. In PHP, for example, you call “session_start()” and then have access to a collection called $_SESSION. Within the session collection you can store authentication information such as the logged-in user and permissions about what they can do.

In web scale applications, you should not use session variables. In fact, should not store state on the web server at all. All state must be pushed to the browser or stored in the database. To understand why, you need to know how sessions work.

When a session is created, the server creates a record in which session values will be stored. Then it creates a cookie that corresponds with the record and sends that to the browser. With each subsequent request, the server uses the cookie to look up the session record. Each record has an associated timeout, and the session record is discarded when enough time has passed without activity.

The trouble with this how and where the session information is stored. If it’s stored in memory, and you get enough users, you can consume too much RAM and performance will drop as the virtual memory system tries to swap things to disk. If you store session information on the disk, as PHP does, then there’s an extra IO request for every web page and that can slow things way down due to the disk storage bottleneck. In fact, many requests spend more time retrieving session data than delivering content. Either way, you have to set up server affinity on your load balancer because the same user must always be directed to the same server in your pool.

If your web server has no session state, the load balancer is free to shift traffic around and to add servers as needed. It speeds up servers because they don’t have to retrieve state with every request. And it lets you perform maintenance and software upgrades without downtime because you can rotate servers out of the pool, upgrade them, and return them back to the pool without interrupting sessions.

So, if the server has no session state, where does state go? It can be stored in the URL – the path and/or query string portion usually indicate what the user wants to do. It can be stored in Javascript variables (for the duration of the page) or in browser-local storage for cross-page operations. Or it can be stored in cookies. Choice of these depends on the kind of state data being stored.

It’s cookies where web-scale applications keep authentication information. Typically, they create a record that includes the user id and an expiration time. There may be other key information such as access permissions. A keyed-hash algorithm such as HMAC is used to secure the record – preventing others from forging or manipulating it. On each request, all the server must do to authenticate the request is validate the hash. This is strictly a CPU operation, so it is orders of magnitude faster than retrieving records from disk. And it does not consume any memory or disk storage. The result is much greater scalability on the same server hardware. And that results in considerable cost savings.

For any application, the benefits of web-scale design include lower and more predictable hosting costs, more robust operations, high availability, and you're ready to scale when the need is presented.