Last week, guest contributor Spencer Forman argued against default caching by managed web hosts – in his view, the tedium of setting up “ignore cache” rules on pages related to e-commerce and user login outweighs the benefits of speed and server resource optimization.
As sometimes happens around here, the article lit up Twitter. Employees from several major hosting companies, including Kinsta, chimed in with their thoughts and counterpoints. (Quick disclaimer: Several other hosts have bought ads with us recently, including InMotion and Cloudways, and I recently suggested that maybe we shouldn’t all get acquired by web hosts.)
In this post, I’ll take a look at the counterpoint to Spencer’s post. I mostly agree with the “pro-caching” side of things, but I also see the challenge of having a menagerie of plugins that need different URLs or querystrings to be uncached, so I’ll do my best to look at the tradeoffs facing managed WordPress hosts.
Life sucked before caching
Many years ago, I hosted most of my client sites with hosts like Media Temple and Nexcess, on lightly-managed or unmanaged VPS servers in some cases, and on shared hosting in other cases. There was lots of Cpanel, Plesk and WHM, and it was common for sites to suddenly slow down (“because a neighbor is getting a lot of traffic”) or go offline completely due to a traffic surge. You could call or e-mail support – sometimes they’d solve the problem and sometimes you would just have to wait for the issue to mysteriously resolve itself.
WP Engine solved that by introducing awesome WordPress-specific caching and performance (and generally improving on the Plesk/Cpanel interfaces). Eventually everyone adopted a similar caching and performance philosophy, to the point that GoDaddy now sells it for a few bucks a month.
The speed and reliability improvements were dramatic. We can consistently tell clients that moving to WP Engine will speed up their sites (without any other changes to the code), which is a big deal. We’ve solved a lot of problems for a lot of clients with a simple migration, which speaks to the value of caching and other performance optimization features on managed hosting services like WP Engine and Kinsta. I think the critics of Spencer’s post are correct when they say that the juice is worth the squeeze. When you add up all the benefits and drawbacks, caching helps the vast majority of sites. The sites that have functionality that breaks as a result of overly aggressive caching are the minority.
But caching sure can be annoying!
Having multiple layers of caching is the #1 reason that our clients get confused and have trouble updating their own sites – I’d argue it is one of the major (largely unaddressed) barriers to WordPress competing with Squarespace for DIY users. When you click the Save button, your site should change!
Our typical setup has a Cloudflare cache first, then a WP Engine cache second, sometimes an additional plugin-based Autoptimize or WP Rocket cache, and then the client’s local browser cache on top of that. Elementor and Beaver Builder often add their own layers of compilation in a way that sometimes acts like an uncleared cache. This seven-layer caching shitshow causes a lot of unnecessary anxiety for clients and developers, and in an ideal world we’d be able to serve information over the internet with much less complexity.
If I could get away with never caching anything, I would! For now, it is a necessary evil and the best solution available for keeping the servers humming.
A better way to ignore cache for specific plugins
One of the messages of Spencer’s original article is that by caching nearly all pages by default, hosts are setting the developers (and users) of custom plugins up for failure.
While most hosts have built-in exceptions for popular plugins like WooCommerce and Easy Digital Downloads (which require certain page URLs and cookies to be uncached), this preferential treatment can really hurt the developers of newer or less-popular plugins.
For example, if I wanted to launch a WooCommerce competitor, one of the huge barriers would be that most of my users would not be able to use it without first contacting their web host’s support team and explaining to them exactly which URL patterns and cookies need to be uncached. This would make it much harder to get traction for my new e-commerce plugin – even if it were 100x better on the merits than anything else on the market – because the web hosts’ caching systems effectively act as a filter (gatekeeper?) that causes my new plugin to fail for a majority of my target audience.
Spencer correctly notes that this is annoying for the user. I think it is also a problem for the growth of the ecosystem in general. Spencer’s recommendation is to make caching options more transparent, including allowing you to turn off all caching. His critics correctly point out that this basically would be a button that allows people to crash their own sites (and perhaps others on the same server), bringing us back to the pre-caching 2000s.
I think an interesting middle ground would be for WordPress to allow plugin authors to declare (in a formal and structured way) the URL patterns or cookies they want to leave uncached. The major WordPress managed hosts could then pick up this structured data on the installed plugins and cache appropriately – without requiring a support request or any technical knowledge from the site owner. This also future-proofs things so that new plugins can easily enter the market in the future without having to get “whitelisted” by the hosts’ caching systems.
Eventually, you need to build your own stack
As we move up the WordPress caching hierarchy of needs, we eventually get to a point where a site is just too big for managed hosting. WP Engine’s enterprise offerings, for example, quickly become very expensive compared to a well-built Amazon Web Services stack, and we’ve had some clients (years ago) who just never were able to figure out why the server would crash on managed hosting. (It had something to do with bot traffic hitting uncached querystrings and sending us into infinite loops.) So we spent thousands of dollars to build a custom AWS stack, and it is awesome and blazing fast. That said, this is a “1% of sites” scenario, so I think managed hosting is still appropriate for the vast majority of users – and using something like a custom (but cheap) Lightsail setup is way more trouble than it’s worth. I think the “build your own stack” argument is clearly correct for very large enterprise users (i.e. people who would need to spend $600+/mo with WP Engine or Kinsta). But it is not really correct to say that anyone who is frustrated with a caching system should go get their own VPS – I think there’s clearly a large group of people who need more caching customization but also don’t have the skills or desire to run their own private server. This is why Spencer and his clients are currently banging their heads against the wall with managed hosting.
Is caching worth it?
I think Spencer makes a strong argument that more-granular control of caching would be a big benefit for a significant subset of managed hosting customers. His critics make a strong counterargument that, from a software and business standpoint, we sometimes need to just make opinionated decisions and offer fewer features instead of trying to be everything to everyone.
I personally don’t think that caching customization would persuade me to change hosts, but it would definitely save me (and the support staff at WP Engine) perhaps a dozen hours a year by putting more cache control at my fingertips rather than requiring a support ticket. And I think the structured-data approach would be a very cool, developer-friendly, ecosystem-friendly idea that allows us to permanently solve this problem in the long term.
By the way, did you know that we pay every guest contributor? If you have a cool idea or want to write a rebuttal to an existing article, learn how to get paid to write for us.
