The day I got tired of my feeds breaking

I follow a handful of subreddits and blogs through RSS instead of opening a dozen tabs every morning. It's a nice, quiet way to read things. Except every so often, one of them just... stops. My reader shows an error where a subreddit used to be, or a blog goes blank for a day, and I have no idea if the site is actually down or if it's something on my end. Either way, I lose the thing I was trying to read.

Eventually I looked into it, and it turns out this is just normal life for a website. Servers crash. Hosting has a bad night. And a lot of sites, on purpose, will slow you down or cut you off if you ask them for things too often — that's not a bug, that's them protecting themselves from getting hammered. From where I'm sitting, though, all of that looks identical: the feed just isn't there right now.

Why "just cache it" actually works

The idea that fixed this for me is really old and really simple, and I think that's why I like it. It's the same idea as writing something down. If I call a friend and they don't pick up, I still remember the last thing they told me — I don't just forget it because they didn't answer this time. A cache is that, for a website: the last good answer it gave you, written down somewhere, so a moment of silence from the source doesn't turn into silence for you too.

Don't ask the website directly every time. Ask it quietly in the background, on your own time. Whatever it last told you that made sense — keep that around, and hand it to whoever asks, whether the site is currently up, down, grumpy, or somewhere in between.

Once I thought about it that way, the shape of the thing was obvious. It needed to do two separate jobs, and — this part matters — never let one job wait on the other.

What a 429 actually does to your reader

Here's the part that made it click for me, and it's not what I expected going in. The 429 doesn't usually come from a crowd of strangers hammering the same feed — it comes from your own reader. You don't just subscribe to one subreddit; you've got a dozen. And a normal reader app doesn't refresh them one politely-spaced request at a time — it refreshes all of them together, either on a fixed timer or the moment you open the app, since that feels like the natural "give me everything fresh" moment.

So instead of one request every so often, the site sees a burst: a dozen near-simultaneous requests, all from you, all in the same second. If the site only allows so many requests from one client in a short window, that burst is exactly what trips it — and it has nothing to do with how popular the subreddit is or whether anyone else in the world is reading it right now.

What comes back is a 429 Too Many Requests for some of those feeds, and most readers don't handle that gracefully — they show a red "error" icon, quietly mark the feed as broken, or just show nothing until the next refresh happens to land better. And none of that is really about the content being unavailable. The post you wanted is sitting right there on the server, healthy the whole time — it's the shape of the burst that's the problem, not the content.

A cache fixes this almost by accident. It's the one place doing the refreshing now, and it never needs everything at once the way opening an app does — it can take its time and go one feed at a time, spread out, well within whatever the site is comfortable with. You still get everything the moment you open your reader, because you're reading whatever the cache already has sitting there, not waiting on a dozen live requests to land safely.

A real example: Reddit tightening things up

Reddit made this really obvious to me recently. They tightened up their rate limits on feed refreshes — something like one feed refresh per minute — and honestly, that's a completely fair call on their part. It's their infrastructure, and they get to decide how much polling it can take from any one visitor.

But a one-per-minute allowance and a reader app that refreshes a dozen subscribed subreddits the second you open it are just not compatible. The first one lands inside that minute and comes back fine; the rest land a beat later, blow past the same allowance, and come back as 429s. You open your reader and half your subreddits show an error — not because anything is actually wrong with them, but purely because of how many requests you personally fired off in the same second.

The background-checking approach sidesteps this cleanly: it was never going to burst all twelve requests into the same second in the first place. It spreads them across the minute (or several minutes) it actually has, one feed at a time, comfortably inside the limit. And since you're always reading whatever it already has cached rather than waiting on a live refresh, it doesn't matter to you whether a given feed's turn came up ten seconds ago or four minutes ago — you never see the 429, and you never see a gap.

Checking in on it, without being annoying

The first job is just: every now and then, go take a look at the site and see if it still says something sensible. If it does, that becomes the new "last good copy." If it doesn't — the site's down, or it's telling you to slow down — nothing bad happens. You just... don't update anything, and try again a little later.

I did have to be a bit careful here, though. If you check on a schedule that's too exact — say, exactly every ten minutes, forever — you end up looking suspiciously like a bot hammering the site at the same moment every time, which is exactly the behavior that gets you slowed down in the first place. So the checks happen a little irregularly on purpose, and if a check fails, the next attempt waits a bit longer than the one before, instead of trying again immediately and making things worse.

None of that is visible from the outside. It's just good manners, running quietly in the background.

Not every silence means the same thing

There's one distinction I care about a lot, even though it's a small one: a website not answering at all, and a website answering with nonsense, are two very different situations, and I didn't want to treat them the same.

If a site just doesn't respond — my internet hiccups, or theirs does — that's not really telling me anything about whether the feed itself is fine. It's a connection problem, not a content problem, so it gets shrugged off and tried again later, no harm done. But if the site does answer, and what comes back is clearly broken or wrong, that's worth actually noting, separately from whatever old copy is still sitting there being served. It felt important not to let a random bad moment on the internet make something perfectly fine look broken.

Answering you, without any of that drama

The second job — the part that actually talks to you — doesn't touch the website at all. Ever. It just opens up whatever the last good copy was and hands it over. It doesn't know or care whether that copy is five minutes old or five days old, and it never makes you wait around while it goes and checks. That waiting is exactly what I was trying to get rid of.

That's really the whole trick — keeping those two things from ever touching. The checking can be slow, unlucky, rate-limited, whatever. As long as it doesn't get in the way of the answering, I never notice any of it.

How I actually use it, day to day

The setup itself is almost anticlimactic compared to everything above. I give it a name and the feed's URL — say, name it neovim, point it at reddit.com/r/neovim.rss — and that's it. It goes and checks that URL on its own from then on, and whatever it last managed to fetch successfully shows up at localhost:8080/rss/neovim, always, whether the original site is currently fine or currently having a moment.

Running it on my own laptop at localhost:8080 is fine for testing, but it's obviously only reachable while my laptop happens to be on — which somewhat defeats the point of a thing meant to always be there. So in practice I've got it running on a small VPS instead, sitting behind an actual domain, which makes the whole thing much more useful: instead of a localhost URL, I get something like rss.mydomain.com/rss/neovim that's reachable from anywhere, all the time.

Either way, the last step is the same one you'd expect: take that URL — local or public — and add it to whatever RSS reader you actually use, in place of the original subreddit or blog link. The reader has no idea anything is different. It just asks for a feed and gets one back, instantly, every time. Voilà — everything works, even on the days the original site really, really doesn't.

I have a project that i vibe coded for this prototype at https://github.com/cksidharthan/rss-cacher Maybe one day i will make it production ready with tests and other stuff.