The best presentations tell a story — this is a story of our journey hosting the Fediverse at SpaceHost:
- What we tried first.
- How & why that failed.
- How we solved the problem.
- And, an unexpected invention that could reshape the Social Web.
If you care about the Fediverse, then this is a story for you.
NOTE: This article is a written adaptation of the presentation I delivered at the Fediverse House during SXSW 2025.
But — a story needs a beginning. So, I will start with an introduction.
I am Charles Iliya Krempeaux.
I have been a product leader, an engineering leader, and a founder.
But, I will skip the long bio — here is just one highlight from my career:
I was one of the first engineers at Hootsuite — a social-media company which at one time the most popular Twitter app in the world. Over time, Hootsuite expanded to also became a Facebook, Google+, Instagram, LinkedIn, TikTok, YouTube, WhatsApp, and every other social-media platform app. Everyone from the U.S. White House, to IBM, to tens-of-millions of people all over the world, and 80% of fortune 1,000 companies have used the tool I created with others.
I have been in Tech for more-than 30 years.
And, from engineering, to product, to enterprise, and to company — I have a lot of experience with social-media.
Some of you might know me from — The Fediverse!
From my @reiver@mastodon.social account.
Where I talk a lot about — you guessed it — the Fediverse. I also talk about other decentralized social-media (DeSo) — including Bluesky, Farcaster, Nostr, and others.
And, I talk about the Fediverse related projects I work on, too. For example…
… GreatApe!
GreatApe is a conversations platform for the Fediverse, and decentralized social-media.
Here is an example of myself and atomicpoet (Chris Trottier) using GreatApe back in October 2023.
Here, we were using GreatApe to do a live video podcast.
Now, you don't see it in this image, but — we can also invite people watching in the audience to join the video conversation.
Another thing some people might know me from is — SpaceHost.
And, for this story — SpaceHost is probably the more important thing — as it partially motivates what I am soon going to show you.
But, what is SpaceHost —
We need to know this for the story.
What is SpaceHost —
To regular people, I tend to describe SpaceHost as —
SpaceHost is an app-directory for the Fediverse
Or maybe –-
SpaceHost is an app-directory for the decentralized social-media
To people who are a little more tech savvy — I might describe SpaceHost as —
SpaceHost is fully-managed hosting for all the Fediverse
As far as I know –
- SpaceHost was the second hosting-company in the Fediverse, after Masto.Host, and
- SpaceHost is the first hosting-company in the Fediverse to try to offer all the (other) Fediverse software other than Mastodon – although, we also offer Mastodon.
We learned a number of things from our experience with hosting so many people’s Fediverse servers. Here is one thing we learned —
We learned what seems to be the number 1 cause of Fediverse servers crashing — including Mastodon, Misskey, Pixelfed, and every other type of Fediverse server.
Before you read what that is – guess to yourself what you think it might be.
(Scroll down once you have your guess.)
Here is the answer —
Caching
And, in particular —
The storage drive of Fedivere servers filling-up – due to continued caching.
Some of you might ask —
- What is being cached?
- Some of you might even be ask, what is caching?
Those are both good questions 🙂
Caching is a technical term — in computer science, computer hardware engineering, and software engineering. It (caching) basically means — temporarily storing things.
So, what is being temporarily stored?
With Fediverse servers, they temporarily store user-data from other servers.
- posts,
- media from posts,
- user-profile text,
- avatar images,
- header images,
- and more.
Now, temporarily storing things (caching) is normal.
It (caching) is a core technique with computer programming & (other) engineering –
Everything from the CPU to the Web, to applications all cache. And, would not work without caching.
So – caching is important, caching makes things work, but – caching is a difficult to do well.
So – it is not surprising that caching is challenging for Fediverse servers to do well.
At SpaceHost we wrestled with this problem.
Should we just automatically clear the cache for our customers?
Or, should we tell them their storage drive is filling up – and, let them decide.
The challenge is – clearing the cache affects user-experience. It can “break” search. It can also "break" the home-feed.
So –
At first – we decided to just e-mail our customers to tell them that their storage drives were filling-up and let then decide.
What we found-out was that –
Most customers seemed to never notice the e-mail alert we sent them!
In hindsight, that was not a good strategy.
So –
This was a problem!
It created a bad experience.
Their server got slow. And sometimes crashed.
It doesn't matter if we sent them an e-mail or not. It wasn't being noticed. And, they were unhappy.
Which made us unhappy. We wanted to fix this. So –
We changed our strategy.
We started automatically clearing our customers caches.
This is better-than servers crashing, but – there is still a problem –
Search is sort of broken for them.
So –
We need to separate search from the back-end server.
We need to make it so accumulating user-data doesn’t crash the back-end.
In fact – multiple back-end servers could share the same search-server to reduce costs.
We also need a Fediverse API for search. (More on that in a future article.)
Here is the thing – we already built a lot of it.
We started working on this back in August 2024.
In fact – we start working on search plus custom feeds back in August 2024.
I didn’t mentioned custom feeds yet, but I am mentioning it now.
But back to search –
Because no Fediverse software uses this search API yet –
We created a real-time search-engine (named RobinLive) so you can see it in action.
You can try it for yourself at:
Now, although RobinLive exists – the idea is that there would be multiple search-servers.
And, not just RobinLive.
That is right – let's decentralize them. Let's decentralize search!
RobinLive will be one of many.
It is difficult to overstate the significance decentralized search – not only for the Social Web but for the (classical) Web and the Internet as a whole. Decentralization will be just as transformative for search and the Web as it has been for social-media.
If you were looking closely at the previous RobinLive screen, you might have noticed that – in addition to showing results from the Fediverse – we were also showing results from Bluesky.
We on the Fediverse have more in common with those on other decentralized social-networks than the centralized ones. I don't think we are enemies. I think we are all fellow-travelers. We are all reacting to the same problems (with the Internet, the Web, and social-media) in a similar way — we are just choosing different technologies.
I have my own opinions about the Tech. I have my own preferences. But –
I want the user to not have to care about these differences.
I think Danielle Foré said it best –
When talking about the Fediverse – she didn’t say "decentralized", or "open-source", or "server", or even mention "e-mail". She just said –
you can follow anyone from any social-network.
I want to make that as true as I can.
Thus – users should not care about the differences between – the Fediverse, Bluesky, Nostr, and others.
I also mentioned – custom feeds.
I have been talking about and meme'ing about Fediverse custom feeds for a while – and thinking about them even longer.
And, guess what – we already built a lot of that, too.
In fact — the technology for real-time search and custom feeds are related.
The feed most people are probably familiar with is – the home-feed.
On Mastodon, the home-feed uses a simple algorithm – reverse chron. This (reverse chron) is what some people inaccurately call "no algorithm".
A custom feed lets you pick an alternative algorithm to this. For example –
What if you wanted a feed that – only showed posts from people you follow who do not post very often. (This was a 3rd party app for Twitter decades ago.)
What if you are a parent and want a feed for you child that – shows posts from the people your child follows, but – removes all posts that contain pornographic images.
What if you want a feed – that only shows posts from mutuals.
What if you are a cat lover and follow a bunch of (other) people who post photos of cats but also post stuff unrelated to cats that you do not care about, and want a feed that – only shows posts from the people you follow that includes a photo of a cat.
What if you are a scholar, and you want a feed that only shows scholarly papers shared by people you follow.
What if you use your community as a human-powered recommendation engine, and want a feed that – shows popular posts and links drawn from the various communities you are in.
Etc.
This is RobinFeed – RobinFeed is a custom feed generator.
RobinFeed actually already has users — people are already using RobinFeed!
Here (in the previous screenshot) are RobinFeed’s current choices for creating a custom feed. (More options will be available in the future.)
There is a saying:
Show, don't tell.
So I am going to show you some of what RobinFeed is like – if you were to choose the "By Search Terms" button (in the previous screenshot), you would then see the following –
Here (in this screenshot), RobinFeed lets you choose what search terms you want to base your custom feed on. (Again, there are other ways of creating a custom feed, but — in this demo I am taking you through just one of them.)
I can fill this form in as follows (in the following screenshot) to get the following –
Here I am just made making RobinFeed search for "fediverse house".
I can also filter-out things by choosing "Ad Filter" –
But, I will leave that empty for this demo.
(BTW, one upcoming filter we plan to add to RobinFeed is — the ability to detect and filter-out pornographic images. Something that can help people create both child-safer and work-safer feeds.)
We turned the Fediverse results in RobinFeed off, just for now.
Pressing the preview button currently only shows results from Bluesky. (Although, by the time you read this, RobinFeed Fediverse custom feeds may have been turned back on.)
Again – Fediverse results are in the system. You can see them in the real-time search-engine – RobinLive
But –
Before the Fediverse can have its own custom feeds, we need something from Fediverse developers – from the people who build the Fediverse.
The Fediverse needs front-end application support for Fediverse custom feeds, as a user-experience.
The Fediverse also needs front-end application support for (and this is technical) an ActivtyPub "Feed" sub-actor-type (of "Application"), as a way of encoding Fediverse custom feeds.
Which means – I have more work to do, and I need to have more conversations with more Fediverse developers, to make this happen.
And – I will end this article here.
Thank you for reading all of that. To summarize:
You can try the real-time search-engine at:
Hopefully, there will eventually be other search-engines like it – so that we can decentralize search.
And – although more work needs to be done to support Fediverse custom feeds –
You can try out the custom feed generator at:
And – if you want to take a look at what motivated this story – you can give SpaceHost a try at:
If you would like to talk to me more about any of this, or just want to follow along – you can follow me at –
Thank you Evan Prodromou for letting me present at the Developer Meetup at the Fediverse House at SXSW 2025.