self-hosting is great, but...

I am a very big fan of self-hosting services I use on a daily basis or that hold my data. Some services are mostly a backup, some are in day to day production use for my company, some are to toy around. Whenever one of the big providers messes up, has a data breach or is bought and shuts down people advocate for self-hosting similar services, and you might think I’d be one of the,. Sadly it is not that easy.

I am not self-hosting because I do not trust Apple or Google to keep my data out of third parties hands. However, I do not trust them to keep my accounts available and my access to my data intact. I have read their privacy policies and I know that Google also uses my data in ways I am not too happy about. I am not too worried that they will all out leak the whole dataset. Their security teams, processes and procedures are really good. But never say never.

There are a few Apple services I use on a daily basis. Not for the most important data I am dealing with. And an account loss would suck, but I would still have a local alternative I can fall back on that holds all my data. And this is the first thing to be aware of when self-hosting: OS integration and often UI will not be as good as the commercial alternative.

User Experience

There are many open source projects out there doing a really good job with their UI and UX. But there are many more that are "okay" to use - and the "okay" is sometimes generous. This includes everything I am building. I am no UI / UX designer and I cannot justify pulling company resources (aka asking my wife to make the thing look pretty) for weeks onto a random OSS project I feel like doing in my spare time. For most people a pleasant UI / UX is not negotiable and will determine which product will be used.

For the last five or six years I didn’t face any big challenges hosting various services. I think one time a database version conflicted when upgrading the docker compose file. One project changed the default port, stopping the reverse proxy from forwarding requests. Nothing I would consider a big deal. But I am not considering it a big deal because I’ve been dealing with this stuff for nearly 30 years now.

The bar to even host a simple note taking application is insanely high for someone just starting out and not having a lot of background knowledge. Assuming you get a brand new computer or VM. You need to install the OS. Docker. Setup containers. Get the ports exposed. Add a web server. Configure the reverse proxy and SSL. And we already lost half the people who do not think this sounds like fun just reading the requirements list.

Note from proofreader (a.k.a. wife): I would’ve noped out at exposing ports. Sounds dangerous!

The complexity of running even the most simple, single binary is outside of what many people who would like to have ownership over their data are willing to learn. And that is okay. My car mechanic could also tell me I can learn how to do an oil change. "It is not that hard." Except there are 50 professions I should know the basics of - and the failure / recovery scenarios, obviously - while being an expert in mine.

What happens when disaster strikes? Is the backup working? Did anyone ever try to restore the backup? Are all the images needed available or is Docker pulling a newer version that is not directly compatible? You know what happens when my MacBook dies? I login to another Apple device, turn on iCloud sync and have access to my documents. Call it comfort. Call it business continuity.

Why am I doing this to myself?

Here is the list of software running on my internal network.

timo@alextrasza:~/compose$ ls
acme
caddy
excalidraw
garagehq
immich
linkwarden
nodered
opencloud
redlib
sponsorblock
wyoming-piper
ampache
diun
floatplane
grafana
jellyfin
mailarchiver
ollama
openwebui
roundcube
sshwifty
gluetun
wyoming-whisper
bravesync
drawio
forgejo
homedash
komga
memos
omadav6
prometheus
searxng
stalwart
vaultwarden
yarr

Some of these like gluetun have a few more services such as qbittorrent and metube included in their compose.yaml. Add a few more Forgejo instances, mail servers, IDP, wiki,... some. one cloud VMs, some on bare metal in the rack next to the one housing these services. The internal list alone shows how much you have to deal with.

My wife can use some of these as well. The other rack powers most of our business. Some services are used by friends and their friends as well (like DeltaChat). From my experience people are willing to accept a tradeoff or two. But not five or ten.

Now all of this sounds pretty bleak, doesn't it? Let me give a counter point: Most people do not need to know all of this. They do not need to self-host all services. What they need is someone doing it for them. They want to own their data or at least not have their data used to train LLMs. Have someone to talk to if there is a problem. Many are likely willing to pay for it. But they do not want to become sysadmins.

Make it a business

One viable option is providers like Hetzner hosting NextCloud for you. I personally cannot stand its UI and think some of their decisions are more than questionable. But it is a rather complete solution for file sharing, calendars and contacts that will get many people relatively far and off of major cloud providers for a few Euro a month.

Self-hosting is great, but it is not the answer if the question is how we can help people to not rely on two or three large companies.

Making self-hosting services easier is an admirable goal. But it will never be easy enough that you click a button and you are done without learning anything about the stack. What I really hope for is seeing many more small providers pop up with compelling offers to do the work for people. Back in my day we called them businesses offering hosted services.

The price has to be right. So the easier self-hosting becomes, the more these providers can compete on price while still having a viable business. People will not go from paying nothing to 500 a month. But if they are Apple or Google subscribers 20 to 30 are not outlandish if they are worried about data ownership and being dependent on providers with no human to speak to if there is an issue.

I like the self-hosting community. Mostly good spirited enthusiasts getting their hands dirty on weekends with stuff others consider work. I like seeing people being less dependent on large providers who have repeatedly shown to not care about customers. But these two groups have and will have a very small overlap in a venn diagram. I hope long term we manage to work towards a world where the latter group has more options and is better served than they are today instead of telling them "become a sysadmin".