It’s a crapload more work to support XMPP/Matrix/whatever messaging on any platform than…SMTP
It’s absolutely not.
And you know this since you’ve written code to manage both on different environments, right?
Also, whatsapp supports all kinds of “bots” and it has absolutely massive userspace compared to pretty much any other instant message application. It doesn’t matter if you create the perfect protocol and platform for this kind of thing if there’s 7 people globally using it.
It’s a whole lot less work than configuring email.
It’s a crapload more work to support XMPP/Matrix/whatever messaging on any platform than just using a robust, reliable, resilient, widely supported good old SMTP. For you it might be easier to input your account (which at least on XMPP resemble quite a bit of email address) but for the developer it’s totally different thing. Also practically everyone accessing a website has an email address and if they’d decide to support some mesaging platform it’d make more sense to use whatsapp than XMPP since it’s vastly more popular.
Self hosting is not just one thing. You are system adminstrator, network engineer, security specialist, service architect and many other things, specially if you expose anything to anyone outside your very private network. And to get anything even running on that complex mess requires some knowledge on a lot of things. Making them run securely with proper backups requires even more knowledge on things.
Sure, you can just throw some docker images on your old desktop and be happy, even forward ports from the public internet to your things if you like. But that exposes your stuff to quite a lot of dangers and if you just click buttons without any understanding you’ll soon be a part of a botnet or lose your data or lose money if someone decides to mess around with your home automation or something else.
I get what you’re saying, not all of us are very polite and answers can be pretty harsh, but more often than not the generic idea behind those answers is not trying to be an asshole or gatekeep anything. It’s just that there’s a skillset you need to build things safely and if it’s clear from the start that someone looking for answers is way over their head it’s better for everyone to get them take a step back and learn instead of trying to create a meaningful answer since there’s too many variables or it’d just take immense effort to write down comprehensive guide on what to do, why and how for everything from the ground up.
I know for a fact that in my area there’s a bunch of surveillance cameras, home automation stuff and even some farm equipment directly open to the public network just because someone just plugged things in without any idea on the whole picture. Sometimes the correct answer is ‘stop shooting yourself on the foot and learn the basics first, then come back’.
Just for the sake of conversation, I recently did some crude math on this. I have few friends around who are well capable of running a backup server for me (hardware maintenance and stuff is always needed anyways) and at first it seemed like a good plan. Just get a 4TB SSD/NVME and throw that on a Raspberry Pi (or something small to keep electricity consumption low and setup silent), set up encryption, connect that to my network with wireguard or some other VPN and let it do it’s thing.
But I’d need to purchase everything as setting up a remote location with old hardware is just asking for trouble. The drive alone is 300€ (give or take) and the rest is easily another 100€. Currently my storagebox costs ~10€/month for 5TB. Even if I scored a fantastic black week offer and got everything for -50% discount that hardware with multiple single point of failures would cost nearly 2 years worth of cloud backups. And I’d still owe at least few beers to the friend for the trouble.
Your mileage may obviously vary, there’s a million different scenarios, but for me with my current setup it just makes sense to pick couple cloud providers and let them store my bits instead of getting more hardware to maintain and upgrade.
With backups two is one and one is none, so you are very much in a right track. Personally I have my stuff running on proxmox VMs with a proxmox backup server (VM as well) storing backups to Hetzner Storagebox. I’m planning to set up a another host in garage to have “local” backups too, as mine is detached as well the risk of both going up in flames in event of fire is pretty low. However, a voltage spike due to lightning on the grid or something else might blow up both hosts so that’s a threat model to be aware of. Also if your connection to garage is over copper it can cause other problems, fibre or wireless is highly recommended.
With backups it’s largely about the bandwidth available. I personally have enough so uploading to cloud is not an issue, but backing up a terabyte of data over 10Mbps connection might not work out at all.
For more info search for 3-2-1 strategy, that should give you plenty of ideas what you need to think about and what are industry best practises about making sure backups are in order.
So it is always DNS
I actually did something for quite a while. Finished long overdue wiring for outdoor access point and one more camera, replaced a main switch since the old one started to behave unreliably, installed frigate (which still needs some work), cleaned up some wiring while messing around, updated a bunch of firmwares, replaced switch in garage to managed one and made some changes on my workstation and some other minor stuff.
Next would be to move cameras into their own VLAN and harden that setup a bit. And I really should get around on better backups for my VPS. But it’s a new week coming up, if the work isn’t too busy I might get something more done.
Another happy Hetzner customer here. ~10 years so far, both for business and for personal use. 1€/month won’t happen, but they’re not that expensive either.
DNS PTR records belong to the entity who owns the IP addresses, you can’t make reverse records for arbitary addresses like you can with forward zones. I haven’t heard about any residential ISP which would give access to PTR records and even on business lines that’s usually a premium.
What you could do is to get a VPN service which gives you these options, if there is one, I don’t know. Most likely you’re looking for a VPS for that and tunnel traffic with some kind of VPN-setup to your local instance. And at that point you might as well run the whole thing on VPS unless you happen to need a ton of storage or some other reason makes pure VPS server too expensive.
Depends heavily on what you need/want. My current installation doesn’t have anything extra, mail and calendar with “standard” file storage (with sync agents on desktop/laptop) is well enough on what I use it for as my photos are in Immich instance.
Nextcloud offers practically everything you ask for besides the family tree and even that might be available as a plugin (or “App” as Nextcloud calls tem). Or if you’re willing to split photos in a different app, Immich works great.
I do it. Postfix+dovecot+spamassassin managed with ISPConfig running on a VPS. Works just fine, but my domains already have a long-ish good reputation so that may play a part on my experience. Biggest headache is to keep things running, which occasionally means jumping trough hoops microsoft(mostly) and others throw at you by flagging your server as spam for no apparent reason.
It’s quite likely that any given IP, unless you get one from shady VPS provider or something, is “clean”. And if it’s not it’s usually not that big of a deal to get it cleared from major blacklists (spamhaus, google and microsoft covers quite a lot). You just need to dig up proper forms to tell them that you’re a new owner of said IP and promise to play nice.
Same goes with domain names, but if you get a new one that’s a non-issue. Just set up SPF-records properly (and preferably DKIM/DMARC, but those aren’t strictly necessary and need a bit more than a single TXT-record) and you’re good to go.
And then you of course need to stay away from those lists. If you configure your SMTP to act as a open proxy you’ll be on every shitlist on the planet pretty quickly. So, reasonable measures against compromised account (passwords, firewalls, rate limits…) and against other threats (misconfigured/unsafe web service used for spam and stuff like that). Any of those alone are not too difficult to accomplish, but there’s quite a few things you need to get right.
Maybe easier to get anything runnin quickly. But it obfuscates a lot of things and creates additional layer of stuff which you need to then manage. Like few days ago there was discussion about how docker, by default, creates rules which bypass the “normal” INPUT rules on many (most?) implementations. And backup scenario is different, it’s not as straightforward to change configuration than with traditional daemon and it’s even more likely to accidentally delete your data as a whole.
As I already said, docker has its uses, but when you’re messing around and learning a new system you first need to learn how to manage the ropes with docker and only after that you can mess around with the actual thing you’re interested of. And also what I personally don’t really like is the mindset that you can just throw something on a docker and leave it running without any concern which is often promoted with ‘quickstart’-type documentation.
You absolutely can run services without containers and when learning and trying things out I’d say it’s even preferable. Docker is a whole another beast to manage and has a learning curve of it’s own.
Containers can of course be useful but setting everything up, configuring networking, managing possible integrations with other components (for example authentication via LDAP) it’s often simpler just to run the thing “in traditional way”. With radicale you can just ‘apt install radicale’ (or whatever you’re using) and have a go with it without extra layer of stuff you need to learn before getting something out of the thing. And even on production setups it might be preferred approach to go with ‘bare metal’, but that depends on quite a few variables.
On residential connections it’s a bit pain in the rear, but if you get VPS (or something similar) it’s perfectly manageable. You just need to maintain stuff properly, like having proper DNS records, and occasionally clear false positives from spam lists. The bigger issue is to have proper backups and precautions, I’ve hosted my own emails for over 10 years and should I lose all the data and ability to receive new messages it would be a massive personal problem.
Obviously we’re talking about hobbyist level stuff and with that there’s always something what can go wrong and it’s not always obvious what it is. So if the ‘remote end’ doesn’t have someone who can do at least very basic troubleshooting it can be nearly impossible to fix the setup over the phone unless you just replace the whole thing and ship whole units back and forth.
But in this particular case the remote end has someone who knows their stuff so it’s taken care of, with or without a KVM. I’ve been thinking a similar setup with my relatives and on my case the distance isn’t an issue but it’s still something I’d need to bother family members with and, for me, it was simpler to get a storage box from hetzner and run backups to that instead of getting more hardware.
Maintenance is anyways something you need to consider and viable options for that vary on a case-by-case basis, so there’s no ‘one size fits all’ solution.
It won’t help if your power supply breaks and KVM itself can malfunction too. It’s of course nice to have, but it has limitations.
I would consider also the case where something goes wrong. Maybe the whole thing crashes, maybe you misconfigure something, maybe there’s a power outage or something else happens and you lose the connectivity. Is there someone on site who can do anything to your hardware as you can’t easily just go and figure it out by yourself?
If the answer is ‘no’ then I would strongy reconsider the whole approach. On a worst case scenario the system goes down before you’re even back home from the trip and then your hardware is just gathering dust.
How you imagine things send messages to reset your passwords, sending notifications and whatever is currently managed via email than some piece of code creating and sending messages, managing possible errors with them and potentially also monitoring/logging the message traffic for statistics or debugging?
User adoption matters if you want your thing to be actually useful for the actual users. And supporting any messaging system requires effort, so it makes sense to spend limited resources on a thing which has the biggest userspace. If you want to run matrix server which has you and your dog using it, go ahead, but don’t be surprised if you want to contact your neighbor and he’ll look like you have two heads when you start to explain how to reach you.