• 0 Posts
  • 20 Comments
Joined 2 years ago
cake
Cake day: June 20th, 2023

help-circle
  • I recently decommissioned my old poweredge T620. Beast of a thing, 5U heavy af. It had 8x10T drives and was the primary media server.

    Now that it is replaced I bought 2x Synology RS822+ and filled them with the old disks. Using SHR2. They are mixed brands bought at different times so I’ve made sure each NAS has a mix of disks.

    Lowest is 33k hours, highest is 83k.



  • In 2010 I self hosted a Xen hypervisor and used it for everything I could. It was fun!

    I had a drive failure in my main raid 5 array so bought a new disk. When it came to swap it out, I pulled the wrong disk.

    It was hardware raid using an Adaptec card and the card threw the raid out and refused to bring it together again. I couldn’t afford recovery. I remember I just sat there, dumb founded, in disbelief. I went through all the stages of grief.

    I was in the middle of a migration from OVH to Hetzner and it occurred at a time where I had yet to reconfigure remote backups.

    I lost all my photos of our first child. Luckily some of them were digitised from developed physical media which we still had. But a lot was lost.

    This was my lesson.

    I now have veeam, proxmox backup server, backuppc and Borg. Backups are hosted in 2 online locations and a separate physical server elsewhere in the house.

    Backups are super, SUPER important. I’ve lost count how many times I’ve logged into backuppc to restore a file or folder because I did a silly. And it’s always reassuring how easy it is to restore.














  • Monitoring is the key. I use Zabbix, but essentially you want to gather metrics and report on issues.

    Once things are set up and working, even with 10s of VMs and applications, it’s quite reliable. The biggest things that catch you out are updates breaking functionality, updates requiring additional manual steps, running out of disk space or expired certificates.

    I find I get a spurt of energy to recreate or implement a new system every few months but things just tick over in the meantime.