I’ve had a Lemmy instance running on a VPS with 100 GB of storage for a few months and it has filled up. I’ve been searching for ways to reduce the amount of storage used but so far I am coming up empty. Can anyone point me in the right direction?
Firstly move pict-rs to object storage. My instance’s pict-rs uses 150GB alone. I pay less than $2/mo to put it on Cloudflare R2. Backblaze B2 might be even cheaper. Instructions: https://crates.io/crates/pict-rs#filesystem-to-object-storage-migration
If that doesn’t help enough and you’re comfortable with SQL, you can purge the unnecessary entries in
received-activities.Command:
delete from received_activity where published & NOW() - INTERVAL '3 days';(Lemmy has problems with ampersands so you’ll have to edit it)Then do a
vacuum full received_activity;to reclaim the space.This deleted 98 million entries for me and reduced my database size from 49GB to 20GB a week ago when I started running out of space. No other effect as far as I can tell. Thanks @illecors@lemmy.cafe
Thank you, that SQL command looks like exactly what I’m after! I’m going to give that a shot. I appreciate the help!
Instead of using the hard drive for pictrs, I suggest using it’s S3 capabilities and migrating to bucket based storage. You’ll save way more money and keep the expensive VPS hard drive just doing text and DB things. I think I spend maybe a dollar a month in S3 storage.
Is there a guide for switching over?
Thanks for the suggestion! As Nix asked, do you happen to know of a guide or any documentation I could reference for this?
I used the same one @bdonvr@thelemmy.club used - https://crates.io/crates/pict-rs#filesystem-to-object-storage-migration
pict-rs has the option to compress images. Ours is set to WEBP with 1280 pixels either side max.
That sounds like a good idea. Do you know of any documentation for this? I’d like to give it a try.
I ran into the same problem and ended up switching to an S3 with Vultr. It’s been a while since I did it but here are the links that I used to figure it out. I’m deployed using Lemmy-Easy-Deploy.
I used a combination of:
https://lemmy.world/post/538280
https://github.com/ubergeek77/Lemmy-Easy-Deploy/blob/main/ADVANCED_CONFIGURATION.md
https://git.asonix.dog/asonix/pict-rs/#user-content-filesystem-to-object-storage-migration
Good luck!
Wow, thank you for these great resources! I will check it out. I really appreciate it!
So I wanted to try it and found that Synology offers 15 GB for free so I tried it and it was extreamly easy to do. Set up a bucket and then fill in the ENV variables in docker-compose restart and it works, impressive. I might really need to think getting a good deal and doing that for my other fediverse stuff like Mastodon, Peertube and Matrix too, I think it’s equaly easy there.
Ah, but all my old pictures are not available anymore …
You’ve got to go through the migration process: https://crates.io/crates/pict-rs#filesystem-to-object-storage-migration
in v0.19.0 you could try disabling pictrs caching https://github.com/LemmyNet/lemmy/commit/1d23df37d86cc5cb6d7d9efaaf4360ecc9a9796f
cache_external_link_previews: falseI don’t think that will clear the existing cache though
Isn’t this only on the as yet unreleased version 0.19?
Asking as someone with a 165GB picts cache…
yes you’re right, also that config flag was renamed to
cache_external_link_previews
I love how OP doesn’t say it, but everyone immediately goes “it’s the pictures”
Honestly, what else would it be? Text takes ridiculously little storage compared to a single picture of a decent resolution.
Are you on the latest version?
Is the space used by pictrs, or your db?
Sadly, no. My server has been a bit neglected but it’s been plugging along and working fine for the most part. I need to upgrade though. And I assume pictrs but to be honest I haven’t checked. I just noticed today it was running poorly and checked and the drive is full.
The last release (or maybe the one before) did a bunch of DB cleanup that reduces the db size by about 20gb.
