this post was submitted on 14 Jul 2022
18 points (100.0% liked)

Lemmy

12443 readers
44 users here now

Everything about Lemmy; bugs, gripes, praises, and advocacy.

For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.

founded 4 years ago
MODERATORS
nutomic@lemmy.ml
18
Storage size on instance servers (midwest.social)
submitted 2 years ago by seahorse@midwest.social to c/lemmy@lemmy.ml
23 comments fedilink hide all child comments
 

I recently had to expand my instance's storage size from 25 GB to 50 GB because it was out of space. How much storage do you all use for your instances? I'm a little surprised that it ran out of space as my instance isn't heavily used.

Am I doing something wrong? I prune all the unused docker images, so it's not that.

all 25 comments
sorted by: hot top controversial new old
[–] sexy_peach@feddit.de 8 points 2 years ago (1 children)

@dessalines@lemmy.ml how much space does lemmy.ml use?

[–] dessalines@lemmy.ml 8 points 2 years ago* (last edited 2 years ago) (2 children)

Pretty much nothing, postgres volumes folder is ˜500MB, but that really doesn't change much. Remember that text data really doesn't take up much space: all of english wikipedia is about 20 GB.

Picture storage in pictrs is the main issue. The only thing I could suggest there, is to use either a remotely running pictrs, or to use sshfs to mount the pictrs volumes folder to a remote machine (non-VPS) that has a ton of storage.

[–] NormieGirl@lemmy.perthchat.org 3 points 2 years ago (2 children)

You could also delete old pictures. I think mastodon does this.

[–] electrodynamica@mander.xyz 2 points 2 years ago

I'm not familiar with pictrs and how it works, but software I wrote which contains thumbnails and full copies of every image on the fediverse along with fingerprints, full database metadata, etc, doesn't take up even a tiny fraction of that space. So...

[–] sexy_peach@feddit.de 1 points 2 years ago (1 children)

mastodon only does this for remote content!

[–] NormieGirl@lemmy.perthchat.org 1 points 2 years ago (1 children)

TIL! What's your mastodon acct username?

[–] sexy_peach@feddit.de 1 points 2 years ago (1 children)

sorry I am not saying, I want to keep the identities separate.

[–] NormieGirl@lemmy.perthchat.org 2 points 2 years ago (1 children)

Same deal here lol

[–] sexy_peach@feddit.de 1 points 2 years ago

good! It's better to be safe than sorry ^^

[–] wintermute@feddit.de 7 points 2 years ago (1 children)

I do

  • remove pictrs temp files /tmp/magick-*
  • recompress overly large image files
  • truncate docker logs which frees up a lot of space
[–] sexy_peach@feddit.de 1 points 2 years ago

recompress overly large image files

When I was looking at some very old posts from feddit.de they had terrible jpg compression artifacts, is this the reason?

[–] sexy_peach@feddit.de 6 points 2 years ago (1 children)

Wow yeah that is pretty fast growth!

@wintermute@feddit.de how much space does our server use?

[–] wintermute@feddit.de 7 points 2 years ago (2 children)

Right now 13GB

[–] seahorse@midwest.social 6 points 2 years ago (1 children)

I just saw in the Lemmy matrix chat that you cleared up a lot of space by truncating logs. Where are those?

[–] wintermute@feddit.de 7 points 2 years ago (1 children)

you could truncate -s 0 /var/lib/docker/containers/*/*-json.log (which might break logs currently being written), configure logging via docker-compose.yml, or even better /etc/docker/daemon.json

https://github.com/LemmyNet/lemmy-ansible/pull/49

[–] seahorse@midwest.social 5 points 2 years ago (3 children)

Thanks. I'll have to look up how to configure daemon.json. I just truncated the logs for now. I wish I had known about that earlier! It cleared up about 15 GB of space.

[–] Redpandalovely@midwest.social 4 points 2 years ago

Wow, logs take up a ridiculous amount of space!

[–] wintermute@feddit.de 3 points 2 years ago (1 children)

you could also free up space on pictrs docker exec -it YOUR_PICTRS_CONTAINER /bin/sh -c "rm /tmp/magick-*"

[–] seahorse@midwest.social 2 points 2 years ago (1 children)

Hmm, tried that. It said no magick-* files exist.

[–] wintermute@feddit.de 2 points 2 years ago (1 children)

hm, maybe this problem has been fixed

[–] seahorse@midwest.social 4 points 2 years ago

I created this instance with docker directly instead of using Ansible. Not sure if that could be why.

[–] sexy_peach@feddit.de 3 points 2 years ago

okay that's not too bad.