Ugh, apparently yesterday a bot visited my Forgejo instance and queried everything, which caused Forgejo to create repo archives for everything. Git on the instance is 2.1 GB in size, but the repo archive filled up everything and is 120 GB. I really didn’t expect such a spike.

That meant that it filled up the whole hard drive and the server and all the services and websites on it went down while I was sleeping.

Luckily it seems that just deleting that directory fixes the problem temporarily. I also disabled the possibility of downloading archived from the UI but I’m not sure if this will prevent bots from generating those archives again. I also can’t just make the directory read only because it uses it for other things like mirroring, etc too.

For small instances like mine those archives are quite a headache.

  • fireshell@kbin.earth
    link
    fedilink
    arrow-up
    2
    arrow-down
    5
    ·
    1 day ago

    Script for monitoring disk space in Linux

    The script below is designed to monitor disk space usage on a specified server partition. Configurable parameters include the maximum allowable percentage of disk space usage (MAX), the e-mail address to receive alerts (EMAIL) and the target partition (PARTITION).

    The script uses the df command to collect disk usage information and sends email alerts if the current usage exceeds the specified threshold

    #!/bin/bash
    # Script: ./df_guard.sh [config_file]
    
    # Set the maximum allowed disk space usage percentage
    MAX=90
    
    # Set the email address to receive alerts
    EMAIL=user@example.com
    
    # Set the partition to monitor (change accordingly, e.g., /dev/sda1)
    PARTITION=/dev/sda1
    
    # Get the current disk usage percentage and related information
    USAGE_INFO=$(df -h "$PARTITION" | awk 'NR==2 {print $5, $1, $2, $3, $4}' | tr '\n' ' ')
    USAGE=$(echo "$USAGE_INFO" | awk '{print int($1)}') # Remove the percentage sign
    
    if [ "$USAGE" -gt "$MAX" ]; then
    # Send an email alert with detailed disk usage information
    echo -e "Warning: Disk space usage on $PARTITION is $USAGE%.\n\nDisk Usage Information:\n$USAGE_INFO" | \
    mail -s "Disk Space Alert on $HOSTNAME" "$EMAIL"
    fi
    

    Installation

    sudo install -m 0755 df_guard.sh /usr/local/bin/df_guard.sh
    

    Make the script executable:

    sudo chmod +x /usr/local/bin/df_guard.sh
    

    Launch examples

    • Every 15 minutes.

    In crontab (root)

    */15 * * * * * /usr/local/bin/df_guard.sh
    
    • Jeena@piefed.jeena.netOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      23 hours ago

      I have monitoring of it, but it happened during night when I was sleeping.

      Actually I saw a lot of forgejo action on the server yesterday but didn’t think it would go so fast.