Skip to content

Instantly share code, notes, and snippets.

@JustinShenk
Last active January 22, 2024 20:45
Show Gist options
  • Save JustinShenk/312b5e0ab7acc3b116f7bf3b6d888fa4 to your computer and use it in GitHub Desktop.
Save JustinShenk/312b5e0ab7acc3b116f7bf3b6d888fa4 to your computer and use it in GitHub Desktop.
Google Cloud Platform (GCP) instance idle shutdown
#!/bin/bash
# Add to instance metadata with `gcloud compute instances add-metadata \
# instance-name --metadata-from-file startup-script=idle-shutdown.sh` and reboot
# NOTE: requires `bc`, eg, sudo apt-get install bc
# Modified from https://stackoverflow.com/questions/30556920/how-can-i-automatically-kill-idle-gce-instances-based-on-cpu-usage
threshold=0.1
count=0
wait_minutes=60
while true
do
load=$(uptime | sed -e 's/.*load average: //g' | awk '{ print $1 }') # 1-minute average load
load="${load//,}" # remove trailing comma
res=$(echo $load'<'$threshold | bc -l)
if (( $res ))
then
echo "Idling.."
((count+=1))
fi
echo "Idle minutes count = $count"
if (( count>wait_minutes ))
then
echo Shutting down
# wait a little bit more before actually pulling the plug
sleep 300
sudo poweroff
fi
sleep 60
done
@inossidabile
Copy link

inossidabile commented Nov 12, 2022

#!/bin/bash

threshold=0.1
count=0
wait_minutes=60

while true
do

load=$(uptime | sed -e 's/.*load average: //g' | awk '{ print $1 }') # 1-minute average load
load="${load//,}" # remove trailing comma
ssh_flag=$(ss | grep -i ssh | wc -l)
load_flag=$(echo $load'<'$threshold | bc -l)

if (( $load_flag ))
then
    echo "Idling CPU"
    if ! (( $ssh_flag ))
    then
        echo "Idling SSH"
        ((count+=1))
    else
        count=0
    fi
else
    count=0
fi
echo "Idle minutes count = $count"

if (( count>wait_minutes ))
then
    echo Shutting down
    sleep 300
    sudo poweroff
fi

sleep 60

done

@OptogeneticsandNeuralEngineeringCore
Copy link

Thanks for this! Here is a version that doesn't require bc. It also has flags on top, requiring user input to ensure some understanding (useautoshutdown=false). It also checks if there is a SSH connection (such that you can still use it but then only enable shut down after the SSH is closed). It also looks at GPU usage (via nvidia-smi) in case that is your thing.

#!/bin/bash
# Add to instance metadata with `gcloud compute instances add-metadata \
#   instance-name --metadata-from-file startup-script=idle-shutdown.sh` and reboot
# NOTE: This version does not require `bc`.
# Modified from https://stackoverflow.com/questions/30556920/how-can-i-automatically-kill-idle-gce-instances-based-on-cpu-usage
# ONE Core 5.Jan.2023

# User modification settings
## Flags
useautoshutdown=false # Should this script be used at all?? Default to no. Change to true to use
check_ssh=true        # Flag to enable/disable SSH connection check. Defaults to true, so if a SSH is open, it will not shut down
check_gpu=true        # Flag to enable/disable GPU utilization check
check_cpu=true        # Flag to enable/disable CPU utilization check
## Settings
threshold_cpu=10     # Average (over 1 min) of CPU usage. Defaulted as 10%
threshold_gpu=10      # GPU utilization percentage threshold
wait_minutes=10       # Time, in minutes, that the CPU/GPU usage must be under before the VM is shut down. Note that the script will wait for 30 seconds after this time to shut down to allow the VM to sort itself out a bit.

# Initialization of variables
count=0
ssh_resolution_flag=false
cpu_resolution_flag=false
gpu_resolution_flag=false

# Code
if [ "$useautoshutdown" == false ]; then # Check if useautoshutdown is false, and if so, exit the script
    echo "Auto shutdown is disabled. Exiting script."
    exit 0
else # Else infinate loop this
    while true
    do
    if [ "$useautoshutdown" == true ]; then
        current_time=$(date +"%Y-%m-%d %H:%M:%S")

        if $check_ssh; then
            active_sessions=$(ss | grep -i ssh | wc -l) #who | grep -c "pts/")
            if [ "$active_sessions" == 0 ]; then 
                ssh_resolution_flag=true
                echo "SSH flag set to true." 
                echo "  No SSH detected."
            else # SSH is connected
                ssh_resolution_flag=false
                echo "SSH flag set to true." 
                echo "  Found SSH connections. Number of SSH detected: $ssh_resolution_flag"
            fi
        else # Skip SSH check if the flag is disabled
            ssh_resolution_flag=true
            echo "SSH flag set to false, will not check"
        fi
        
        if $check_cpu; then
            cpu_utilization=$(uptime | sed -e 's/.*load average: //g' | awk '{ printf("%.0f", $1 * 10) }')
            resolution_cpu=$((cpu_utilization < threshold_cpu)) # Set to 0 if CPU utilization is less than the threshold, and 1 if it's not
            echo "CPU flag set to true."
            echo "  At time: $current_time, cpu load: $cpu_utilization %"
            echo "  CPU threshold set to: $threshold_cpu %"
            if [ "$resolution_cpu" -eq 0 ]; then # CPU above thresh
                cpu_resolution_flag=false
                echo "  CPU found to be above threshold. Not idling"
            else # CPU found to be below threshold
                cpu_resolution_flag=true 
                echo "  CPU found to be below threshold. Considered idling"
            fi
        else # Skip CPU check if the flag is disabled
            cpu_resolution_flag=true
            echo "CPU flag set to false, will not check. Considered ideling"
        fi

        if $check_gpu; then
            gpu_utilization=$(nvidia-smi --query-gpu=utilization.gpu --format=csv,noheader,nounits)
            resolution_gpu=$((gpu_utilization < threshold_gpu)) # Set to 0 if GPU utilization is less than the threshold, and 1 if it's not
            echo "GPU flag set to true."
            echo "  At time: $current_time, GPU load: $gpu_utilization %" # Set to 0 if GPU utilization is less than the threshold, and 1 if it's not
            echo "  GPU threshold set to: $threshold_gpu %"
            if [ "$resolution_gpu" -eq 0 ]; then
                gpu_resolution_flag=false
                echo "  GPU found to be above threshold. Not idling"
            else # GPU found to be below threshold
                gpu_resolution_flag=true
                echo "  GPU found to be below threshold. Considered idleing"
            fi
        else # Skip GPU check if the flag is disabled
            gpu_resolution_flag=true
            echo "GPU flag set to false, will not check"
        fi

        if $ssh_resolution_flag && $cpu_resolution_flag && $gpu_resolution_flag; then
            ((count+=1))
            echo "Because of settings and observed loads and SSH connections, the VM is considered ideling. Will increase idel time"
            echo "  Time in minutes in idel: $count"
        else # If ANY flag is found to be false, we reset the counter
            count=0
            echo "Because of settings and observed loads and SSH connections, the VM is considered to be working. Will reset the timer"
        fi

        if [ $count -gt $wait_minutes ]; then
            echo "Shutting down. Peace out"
            sleep 5 # wait a little bit more before actually pulling the plug
            sudo poweroff
            fi

        sleep 60 # Sleep for 1 minute to check CPU usage and SSH connection status every minute
    fi
    done
fi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment