Skip to content

Instantly share code, notes, and snippets.

@martyphee
Created February 15, 2019 15:30
Show Gist options
  • Save martyphee/7a6ec3d728615dd64bda45c5f0a31ba8 to your computer and use it in GitHub Desktop.
Save martyphee/7a6ec3d728615dd64bda45c5f0a31ba8 to your computer and use it in GitHub Desktop.
We never want to abruptly restart production.
The Heroku deploy hook is setup to do a rolling restart of the dynos: it will gracefully replace the old generation with the new one, and will only stop dynos after they're done serving the current request and if there are new ones ready to take their place.
Sometimes we need to restart production manually. The manual command, however, will kill and restart all the dynos at the same time.
Use this shell script to automate a gradual restart of the production dynos.
```shell
#!/usr/bin/env sh
getAllDynos() {
local target_app="$1"
heroku ps --app $target_app | egrep -o '^\w+\.\d+' | egrep -v 'run';
}
restartDyno() {
local target_app="$1"
local dyno="$2";
if [ -n "$dyno" ]; then
local cmd="heroku ps:restart $dyno --app $target_app";
echo "> $cmd";
eval $cmd;
fi
}
rollingRestart() {
local target_app="$1"
local dynos=$(getAllDynos $target_app);
local count=$(echo $dynos | wc -w | tr -d '[:space:]');
local last=$(echo $dynos | egrep -o '[^ ]+$');
echo "Rolling restart of $target_app for $count Dynos, 3 at a time, with 30 second pauses.";
echo $dynos | xargs -n 3 | while read d1 d2 d3; do
restartDyno $target_app $d1;
restartDyno $target_app $d2;
restartDyno $target_app $d3;
if [[("$d1" != "$last") && ("$d2" != "$last") && ("$d3" != "$last")]]; then
echo "---- 30 second pause ----";
sleep 30;
fi
done
}
rollingRestart "deliveroo-production"
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment