My Sidekiq 3 installation seams to be very hungry in respect of memory. If you dig through the Net you'll find a lot of similar reports.
For e.g. the unicorn web worker a memory-killer script exists which should prevent running into Madland.
So I thought I take an example and do a similar thing for the sidekiq workers. If you give them time and send a graceful kill signal there shouldn't be any data-loss either.
#!/bin/bash
# check if script is already running
SELFCHECK=$(ps -eo pid,command |grep -v grep |grep $0 |wc -l)
if [ "$SELFCHECK" -gt 2 ]; then
echo "$0 is already running!";
exit 1;
fi;
NAPTIME=120
MAXRSS=$(( 4200 * 1024 )); # 4.2GB
FOUND=$(ps -eo rss,pid,command |grep sidekiq |grep -v grep |grep -v $0);
while read -r line; do
echo -n "$line ...";
USAGE=$(echo $line |cut -d' ' -f1);
PID=$(echo $line |cut -d' ' -f2);
if [ "$USAGE" -gt $MAXRSS ]; then
echo -e " \033[0;31mrestart\033[0m";
kill -SIGUSR1 $PID;
# wait before we forget about the graceful stuff
sleep $NAPTIME;
kill -SIGKILL $PID;
# prevent killing all worker at once
sleep $(( $NAPTIME / 2 ));
else
echo -e " \033[0;32mok\033[0m";
fi;
done <<< "$FOUND"
Currently it starts killing if it exceeds 4GB but you can adjust it by touching following line: MAXRSS=$(( 4000 * 1024 )); # 4GB
But there is .. so to speak .. light at the end of the tunnel. They improved sidekiq massively in respect of memory consumption. In a similar installation I use Sidekiq 4 and for this I don't need any nasty killer scripts :)
Cheers,
Lukas