Tuesday, 13 August 2019

bash - Linux trap signals to handle SSH connection dropped/killed


I’ve got a command in an npm script that SSHes into a remote build server and runs a Bash script. The script sets up a lockfile and a trap call to delete the lockfile when the script exits.


I’m cancelling the operation with ctrl+C in both cases.


LOCKFILEPATH="/tmp/env_app.lock"
# cleanup function just deletes $LOCKFILEPATH
function mutex() {
if [ -f "$LOCKFILEPATH" ]; then
echo -e "\n\n${redtext}Build already in progress! Exiting.${resettext}\n\n";
exit 1;
else
touch $LOCKFILEPATH;
trap cleanup EXIT;
fi
}

This works fine when you first SSH into the host to run it, but the trap is not working when you send the command over SSH with


ssh hostname command

I tried adding to the trap command to run for more signals, but these don't seem to work either:


 trap cleanup EXIT SIGHUP SIGKILL SIGTERM SIGINT

What should I be doing here?


I also set up a simpler script and it seemed to work fine when executing it manually over SSH. Maybe there's added layers when I’m running it using an npm script? The npm script is:


"deploy": "ssh HOSTNAME ''deploy-script $(git rev-parse --abbrev-ref HEAD) stage $npm_package_config_deploy_target yes''",

which just checks the current branch name and uses that to deploy on the build host. Same as


"deploy": "ssh HOSTNAME ''deploy-script CURRENTBRANCH stage APPNAME''",

UPDATE: Adding a force tty -t to the npm script seems to have fixed it. Confusing since I didn't need that for the simple script case. Maybe I'm spawning too many sub processes in the large script (too much to paste here without redacting a bunch) so it requires a tty to trigger the cleanup trap.


"deploy": "ssh -t HOSTNAME ''deploy-script CURRENTBRANCH stage APPNAME''",

Answer



When you do


ssh hostname command

and then Ctrl+C, you terminate the local ssh. The command still runs on the remote side, it never gets your keystroke. This is different with -t. See this answer for explanation. The relevant fragment:



On the client side, ssh will try to set the tty used by stdin to "raw" mode, and sshd on the remote host will allocate a pseudo-tty […] if you type Ctrl+C, it will be sent to the remote host, where the command will likely receive SIGINT […]. The remote sshd then closes the connection, and ssh reports Connection to remotehost closed.



Therefore use:


ssh -t hostname command

and your command (not the local ssh) will get SIGINT when you press Ctrl+C.




This may get even more interesting. Compare ssh + here document – does Ctrl+C get to remote side?


No comments:

Post a Comment

How can I VLOOKUP in multiple Excel documents?

I am trying to VLOOKUP reference data with around 400 seperate Excel files. Is it possible to do this in a quick way rather than doing it m...