Managing SSH Sockets in GNU Screen


One slick thing about the AppNexus cloud is that I can SSH from my remote server to my coworkers’, making actions like sending files to their servers very easy.  After connecting via SSH from my laptop to my remote server, it’s just

# In a terminal
01:~ jhuttner$ scp branching_proposal.php
branching_proposal.php      100% 4322     4.2KB/s   00:00
01:~ jhuttner$

Logging in to another server is easy, too

01:~ jhuttner$ ssh
[ ~]$

Observe that I am not asked for the passphrase to my SSH private key when connecting to co-worker. Here’s why: my public key is forwarded by ssh-agent to co-worker, which sends a challenge back to my remote server (and thus to my laptop since I have agent forwarding on). Ssh-agent running on my laptop responds correctly to the challenge because the active socket on my laptop has access to my private key. And so co-worker authorizes the connection request, and the connection is opened.

Sadly, the beauty of this workflow gets hit with the ugly stick when attempted within an instance of GNU Screen. Specifically, the above commands prompt me for the passphrase to my SSH private key. What gives?

The answer is that Screen is doing what it does best: maintaining state. The SSH socket that my screen tries to use to connect to co-worker is the socket that was in the environment when that screen was created, which was months ago in my case. (I have been re-attaching to this same screen since.) Though the screen lives on, the SSH socket from the initial connection is long dead. When I type “ssh …” or “scp …”, the screen tries to use that old socket, which of course fails. Without an active SSH socket, ssh-agent has to create a new one. This new socket needs access to my private key so that it can authenticate with co-worker. To gain access to my private key it needs the passphrase, hence the prompt.

Continually typing passphrases is annoying. Ideally, I want the screen to maintain state except for SSH connection variables. Every reattachment to an existing screen should update the SSH socket used within that screen to be the active socket. Fortunately, this is possible via the following steps.

Ssh-agent running in GNU Screen relies on an environment variable called SSH_AUTH_SOCK to determine which socket it should use when requesting new connections. I will tell all screens that this variable’s value is actually found in a file I will create in a bit in the .ssh directory. The first step is to add this line to your ~/.screenrc

setenv SSH_AUTH_SOCK $HOME/.ssh/ssh_auth_sock

Or, alternatively, add this line to your ~/.bashrc

export SSH_AUTH_SOCK=$HOME/.ssh/ssh_auth_sock

SSH_AUTH_SOCK now resolves to whatever ssh_auth_sock is. To finish up, ssh_auth_sock has to point at the current, actual socket identity. This is done by creating an rc file for SSH, which will automatically run every time a new SSH connection is established on my remote server. Here is the contents of ~/.ssh/rc

if test "$SSH_AUTH_SOCK" ; then
    ln -sfv $SSH_AUTH_SOCK ~/.ssh/ssh_auth_sock

Note that SSH_AUTH_SOCK is the active socket identity before we mess with it.

To test your new trick, close all the terminals on your laptop, then SSH to your remote server. You should see a message like this

create symbolic link `/home/jhuttner/.ssh/ssh_auth_sock' to

The output of ls -l in the ~/.ssh directory shows the symlink is setup correctly

ssh_auth_sock -> /tmp/ssh-rUxLEl9056/agent.9056'

Re-attach to an existing screen, and then SSH to another server. You’ll know you’ve got it working when you are not prompted for the passphrase to your private key. It just goes.

Note: If you want this to work on a screen you already have setup (and do not want to kill), you will need to go the bashrc route. In each window in your screen, source your updated bashrc file. If you are starting with a fresh screen instance, the screenrc line is all you need.

Good luck!

About Joseph

Software Engineer at AppNexus

This entry was posted in Developer Tools, Development Process. Bookmark the permalink.

  • Dmitry

    the best ssh-forwarding for screen i’ve seen, thank you!

  • Andre Arko

    This is by far the best process I’ve ever seen for this… but once you add a file to ~/.ssh/rc, SSH calls that file instead of xauth, and X11 forwarding no longer works. :( Anybody have a version of the rc script that replicates the call to xauth correctly?

  • Keb

    This is very helpful and solves the problem I’ve had in a way I hadn’t thought of. Thanks.

    One trick that you missed in your final note is to use “C-a :” in screen then type “source .screenrc” to reapply any changes made to .screenrc to the current screen session. Of course, “setenv” in screen only applies to new windows, so any open windows will have to have their environment updated manually via sourcing .bashrc as you suggest, or more simply, just running “export SSH_AUTH_SOCK=$HOME/.ssh/ssh_auth_sock”. My point is that changes to .bashrc can be avoided in all cases, hence reducing potential maintenance costs, however insignificant.