› Articles › Contact › Download › Misc › Home

Securing distcc with chroot and OpenVPN

Email: thomas[at]northernsecurity.net
Url: www.northernsecurity.net
Updated: 050601

distcc: distcc.samba.org
OpenVPN: openvpn.net

A short guide how to set up distcc to run in chroot and send/receive traffic over an encypted tunnel using OpenVPN. Note that the instructions and scripts are tested only on Debian.

Benefits:

both distcc and OpenVPN runs as a unprivilaged user in chroot, which restricts the commands available to the client
distcc accepts jobs only from clients that has a certificate issued by the OpenVPN server
distcc listens for job on a private address space (10.8.0.0/24)
no need to create additional user accounts on the server, which you had to do if you were using SSH

Drawbacks:

since the connections are made over OpenVPN it's slower
you need to manually update the chroot when a new version of distccd or gcc is available, this is however made easier with the available script
you need to issue certificates and distribute them to the client(s)

Setting up OpenVPN

Since openvpn.net/howto.html#pki covers all the steps in a detailed manner, we're only going to touch this subject briefly.

Install OpenVPN with aptitude.

aptitude install openvpn

Copy the easy-rsa directory to /etc/openvpn and change permissions.

cd /etc/openvpn
cp -R /usr/share/doc/openvpn/examples/easy-rsa/ ./
chmod 0700 easy-rsa/

Edit /etc/openvpn/easy-rsa/vars.
Gunzip the openssl.cnf.gz file.
Create the CA key.

. ./vars
./clean-all
./build-ca

Generate a certificate and private key for the server.

./build-key-server server

Generate client certificate(s).

./build-key client1

Always use a unique common name for each client.

Diffie Hellman parameters must be generated for the OpenVPN server.

./build-dh

Copy the necessary keys to /etc/openvpn:

cp easy-rsa/keys/{ca.crt,server.crt,server.key,dh1024.pem} ./

Server configuration:

tls-server

port 1194
proto udp 
dev tun

ca ca.crt
cert server.crt
key server.key 
dh dh1024.pem

server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt

keepalive 10 120

comp-lzo

chroot /etc/openvpn
user nobody
group nogroup

persist-key
persist-tun

status openvpn-status.log
log openvpn.log

verb 3
mute 20

Client configuration:

dev tun
proto udp

remote distcc.example.com
ns-cert-type server

client

ca ca.crt
cert client1.crt
key client1.key

chroot /etc/openvpn
user nobody
group nogroup

comp-lzo

persist-tun
persist-key

status openvpn-status.log
log openvpn.log

verb 3
mute 20

Setting up distcc

aptitude install distcc

Change the ALLOWEDNETS in /etc/default/distcc to read

ALLOWEDNETS="127.0.0.1 10.8.0.0/24"

Run the chroot script available at www.northernsecurity.net/download/distccd-chroot and copy the init (www.northernsecurity.net/download/distccd-chroot.init) file to /etc/init.d/distcc.

If everything went well, /var/chroot/distccd/ was created containing all necessary files.
Start distccd with /etc/init.d/distcc start and verify that it runs in chroot.

# ps -fe|grep distccd|awk '{print $2}'
29302
# ls -l /proc/29302/root 
lrwxrwxrwx  1 distccd nogroup 0 2005-06-01 01:41 /proc/22628/root -> /var/chroot/distccd

From the distccd logfile:

distccd[29301] (dcc_listen_by_addr) listening on 10.8.0.1:3632

The client should then be able to run distcc just like normal, except that the server is located at 10.8.0.1.

$ export DISTCC_VERBOSE=1	# We want details
$ export DISTCC_HOSTS='10.8.0.1'

Watching with distccmon-text:

18929  Preprocess  util_ebcdic.c                                  10.8.0.1[0]
11311  Compile     util_time.c                                    10.8.0.1[0]

Log on the client, if things work out:

distcc[191] exec on 10.8.0.1: cc -g -O2 -pthread -c modules.c -o modules.o
distcc[191] (dcc_note_state) note state 2, file "modules.c", host "10.8.0.1"
distcc[191] (dcc_connect_by_name) connecting to 10.8.0.1 port 3632
distcc[191] (dcc_connect_by_addr) started connecting to 10.8.0.1:3632

Server log:

distccd[32373] (dcc_check_client) connection from 10.8.0.6:4882
distccd[32373] compile from modules.c to modules.o
distccd[32373] (dcc_r_file_timed) 181798 bytes received in 0.137825s, rate 1288kB/s
distccd[32373] (dcc_collect_child) cc times: user 0.100006s, system 0.008000s, 
	2000 minflt, 0 majflt
distccd[32373] cc modules.c on localhost completed ok
distccd[32373] job complete