FreeBSD Jails

uWSGI 1.9.16 introduced native FreeBSD jails support.

FreeBSD jails can be seen as new-generation chroot() with fine-grained tuning of what this “jail” can see.

They are very similar to Linux namespaces even if a bit higher-level (from the API point of view).

Jails are available since FreeBSD 4

Why managing jails with uWSGI ?

Generally jails are managed using the system tool “jail” and its utilities.

Til now running uWSGI in FreeBSD jails was pretty common, but for really massive setups (read: hosting business)where an Emperor (for example) manages hundreds of unrelated uWSGI instances, the setup could be really overkill.

Managing jails directly in uWSGI config files highly reduce sysadmin costs and helps having a better organization of the whole infrastructure.

Old-style jails (FreeBSD < 8)

FreeBSD exposes two main api for managing jails. The old (and easier) one is based on the jail() function.

It is available since FreeBSD 4 and allows you to set the rootfs, the hostname and one ore more ipv4/ipv6 addresses

Two options are needed for running a uWSGI instance in a jail: –jail and –jail-ip4/–jail-ip6 (effectively they are 3 if you use IPv6)

—jail <rootfs> [hostname] [jailname]

—jail-ip4 <address> (can be specified multiple times)

—jail-ip6 <address> (can be specified multiple times)

Showing how to create the rootfs for your jail is not the objective of this document, but personally i hate rebuilding from sources, so generallyi simply explode the base.tgz file from an official repository and chroot() to it to make the fine tuning.

An important thing you have to remember is that the ip addresses you attach to a jail must be available in the system (as aliases). As always we tend to abuse uWSGI facilities.In our case the –exec-pre-jail hook will do the trick

  1. [uwsgi]
  2. ; create the jail with /jails/001 as rootfs and 'foobar' as hostname
  3. jail = /jails/001 foobar
  4. ; create the alias on 'em0'
  5. exec-pre-jail = ifconfig em0 192.168.0.40 alias
  6. ; attach the alias to the jail
  7. jail-ip4 = 192.168.0.40
  8.  
  9. ; bind the http-socket (we are now in the jail)
  10. http-socket = 192.168.0.40:8080
  11.  
  12. ; load the application (remember we are in the jail)
  13. wsgi-file = myapp.wsgi
  14.  
  15. ; drop privileges
  16. uid = kratos
  17. gid = kratos
  18.  
  19. ; common options
  20. master = true
  21. processes = 2

New style jails (FreeBSD >= 8)

FreeBSD 8 introdiced a new advanced api for managing jails. Based on the jail_set() syscall, libjail exposes dozens of featuresand allows fine-tuning of your jails. To use the new api you need the –jail2 option (aliased as –libjail)

—jail2 <key>[=value]

Each –jail2 option maps 1:1 with a jail attribute so you can basically tune everything !

  1. [uwsgi]
  2. ; create the jail with /jails/001 as rootfs
  3. jail2 = path=/jails/001
  4. ; set hostname to 'foobar'
  5. jail2 = host.hostname=foobar
  6. ; create the alias on 'em0'
  7. exec-pre-jail = ifconfig em0 192.168.0.40 alias
  8. ; attach the alias to the jail
  9. jail2 = ip4.addr=192.168.0.40
  10.  
  11. ; bind the http-socket (we are now in the jail)
  12. http-socket = 192.168.0.40:8080
  13.  
  14. ; load the application (remember we are in the jail)
  15. wsgi-file = myapp.wsgi
  16.  
  17. ; drop privileges
  18. uid = kratos
  19. gid = kratos
  20.  
  21. ; common options
  22. master = true
  23. processes = 2

Note for FreeBSD >= 8.4 but < 9.0

uWSGI uses ipc semaphores on FreeBSD < 9 (newer FreeBSD releases have POSIX semaphores support).

Since FreeBSD 8.4 you need to explicitely allows sysvipc in jails. So be sure to have

  1. [uwsgi]
  2. ...
  3. jail2 = allow.sysvipc=1
  4. ...

DevFS

The DevFS virtual filesystem manages the /dev directory on FreeBSD.

The /dev filesystem is not mounted in the jail, but you can need it for literally hundreds of reasons.

Two main approaches are available: mounting it in the /dev/ directory of the roots before creating the jail, or allowing the jail to mount it

  1. [uwsgi]
  2. ; avoid re-mounting the file system every time
  3. if-not-exists = /jails/001/dev/zero
  4.   exec-pre-jail = mount -t devfs devfs /jails/001/dev
  5. endif =
  6. ; create the jail with /jails/001 as rootfs
  7. jail2 = path=/jails/001
  8. ; set hostname to 'foobar'
  9. jail2 = host.hostname=foobar
  10. ; create the alias on 'em0'
  11. exec-pre-jail = ifconfig em0 192.168.0.40 alias
  12. ; attach the alias to the jail
  13. jail2 = ip4.addr=192.168.0.40
  14.  
  15. ; bind the http-socket (we are now in the jail)
  16. http-socket = 192.168.0.40:8080
  17.  
  18. ; load the application (remember we are in the jail)
  19. wsgi-file = myapp.wsgi
  20.  
  21. ; drop privileges
  22. uid = kratos
  23. gid = kratos
  24.  
  25. ; common options
  26. master = true
  27. processes = 2

or (allow the jail itself to mount it)

  1. [uwsgi]
  2. ; create the jail with /jails/001 as rootfs
  3. jail2 = path=/jails/001
  4. ; set hostname to 'foobar'
  5. jail2 = host.hostname=foobar
  6. ; create the alias on 'em0'
  7. exec-pre-jail = ifconfig em0 192.168.0.40 alias
  8. ; attach the alias to the jail
  9. jail2 = ip4.addr=192.168.0.40
  10.  
  11. ; allows mount of devfs in the jail
  12. jail2 = enforce_statfs=1
  13. jail2 = allow.mount
  14. jail2 = allow.mount.devfs
  15. ; ... and mount it
  16. if-not-exists = /dev/zero
  17.   exec-post-jail = mount -t devfs devfs /dev
  18. endif =
  19.  
  20. ; bind the http-socket (we are now in the jail)
  21. http-socket = 192.168.0.40:8080
  22.  
  23. ; load the application (remember we are in the jail)
  24. wsgi-file = myapp.wsgi
  25.  
  26. ; drop privileges
  27. uid = kratos
  28. gid = kratos
  29.  
  30. ; common options
  31. master = true
  32. processes = 2

Reloading

Reloading (or binary patching) is a bit annoying to manage as uWSGI need to re-exec itself, so you need a copy of the binary, plugins and the config filein your jail (unless you can sacrifice graceful reload and simply delegate the Emperor to respawn the instance)

Another approach is (like with devfs) mounting the directory with the uwsgi binary (and the eventual plugins) in the jail itself and instructuWSGI to use this new path with –binary-path

The jidfile

Each jail can be referenced by a unique name (optional) or its “jid”. This is similar to a “pid”, as you can use itto send commands (and updates) to an already running jail. The –jidfile <file> option allows you to store the jid in a filefor use with external applications.

Attaching to a jail

You can attach uWSGI instances to already running jails (they can be standard persistent jail too) using –jail-attach <id>

The id argument can be a jid or the name of the jail.

This feature requires FreeBSD 8

Debian/kFreeBSD

This is an official Debian project aiming at building an os with FreeBSD kernel and common Debian userspace.

It works really well, and it has support for jails too.

Let’s create a jail with debootstrap

  1. debootstrap wheezy /jails/wheezy

add a network alias

  1. ifconfig em0 192.168.173.105 netmask 255.255.255.0 alias

(change em0 with your network interface name)

and run it

  1. uwsgi --http-socket 192.168.173.105:8080 --jail /jails/wheezy -jail-ip4 192.168.173.105

Jails with Forkpty Router

You can easily attach to FreeBSD jails with The Forkpty Router

Just remember to have /dev (well, /dev/ptmx) mounted in your jail to allow the forkpty() call

Learn how to deal with devfs_ruleset to increase security of your devfs

Notes

A jail is destroyed when the last process running in it dies

By default everything mounted under the rootfs (before entering the jail) will be seen by the jail it self (we have seen it before when dealing with devfs)