Here are documented brief instructions how to run 32-bit chroot’ed environment on 64-bit Ubuntu server. chroot means that you run re-rooted and jailed system inside another system.
What we do here is enabling 32-bit chroot’ed userland on 64-bit server. 32-bit userland and 32-bit Python environment reduces the memory usage of heavy website applications we are running (read: Plone/Zope), since 32-bit has only half of the pointer size and object-oriented programming is all about pointers. Zope is especially memory hungry, because it uses ZODB object database. The developer does not need to worry about when doing queries, updates or caches that much as the persistent site state is transparent to Python (objects are automatically loaded from database or cache when they are referred). Easy persistency means that almost everything is in the database and you need to have big object cache. Plone has huge client-side, in-process, cache for persistent objects. The default setting is for the cache size 5000 objects. (sidenote: since ZODB cache is in-process and Python does not do threading too well, running big Plone sites means that you need run several processes to handle parallel requests – having multiple processes with big in-process caches means loads of memory consumption)
32-bit userland is especially useful if you need to run Plone on 64-bit VPS (virtual private server) with low amount of available memory (512 MB or 1 GB).
There are some brief measurements about 32-bit vs. 6-bit Python memory usage at the end of this article.
Unless otherwise specified, all command here should be run as the root user of the host system. Commands here are for example only and you need to know what you are doing. If you lack advanced UNIX administration skills we gladly arrange you some commercial training or hosting support.
Basic schroot installation instructions for Ubuntu can be found here https://help.ubuntu.com/community/DebootstrapChroot . We also install ZopeSkel in the chroo’ed environment for starting creating Plone sites. Note that we are using Ubuntu 8.04 which still ships with Python 2.4 – for later Ubuntus you need to compile Python 2.4 from the scratch.
apt-get install debootstrap apt-get install schroot # Old schroot uses global schroot.conf, new versions have # chroot.d directory # This is a heredoc, but use what ever editor you like # to create the configuration cat <<EOF > /etc/schroot/chroot.d/hardy_i386.conf [hardy_i386] description=Ubuntu 8.04 Hardy for i386 location=/srv/chroot/hardy_i386 personality=linux32 root-users=bob run-setup-scripts=true run-exec-scripts=true type=directory users=bob,john,alice,ploneuser EOF mkdir -p /srv/chroot/hardy_i386 debootstrap --variant=buildd --arch i386 hardy /srv/chroot/hardy_i386 \ http://archive.ubuntu.com/ubuntu/ # Check that the chroot is created and working schroot -l # Enter the chroot (logged in as bob) schroot -c hardy_i386 -u root # Once inside, install python2.4-dev and other needed tools # Installing PIL with easy_install didn't work for some # reason, so we use python-imaging package. apt-get install python2.4-dev python-setuptools python-imaging easy_install-2.4 ZopeSkel
And that’s about it.
Chroot’ed environment will have
- It’s own application binaries (and userland bitness)
Chroot’ed environment will share with the host system
- User accounts
- Can’t list filesystem outside chroot
…giving additional safety for shared hosting in the case of chroot’ed environment is compromised.
2. Entering chroot’ed environment as a specific user
schroot -c hardy_i386 -u root
schroot -c hardy_i386 -u plone_user # After you have set-uped normal user for chroot'ed environment
All background processes you leave running in chroot’ed environment are terminated when you exit this environment, unless you create sessions as described below.
3. Creating chroot sessions
Sessions enable running commands in chroot without the need to have it constantly open.
# Create a new schroot session schroot --chroot=hardy_i386 --user=ploneuser \ --session-name=plonesession \ --begin-session # Run commands in the created session schroot --chroot=plonesession --user=ploneuser --run-session \ /srv/plone/yourplonesite/bin/instance start # Ending session schroot --chroot=plonesession --user=ploneuser --end-session
Note that --chroot parameter takes in both actual chroot installations and session ids.
4. Doing resets for chrooted environment
The session processes exist as long as the session exist. Unless you explicitly start a new session with –begin-session the processes are terminated as soon as you log out from the chroot’ed environent. chroot’ed environment is temporary unless you explicitly specify it not be Thus if you want to run daemonized services in chrooted environment you need to take care of session handling manually. Here is an example how do you construct a session (as a real root user) and then launch a shell script which will take care of launching applications inside the chroot.
# Terminate previous session if any schroot --chroot=hardy_i386 --user=plone_user --session-name=plone_session --end-session
#Start the session (again) schroot --chroot=hardy_i386 --user=plone_user --session-name=plone_session --begin-session
# Run a start script inside the chroot'ed environment which will start Plone # NGINX and other necessary 32-bit services schroot --chroot=plone_session --user=plone_user --run-session /srv/plone/myplonesite/restart-all.sh
5. Running sessions at startup
You can add schroot bootstrap in real /etc/rc.local:
schroot --chroot=hardy_i386 --user=ploneuser \ --session-name=plonesession --begin-session \ && schroot --chroot=plonesession --run-session \ /srv/plone/inst/bin/instance start
Remember that the users have to be created outside the chrooted environment. If you set the home directory to something that exists only in the chrooted environment, use something like this
adduser --no-create-home --home HOMEDIR_IN_CHROOT ploneuser
Then to create the directory inside the chroot and set its ownership to the newly created user and group.
6. 32-bit vs 64-bit memory consumption
Reason why we even tried this is that some python applications, like Zope, use references heavily and moving from 32bit to 64bit references increases memory usage. (J Stahl 2010)
Memory figures from a development Zope/Plone 3.3.5 server
|After startup||112 MiB RES 116 MiB VIRT||175 MiB RES 342 MiB VIRT|
|After normal usage||159 MiB RES 194 MiB VIRT||236 MiB RES 487 MiB VIRT|
This is far from a complete study, but it would seem that the chroot does pay off even though it has to load 32bit versions of basic libraries along. If running more than one instance on same server memory savings should increase.