These instructions help you to copy and transfer production server ZODB database (Data.fs) to your local computer for development and testing. This allows you to do the testing against the copy of real data and the production server Plone instance set up.
Data.fs is ZODB file storage for transactional database. Journal history takes quite a lot of disk space there. Packing, i.e. removing the journal history, usually reduces the size file considerably, making the file lighter for wire transfer. Depending on the database age the packed copy is less than 10% of the original size.
These instructions apply for Ubuntu/Debian based Linux systems. Apply to your own system using the operating system best practices.
We need ZODB Python package to work with the database. To use it, we’ll create virtualenv Python installation in /tmp. In virtualenv installation, installed Python packages do not pollute or break the system wide setup. Note that you might use easy-install-2.4 depending on the OS. The latest stable ZODB can be picked from PyPi listing. Plone 3.x default is ZODB 3.7.x, which is not available as Python egg, but you can use ZODB 3.8.x.
sudo easy-install virtualenv
/tmp/packer/bin/easy_install ZODB=3.8.3 Data.fs cannot be modified in-place. You must create a copy of it to work with it. Data.fs copy can be created from a running system without the fear of corrupting the database, since ZODB is append only database.cp /yoursite/var/filestorage/Data.fs /tmp/Data.fs.copy
Then create the following script snippet /tmp/pack.py using your favorite terminal editor.import time import ZODB.FileStorage import ZODB.serialize storage=ZODB.FileStorage.FileStorage('/tmp/Data.fs.copy') storage.pack(time.time(),ZODB.serialize.referencesf)
And run it using virtualenv’ed Python setup with ZODB installed:/tmp/packer/bin/python /tmp/pack.py
Lots of patience here… packing may take a while, but it’s still definitely faster than your Internet connection transfer rate.
Verify that the file is succesfully packed:ls -lh Data.fs.copy -rw-r--r-- 1 user user 30M 2009-09-01 13:24 Data.fs.copy
Woohoo 1 GB was shrunk to 30 MB. Then copy the file to your local computer using scp and place it to your development buildout.scp user@server:/tmp/Data.fs.copy ~/mybuildout/var/filestorage/Data.fs
You just saved about 30-90 minutes of waiting of file transfer.