Using paster local commands with buildout and avoiding the infamous dependency issue

1) Introduction

Paste script is a part of Python Paste web development utilities.

Its paster create command is used by various Python frameworks to generate skeleton code for a development project.

2) Paster and local commands

Besides generic project templates, paster provides local commands which are project aware commands to add more modules into an existing project. Local commands are made available by paster extensions. For example, ZopeSkel product has various local commands to generate skeletons into Plone/Zope projects automatically

  • Views
  • Content types
  • Forms
  • Portlets

… and so on.

For example, you could generate a project template for Plone add-on module and then create content types there using a local paster command. The local commands become available when you execute paster command in the folder of your project.

Example:

paster create -t archetype myorg.customploneaddon
cd src/myorg.customploneaddon

# Now new paster commands are available
paster

Usage: ../../bin/paster COMMAND
usage: paster [paster_options] COMMAND [command_options]

...

Commands:
  ...

ZopeSkel local commands:
  addcontent   Adds plone content types to your project

Above, ZopeSkel paster template adds its addcontent templates. Now you can use addcontent local command to contribute to the existing project

paster addcontent -t contenttype MyShinyWebPage
2.1. ZopeSkel

For more information how to use paster to create add-ons and add-on submodules for Plone, see here.

To see list of available paster local commands, run paster command

../../bin/paster addcontent --list

… in your development project. For ZopeSkel specific projects the output should be something like this:

Available templates:
    atschema:     A handy AT schema builder
    contenttype:  A content type skeleton
    form:         A form skeleton

3) How paster local commands work

paster reads (evaluates) setup.py file which declares a Python egg. If it founds paster_plugins section there, it will look for local commands there. For example, Plone project templates declare the following paste_plugins in setup.py:

paster_plugins = ["ZopeSkel"]

4) setup.py install_requires

Python modules can dependencies to other modules using setup.py and install_requires section. For example, a Plone add-on might read:

install_requires=['setuptools',
                  # -*- Extra requirements: -*-
                  "five.grok",
                  "plone.directives.form"
                  ],

This means that when you use setuptools/buildout/pip/whatever Python package installation tool to install a package from Python Package Index (PyPi) it will also automatically install dependency packages declared in install_requires.

4.1. paster and install_requires

This is where things usually go haywire.

Let’s assume you are using paster in a project which contains N python packages. You probably use an external configuration system to manage your installed applications and their versions to make repeatable deployments possible (hint: buildout is gaining traction in Python community lately).

Paster is not aware of this external Python package configuration set (paster cannot see them in its PYTHONPATH). So what happens when you try to execute paster create which reads setup.py containing install_requires and encounters dependencies?

Paster will try automatically download and install them locally in that folder.

Plone and Zope ecosystem contains over hundreds of reusable components, in nice dependency hierarchy. paster create would try to pull all them in to your source tree as *.egg folders. See discussion here.

Warning

Do not never use system paster command.

Do not ever run sudo easy_install ZopeSkel. Do not ever run paster local commands using a paster command from your system-wide Python installation.

Warning

The internet is full of tutorial saying easy_install ZopeSkel. If you ever encounter this kind of tutorial, it’s wrong.

5) Paste and buildout

If you are using buildout to manage your Python application deployment, you can integrate paster nicely with it.

Add to your buildout.cfg:

parts =
    ...
    paster

[paster]
recipe = zc.recipe.egg
eggs =
        PasteScript
        ZopeSkel
        ${instance:eggs}

After rerunning buildout, buildout adds paster command to bin folder.

Then you can run paster from buildout folder:

bin/paster

… or in a buildout managed project under src folder…

../../bin/paster

This way paster is aware of your deployment configuration and local commands won’t explode on your face anymore.

Thanks Martin Aspeli to helping with how buildout + paster should be done.

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

To gzip or not to gzip CSS and Javascript?

Dear Lazyweb,

Is it wise to gzip static resources (javascript and CSS) before sending them to client?

The opinion of Internet seems to be divided

  • GZip decompression takes too much time at the client end and thus it is not wise (latency at the clients end)
  • Bandwidth save is enough to counter the decompression latency

So if we put this to context of

  • Compressed and merged CSS and Javascript files of Plone
  • Assuming the users are using the state of the art browsers: Safari 4, Chrome, Firefox 3.6 and IE8
  • Connections are faster than 384 Kbit/s

…and forget…

  • Recompressing images (GIF, PNG and JPEG) using GZip as there is no notable save

should I enable GZip compression on the front-end server (Apache) with disk cache enabled?

Of course, I could do testing and timing myself, but I’ll simply ask for your experiences first before investing few hours for this. Also, hints how to measure how fast GZip decompression is, are welcome.

Thank you.

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Could not open a connection to your authentication agent on Ubuntu 9.10 Linux

I just run into this when tried to enable Github SSH to perform git push.

The workaround is to run command

`eval ssh-agent`

Which sets series on environment variables making ssh-add to work.

I don’t know why this doesn’t work anymore – I am quite sure it worked on earlier Ubuntu versions out of the box.

Related bug report.

Plone Go Mobile – preparing a release

Enter the mobile world with Plone! Go mobile for Plone add-on is now “feature complete” and we are preparing a release.

For Easter holidays we bring you a trunk release instructions. Note: you might want to change checkout protocol from “https” to “http” in the buildout cfg. The most able and most inpatient might want to test this now when it is hot.

There exists three production site where you can see the product in action

See the feature set which will beat crap out of other mobile CMSs.

What’s missing for the real release is

  • packaging it as a eggs
  • making microsite
  • clean up documentation (YES. It has documentation since day 0)

It has not been tested on Plone 4.

Thanks Karl Horak for bringing up the issue.