Try your Python application with PyPy, the new hot Python JIT compiler
- Your program will run faster…
- … faster…
- and ..faster!
Here are quick instructions for using PyPy on OSX. You should be able to apply these instructions to other UNIX systems as well.
wget http://pypy.org/download/pypy-1.4.1-osx64.tar.bz2 tar -xjf pypy-1.4.1-osx64.tar.bz
Install Distribute / easy_install command (setuptools fork) in order to install third party libraries.
cd pypy-1.4.1-osx64 wget http://python-distribute.org/distribute_setup.py bin/pypy distribute_setup.py
Install libraries – easy_install seems to run just fine on PyPy as long as you don’t try to install Python native extensions
bin/easy_install plac # command line parser bin/easy_install iso8601 # date format parser
Then run your application
~/code/pypy-1.4.1-osx64/bin/pypy exceptionanalysis.py -s=2010-01-01T00:00:00 /Users/moo/xxx/log/client1.log
In my case the application was a simple log analyzer based on Python generators (code will be posted later). It reads a text file and tries to interpret it. The script utilizes only one CPU core and thus is very CPU bound. The speed comparison results are
- With Python 2.4: Completed in 111.08 seconds
- With PyPy: Completed in 30.39 seconds
That’s almost 4x speed-up!
Things did not work with PyPy
- import multiprocessing – was not hard dependency, needed to comment out in plac_ext
Here is the traceback if anyone can help:
Traceback (most recent call last): File "app_main.py", line 53, in run_toplevel File "exceptionanalysis.py", line 20, in <module> import plac File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/plac-0.8.0-py2.5.egg/plac.py", line 35, in <module> from plac_ext import Interpreter, import_main, ReadlineInput, stdout, runp, Monitor File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/plac-0.8.0-py2.5.egg/plac_ext.py", line 7, in <module> import itertools, traceback, multiprocessing, signal, threading File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/multiprocessing-18.104.22.168-py2.5-macosx-10.6-i386.egg/multiprocessing/__init__.py", line 87, in <module> import _multiprocessing File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/multiprocessing-22.214.171.124-py2.5-macosx-10.6-i386.egg/multiprocessing/_multiprocessing.py", line 7, in <module> __bootstrap__() File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/multiprocessing-126.96.36.199-py2.5-macosx-10.6-i386.egg/multiprocessing/_multiprocessing.py", line 6, in __bootstrap__ imp.load_dynamic(__name__,__file__) AttributeError: 'module' object has no attribute 'load_dynamic'