Try your Python application with PyPy, the new hot Python JIT compiler
- Your program will run faster…
- … faster…
- and ..faster!
Here are quick instructions for using PyPy on OSX. You should be able to apply these instructions to other UNIX systems as well.
Install PyPy by getting binary tarball.
wget http://pypy.org/download/pypy-1.4.1-osx64.tar.bz2 tar -xjf pypy-1.4.1-osx64.tar.bz
Install Distribute / easy_install command (setuptools fork) in order to install third party libraries.
cd pypy-1.4.1-osx64 wget http://python-distribute.org/distribute_setup.py bin/pypy distribute_setup.py
Install libraries – easy_install seems to run just fine on PyPy as long as you don’t try to install Python native extensions
bin/easy_install plac # command line parser bin/easy_install iso8601 # date format parser
Then run your application
~/code/pypy-1.4.1-osx64/bin/pypy exceptionanalysis.py -s=2010-01-01T00:00:00 /Users/moo/xxx/log/client1.log
In my case the application was a simple log analyzer based on Python generators (code will be posted later). It reads a text file and tries to interpret it. The script utilizes only one CPU core and thus is very CPU bound. The speed comparison results are
- With Python 2.4: Completed in 111.08 seconds
- With PyPy: Completed in 30.39 seconds
That’s almost 4x speed-up!
Things did not work with PyPy
- import multiprocessing – was not hard dependency, needed to comment out in plac_ext
Here is the traceback if anyone can help:
Traceback (most recent call last): File "app_main.py", line 53, in run_toplevel File "exceptionanalysis.py", line 20, in <module> import plac File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/plac-0.8.0-py2.5.egg/plac.py", line 35, in <module> from plac_ext import Interpreter, import_main, ReadlineInput, stdout, runp, Monitor File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/plac-0.8.0-py2.5.egg/plac_ext.py", line 7, in <module> import itertools, traceback, multiprocessing, signal, threading File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/multiprocessing-2.6.2.1-py2.5-macosx-10.6-i386.egg/multiprocessing/__init__.py", line 87, in <module> import _multiprocessing File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/multiprocessing-2.6.2.1-py2.5-macosx-10.6-i386.egg/multiprocessing/_multiprocessing.py", line 7, in <module> __bootstrap__() File "/Users/moo/code/pypy-1.4.1-osx64/site-packages/multiprocessing-2.6.2.1-py2.5-macosx-10.6-i386.egg/multiprocessing/_multiprocessing.py", line 6, in __bootstrap__ imp.load_dynamic(__name__,__file__) AttributeError: 'module' object has no attribute 'load_dynamic'
Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+
The external multiprocessing lib is a C extension and so we wouldn’t expect it to work. However, starting in Python 2.6 it is part of the standard library and thus will be a part of the next PyPy release, which implements Python 2.7, and will be out as soon as we finish the last few bugs.
Regarding multiprocessing, you should download a nightly, where it’s a built-in module. If there are none available for OSX64, try translating it yourself. It should also be faster in general: http://bit.ly/i3osmK
the nightly builds include support for python2.7, you might want to try those
load_dynamic() loads native code:
http://docs.python.org/library/imp.html#imp.load_dynamic
There’s a stub function added after last December’s 1.4.1 release, but don’t think it’ll help much!
https://bitbucket.org/pypy/pypy/src/52d990937d2e/pypy/module/imp/app_imp.py
It’s saying that imp.load_dynamic doesn’t exist. It looks like that was just added to pypy:
https://bitbucket.org/pypy/pypy/changeset/ac4d1d5d2c14
You should try again with the latest pypy code:
hg clone http://bitbucket.org/pypy/pypy
hi, can i use pypy on windows 7?
right now im using python 2.6.7…i ve already written code using it..
i just need to make the prog run fast…
how can i do that.?
please help