How to split() strings of indefined item count to Python variables elegantly?

A common problem for  (space) separated string parsing is that there are a number of fixed items followed by some amount of optional items. You want to extract parsed values to Python variables and initialize optional values with None if they are missing.

E.g. we have option format description string with 2 or 3 items value1 value2 [optional value]

s = "foo bar" # This is a valid example option string
s2 = "foo bar optional" # This is also, with one optional item

Usually we’d like to parse this so that non-existing options are set to None. Traditonally one does Python code like below:

# Try get extration and deletion
parts = s.split(" ")
value1 = parts[0]
value2 = parts[1]
if len(parts) >= 3:
    optional_value = parts[2]
else:
    optionanl_value = None

However, this looks little bit ugly, considering  if there would always be just two options one could write:

value1, value2 = s.split(" ")

When the number of optional options grow, the boiler-plate code starts annoy greatly.

I have been thinking how to write a split() code to parse options with non-existing items to be set None. Below is my approach for a string with total of 4 options of which any number can be optional.

parts = s.split(" ", 4) # Will raise exception if too many options 
parts += [None] * (4 - len(parts)) # Assume we can have max. 4 items. Fill in missing entries with None.
value1, value2, optional_value, optional_value2 = parts

Any ideas for something more elegant?

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

ssh-copy-id for OSX

ssh-copy-id is a handy shell script which allows you to easily copy your public key to a remote server, so that you don’t need type in password every time you take SSH connection into that box. Ubuntu and the latest Linux distros ship ssh-copy-id with the ssh client installation. However, for OSX you need to manually drop this little script into your /usr/bin.

The usage is simple. Just run:

ssh-copy-id remotebox.com

to copy your public SSH key to remotebox.com. After that

ssh remotebox.com

shoud ask no password.

Working installation instructions for ssh-copy-id on OSX are in Chris Pitzer’s blog.

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Why C# / Javascript will be allowed and Flash won’t be as App Store programming language

All the Flash developers of the world wet their pants on 8th April 2010 when Apple announced that Flash based application won’t be approved in the Apple’s App Store. This essentially destroyed the easy opportunity to monetize existing Flash development skills and the huge Adobe expansion potential into mobile markets, pissing a lot of folks who had bet on Adobe’s technology, leading developers’ rage on Internet forums.  Bad news for those who had already building iPhone apps on Flash CS5 even before it was released…

Unfortunately it was not only Adobe and Flash who got in the line of fire: popular frameworks like Appcelerator, Unity 3D and Monotouch apps are theoretically excluded to enter App Store by the new developer agreement which states that the original source code of the application must be C, C++, Objective-C or WebKit/Javascript.

I belive Apple does not hate certain programming languages. Apple does not either want to force you to use their development tools. Only by understanding what Apple really meant with the infamous 3.3.1  clause in the developer agreement, we can start building a bridge to get MonoTouch et. al. to be Apple’s favorites again.

1. History of Flash (Lite)

Adobe (Macromedia) and Flash have been into mobile business far before iPhone was published. The thing is called Flash Lite run-time. It runs on billions of phones.

The problem of Flash Lite is that it does not very well integrate with its host environment. It does not even have so simple user interface component as a “menu” out of the box. Which means that every developer must create their own menu implementation. Which means that there exist hundreds of shitty Flash Lite menu implementations out there, each behaving differently and each not resembling the native menu component. This is not a very nice thing if you consider the user experience of the application and iPhone  is all about user experience. Note that this is not so big thing for games, as games have radically different user experience anyway.

Flash does not communicate with its host platform either. You wouldn’t be able to access native API features even if you fully controlled the deployment environment of Flash application, as Flash is a binary blob into which you cannot plug-in more parts. There even exists a product for mobile Flash whose sole purpose is circumvent the limitations of  Flash by using a localhost TCP/IP socket connection and a native server application. If developers choose Flash they choose to lock themselves into Flash and what Adobe gives for them.

This crippling of your platform potential with Flash is not limited to Flash Lite and mobile. I have personal experiences from a project done with Adobe Air for Windows. We wisely chose Adobe Air as a desktop application development platform, because it would guarantee the future portability of our code. However, this nice idea did not really interest in the point when we noticed that we e.g. couldn’t control how File Open dialogs behave (file mask, remember start folder, etc.), severely reducing the user experience of the application.

So, Steve Jobs definitely is on something when he says “intermediary translation or compatibility layer” is bad for your platform.

Note that the same limitation concerns Java ME also. Desktop Java has JNI interface for building extensions, but mobile Java doesn’t.

2. How other frameworks differ from Flash

Let’s take a MonoTouch for example. MonoTouch is Novell’s open source Mono project based development tool which allows you to create iPhone applications in C# language. Compared to Objective C, development using MonoTouch  has several advantages for certain audience: you don’t need to learn new programming language, C# is closer to traditional Java/C++ languages than Objective-C, you can leverage the full potential of existing C# ecosystem out there, the standard library has more functionality and of course, it is easier to port the core of the application from a platform to another.

Note that the porting part concerns core i.e. application logic only. MonoTouch does not try to separate you from Apple’s platform. It does not reinvent platform services or user interface building blocks, or force something like Windows user interface into your shiny iPhone. In fact, MonoTouch seamlessly integrates with Apple’s platform. You even need to use Apple’s own Interface Builder tool to create user interfaces, which will be exposed to MonoTouch’s C# code. Binding with native API is breeze: below ten lines of code guaranteeing that whenever Apple releases a new platform feature it will be instantly available for all MonoTouch developers.

MonoTouch embraces the platform. You can pick Objective C or C# depending on taste. The resulting source code is similar in idea, different in syntax. There is no “compatibility layer” so to say. Not even technically, as C# is compiled to native ARM binary. There is no way how a person could distinguish a MonoTouch application from an application build using Objective-C.

3. Open source philosophy and platforms

Appcelerator, PhoneGap and other open source / Javascript frameworks are also protected from Apple’s wrath. They are open source which means that you can tap the full potential Apple’s development platform as long as someones writes a little binding code. Also, they try to use native look and feel and components as much as possible, just to make the applications slick. The frameworks do not have conflicting interest with Apple; the frameworks provide portability to a certain point, but they do this respecting their master.

Flash does not enjoy this freedom; developers can’t change Flash or venture outside Flash’s sandbox.

4. The Future

There is no point of technically counter App Store’s developer agreement, like saying “hey I’ll just compile all my Flash source code to C so it is C code.” It’s Apple’s game. Apple can do anything they wish and they can also change the developer agreement. So when Apple sees something happening in its ecosystem which might damage it, it simply pulls the rug under your feet again. But Apple can also change the agreement in a positive way. PhoneGap has already went through process of becoming App Store approved framework once. I hope that development communities will not burn its bridges with Apple, but try to communicate this matter with a meaningful manner and come to a development agreement resolution based on ideas given in this blog post. Below is my proposition (IANAL. It is left to the reader to come up with something smarter):

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be written using Apple’s guidelines and best practices to express Apple’s intend how the application should behave.

iPhone is not a monopoly. You are free to build and run your Flash application on a device like N900 on any day and sell it in Nokia’s OVI store. If this is a problem for your business then maybe you should reconsider your business model to be less iPhone centric and promote heterogenicity of mobile platforms and application stores.

Until Flash is fixed so that you can mix-in native code, instead of it being a barrier between the expression and the hosting platform, I find it unlikely any company based their reputation around the user experience would allow Flash on their platform.

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Using paster local commands with buildout and avoiding the infamous dependency issue

1) Introduction

Paste script is a part of Python Paste web development utilities.

Its paster create command is used by various Python frameworks to generate skeleton code for a development project.

2) Paster and local commands

Besides generic project templates, paster provides local commands which are project aware commands to add more modules into an existing project. Local commands are made available by paster extensions. For example, ZopeSkel product has various local commands to generate skeletons into Plone/Zope projects automatically

  • Views
  • Content types
  • Forms
  • Portlets

… and so on.

For example, you could generate a project template for Plone add-on module and then create content types there using a local paster command. The local commands become available when you execute paster command in the folder of your project.

Example:

paster create -t archetype myorg.customploneaddon
cd src/myorg.customploneaddon

# Now new paster commands are available
paster

Usage: ../../bin/paster COMMAND
usage: paster [paster_options] COMMAND [command_options]

...

Commands:
  ...

ZopeSkel local commands:
  addcontent   Adds plone content types to your project

Above, ZopeSkel paster template adds its addcontent templates. Now you can use addcontent local command to contribute to the existing project

paster addcontent -t contenttype MyShinyWebPage
2.1. ZopeSkel

For more information how to use paster to create add-ons and add-on submodules for Plone, see here.

To see list of available paster local commands, run paster command

../../bin/paster addcontent --list

… in your development project. For ZopeSkel specific projects the output should be something like this:

Available templates:
    atschema:     A handy AT schema builder
    contenttype:  A content type skeleton
    form:         A form skeleton

3) How paster local commands work

paster reads (evaluates) setup.py file which declares a Python egg. If it founds paster_plugins section there, it will look for local commands there. For example, Plone project templates declare the following paste_plugins in setup.py:

paster_plugins = ["ZopeSkel"]

4) setup.py install_requires

Python modules can dependencies to other modules using setup.py and install_requires section. For example, a Plone add-on might read:

install_requires=['setuptools',
                  # -*- Extra requirements: -*-
                  "five.grok",
                  "plone.directives.form"
                  ],

This means that when you use setuptools/buildout/pip/whatever Python package installation tool to install a package from Python Package Index (PyPi) it will also automatically install dependency packages declared in install_requires.

4.1. paster and install_requires

This is where things usually go haywire.

Let’s assume you are using paster in a project which contains N python packages. You probably use an external configuration system to manage your installed applications and their versions to make repeatable deployments possible (hint: buildout is gaining traction in Python community lately).

Paster is not aware of this external Python package configuration set (paster cannot see them in its PYTHONPATH). So what happens when you try to execute paster create which reads setup.py containing install_requires and encounters dependencies?

Paster will try automatically download and install them locally in that folder.

Plone and Zope ecosystem contains over hundreds of reusable components, in nice dependency hierarchy. paster create would try to pull all them in to your source tree as *.egg folders. See discussion here.

Warning

Do not never use system paster command.

Do not ever run sudo easy_install ZopeSkel. Do not ever run paster local commands using a paster command from your system-wide Python installation.

Warning

The internet is full of tutorial saying easy_install ZopeSkel. If you ever encounter this kind of tutorial, it’s wrong.

5) Paste and buildout

If you are using buildout to manage your Python application deployment, you can integrate paster nicely with it.

Add to your buildout.cfg:

parts =
    ...
    paster

[paster]
recipe = zc.recipe.egg
eggs =
        PasteScript
        ZopeSkel
        ${instance:eggs}

After rerunning buildout, buildout adds paster command to bin folder.

Then you can run paster from buildout folder:

bin/paster

… or in a buildout managed project under src folder…

../../bin/paster

This way paster is aware of your deployment configuration and local commands won’t explode on your face anymore.

Thanks Martin Aspeli to helping with how buildout + paster should be done.

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+