Vote Mr. Moo for Ploneconf2010 speaker and see how get free beers

I am running for a speaker seat in Plone Conference 2010. As a strategy to gain this glamored position I have submitted several topics which might interest you.

Vote here.

  • Backpacking with Plone: how our team has redefined North Lapland travel industry with Plone sites. I promise to bring a genuine Lapland snowball with me and toss it to the first person asking a question.
  • Mad about mobile: How to turn your Plone web site to a mobile site in 15 minutes (* the actual time varies on the status of PyPi and buildouting speed). If you can’t afford iPhone 4 yet, I promise the site will be 100% functional on a 40$ budget phones also.
  • Culture of good documentation: Plone is hated because newcomers cannot grasp how to customize it. I belive this is an attitude problem, not with the newcomers, but with module authors. This manual is now 86k words. If we can make it to go up to 100k words in the conference I’ll buy a beer everyone who participates the effort.
  • The world outside Plone: Why there exist 13 600 000 Joomla! sites, but 20 000 Plone sites? Why single Joomla! freelancer-entrepreneur can make 250k€ a year alone? How we could make Plone more succesfully by learning what others are doing? Our team has some cross-system experience and we want to give you the best bits what others are doing right (and Plone might be doing wrong).

Vote here in the case you missed the link above.

Ps. If I am too boring I promise I will do only PHP/Joomla! jobs in the future, so vote me now or I will forever hold my peace :<

because Plone community has had problems with the attitude

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Insider jokes are a sign of healthy community or….?

Is this the stage when  community starts to becoming a culture?

perrito666: Moo^_^: good evening
Moo^_^: how does it go, my hellish perrito666?
dahoste: Moo^_^, ...wait a min... didn't you "quit" plone?  :)
Moo^_^: dahoste: quit does not belong to my vocabulary
Moo^_^: or should I say, my <vocabulary>
kojiro: bwahaha
kojiro: Moo^_^: I get your Reference. You had no Choice?
-*- kojiro runs
-*- dahoste groans.
davisagli: kojiro: your puns have only Token Value, in my View ;)
kojiro: WYSIWYG.

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

BitReader – Python module for reading bits from bytes

I worked on a project that involved working with MPEG Transport Stream and Digi-TV(DVB). EPGReader to be exact.

The MPEG TS is a binary format, where multiple fields can be defined within a single byte. I could not use Python’s struct module because it only works with bytes or larger and the fields I had were just a couple of bits. First I started with regular bitshifts and bitmasks, but I soon realised it was very error prone task for me. It was very easy to make mistakes and the code was not very readable either.

For example, first 3 bytes of MPEG TS header contains a SYNC_BYTE, which is 1 byte in size and always has the value 0x47. This byte is used to detect packets from the stream. Each packet is 188 bytes long. The next bit is “transport error indicator”, which is set by receiver hardware to flag errors in demulation( analog signal to bits ), next bit is “payload unit start indicator” indicating that the current packet starts a new payload of data, then comes “transport priority” bit and finally “packet id”. Normally I’d write code something like following:

data      = read(3)          # Get 3 bytes of data
sync_byte = data >> 16       # Get bits 24-16
tei       = data >> 15 & 0x1 # 16. bit
payl_start= data >> 14 & 0x1 # 15. bit
tp        = data >> 13 & 0x1 # 14. bit
pid       = data & 0x1FFF    # Get last 13 bits

On top of that I needed to store the values in dictionary. As you can see this is not very readable nor convenient. So I figured there must be something easier.

1. Meet BitReader

spec = (
    # Name of the data to read
    'sync_byte',
    # How many bits to read( 8 bits = 1 byte )
    8,
    'tei',
    1,
    'payl_start',
    1,
    'tp',
    1,
    'pid',
    13
)

reader = BitReader(spec)
data   = reader.read(read(3))
assert data.sync_byte == 0x47

And if and when one needed to add one more byte and couple of variables, that’s when the code starts to break with bitshifts. Any change to the original data size requires you to change the bitshifts accordingly. Also adding new values to the middle requires changes to bitshifts, in case you missed it in the spec the first time etc.

data      = read(4)          # Get 4 bytes of data
sync_byte = data >> 24       # Get bits 32-24
tei       = data >> 23 & 0x1 # 24. bit
etc... not very interested on getting this right, but you'll get the idea

When using BitReader, I just give the variable name and how many bits it takes. Simple as that. No need to touch the other variables.

spec = (
    # Name of the data to read
    'sync_byte',
    # How many bits to read
    8,
    'tei',
    1,
    'payl_start',
    1,
    'tp',
    1,
    'pid',
    13,
    'scrambling',
    2,
    'has_adapt',
    1
    'has_payload',
    1,
    'continuity',
    4
)

reader = BitReader( spec )
data   = reader.read(read(4))

And it doesn’t matter if the new values are added to the beginning, middle or at the end.

2. About performance & syntax

BitReader is a bit slower than using bitshifts, but it was still easily fast enough for the task I worked on. And if compiled using Cython the performance nearly doubles without any code change.

Is it faster? – Performance, no, but you are faster. Have a cup of C if you want speed. Is it more readable? – Yes. Makes life easier? – You bet!

Somebody might look at the specification syntax and quickly note that I could have used dictionary instead. Unfortunately it is not possible because the order is needed and dictionary does not preserve it.

And what about using 2-tuples ( variable, bits )? Is it more readable and less error prone? Not sure, maybe, but I thought I’ll save myself from typing parenthesis 🙂

The specification syntax was inspired by domgen… or the other way around. Can’t remember which came first.

You can also convert the data back into binary format. The read returns a BitData object, which implements ‘dump()’ method, which returns an array.array(‘B’) containing the bytes. You can change the attributes of the BitData and then dump the data back into array and easily write it to a file using array.tofile(f) or send it to network.

A Javascript port might be interesting for web apps… Especially mobile apps, which often have slow connections. And a hand made C module would probably be at least as fast as the bitshifts on Python.

3. Project location

Get the code from bitbucket.

Samsung Galaxy S i9000 and Android 2.1 – enable file copy over USB as mass storage mode on OSX and Linux

Samsung Galaxy S i9000 (Android 2.1) comes with Samsing Kies Windows software which is supposed to be used for copying files between your computer and the phone.

Naturally, this Windows software 1) is not very good 2) works on Windows only.

However, you can still mount  the device as normal USB mass storage. Just set Settings -> Applications -> Development -> Allow USB debugging checkbox on and replug the USB cable. Then open the notifications menu (pull down the top bar on the home screen). Now there is new option USB Connected. Choose it and then choose Mount.

You should be able to access both the phone internal storage and microSD memory card in Finder / Nautilus.

I created directory /media/mp3 where I copied few music files from my computer. I was able to achieve 5 MB/s transfer rate from my Asus Eee 1005HA PC when writing files to phone internal memory. The phone music player was able to pick up them fine.

More info

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+