2010 January | Open Source Hacker

Python 2 has a thing called “default encoding” to automagically encode Unicode strings when they are presented as byte strings. This is evil and has been discussed various times before.

What could be even more evil? Something in your development environment messes this setting set for you, without telling you that. This way you never encounter Unicode problems on your development computer and when you roll out your seemingly working code to production, the world goes haywire.

Evil. Evil. Evil. Thousands of curses and overworking hours to fix the problems.

I encountered this problem. And this is the code I used to track the problem down in site.py:

# Trap the bastard messing with the default encoding
# using a monkey patch
old_set_default_encoding = sys.setdefaultencoding

def aargh(x):
    import pdb ; pdb.set_trace()

sys.setdefaultencoding = aargh

And the result was surprising:

--Return--
> /home/moo/py24/lib/python2.4/site.py(485)aargh()->None
-> import pdb ; pdb.set_trace()
(Pdb) bt
/home/moo/py24/lib/python2.4/site.py(613)?()
-> main()
/home/moo/py24/lib/python2.4/site.py(604)main()
-> execsitecustomize()
/home/moo/py24/lib/python2.4/site.py(514)execsitecustomize()
-> import sitecustomize
/home/moo/Desktop/Aptana Studio 2.0/plugins/org.python.pydev_1.5.3.1260479439/PySrc/pydev_sitecustomize/sitecustomize.py(99)?()
-> sys.setdefaultencoding(encoding) #@UndefinedVariable (it's deleted after the site.py is executed -- so, it's undefined for code-analysis)
> /home/moo/py24/lib/python2.4/site.py(485)aargh()->None
-> import pdb ; pdb.set_trace()
--Return--> /home/moo/py24/lib/python2.4/site.py(485)aargh()->None-> import pdb ; pdb.set_trace()(Pdb) bt  /home/moo/py24/lib/python2.4/site.py(613)?()-> main()  /home/moo/py24/lib/python2.4/site.py(604)main()-> execsitecustomize()  /home/moo/py24/lib/python2.4/site.py(514)execsitecustomize()-> import sitecustomize  /home/moo/Desktop/Aptana Studio 2.0/plugins/org.python.pydev_1.5.3.1260479439/PySrc/pydev_sitecustomize/sitecustomize.py(99)?()-> sys.setdefaultencoding(encoding) #@UndefinedVariable (it's deleted after the site.py is executed -- so, it's undefined for code-analysis)> /home/moo/py24/lib/python2.4/site.py(485)aargh()->None-> import pdb ; pdb.set_trace()

Looks like the culprint was PyDev (Eclipse Python plug-in). The interfering source code is here. Looks like the reason was to co-operate with Eclipse console. However it has been done incorrectly. Instead of setting the console encoding, the encoding is set to whole Python run-time environment, messing up the target run-time where the development is being done.

There is a possible fix for this problem. In Eclipse Run… dialog settings you can choose Console Encoding on Common tab. There is a possible value US-ASCII. I am not sure what Python 2 thinks “US-ASCII” encoding name, since the default is “ascii”.

Begun, this mobile browser war, has. When mobile internet is growing 8x faster than desktop internet everyone wants to have a share of it. In the core of this fight is the mobile browser – the doorway to the mobile internet.

Usually phone comes with a browser from the phone manufacturer: Safari ships with iPhone, Android ships with WebKit based browser and Maemo comes with Nokia microB. Besides the default browser, open platforms have seen third party browsers created for them: Opera Mini has 30 million users and several browsers have been created for Symbian platforms. (Note that iPhone is not really open platform regarding this as Apple developer terms specifically forbid creating alternative browser engines for their Safari – all iPhone “browsers” are just the same Safari with new toppings).

Now Mozilla foundation is releasing Firefox Fennec (RC1 version is available for Nokia N900), touted as the most innovative mobile browser this far. New user interface ideas, desktop syncronization and vibrant add-on community are something yet to be seen for mobile browsers. Mozilla did an amazing thing with Firefox when it actually managed to push Internet experience forward and compete against Microsoft’s bundled Internet Explorer with sheer quality. Can Mozilla repeat the same thing it did for desktop browsing for mobile browsing too?

Is Fennec good? I installed the release candidate and conducted some tests by visiting on popular sites. It is especially fruitful to compare Fennec against Nokia’s own microB browser as they both are based on the same Gecko rendering engine beneath the hood.

The differences of the browsers are, actually surprisingly, not limited to branding and user interface shell. Fennec is portable browser – Mozilla hopes to run Fennec on other mobile platforms beside Maemo in the future. Fennec user interface is based on Mozilla’s XUL library and you can actually run Fennec on your desktop computer too. Nokia’s interest, on the other hand, is have an optimized browser for their own mobile phones: microB user interface is using native Maemo user interface components.

Below are some aspects of the browsers compared against each other.

1. Start up time

microB: instant
Fennec: About ten seconds (warm start-up is little bit faster, but it is still slooooow….)

This is a pain for Fennec. Loading all that XUL Javascript needed to run Fennec is just too much. You really don’t want to start Fennec for a quick browsing session, unless you have the patience of a cow. I am not sure whether N900 keeps microB loaded on the background all the time or what’s causing the difference.

2. User interface

This is really where Fennec shines. Nokia enjoys some reputation of being a boring engineer house with little innovation left to stir. After learning the trick of left and right sweep, which is cleverly demostrated on the start page, Fennec user interface instantly feels intuitive. microB, on the other hand, uses somehow clumsy “bottom right corner full-screen button” to access buttons and left-right sweep is not very well thought. For example, switching a tab/browser window takes three “clicks” on microB (show menu – switch application – choose next browser window) when Fennec does it with one sweep and click. Also, backward navigation is much more intuitive on Fennec and takes too many gestures on microB.

Both browsers have search integrated to the navigation bar. Fennec start screen is more clever, showing the history and shortcuts, while microB shows only the bookmarks. Fennec navigation bar also is a combination of title and navigation bar, saving the precious screen estate on small physical form factor. Fennec zooms to text fields automatically when you start to input text into them and also have soft “tab keys” to navigate to next and previous input field.

3. Page reading and speed

On sites with above average layout complexity, Fennec is unbearable slow compared to microB, up to the point the browser is next to unusable in its current incarnation. As they both use the same rendering engine, I have hard time to understand how microB manages even the heaviest dynamic pages (Facebook profile page) when Fennec becomes unusable even on a moderate complex page (slashdot.org).

The thing with Fennec is that for some of the the time it does not register your interaction and does not have any indicator showing if it is responding – it has grinded to halt, little bit like desktop computer when swapping. And even when Fennec is responding the scrolling of the page refreshment is sluggish compared to microB. This makes the page reading experience unusable. A normal user won’t stand 1-3 second frequent responsivity pauses or page movement which cannot be controlled.

microB must do the rendering somehow different – is it hardware acceleration on font rendering, smarter management of images or some other trick?. However, until Fennec reaches the smoothiness of microB, there is no way I would switch to Fennec over microB.

(Note: You can press CTRL-Backspace from N900 keyboard to force application switch if you cannot exit from halted Fennec otherwise)

4. Mobile browsing

Though N900 has 800 pixel wide screen, it is still a mobile phone. Small physical size, low bandwidth with high latency and limited CPU power might make you to pick a mobile internet version of the site when it is available. However, since the screen has exceptional high Dots-Per-Inch value, this poses a problem for rendering sites with the default font sizes.

Fennec does not seem to have a shortcut for setting a large text size. This is something one would hope to see on such high DPI device as the most of the time default web site fonts are too small to be usable. Also, Fennec does not use the shoulder plus and minus volume buttons for zooming – microB does it and it is very natural place for this function.

Fennec seems to have some difficulties with mobile site rendering: for example touch.facebook.com and yle.mobi are not scaled to full width. Instead a narrow colum of 1/3 screen width is displayed.

5. Bugginess

microB is very solid piece of software. It crashes more rarely than Safari on iPhone (might this be because of more memory – low memory conditions seem to be a normal crashing condition for Safari?). Fennec is still in its first version and have some issues.

(Note: I managed to get Fennec to zombie state – I had to go to terminal and type killall fennec command to make the browser become launchable again).

6. Sites tested

Slashdot.org

Geek discussion site

microB: no problems

Fennec: slow, frequent pauses, not smooth scrolling

slashdot.org/palm

Very simple mobile version of the above.

microB: Font too small

Fennec: Scales correctly

Facebook.com

High profile social networking site

microB: Sometimes little slow, but seems to work perfectly

Fennec: Unusable slow

touch.facebook.com

microB: Perfect (at least when scaling font up a little)

Fennec: Does not scale correctly (default scale uses only 1/3 of screen width, double click zooming scales too much)

yle.fi

Finnish national broadcasting company site

microB: Ok. Readable and usable with text size large.

Fennec: Ok. The default view is navigable, but not readable. You need to double-click zoom to read the text (Fennec doesn’t seem to have text size large option)-

yle.mobi

The mobile version of above.

microB: Perfect with text size large, ok otherwise (need to double click to zoom and then click to choose a link to follow).

Fennec: Ok – font size too small

GMail HTML version

The default Javascript version of GMail is too heavy for both the browsers. GMail still provides “Basic HTML” view as the fallback for devices with less CPU power and network bandwidth.

microB: Ok – you can do some basic emailing

Fennec: Ok. Does not seem to be affected by as much of “slugginess” as other sites are (might the slugginess be a Javascript issue?)

Youtube.com

The web version of flash based video sharing site.

microB: Plays Flash movies ok – smooth scrolling even whilst a Flash movie is playing

Fennec: Frequent grinds to halt, sluggish, unusable. Manages to open Flash video, though.

m.google.com/youtube

The mobile version of above.

microB: Youtube claims the browser is unsuppoted

Fennec: Cannot enter the site – shows only the page of Youtube Mobile instructions

twitter.com (web site)

microB: Perfect

Fennec: Ok. Sluggish when opening new pages, but still usable. Fennec start view ships with Twitter button, so one might assume this site is well tested for Fennec.

m.twitter.com

The mobile site of above.

microB: Ok – the default font size too small, but when settings text size large works well

Fennec: Ok – the default font size too small. Double click zoom does not work well on the twit feed, making reading difficult.

plone.org

A community site with relatively simple layout.

microB: Ok – minor rendering errors

Fennec: Ok – minor rendering errors

iltalehti.fi

Finnish tabloid web site with lots of images.

microB: Ok

Fennec: Grinds to halt, unusable slow

7. Summary

Though having nice promise of innovation, the advise for Fennec development team would be “back to the basics”. The slugginess and response times of Fennec are such an issue that one would not yet consider it as an real alternative for Nokia’s default microB browser.

With Fennec’s user interface and microB’s speed one could have a near perfect mobile browser. Depending what kind of future co-operation Nokia and Mozilla foundation will have, we might live to see it.

Open Source Hacker

Pushing the boundaries of free technology

Monthly Archives: 2010-1

PyDev, Python and system default Unicode encoding problem

Creating a drag’n’drop basket with jQueryUI

Mobile browser wars: Nokia microB vs. Firefox Fennec