Monthly Archives: 2012-10

Converting community black knights to white knights

Posted on by
Reply

This is how it happens in IRC:

<x^> html5 sux
<moo-_-> x^: thank you for sharing this information. how can we make it better for you?
<x^> please improve html5
<x^> make it using xaml
<moo-_-> x^: if you wish to contribute to the development of HTML5 or related standards a good starting point for doing it would be on WHATWG mailing lists
<x^> moo-_-, can i actually do that?
<x^> great :D
<moo-_-> x^: try here http://www.whatwg.org/
<x^> ok thx

Tools

  • Politeness
  • Mutual respect
  • Giving opportunity
  • Respecting individual
  • Lowering your noise floor and putting more effort to signal decoding

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

SSH key and passwordless login basics for developers

Posted on by
18

SSH keys are useful to login over ssh (secure shell) without typing a password. They are also used by Github and other version control systems for passwordless authentication. Here is some basic information from the software developer point of view how to use SSH keys for maximum comfort and security.

These instructions apply for OSX and Linux (tested on Ubuntu).

SSH keys are not used only by shell sessions, but also by remote file copy (rsync, scp) version control system authentication (Git, Subversion).

Table Of Content

1. Creating your first key

2. Storing the server (UNIX) passwords

3. Private and public key file

4. Managing and using keys with passphrases

5. Desktop and login integration

6. SSH agenting and passwordless git and svn

7. Online compression of the SSH traffic

8. Configuration files

9. Tunneling

10. Freeing yourself typing password for sudo

1. Creating your first key

SSH keys are created using ssh-keygen command.

You should (really must!) add a passphrase on your key when asked. You don’t need to be typing this, but it prevents someone using your keys from a cold storage like stolen hard-disk. See more information below how SSH keys are integrated to your desktop computer login for avoiding passphrase typing.

2. Storing the server (UNIX) passwords

On the first login to a new server change your UNIX / SSH password to something random using passwd command. You are going to need this password for sudoing, but not for login. You can use software like KeePassX (OSX, Linux, Windows, Android, others) to generate and manage passwords for you in an encrypted file and then sync the file to Dropbox for back-up.

3. Private and public key file

SSH keys consist of two parts. You have SSH private keys only on your personal computer. You place the public key file on a remote server, in a home directory in plain-text file ~/.ssh/authorized_keys. In this file, one line represents one allowed public key for this user to login.

You can add the keys to this file by hand editing the file or using ssh-copy-id command (Ubuntu has it by default, download for OSX).

Example of login on the server with password for a test, placing a key (github-pkunk is the private key file name) on the server (you need to create it first using ssh-keygen) and then doing a passwordless login:

4. Managing and using keys with passphrases

You can have multiple keys. For example I have one for Github and one for corporate servers.

After each local computer boot you add the private keys to your running SSH agent (a local daemon application) using command ssh-add .ssh/my-private-key-name This will ask you to type the passphrase of the key.

After this you can login to any server where you have placed the corresponding public key without a password.

You can add / change passphrases to they private keys like this:

                                                                                                                                                                                                                                                    [moo@Kohr-Ah][23:23]
[~/.ssh]% ssh-keygen -p -f github-pkunk
Key has comment 'github-pkunk'
Enter new passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved with the new passphrase.

(More info)

5. Desktop and login integration

On OSX, adding keys to the SSH agent can done automatically by OSX Keychain on login. OSX Keychain stores the passphrase of the key on an encrypted storage unlocked by your login on the computer. The default key ~/.ssh/id_rsa is added automatically by OS, but you can add more keys to Keychain like this:

[~/.ssh]% ssh-add -K github-pkunk
Enter passphrase for github-pkunk: 
Passphrase stored in keychain: github-pkunk
Identity added: github-pkunk (github-pkunk)

The latest Ubuntu / Gnome keyring seems to be able to do this also, but I am not sure about the details whether the process is the same.

(More info)

6. SSH agenting and passwordless git and svn

Never copy your private key anywhere else besides 1) your own computer 2) back-up. You should never place your private key on a server controlled by a third party. If you need to access SSH keys on a remote server, for example when you need to do a git pull from Github, use SSH agent forwarding option.

SSH agent forwarding allows the remote server shell session to use keys from your local computer. It is enabled with ssh -A command line switch or in an option in the configuration file (see below). When enabled, all keys registered with ssh-add will be available on the remote server.

Subversion supports SSH. This means that, when enabled, you can do svn up on a remote server without need to give or store your SVN password on the server. The svn+ssh protocol support for Subversion must be enabled and it may not provide per-user repository access control, as oppose to https protocol with Apache.

7. Online compression of the SSH traffic

Enable compression when working with servers far-away with low bandwidth. See below how to enable this in the ~/.ssh/config file.

This will somewhat increase the speed of long command output like ls.

8. Configuration files

Store the servers you access often in .ssh/config file. You can store per-server options like enable agent forwarding, server alias (shorter to type) and  default username. Here is an example snippet from my config:

Host xapsi 
Hostname lakka.xapsi.fi
User miohtama # This is my username on the server
LocalForward 8889 localhost:8889 # Build tunner for Quassel IRC core
Compression yes # Enable compression
CompressionLevel 9
IdentityFile=/Users/moo/.ssh/foobar # Always use this key

The configuration file also enables the tab competion on some shells (zsh examples). With the above snippet in the config I could do:

ssh kap[TAB][ENTER]

Login with 6 letters – no passwords or usernames asked!

9. Tunneling

SSH key can forward TCP/IP ports between the server and your local computer. This is useful e.g. testing firewalled servers from a local computer or give a public IP / port for your local development server.

More info about SSH tunneling.

10. Freeing yourself typing password for sudo

For the state of the art UNIX system the direct root SSH account is disabled for the security reasons. Instead you ssh in as a normal user and then elevate your privileges using sudo command.

Normally sudo is configured to ask password for every time sans sudo grace period. However if you primarily use SSH keys as your security measure it is recommended that

  • You randomize and store the real UNIX passwords in an encrypted safe as described above
  • You put sudo users to a specific UNIX system group (on Ubuntu this is group admin, but also wheel is used on some systems)
  • Give passwordless sudo access for this group and this group only

Example how to add an account to a specific group on Ubuntu Linux:

usermod -a -G admin moo

Then you can whitelist this group in /etc/sudoers file:

%admin ALL=(ALL) NOPASSWD: ALL

The recommended safe way to edit /etc/sudoers file is visudo command. Example:

 export EDITOR=nano && sudo visudo

Please note that you MUST use only passphrase protected private keys or otherwise anyone getting access to a private key file can get root on your server.

Please note that you should give passwordless sudo only on specific users on your server. Otherwise in the case of a compromised web server (PHP anyone?) the attacker could get root access through www-data or other UNIX system account.

Please share your own tips in the blog comments.

 

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

Opening files from Firebug in Sublime Text 2 or any text editor

Posted on by
2

I have the following use case in my web application development workflow

Effectively you right click any source code line in Firebug on any web server and you can open the corresponding source code line it in your favorite text editor. This also works for Javascript stack traces.

Firebug has an external editor feature. However the external editor configuration does not support mapping URLs to files. Since I do stuff like WebGL and AJAX I cannot work with file:// protocol, but I use Python’s SimpleHTTPServer as a development server. I had to figure out a way to map URLs (http://localhost:8000) from Firebug to my local files on the hard drive (/Users/mikko/code/xxx) .

Thus, I created the Python script below. It takes four arguments URL (supplied by Firebug), line number (supplier by Firebug), base URL and base directory to map URLs to files on the hard disk. You can create several Firebug external editor configuration entries for different URLs.

On OSX, Firebug is picky and does not allow pass arbitrary UNIX binaries as external editor. Thus, you need to wrap the script using py2app or download the ready firebug-subl app (Mountain Lion) from Github and drop it in /Applications.

I also run for the two following bugs in Firebug 1.10.4 (not reported by me yet)

  • Cannot open the external editor if the URL starts with http://localhost (see the script below how to configure /etc/hosts for a workaround)
  • Cannot open the external editor in some cases if there are spaces in the command line (had to use | as the separator)

… caused some hair pulling.

Firebug’s external editor configuration menu can be hard to find:

And here is a sample config:

Further instructions in the script itself.

Note that the script outputs to /tmp/firebug-subl.log as there didn’t seem to be a way to get console output from the external editor otherwise. If this file doesn’t get created Firebug doesn’t even try to run the script (or the wrapper app) as with the bugs mentioned above.

And then the cake, firebug-subl.py. A possible updated version will appear on ztanesh repo on Github:

"""

    Firebug - Sublime Text 2 connector.

    Allows you to open any Firebug files in Sublime Text 2 text editor.
    Maps URLs to files.

    Usage::

        firebug-subl [url]|[line-no]|[base-url]|[base-directory]

    Note that we use pipe separation because of Firebug 1.10.4 seems
    to have a bug with space separation.

    Example configuration line in Firebug editor settings::

        %url|%line|http://firebugbugs:8000|~/code/mixnap-base/krusovice-src

    ... this will open all files from http://firebugbugs:8000 as they
    were on the directory /Users/mikko/code/mixnap-base/krusovice-src on the disk.

    Note: Firebug seems to have a bug that if the domain name in the URL
    is localhost it doesn't even try to start the editor. Thus you need
    to resolved /etc/hosts tricks to spoof your localhost with some other
    name if you run a local development server::

        127.0.0.1   localhost firebugbugs

    Shell expansion supported.

    Starting Firefox from command-line for debugging::

        /Applications/Firefox.app/Contents/MacOS/firefox

    Because of Firebug's Select application retardness this script must be wrapped with py2app on OSX
    before Firebug allows you to pick this script as a legal choice.
    In clean virtualenv::

        # We need py2app trunk version for OSX Mountain Lion as the writing of this (altgraph > 0.10)
        # First install hg command (mercurial)
        pip install setuptools-hg
        pip install -e hg+https://bitbucket.org/ronaldoussoren/altgraph#egg=altgraph
        pip install -e hg+https://bitbucket.org/ronaldoussoren/macholib#egg=macholib
        pip install -e hg+https://bitbucket.org/ronaldoussoren/modulegraph#egg=modulegraph
        pip install -e hg+https://bitbucket.org/ronaldoussoren/py2app#egg=py2app
        py2applet firebug-subl.py && cp -r firebug-subl.app /Applications

"""

__author__ = "Mikko Ohtamaa <http://opensourchacker.com>"
__license__ = "MIT"

import os
import sys
import subprocess
import logging
import urllib

# Write debug output to a file as we don't otherwise get any feedback
# if this script fails
logger = logging.getLogger("firebug-subl")
hdlr = logging.FileHandler('/tmp/firebug-subl.log')
logger.addHandler(hdlr)
logger.setLevel(logging.INFO)

# Add your installation here if missing
LOCATIONS = [
    "C:\\Program Files\\Sublime Text 2\\sublime_text.exe",
    "/home/ed/apps/sublime_text_2/sublime_tex",
    "/Applications/Sublime Text 2.app/Contents/SharedSupport/bin/subl"
]

def guess_subl():
    """
    Guess the editor location.
    """
    for loc in LOCATIONS:
        if os.path.exists(loc):
            return loc

    return None

def main():
    """
    Main magic.
    """

    logger.info("Got command-line: %s" % sys.argv)

    # Where subl lives?
    editor = guess_subl()
    if not editor:
        logger.error("Could not find Sublime Text 2")
        return

    # Read command-line
    # Here we need to have some hacks as for some reason space
    # space separated command line was broken on Firebug 1.10.4
    mega_arg = sys.argv[1]

    # Seems to be urlfied...
    mega_arg = urllib.unquote(mega_arg)

    url, line, base_url, base_dir = mega_arg.split("|")

    # Map Javascript file location from URL to a file on a disk
    path = url.replace(base_url, base_dir)

    # Replace tilde with the user home dir
    path = os.path.expanduser(path)

    if not os.path.exists(path):
        logger.error("Tried to open non-existing file %s - please check your URL-directory mapping" % path)
        return

    # Create a Sublime Text 2 style direct line in a file pointer
    hint = path + ":" + line

    logger.info("Launcing %s with %s" % (editor, hint))

    # Call Sublime Text to open the file in the current project.
    # Create process with shell variable expansion
    subprocess.call([editor, hint], shell=False)

try:
    main()
except Exception as e:
    logger.exception(e)

–

 

 

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+

True lies and falsy values in Python and Javascript

Posted on by
4

In Python and Javascript programming you can compare different values against true or false even if they are not boolean values as is.  By knowing this, writing some code paths become simpler.

The most used example is None in Python (None would be null in other programming languages):

>>> spoon = False
>>> print "Porridge time" if spoon else "There is no spoon"
There is no spoon

>>> spoon = None
>>> print "Porridge time" if spoon else "There is no spoon"
There is no spoon

None is not same as False in Python, but if truth comparison treats it is as were.

In Python, the values which have falsy comparison include, but are not excluded to

  • empty string
  • empty lists and tuples
  • number zero

Further examples:

>>> spoon = ()
>>> print "Porridge time" if spoon else "There is no spoon"
There is no spoon
>>> spoon = []
>>> print "Porridge time" if spoon else "There is no spoon"
There is no spoon
>>> spoon = ""
>>> print "Porridge time" if spoon else "There is no spoon"
There is no spoon

So it actually doesn’t make then sense to test both for None or empty string as you could get away with one condition.

>>> banana = ""
>>> print "I am banana" if (banana != '' and banana is not None) else "Muttia miehelle"
Muttia miehelle

The above if could have been written just as if banana.

As a word of warning, some framework classes may implement their own __eq__() and related method causing surprising when doing “if something:” and then expecting it to check for None values only. In these cases, you need to explicitly use is operator to compare against None. is tests if the values point to the same object (a reference comparison) and there exists only one None object in Python. Example:

>>> banana = ""
>>> print "I am banana" if (banana is not None) else \
    "Should not be reached as we explicitly used is comparator"
I am banana

Sometimes you need to process the values which can be string or None, depending on what your favorite form framework have spit out. Then you can use the following or comparison trick to shorten the code not to have extra if: If the left side of or evaluates to false like value (None) then the right side is used. We can use this to force falsy values to be empty string always. Example:

>>> string_like_value = None
>>> print string_like_value.strip()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'strip'

>>> string_like_value = None
>>> print (string_like_value or "").strip()

>>> string_like_value = "  foobar  "
>>> print (string_like_value or "").strip()
foobar

Unlike Python, Javascript does not raise error when accessing a missing attribute on an object. Instead, accessing an undefined variable gives you back undefined value which equals to false in boolean comparisons.

undefined values result to hilarious debugging sessions as a mistyped variable does not cause error or stack trace where it is being accessed, but somewhere deeper in the code not related to the place of an actual error. You can mitigate the risk of this by spreading asserts all around your code to check possible bad function arguments in your own code.

However, undefined as a perverted language feature can be exploited for creating a configurator design pattern (not sure if this pattern has any better name?) as shown below. In Javascript, this pattern is commonly used to give configuration parameters to library functions. Several values are read from layered configuration objects and the highest set value is used – falling back to the defaults if no value is given.

var globalConfig = { x : 3 }
var foobar = function(config) { 
    this.x = config.x || globalConfig.x || "some default value if even global config is missing"; 
    console.log(this.x); 
}
// No x given, so function above should use 
// the default x value from globalConfig
foobar({y:4});
3

Javascript also features === and !== comparisons which equals to Python is comparison. This can be used to check against undefined explicitly.

>> foobar = "";
>> if(!foobar) console.log("Has no foo");
Has no foo

>> foobar = undefined;
>> if(!foobar) console.log("Has no foo");
Has no foo

>> foobar = "";
>> if(foobar !== undefined) console.log("Has no undefined foo");
Has no undefined foo

To make things even more hilarious, undefined could be redefined before ECMAScript 5…

Please share your favorite boolean like comparison trick 🙂

Then, True Lies (1994):

\"\" Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+