New Blog

OK so Blogger are now officially shutting down FTP support. That's how I publish this blog. I understand why they're doing it, makes sense, but I don't intend to migrate this blog to their servers.

So, I won't be posting here again, but that's OK because I stopped posting anyway. I have a new blog at jouire.com these days, so update your feed readers and bookmarks accordingly.

Towards a manageable feed reader

3 years ago I made a comment on a 37signals post: Taming the RSS beast.

I was just editing and expanding it in an email and thought I'd post it here instead.

Most people group their feeds by topic. Here’s a suggestion that helps me manage my feeds more effectively: Group your feeds by a small number of categories that match your behaviour, rather than subject matter. These are my categories:

• Quick – anything that’ll take me two seconds to read (photos, one liner blogs)
• Links – link blogs (these often lead to heavy tab spawning)
• Light – blogs that generally consist of shortish pithy posts.
• In-depth – blogs that usually have long in-depth posts
• Me – ego stuff: my flickr comment feeds, twitter mentions etc (in case they slip through my twitter client)
• Friends - people I have a stronger emotional connection to and care more about
• Announce – svn commits, web host status feeds, app release feeds

Grouped like this, it’s easier to pick off the low hanging fruits and slowly chip away at the backlog rather than sticking rigidly to subject taxonomies that are usually inadequate and give you no sense of how far through the job you are or about what you actually care about.

Try to put every new feed you subscribe to in one of those categories, and don't create too many new ones or it starts to become unmanageable. Then collapse as many as you can so it doesn't look like you have too much to go through.

NEVER go through all your feeds as one big bucket of unread items.

Other than that, a hard trick is to actually notice when something is no longer interesting. This is hard because we don't like to admit we made a mistake, and will often stick with things for much longer than usual after making an initial value judgment and have invested in something. This is the "sunk cost" mind hack and is very powerful if you can pull it off.

My feed reader already marks feeds that haven't updated in a while as "Dinosaurs". If it also marked those that I hadn't clicked through from in a while as fluff that'd be ace.

'import site' failed; use -v for traceback

If you suddenly start seeing this error every time you try to run Python:

'import site' failed; use -v for traceback

Chances are you have an erroneous file in your site-packages directory that's screwing up Python's import path initialisation and could lead to you being unable to import some installed modules.

To figure out which file this is you can use the Python debugger, pdb:

> python -v
>>> import pdb
>>> pdb.pm()
> [python_lib_path]/posixpath.py(173)exists()
-> return False
(Pdb) u
> [python_lib_path]site.py(146)addpackage()
-> f.close()
(Pdb) f
<closed file '[site-packages]._blah.pth', mode 'rU' at 0x52c38>

Your actual output will be quite a bit different with a lot of debugging information spewed into the console but let's explain what's going on here:

1. Run Python in verbose mode to get a traceback.
2. import pdb
3. Run the pm (postmortem) function to enter the (Pdb) shell and inspect the exact point at which the error occured.
4. Go up a level in the stack with u.
5. Inspect the f file object.

This tells us that ._blah.pth is causing Python's site import routine to choke. [bug report]

Move this file out of site-packages and try again, repeating the steps above if you get the same error.

When you run Python, it runs through all the directories specified by your PYTHONPATH environment variable, looking for .pth files. These can contain further directories to be added to the import path and are typically created by python packaging systems like easy_install. [more info on .pth files]

So what's up with this one?

The so called "dot-underscore" or AppleDouble files are actually created by Mac OS X. They contain resource forks and extra file metada for their corresponding file (the example we found earlier contains metadata for a legitimate file blah.pth).

Mac filesystems like HFS+ support resource forks natively and don't need these files, so they aren't usually created when you're accessing your machine locally. They can get created when you're accessing it over a network using the Samba protocol or if the connecting Mac otherwise thinks you don't support resource forks—usually if you're on Windows or an NFS filesystem.

You can safely delete these files from the site-packages directory and Python should be back to normal.

Safari 3.2.1 bug: Random unicode characters munged into select drop down

I recently encountered a very bizarre browser bug involving a <select> drop down in Safari 3.2.1.

Changing the selection was resulting in random characters overwriting the text contents of the selected <option> element.

Since this behaviour wasn't happening in the latest version of WebKit, my bug report was resolved WORKSFORME.

This wasn't particularly helpful as I still needed to figure out a workaround for a bug that was affecting people using the current shipping version of Safari.

I eventually managed to isolate a minimal test case here:
http://james.wheare.org/stuff/bugs/safari/fixedselect

The setup looks like this:

<div id="relative" style="position: relative;">
    <div id="fixed" style="position: fixed; top: 0; left: 0;">
        <select>
            <option>Choose another option</option>
            <option>Garbled</option>
        </select>
    </div>
</div>

The bug is triggered by javascript:

// Set the fixed container's positioning to absolute
var element = document.getElementById('fixed');
element.style.position = 'absolute';
// Access its clientWidth or clientHeight properties
element.clientWidth;
element.clientHeight;
// Restore the positioning to its original value of fixed
element.style.position = 'fixed';

Now, just select a different option from the drop down and it's text content will be munged with random characters.

This may seem like quite a strange sequence of operations but it's actually used in the Prototype library's Element.getDimensions method [source].

When I discovered this bug, it was additionally causing Safari to crash if you continued to use the drop down. This test case doesn't trigger the crash, but it's almost certainly the same root cause.

There a few options to workaround this issue, depending on your particular situation:

1. Avoid Element.getDimensions.
2. Modify Prototype's Element.getDimensions function.
3. Change the positioning of your containers.
4. Wrap the select in another position: fixed element and don't call Element.getDimensions on it.
5. Wait for Apple to release a new version of Safari.

If you choose to modify Element.getDimensions, there's a handy way to do that without altering the Prototype source code. Simply define your own version and then extend Element with it by using Element.addMethods. This will override the built in method.

Element.addMethods({
    getDimensions: function(element) {
        // Your version in here
        // ...
    }
});

If you're wondering how to patch Element.getDimensions, I forked Prototype on github with a fix for this issue:
Fix for Element.getDimensions

I also filed a ticket for Prototype to add a fix for this themselves:
Element.getDimensions causes issues in certain conditions in Safari
Update: This fix has been accepted into Prototype and should make its way into a future release.

Ordering related objects in Django

Django models have a meta option order_with_respect_to that allows you to order objects within the scope of a related ForeignKey object.

This option adds an _order column to the model's database table to keep track of this ordering.

That's all very well, but if you need to change the sequence order you may well feel at a loss. The official documentation makes no mention of this option beyond a basic explanation of its purpose , and editing your objects in the built in admin app reveals no user interface for changing the ordering either. [1]

Update Feb 2009: There's now a ticket for this functionality to be added to the docs.

I managed to uncover a Python interface by delving into Django's API internals [2]. An object with related models that order with respect to it is given two handy methods:

get_RELATED_order()
set_RELATED_order()

where RELATED is the lowercased model name of the ordered objects

class Entry(model.Model):
    # ... fields

class Comment(model.Model):
    entry = model.ForeignKey(Entry, related_name='comments')
    # ... more fields
    
    class Meta:
        order_with_respect_to = 'entry'

Given the above example models, you can retrieve the order of an entry's comments:

>>> entry = Entry.objects.latest()
>>> entry.get_comment_order()
[1, 2, 3, 4]

And change the ordering by passing a list of comment ids back in.

>>> entry.set_comment_order([2, 1, 4, 3])

N.B. Be sure to pass in the same ids returned by get_comment_order

Two other handy methods exist for the Comment objects, get_next_in_order and get_previous_in_order [3]

>>> comment = Comment.objects.get(pk=3)
>>> comment.get_next_in_order()
<Comment: 4>
>>> comment.get_previous_in_order()
<Comment: 2>

1. A very early ticket is still open to restore this functionality to the admin app, and recent activity suggests it may well reappear soon. See also this thread on the django-developers mailing list. back


2. Look particularly at the two helper functions method_set_order and method_get_order in django.db.models.base. back


3. Theoretically at least, there's still a bug in get_previous_in_order but it should be fixed soon. Update: Fixed by the queryset-refactor back

Barcelona 11/07 Photos

Bus times back online

LiveBus.org is currently the first hit for "mash up offline" on Google. Not a particularly delightful situation, but I'm happy to say that the DeadBus has risen!

I've moved the site to a sultry new virtual private server at RimuHosting and real time bus updates are flowing once more. You shouldn't notice a difference, but I've now got a lot more freedom to tweak performance behind the scenes, and future downtime of this sort should be rare.
Disclaimer: Rare, not impossible

Do let me know if you're seeing funny stuff, things may have become muddled in the move.