del.icio.us direc.tor: Delivering A High-Performance AJAX Web Service Broker
June 20, 2005
What is it?
del.icio.us direc.tor is a prototype for an alternative web-based rich UI for del.icio.us. It leverages the XML and XSL services of modern browsers to deliver a responsive interface for managing user accounts with a large number of records.
The main features are:
- In-browser handling of del.icio.us bookmarks (tested up to 12,000 records)
- Find-as-you-type searching of all your bookmarks, with basic search operators
- Sort by description, tags, or timestamp
- Ad-hoc tag browser
How do I use it?
- Create a bookmarklet by bookmarking the following link:
- Go to api.del.icio.us
- Launch the bookmark you just created while you are still on the del.icio.us page
- Login to del.icio.us, if prompted
(Try the static demo if you don't have a del.icio.us account.)
- Type in a search term or click a tag in the browser; click the column headings to sort
direc.tor supports the following operators:
|Search only in tag field|
|Search only in description field|
|Exclude results containing search term|
Combining operators, like
-t:nonsense, is currently not supported.
How does it work?
Other brokering services like the Google Maps hack are not entirely self-contained and require the broker host to proxy information between the main server and the client, thus doubling the amount of network traffic and degrading the overall performance. direc.tor eliminates the need for the broker host to proxy requests by instructing the client to directly communicate with the main server. This approach is very similar to the way Greasemonkey scripts are loaded, except that it is largely platform independent and does not require additional client-side extensions like Greasemonkey. However, the major pitfall to this approach is that users are required to manually create the bookmarklet.
In a standard service, the Client Browser makes a request to the Service Broker (1), which in turn makes a request to the Web Service (2). The response from the Web Service is then transformed by the Service Broker, and presented to the Client Browser (3). In a client-side service, the Client Browser gets the entire service logic from the Service Broker (1), and then communicates directly with the Web Service (2).
Loading the service
http://del.icio.us/api/posts/all to get the XML listing of the user's bookmarks, which is persisted through the lifetime of the direc.tor page. Because del.icio.us uses the standard HTTP basic authentication, the browser will automatically ask for credentials if it has not been established yet. Since the client is communicating directly with del.icio.us, those credentials never pass through this site. For more information about this, and cross-site scripting concerns, see Creating a client-side web service broker.
Filtering and sorting the bookmarks
Implementing the tag browser
The major hurdle is not doing the initial retrieval of tags, but finding tags that have more than one other tag in common, i.e. "Show me all the tags that appear with the tag 'blog' and 'photo'." The brute force approach would cycle through all of the records, return those that contain the tag 'blog', cycle through that subset in search of 'photo', and then finally list and count all of the remaining tags that aren't 'blog' or 'photo'. Instead, direc.tor uses a single XPath query to pull out the subset of tags:
//posts/post[contains(string(@tag),'blog') and contains(string(@tag),'photo')]
Example: This diagram represents a bookmark collection that has a total of 6 unique tags: blog, design, css, photo, cool, politics. The outer hashtable (left) uses each of those tags as its keys, while its values are hashtables that contain the key tag and any tag that is related. The values of those inner hashtables represent the total count of each tag and its occurence with the outer tag.
The hashtable's fast key-based retrieval makes it an ideal indexer for storing the tag counts, and fulfilling the tag requests from the user. Getting a list of tags is accomplished by enumerating over the hashtable keys, and getting the tag counts involves retrieving the values from the inner hashtables.
Highlighting the search terms
What else could this do?
There are probably hundreds of other features that would be cool to implement, so here are some that I would implement if I had more time:
- Bulk tag editing: Enable tag addition and removal from multiple bookmarks (which would then give me an excuse to implement the fancy fade anything technique).
- Labeling: Designate a tag as a special UI flag that mimics Gmail's "starred" functionality.
- Other operators: Add
- Media detection: Expose media player controls for registered types, like MPG, MOV, WMV, etc.
- RSS import: Expand the input processor to parse RSS feeds, i.e. other people's tags.
- XML export: Allow download of an XML version of a current search, or all bookmarks.
- Link autopreview: Enable an Outlook-style bookmark preview pane.
- OS X Dashboard integration: Port direc.tor to a Dashboard widget.
The ultimate feature, though, would be to integrate some of this project directly into del.icio.us in order to eliminate the bootloading process altogether. I'm sure all you cats on delicious-discuss can come up with a collective feature list. I did this project to research different interface possibilities on other projects, so by all means, let me know if you're interested in helping make this a more mature service.
myObject.myProperty can alternatively be accessed using
myObject["myProperty"]. This leads to the common misconception that typing
myArray["myKey"] creates an associative array called
myArray with an element indexed under
myKey. It actually assigns a new property called
myKey to the object instance of