0xDECAFBAD - Tag: greasemonkey

Writing a Delicious command for Ubiquity

2008-09-01T04:37:03+00:00

In my last post, I got all fluffy about how cool Ubiquity is but didn't share any code to prove the point. As it happens, I have come up with at least one useful command that I'm starting to use habitually in posting bookmarks to Delicious. You can subscribe to my command or check out the full source—this post will serve as a dissection of the thing. Since this will be fairly lengthy, follow along after the jump.

Oh, and it's been awhile since I posted something this in-depth around here, so feel free to let me know how this first draft works. And, bug reports and patches are of course welcome.

To begin, consider the following code starting off the command source code:

/**
 * share-on-delicious - an Ubiquity command for sharing bookmarks on
 * delicious.com
 *
 * l.m.orchard@pobox.com
 * http://decafbad.com/
 * Share and Enjoy!
 */
var uext = Application.extensions.get('ubiquity@labs.mozilla.com');

var cookie_mgr = Components.classes["@mozilla.org/cookiemanager;1"]
    .getService(Components.interfaces.nsICookieManager);

The first thing to note here is that a short header comment introduces the command. This isn't required, but it's a good idea. It's also something you can't really do with bookmarklets. On the other hand, Greasemonkey user scripts expect metadata about the script to be provided here, but Ubiquity doesn't use this convention.

Second, notice that the code accesses some chrome-level resources. Again, this is something unavailable to bookmarklets and Greasemonkey user scripts. Just take a look at the FUEL library documentation to get a quick sense of what's available using that simplified API, not to mention what's available using the lower-level browser APIs.

Now, check out this next chunk of code, which begins the construction of an Ubiquity command:

CmdUtils.CreateCommand({
    
    name:        
        'share-on-delicious',
    icon:
        'http://delicious.com/favicon.ico',
    description: 
        'Share the current page as a bookmark on delicious.com',
    help:        
        'Select text on the page to use as notes, or enter your own ' + 
        'text after the command word.  You can also assign tags to the '+ 
        'bookmark with the "tagged" modifier, and alter the bookmark ' + 
        'default page title with the "entitled" modifier.  Note that ' + 
        'you must also already be logged in at delicious.com to use ' +
        'this command.',

    homepage:   
        'http://decafbad.com',
    author: { 
        name: 'Leslie Michael Orchard', 
        email: 'l.m.orchard@pobox.com' 
    },
    license:
        'MPL/GPL/LGPL',

Whereas Greasemonkey scripts support metadata in the header comment, the Ubiquity command script API works a little differently.

The CmdUtils module provided by Ubiquity offers a CreateCommand function, which expects an object as a parameter. The object literal whose construction is begun in the code above serves as a self-contained package for the command, bearing metadata describing the command as well as containing all the code necessary to implement it.

So, in the above code block, you can see the machine-readable description of the command—including a command name, display icon, home page URL, author information, and license. The command name (share-on-delicious) will be used by the Ubiquity command parser, but the rest of the description will also be used in the list of commands available to the user, invoked by the command-list command, like so:

Moving along, this next chunk of code introduces the first functional bits of the command:

    takes: { notes: noun_arb_text },
    modifiers: { 
        tagged:  noun_arb_text,
        entitled: noun_arb_text
    },

Like smart keyword shortcut bookmarks, Ubiquity commands accept user-supplied input. But, what's unique to Ubiquity is that it employs a parser whose goal is to support something approximating natural language. At present, this results in commands that support a single primary argument—declared above with the takes property—and any number of additional keyword modifiers—declared above by the modifiers property.

For the command under construction here, this establishes a pattern something like the following:

share-on-delicious {notes} [tagged {tags} entitled {title}]

Content for the {notes} argument can either be typed directly by hand, or it can be supplied by text highlighted on the page. To use highlighted text, you can either issue the command alone, or use the word this for the {notes} argument before including further modifiers.

The modifiers tagged and entitled are optional, and can be used in any order. Each of these keywords signifies the start of a different argument—which unfortunately can collide with the literal data supplied for notes, which will hopefully be a rare occurrence.

All of this adds up command invocations including the following:

share-on-delicious
share-on-delicious I really like this page tagged nifty amusing
share-on-delicious this entitled This bookmark has no tags
sh this tagged osx software apple entitled This is good OS X software

That last example is important—since I have no other commands starting with "sh", I can abbreviate the full command. Ubiquity only requires enough of a command name to disambiguate it within your collection of commands.

Another thing to note is the use of the constant value noun_arb_text, which declares that these arguments should expect any arbitrary text as input.

This facility is not exploited for the present command, but Ubiquity defines noun types. These include concepts such as plain text, dates, address book contacts, browser tabs, bookmark tags, and more. You can define your own noun types, as well as implement suggestion schemes that help guide the user toward constructing useful input values in the command interface. You can read more about this in the official author tutorial.

Next up is a quick bit of command-specific configuration:

    /**
     * Command configuration settings.
     */
    _config: {
        // Base URL for the delicious v1 API
        api_base:      'https://api.del.icio.us',

        // Domain and name of the delicious login session cookie.
        cookie_domain: '.delicious.com',
        cookie_name:   '_user'
    },

Since this command will be posting to Delicious via the V1 API, it's handy to declare the base URL for the API in an easily changed spot. That way, you could change this value later on to point the command at another implementation of the API.

Additionally, this command will employ a little-known authentication trick supported by the Delicious API that accepts the user's login cookie set by the Delicious website—this "cookie god" auth is used by the official Delicious addon for Firefox. It's handy for piggybacking on the website login and removing the need to ask the user for their username and password again and possibly storing it in an insecure manner.

In fact, this next chunk of code defines a utility method to rummage through the cookie jar:

    /**
     * Dig up the Delicious login session cookie.
     */
    _getUserCookie: function() {
        var iter = cookie_mgr.enumerator;
        while (iter.hasMoreElements()) {
            var cookie = iter.getNext();
            if( cookie instanceof Components.interfaces.nsICookie && 
                cookie.host.indexOf(this._config.cookie_domain) != -1 && 
                cookie.name == this._config.cookie_name) {
                return decodeURIComponent(cookie.value);
            }
        }
    },

The method defined above, ._getUserCookie(), uses the browser's cookie manager and the values defined in the previous configuration section to find the login session cookie set for Delicious. Take note that this is far beyond the allowed capabilities of bookmarklets and Greasemoney user scripts—this is digging straight into the browser itself, skipping past the usual content-space security restrictions.

In other words: In Ubiquity, the gun is loaded and you should be careful.

Moving along, consider this next utility method:

    /**
     * Given input data and modifiers, attempt to assemble data necessary to
     * post a bookmark.
     */
    _extractBookmarkData: function(input_obj, mods) {
        return {
            _user:
                this._getUserCookie(),
            url:
                context.focusedWindow.location,
            description:
                mods.entitled.text || context.focusedWindow.document.title,
            extended: 
                input_obj.text,
            tags:
                mods.tagged.text
        };
    },

Named ._extractBookmarkData(), this utility method accepts the results of Ubiquity's parser interpreting the primary argument and modifier arguments supplied by the user. Using these data structures, it attempts to build a structure representing the fields of a Delicious bookmark.

The _user field is used for authentication via the site login cookie. The url is set from the location bar of the current page. The description, or title, field of the bookmark is taken from either the entitled modifier or the title of the current page. The tags, if any, come from the tagged modifier. And, finally, the extended notes for the bookmark are taken from the primary input argument of the command.

As you'll see shortly, this utility method will be used in both the preview and the execution of the command.

Next, there's one more utility method to cover:

    /**
     * Given an object, build a URL query string
     */
    _buildQueryString: function(data) {
        var qs = [];
        for (k in data) if (data[k]) 
            qs.push( encodeURIComponent(k) + '=' + 
                encodeURIComponent(data[k]) );
        return qs.join('&');
    },

In anticipation of using the Delicious V1 API, the ._buildQueryString() method accepts an object and constructs a URL query string from the encoded properties of the object. This will be paired with the ._extractBookmarkData() method to supply data for API calls.

Moving along, it's time to start digging into the meat of this Ubiquity command:

    /**
     * Present a preview of the bookmark under construction during the course
     * of composing the command.
     */
    preview: function(pblock, input_obj, mods) {

        var bm          = this._extractBookmarkData(input_obj, mods);
        var user_cookie = this._getUserCookie();
        var user_name   = (user_cookie) ? user_cookie.split(' ')[0] : '';

        var ns = { user_name: user_name, bm: bm };
        var tmpl;

With this code, the implementation of command method .preview() has begun. This method is used by Ubiquity to generate a live preview of the command. Called with a DOM node (pblock) and partially completed command input (input_obj and mods), this method is expected to build a representation of the command's results in the DOM node. As the user types, this method will be called over and over again, ideally offering feedback as the user composes a command.

Continuing on, consider this next chunk of code checking the validity of command input:

        if (!user_name) {

            // If there's no user name, there's no login, so this command won't work. 
            tmpl = [ 
                'No active user found - log in at ', 
                ' ',
                'delicious.com ', 
                'to use this command.'
            ].join('');

        } else if (!bm.description) {

            // If there's no title, then this is an error too.
            tmpl = [ 
                'A title is required for bookmarks on ', 
                ' ',
                'delicious.com ', 
                ''
            ].join('');

This chunk of code first checks for a user name, which can be extracted from a valid Delicious login cookie, if one was found. If not found, the command will fail—so the preview built here will instruct the user to login at Delicious before going further.

The second precondition for using the command is that the bookmark has been given a title. By default, this is the title of the current page—but, some pages don't offer titles. So, an error needs to be flagged if the user hasn't manually supplied a title in this case.

Finally, notice in both of these error cases, a string of HTML is composed in the variable tmpl. This will be used at the end of the method to populate the DOM node passed in as pblock.

Now, assuming that all the command's prerequisites have been met, it's time to try constructing a proper preview for the results of this command:

        } else {

            // Attempt to construct a vaguely delicious-esque preview of a bookmark.
            tmpl = [ 
                '',
                '',
                    'Share a bookmark at  ',
                        'delicious.com/${user_name}:',
                    '',
                        '',
                        '${bm.description}',
                        '${bm.url}',
                        bm.extended ? 
                            '${bm.extended}' : '',
                        bm.tags ?
                            'tags: ${bm.tags}' : '',
                    '',
                ''
            ].join("\n");

        }

        pblock.innerHTML = CmdUtils.renderTemplate(tmpl, ns);
    },

Building on the notion that previews are built in a DOM node, the code above uses both CSS and HTML to assemble a quick-and-dirty facsimile of a Delicious bookmark—which will be rendered like this:

Also note that Ubiquity provides a template engine for use in generating content—namely the JavaScript Templates engine from the TrimPath project. This engine may eventually be replaced with another, but the notion is that Ubiquity will provide tools to more easily generate previews and more.

The conclusion of the .preview() method uses the template engine with a call to CmdUtils.renderTemplate() to inject content into the preview element by way of the .innerHTML property.

Now that the preview is out of the way, it's time to get down to implementing the execution of the command:

    
    /**
     * Attempt to use the delicious v1 API to post a bookmark using the 
     * command input
     */
    execute: function(input_obj, mods) {
        var bm          = this._extractBookmarkData(input_obj, mods);
        var user_cookie = this._getUserCookie();
        var user_name   = (user_cookie) ? user_cookie.split(' ')[0] : '';

        if (!user_name) {
            // If there's no user name, there's no login, so this command won't work. 
            displayMessage('No active user found - log in at delicious.com ' +
                'to use this command.');
            return false;
        }

        if (!bm.description) {
            // If there's no title, somehow, then this is an error too.
            displayMessage("A title is required for bookmarks at delicious.com");
            return false;
        }

Mirroring the .preview() method, the .execute() method first checks for validity of the arguments given by the user. A missing user name or title result in a notification that the command has failed.

But, if the arguments are all valid, it's time to actually issue a request to the Delicious V1 API:

        var path = '/v1/posts/add';
        var url  = this._config.api_base + path;

        var req = Components.classes["@mozilla.org/xmlextras/xmlhttprequest;1"].
            createInstance();

        req.open('POST', url, true);

        req.onload = function(ev) { 
            displayMessage('Bookmark "' + bm.description + '" ' + 
                'shared at delicious.com/' + user_name);
        }

        req.onerror = function(ev) { 
            displayMessage('ERROR: Bookmark "' + bm.description + '" ' + 
                ' NOT shared on delicious.com/' + user_name);
        }

Using the base URL for the Delicious API declared earlier in the configuration section, the .execute() method constructs an API URL for the /v1/posts/add method. Then, it creates an instance of XMLHttpRequest from the browser to be used in sending the API request. Event handlers are registered with the object to present notifications to the user indicating whether or not the API request was successful.

At long last, it's time to wrap up this method and make the API request:

        req.setRequestHeader('Authorization', 'Basic Y29va2llOmNvb2tpZQ=='); // btoa('cookie:cookie')

        var mediator = Components.classes["@mozilla.org/appshell/window-mediator;1"].
            getService(Components.interfaces.nsIWindowMediator);
        var win = mediator.getMostRecentWindow(null);
        var user_agent = win.navigator.userAgent + ";Ubiquity-share-on-delicious";

        req.setRequestHeader("User-Agent", user_agent);      

        req.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
        req.send(this._buildQueryString(bm));
    },

    EOF:null // I hate trailing commas
});

The login cookie authentication supported by the Delicious V1 API is triggered by supplying a user name / password pair of cookie, which is done by setting the Authorization request header. The login cookie is then expected to be passed in as the POST variable _user, which is done in the ._extractBookmarkData() utility method.

Another bit here that shows more access of browser resources is the construction of a unique User-Agent header for this API call based on the browser's own User-Agent string, something that's suggested in the guidelines for using the Delicious API.

Finally, the .execute() method—and the command itself—is wrapped up with by sending off the bookmark data encoded as POST form variables with the ._buildQueryString() utility method.

And, that's it—a command-driven Delicious browser extension in a little over 200 lines of code. There's still more to be done to really make this thing full-featured, but I think this shows off the basic features of Ubiquity. I'm hoping to dig in deeper and explore further, taking a look at running Greasemonkey-style code at browser startup and page load, as well as playing with some more browser chrome features.

Archived Comments

Jason Clark
2008-09-01T14:47:35
First off- fantastic post. Great to see a lengthy post here again, although I'm one to talk. This is an excellent introduction to Ubiquity command development, and tres useful to boot.

I'm wondering why you chose to construct and post the XMLHttpRequest manually instead of using jQuery, which is included with Ubiquity. I don't know that there's any benefit other than some simplicity, but I took a crack at converting your code to use jQuery, which works nicely. In the 'execute' function, replace everything after "var url = this._config.api_base + path;" with this (hope code blocks work in comments):

var win = context.focusedWindow; var user_agent = win.navigator.userAgent + ";Ubiquity-share-on-delicious";
```
    jQuery.ajax({
      type: "POST",
      url: url,
      data: this._buildQueryString(bm),
      username: "cookie",
      password: "cookie",
      beforeSend: function( req ) {
        req.setRequestHeader("User-Agent", user_agent); 
      },
      error: function() {
        displayMessage('ERROR: Bookmark "' + bm.description + '" ' + 
            ' NOT shared on delicious.com/' + user_name);
      },
      success: function() {
        displayMessage('Bookmark "' + bm.description + '" ' + 
            'shared at delicious.com/' + user_name);
      },
    });
```
Also, with both versions of the code, I'm seeing some unexpected behavior around authentication. Assume I'm logged in to delicious, with "stay logged in" checked, and I restart my browser. Trying to post with the command fails with a 401 unauthorized, even though I can see the cookie was sent (via Live HTTP Headers extension). Going to delicious.com shows me logged in, and once I've viewed the site, the command works. Except that now I can't reproduce; but I know it happened because I've got the headers. At any rate, it is working nicely, but the previous failure is bugging me... feel like I'm overlooking something.

Thanks again for an awesome post. Hope to see more of the same.
Ryan Scott Scheel
2008-09-01T15:07:01

You should be helping with the documentation, if you aren't already. Very nice job with this article;
Aza Raskin
2008-09-02T01:44:13

Hi Leslie,

This is a beautiful tutorial on writing a Ubiquity command. We'd love your help in making Ubiquity's documentation better (especially dev facing). You should totally link to this from the Ubiquity Wiki -- or even add the content in someway.

Anyway, just wanted to say thanks.

-- Aza
seth
2008-09-08T17:30:16

Am I doing something wrong?

When I enter in "sh this tagged tag1 tag2 entitled This is a title" everything after the tagged is added as a tag. It's not seeing the entitled part.
http://spyced.blogspot.com/
2008-09-15T19:12:58

Did something break? I'm getting a 404 accessing http://decafbad.com/hg/UbiquityCommands/file/tip/delicious.ubiq.js
l.m.orchard
2008-09-15T23:07:42

Yeah, looks like I had a small snafu with switching back from Lighttpd to Apache. Left out a rewrite rule - doh!
Tony
2008-10-22T04:56:03

Great article. This is replacing my delicious bookmarklet. Thanks!
dgtlchlk
2009-04-14T01:06:57

Great article and command. Wish it worked correctly with the latest 0.1.8 release though. No matter what text you put in it adds everything as the notes. The tagged and entitled modifiers don't work.
Nolan
2009-04-16T03:10:07

I second that comment. Delicious is actually saying the link given was "chrome://browser/content/browser.xul" and marking it as harmful inside delicious!
Justin
2009-07-06T12:39:39

Hi Leslie,

Thanks for the delicious ubiquity command. Unfortunately, as one of the commenters above mentions, the tagged modifier doesn't seem to work. I'm using Ubiquity 0.5 pre and typing the phrase:

share tagged foo

Adds the bookmark to Delicious with the note text "tagged foo"

Cheers, Justin
Eric
2009-07-20T22:43:17

Echoing others.. I'm using the latest ubiquity (err, next to latest.. 0.1.8?), and doing "share-on-delicious this is a note tagged testing" saves the link with notes "this is a note tagged testing" and no tags. :( Then again, the other delicious command I've found (by someone else) is having the same issue.

delicious 2.0 legacy bookmarklet fix

2008-08-02T22:34:28+00:00

As you've probably seen by now, Delicious 2.0 has launched. It's an all new design and the whole thing has been rewritten from the ground up. Most of the gripes I've seem like general dislike of change—which actually attests to the gargantuan effort put forth to reimplement the original from scratch in a whole new language and architecture.

That said, I found at least one little bug that stops my usual bookmarklet flow. And, what's really annoying is that, it's probably a bug in code I wrote at one point. As it turns out, the original URL parameters for the bookmark posting form don't seem to be accepted anymore, so legacy bookmarklets may be broken. I swore I tested that, since I've got some personal investment in it.

But, although I can't contribute code to the project anymore, I've at least still got Greasemonkey. And, through this quick & dirty user script, I'm back to using my legacy bookmarking bookmarklets. In case you're interested, here's my favorite Firefox keyword shortcut bookmarklet—just select some text, type the keyword in the URL bar with some tags, and hit return.

Archived Comments

Peter Beardsley
2008-08-03T13:52:14

Just curious-- what language was del.icio.us originally implemented in, and what language was it re-implemented in for 2.0?
l.m.orchard
2008-08-03T18:30:58

@peter: The original del.icio.us was built with Mason in Perl, backed by MySQL. The new Delicious 2 is built with Symfony in PHP for the web frontend, with all the out-of-box Symfony database abstraction replaced by a Yahoo!-developed data management and business logic layer written mostly in C++ for the backend. I'm pretty sure I've read all of that in other public releases, so I don't think I'm letting any cats out of bags there :)

I can't get too ticked at O'Reilly and IntelliTXT

2006-01-14T02:00:47+00:00

You know what I've been annoyed by lately? Those contextual ads masquerading as double underlined links in the middle of the page. Aren't these uncomfortably like Microsoft's Smart Tags? Eh, maybe not, since the site owners are willing collaborators—which makes it more of a successful service than a monopolist's sinister plan.

On the other hand, in the midst of writing this little rant, I noticed that they provide a mildly hidden opt-out at the bottom of the FAQ page. Bah. It's annoying, but at least there's that.

And, well, folks gotta make money somehow I guess. The FAQ page is very honest about it:

As we're sure many will appreciate, there is pressure for us to generate revenue from advertising to cover all of the costs, and this is an opportunity for us to do just that.

So, other than the poor user experience of the popup layer when I accidentally roll my mouse over the wrong thing--I can't say they're being crass about it.

Nonetheless: How long until I get itchy enough to make an anti-IntelliTXT Greasemonkey script, I wonder?

Archived Comments

Dominik Schwind
2006-01-14T11:29:31

I found this one: http://dev.upian.com/hotlinks/archives/2005/09/29/#item44519 it seems to be working fine.
Jason Clark
2006-01-14T15:56:26

Lazyweb has beaten you to the punch. I use IntelliTXT Disabler, which I found at userscripts.org, and which is also available from the author. It runs after the page loads, of course, so you see the links appear briefly and then disappear. I've found it quite useful.
Justin
2006-01-14T16:06:53
Greasemonkey is overkill for something you can block with CAPS:
```
user_pref("capability.policy.default.Window.itxtInited", "noAccess");
```
Henning Koch
2006-01-15T15:32:46

Because being fooled by IntelliTXT camouflage makes me want to buy shit every single time.
Brian Donovan
2006-06-18T18:14:00

I'm subbed to your blog but just came across this entry while searching for other IntelliTXT-cleaning GM scripts to help me decide whether it would be worthwhile to tidy up mine and add it to userscripts.org:

http://userscripts.com/scripts/show/4250

The cleaning function in this one is triggered whenever a DOMNodeInserted event fires, so it catches the bogus links immediately after they're implanted. At least for me, this means that I never see a singe IntelliTXT link on any afflicted site.

I tracked down 2 other anti-IntelliTXT user scripts but, while they each work, they both use setTimeout() to trigger their IntelliTXT cleaning functions with the result that you inevitably end up seeing the double-underlined pseudolinks for a moment or two before they're nuked.