0xDECAFBAD - Tag: mashups

Ubiquity cracks open personal mashup tinkering

2008-08-31T04:07:22+00:00

When I was a wee hacker, I lived my digital life though a Commodore 64. I played games on it, did homework, talked to people far away—you know, all the stuff they showed in the pictures on the box. I also took things apart—both the machine itself and software running on it. I grew up learning that my digital environment was ultimately understandable, susceptible to tinkering, and open to being bent to my own purposes.

From the Commodore 64, I graduated eventually to terminals and text editors, opening portals mostly onto computers elsewhere via powerful UNIX command shells. And, of course, over the past decade, this has largely given way to life in a browser.

Yet, for a little while, particularly in the first few years of browsers, freedom to tinker seemed cramped. JavaScript had yet to arrive, and was a little messy when it did. There was no relatively easy addon development. And, though the portals opened by a browser were richer than those provided by terminals, the paths of navigation defined by links controlled by site owners offered less freedom of movement than UNIX commands. I could create my own pages, but I couldn't do much to others' pages.

But then, javascript: URLs came around, dots were connected, and bookmarklets were born. Suddenly, it was possible to customize my browsing environment with arbitrary JavaScript code having access to the current page—no matter whose page it was. And, through the various tricks of the AJAX trade, bookmarklets have only gotten more capable throughout the years.

Smart keyword shortcuts came around a little later, allowing quick access to bookmarks via simple keywords typed into the location bar. The smart part, though, came in the form of bookmarked URLs with placeholders and keywords given arguments to fill the placeholders—allowing not only quick access to bookmarked pages but also search engine forms bookmarked with late-bound fields.

Bookmarklets inherited the benefits of smart keyword shortcuts. The same placeholder in http: URLs can be inserted into the code of a javascript: URL, thus parameterizing the JavaScript code and incidentally turning the location bar into a kind of primitive command line. For example, one of my most heavily used "addresslets" is based on John Resig's original "Super-Fast Delicious Bookmarklet".

Another leap in prying open the browser tinkering space came in the form of Greasemonkey—an addon-powered environment created explicitly for the purpose of end-user scripting applied to others' pages. Greasemonkey user scripts can do more than bookmarklets, and with a much better development environment to boot. And, though a user script can't do quite as much as a proper browser addon, they're much easier to hack on and distribute.

Now, consider one of Mozilla Labs' newest projects, named Ubiquity. This rough and experimental addon for Firefox combines and improves upon everything I've described so far:

Ubiquity is a hackable command line environment, better than bookmarklets and smart keyword shortcuts;
Ubiquity enables persistent customization of others' pages, not unlike Greasemonkey;
Ubiquity facilitates live in-browser creation and web-based subscription to user commands and scripts;
Ubiquity gives access to browser chrome resources without a need for frequent restarts;

So far, most of the commands I see popping up since the 0.1 release have not accomplished much more than smart keyword shortcuts in the location bar could. But, it's early yet, and Ubiquity is far from limited to these commands.

Once the basics have been well-explored, I expect to see more people taking a crack at the broader capabilities offered by Ubiquity. Bookmarklets and Greasemonkey can't access browser chrome—but Ubiquity can. Ubiquity also offers a user interface that's so much more promising than keyword shortcuts, including command previews and typed parameters with suggestions.

Ubiquity promises web-wide mashups directed by a conversational command interface. All in all, the potential of this makes me feel like my digital environment—browser and web as a whole—is getting even more intimately, personally hackable.

It'll be very interesting to see where this project goes.

Archived Comments

Raul
2008-08-31T10:20:58

Hi, was using the original delicious command linked from the Ubiquity wiki, just tried yours and its definitely more polished and functional. Great job with the preview and the extra functionality. Only thing is 'share-to-delicious' is too much to type so I unsubscribed the previous command and changed the namespace in yours. This is clearly going to become a problems as the commands proliferate.
l.m.orchard
2008-08-31T14:43:39

@Raul: "share-to-delicious" is long, but keep in mind is that you only need to type enough of the command to disambiguate it. That is, all I type is "sh this tagged osx software apple" because I have no other commands starting with "sh". Watch the list of commands in the preview as you type. Those tell you what the parser thinks of what you're typing as you type. It's like automatic tab-completion.

Also, I think there's work planned to put some usage based sorting into the command parser, preferring the commands you use most in order of disambiguation.
Abi
2008-08-31T15:33:24

Nice post. I share your sentiment with regards to Ubiquity commands. A lot of commands that I see are just plain simple searches. I hope developers will work on more interesting things. For example, even things like auto-form filling for this comment (possibly even on page load without having to type a command) could be done by Ubiquity. There's still a lot more room for experimentation.
Abi
2008-08-31T15:45:59

But otherwise, I really like your command especially the preivew. We should include it as a builtin command, if you don't mind. :)
Abi
2008-08-31T15:51:20

I noticed a bug in your command. If I select some text in the awesomebar, the bookmark url is chrome. You should use something the command utils to get the url, instead.
l.m.orchard
2008-09-01T06:17:37

@Abi: Making this a built-in command is totally fine by me! It can use more work, though, for sure.

Also, I can reproduce that bug. Ugh. I can't find any methods in the CmdUtils to get the current page URL, though. I'll keep poking a bit though.
Abi
2008-09-01T06:50:23

Your blog seems to be rejecting code (that's why I had so many posts in the first place). You can get the current page url using (with dots):

CmdUtils getDocumentInsecure() location href
Blair McBride
2008-09-02T02:50:12

I recommend against using getDocumentInsecure() - its got "Insecure" in its name for a reason! Instead, you should use:

Application.activeWindow.activeTab.uri.spec

Outsourcing creativity via APIs

2007-11-01T18:17:21+00:00

Tim Faulkner at Valleywag: "I blame Twitter. It's not enough to be a website anymore. Oh no. You must be a platform. Have an API. Court developers. Build an "ecosystem." Whatever. You know what an application programming interface really is? An admission that you're too poor, cheap, or uncreative to build all the features your website needs."

Welcome to Web 2.0, Tim - we've been doing this thing for most of a decade or longer now. I know this is coming from Valleywag and fine-tuned as trollbait and all, but this just struck me as particularly dumb.

It's probably because most of my job right now is working on APIs and feeds for delicious, and I'm partial to the concept in general. Because, yeah, really: An API is an admission that you haven't the creativity or time to build everything your website needs. No one does. Anyone who doesn't make (or denies) that admission is either lying, clueless, or sadly mistaken. Anyone who starts with that admission and manages to open up to let users fill in their own gaps has a chance at hosting something really interesting.

And, yeah, thanks to the swooning hype over Facebook and others, it's a little over-exposed right now and tickling curmudgeonly snark-emitters. But, that doesn't mean the concept's a bad one or that it's going away.

Archived Comments

Aristotle Pagaltzis
2007-11-01T21:46:19

Obviously he is waxing nostalgic about the days when people screen scraped HTML rather than having a formal API. That, or he’s too naïve to realise that every web site is a service, by definition.

Thoughts on Pipes on the Web - Part II

2007-02-15T08:49:56+00:00

In the last post, I expressed concern for my own well-being over a lack of head-over-heels love for Yahoo! Pipes. On the surface, I think it's because it's not a freshly discovered concept for me, and also probably because I'm tired and a bit hacked out right now. But, I think there's a bit more to it.

First of all, I'm not impressed by GUI builders for most things. Don't get me wrong: That Pipes GUI is pretty sweet and quite an impressive implementation — but as a rule, such things don't quite get my geek on. So, there's that.

Another thing is that I've done something kinda like Yahoo! Pipes, called FeedMagick — only, it doesn't have a GUI and I mostly abandoned it after releasing the code and having used it for a project at my old job. It's one of those serial enthusiasms that I've figured I'd circle back to eventually. I still think it's a pretty cool idea.

All kudos to the Yahoo! Pipes team though: Unlike me, they've actually got a living and breathing project — which trumps a paged-to-disk serial enthusiasm most days.

So what else is really curbing my enthusiasm? Well, Yahoo! Pipes looks like a pretty self-contained pipes engine — data goes in one end, frobnication happens in the middle, data comes out the other side.

But, what I like about the notion of the URL-line is that you can take one URL and supply it as a parameter to another URL — making messy pipelined URLs while building a crazy web-wide distributed execution environment powered by HTTP and REST. This is the kernel of the notion that I think really excites me about pipe on the web — I just haven't had a chance to do much with it lately.

Depending on the perspective, true pipes on the web — that is, URLs fetching URLs — look to me like functional programming ala Lisp. Consider an idempotent GET request as a pure function call with no side-effects. Then, consider a GET request that accepts a URL as a parameter — it's a nested function call: the outer GET must make an inner GET to fetch the parameter-supplied URL. Give the inner URL another URL as a parameter, and you've got yet another nested function call.

But, if you like, ignore the theoretical benefits of functional programming — flip the nested function calls inside out and you've got a pipe. And, since you're using HTTP GET, you can get all the benefits of HTTP — like caching of execution results and a web full of distributed processing nodes, among other things.

I haven't explored Yahoo! Pipes deeply enough yet, so maybe I'm missing the features that pipe authors can used to call on other distributed pipe elements out on the web at large. But, I think that's what ultimately gets me psyched about pipes the web and hasn't yet for Yahoo! Pipes.

And, of course, I think I'm just a little hacked out and tired of whizbang new stuff right now. :)

Archived Comments

John Evans
2007-02-15T13:59:01

The thing that struck me about Pipes is that it seemed rather limiting to someone who can code but too geeky to someone who can't. Maybe I am wrong but I wonder that once the hype dies down who exactly the audience will be long term.
Stuart Robinson
2007-02-15T19:35:41

Don't be so apologetic! Conceptually it's cool, but the UI quickly becomes frustrating for anyone with text-based coding experience. I guess this is the web's visual basic and we are all going to be more interested in it's perl.

Thoughts on Pipes on the Web

2007-02-15T08:09:56+00:00

By way of my meme tracker, I found something like 15 entries linking to Yahoo! Pipes from my feed subscriptions. That's crazy — I've usually never seen more than 3-5 feeds linking to anyone thing, maybe as many as 8 if it's something really hot. I read about it over morning coffee and aggregator skimming; I heard about it on TWiT on the bus to work; I was told all about it in the hallway once I got to work. Yahoo! Pipes has been hard to miss.

With that in mind, it seems weird to me that I didn't get as excited about the release of Pipes as everyone else in my local blogosphere seems to have. Pipes has all the ingredients of something that should really set me off. Pipes, filters, mashups, RSS, Atom, RDF — (singing) these are a few of my favorite things! So, what gives? I'm genuinely worried that I'm sick, depressed, losing it, or otherwise replaced by a pod alien. Possibly all the above.

I knew about Pipes as an internal project at Yahoo! quite awhile ago — possibly even in my first week or two after having joined. I'm fuzzy on the timing here, but it's not important. Someone mentioned it to me at some point, I poked at it a bit from the internal network, and thought it was neat / keen. My main impression at the time was, "Cool, someone's doing FeedMagick, but done right and with a slick visual layout tool."

Then, I got really busy with del.icio.us and wandered off. So, the release wasn't big surprising news to me — my reaction was something like, "Huh, so they did a public release of that? Nice." Clearly I need more caffeine.

The thing is that the idea of pipes on the web is not itself a new concept to me. I think my XmlRpcFilteringPipes seems kinda silly in retrospect, and I'm all about the Power of the URL-line nowadays — but suffice it to say that I've been thinking about pipes on the web since as long ago as 2002 or so.

So, while I'm all jaded and apparently incapable of feeling the wonder, this seems to be one of the biggest revelations to people playing with Yahoo! Pipes:

Pipes? On the web? BRILLIANT!

So, it's not you, it's me. I think I've got a serious deficit of enthusiasm for what's really a pretty cool thing. I want to think a bit more about why that is.

Archived Comments

Kenneth
2007-02-15T14:06:37

http://yakubovich.blogspot.com/2007/01/visipipe-visual-version-of-core-utils.html

Might have been buried in all the buzz about Yahoo, but this sounds like a neat idea.

Kenneth Writer
Jeff Daly
2007-02-16T01:08:54

You're not alone. I was excited when I learned about Pipes, but ultimately I was let down by the lack of parsing tools. Pipes is seriously limited without any XPath/regular expression parsing capabilities. Fortunately, there is a post on XPath in the feature request page.

throttling the basement hacker

2006-10-13T19:19:26+00:00

This might just be a phase, related to my recent injury and subsequent period of convalescence - but I feel like doing absolutely nothing extracurricular hack-wise after work. This is an odd state of mind for me, since I'm normally never happy unless I've got both work and home projects spinning. It's been somewhat satisfying to get work done and code checked in during the day, then home for dinner and some WoW on the couch next to The Girl. I'd kind of like to get into something more soon, but at present I feel no compulsion to hack, mashup, reverse engineer, invert polarity, or anything else of that nature. In fact, I'm doing a small measure of thinking about into what I do want to get once the itch returns. I've got a half-dozen unfinished short stories, and a geographical region full of people to meet. Maybe once the leg works again, I'll not feel like being such a full-throttle basement hacker anymore. This is weird.

Archived Comments

Dean Edwards
2006-10-13T23:38:30

Don't worry. This is perfectly normal. Your brain has gone on holiday for a couple of weeks leaving your body in basic life-support mode. Feed it TV and beer and anything you fancy. Normal service will be resumed in a week or two.
l.m.orchard
2006-10-13T23:44:57

Mmm... beer. Luckily, there's a BevMo within walking distance of our apartment. Er, well, theoretical walking distance anyway.
Mad William Flint
2006-10-14T02:02:59

I've got to be particularly careful as a few hours of MMOs can poison my hackish funkitude for weeks at a time.
l.m.orchard
2006-10-14T03:01:12

I suspect, at least for me, that it's the sense of micro-accomplishment. Move a few bars, finish a few quests, get a ding. Woo hoo! I did stuff! :)
Julian Bond
2006-10-14T19:11:26

Me too.

Loads of 10% time projects and I can't be bothered to work on any of them. My excuse to myself was re-reading 13th Valley but now I've finished it...
Len Lynch
2006-10-15T22:08:13

Aquaintances of mine are going through the same thing... so it should pass. Not that I can brag about GTD, personally in a while now.

Raise a glass, I'm toasting to your speedy recovery!

I did hold my breath for a moment when I read that you relocated out west. Others I've known, were changed in an inexplicable way. They lost something they once had and became more like those they live around. It's difficult to explain.

Is there something in the water, the air, the culture, everything?

If you get any insite into this, please share it.