0xDECAFBAD - Tag: atom

The Readerpocalypse, or Occupy Google Reader

2011-11-01T10:31:54+00:00

TL;DR: Google gave Reader a face-lift and removed built-in following & sharing. In exchange, they added +1 buttons and ways to share to Google+. But, the changes seem to have removed most of the value from the service for me, so I'm moving on as I was invited to do.

What happened?

Almost a year ago, I wrote about how my habits had gotten less del.icio.us. One of the main culprits that stole my sharing flow was Google Reader: I went from using a quick bookmarklet to an even quicker icon-click in my feed reader. That meant I left most things untagged, but I shared even more than before. Eventually, Google Reader released a bookmarklet for use off-site, which meant I started sharing everything everywhere to Reader.

I also found a replacement for my old del.icio.us network in Google Reader's social features. I could follow people, and the stream of things they shared and annotated turned into a great source of news curated and pre-filtered by likeminded people.

Well, as it turns out, Google's recent changes to Reader just removed all of the above. In exchange, they added a couple of buttons to +1 and share items to Google+, which streamlines things for them and encourages more use of the new service.

As I wrote on Google+, the net result is nothing to stomp and shout about, but there's nothing there that makes me happier. As I played more, though, I came to realize that most of the value I derived from the service had evaporated.

Item-centric vs People-centric, Efficiency vs Engagement

There's a big difference between the way sharing happened on Google Reader, and the way it happens now on Google+. The best way I can think to explain is that the old way was item-centric and the new way is people-centric. In other words, people seem to be the dominant social objects in Google+, whereas Google Reader used to treat items that way.

Before, we all gathered around the news. The headline and summary were primary, and our comments were collected after the item. The result was a de-duped stream of great news filtered through smart brains. Even if the people I followed never said a word, the fact that they'd clicked "share" was value enough. In fact, it was better that way most of the time, because we all got out of the way of each other and the news.

This, by the way, is what I loved about del.icio.us and now pinboard.in: It's anti-social networking. We can provide low-effort, intelligent inputs to build something useful for each other, but we're not there to assert a presence or distract each other from the things we each came to see.

But now, that filtered news stream is gone. In its place, on a different site entirely, removed from the flow of feeding on news, is a highly-engaging flood of people babbling away, which occasionally includes repeated shares of things that have bubbled up during the day.

What's wrong with that, for me?

I don't want to wade through a heavily-padded, user-friendly flood of duplicated, echoed items.
I don't want to engage with the people—at least, not when I'm in a mood to feed on news. I want to catch up on what's been put out into the infosphere already, and then start talking about it.

The design of Google+—which has spilled over into the Google Reader face-lift—is about engagement and not fast & efficient processing of the day's news. Engagement in general puts a drag on the task—which for me is like a GTD, Inbox Zero, headline skimming, pattern recognition zen thing that might not be everyone's cup of tea.

But, for all I know, avowed weirdo and info freako that I am, these changes may drive Google Reader growth through the roof for mainstream users. But, it's no longer a product for me. The changes in sharing have removed most of the value for me, because I no longer have the warp speed cruise through news that other people used to power for me.

Circles point the wrong way

One of the things Louis Gray touted about the new sharing features reads like so:

In Reader’s previous sharing model, as a follower of your items, I would see every single item you shared, no matter what it was about. Obviously, as I have different interests than you do, not everything you shared was something I cared about, which often led to reduced sharing of off-topic content. Now, you are more in control, so you can share sports items to your sports circles, great recipes with your foodies circle, or local news to family and friends. You can keep sharing, like you always have, but now, you can better select who sees what.

But, for this purpose, Circles point the wrong way.

Circles on Google+ are lists of people controlled by me. They're not topics—again, people-centric and not item-centric. Circles are a targeting mechanism for sharing to people, and a filtering mechanism for items from people. That is, I can use a circle to set who sees something and I can decide who's stuff I see and when.

But, things that Circles don't do include:

Letting me follow Circles without the Circle-owner needing to manage membership - eg. Don't make me send a message saying "I am interested in your ideas and would like to subscribe to your newsletter". RSS doesn't work that way and it's better for it.
Letting me and others filter for what gets sent to specific Circles - eg. I'm in both your "scifi" and "politics" circles, but just show me what you sent to your "scifi" circle.

Google Reader didn't have this to begin with, but what I'm looking for is an equivalent to tagging on pinboard.in. For example, if you like my taste in scifi but could care less about my views in politics, you can subscribe to just the tag feeds you want.

I'm happy to tag my shares with topics; that's beneficial to me for search later, and helps interested people besides. But, the last thing I want to do is manage subscriber lists. That doesn't scale. There's no way for you to even know I have "scifi" or "politics" Circles, and you have no way to join either of them without asking for an add.

Too much coordination, not enough benefit. I just can't see the value in people-centric Circles here.

Other things broke, too.

And beyond the fundamental shift in sharing and reading, these things have broken for me now too:

The bookmarklet is dead, so there goes my off-site sharing.
No sharing at all from the mobile web version, so there goes at least half of my sharing on-site.
Android apps like my favorite gReader Pro still think sharing works, but they're just deluded and use an unsupported API anyway
No more feed to Facebook, so there goes most of my Facebook output. (Though, that might be a boon some friends over there.)

Now what?

The remaining value of Google Reader is as just a plain old hosted feed reader—one of the last of them out there, in fact. The problem here is that I'm a weirdo who wrote a book on feeds and feed readers, and so I'm perfectly capable of providing the remaining value for myself.

I'm sure most people won't want to do that, which is probably why "you're not going anywhere". But, there are alternatives even for people who don't want to D.I.Y. entirely:

Dave Winer's River2 (also open source, in UserTalk on the OPML Editor) runs on my laptop and lets me circle back to the code that ran my second-ever news reader, Radio UserLand. (My first was a thing called News Clipper in Perl from many, many moons ago.) And, of course, River2 is one of a suite of apps Dave's working on to build a decentrallized network of feeds and outlines.
NewsBlur looks mighty fine, and the guy running it only asks US$12-36 (your choice) for a year's worth of premium access. Remember: if you're not the customer, you're the product. And, if you want to try installing it, it's an open source Django site. While not yet very social, NewsBlur's creator has plans to head in that direction.
Tiny Tiny RSS installed on my Wordpress-capable web server in about 10 minutes, gobbled up my list of 800 feeds with no problem, and appears to have 90% of what I wanted from Google Reader. Also, it's an open source PHP site. For sharing, Tiny Tiny RSS lets you "publish" shared items in an RSS feed—kind of like the old reBlog web app—and has some experimental cross-instance sharing features.

At the end, the funny thing is that I've circled back to my old del.icio.us habits and have started tagging items on pinboard.in like never before. I'm sure there aren't nearly as many people watching me as before on either del.icio.us or Google Reader, but it's a place to go at least.

It'll probably be recognized as an illness someday, but my sharing feels compulsive at this point and I get itchy without a good outlet.

Introducing Fireriver, a River of News for Firefox 4

2011-01-27T06:36:52+00:00

TL;DR: I have a history of building news aggregators. Now, I've built one as an addon for Firefox 4.

This is Fireriver, an experimental add-on for Firefox 4. You can find the source code on GitHub.

Fireriver uses Live Bookmarks to build a River of News in your browser. No unread counts, no 3-pane view like an email client. Just paddle down the page with the space bar or your scroll wheel.

Organize the Live Bookmarks into folders to split things up into multiple rivers—which is not a strict River of News, per se. But, I like using them to focus on particular topics or priorities I happen to have time for.

It also displays a more obvious notification when a site has a feed and lets you add a new Live Bookmark subscription to the Fireriver folder with one click. (I'm considering splitting this feature off into its own small add-on.)

Of course, when I say experimental, I mean that this is my first Firefox add-on ever. I'm starting to use it daily, but I'm well-versed in nuking and rebuilding my Firefox profile. I haven't had to do that yet, but I'm expecting to have to do it eventually. I'll be pleasantly surprised if it turns out I haven't totally screwed something up.

Other add-ons you may enjoy

If you like the idea of this add-on, you may also enjoy these:

OPML Support - I just used this to dump in 800 subscriptions from Google Reader
Reliby - This lets you reload all Live Bookmarks on demand
Feedly - A more "magazine" style news aggregator in the browser
Feed Sidebar - Live Bookmarks in a sidebar
LiveClick - Still under development for Firefox 4, but I liked this in 3.6

Archived Comments

fluffy
2011-01-27T18:40:35

That looks like a pretty nice interface to Live Bookmarks, but it still has the inherent problem of not being nice and cloudy. Is there any provision for Live Bookmark synchronization?

Of course, the other problem for me is that I prefer Chrome these days, and of course Chrome solves the sync issue by having nothing to sync to begin with.
- l.m.orchard
  2011-01-27T20:58:53
  
  Yeah, lack of clouds is the drawback here.
  
  There is a bit of a sync story, though. If you're using Firefox Sync, the Live Bookmark subscriptions will carry across machines along with the rest of your bookmarks. Also, I'm using history to hide items from display, and that is covered by sync too.
  
  I need to check, but I don't think items within the Live Bookmarks are sync'd, so each machine is on its own for doing the actual feed polling.
  
  I'm also thinking a bit about Brent Simmons' notions about thin-server RSS sync, and if it has a place in this thing. Google Reader sync might also be nice.
  
  And once I get my head wrapped around add-ons for Firefox on android, I might do a mobile version of this thing so that side of things is covered.

Enter the LizardFeeder

2009-01-06T00:01:54+00:00

[caption id="attachment_1582" align="alignright" width="247" caption="The Mozilla Tree"][/caption]

Behind Firefox is Mozilla, and behind Mozilla is a community. And the Mozilla community acts a lot like an ecosystem, which can be visualized as a kind of living tree—not to confused with the mozilla-central tree. Oh yeah, and Mozilla is the name of both a Foundation and a Corporation.

Confused yet? If not, then we should talk so you can explain it to me, because it all looks pretty tangly and intertwingled to me. Nonetheless, it seems to work, and produces a good chunk of my favorite software and technologies.

There are many efforts to track what's going on—including planets and newsletters and bugzillas and wikis and repositories and tinderboxen. Some of these resources report on, or are driven by, the activity occurring in the others. Some are automated, and others are carefully stitched together by hand. None offer a full picture of what's going on in the Mozilla galaxy in a way that's casually comprehensible by a sane human being.

Of course, that's not a slight against any of these sites or the people maintaining them—extracting an overview from such an organic phenomenon is neither easy nor straightforward. But, it might be fun to try.

As an infovore and avid practitioner of continuous partial attention, my first impulse is to reach for a firehose and stick my head into the stream. Relax, defocus, and try to let my pattern recognizers do their thing—sometimes those pattern recognizers are in my head, and sometimes they're written in Python.

[caption id="attachment_1585" align="alignright" width="225" caption="Firefox Victory!"][/caption]

But, for Mozilla, I couldn't find a stream of sufficient volume or completeness to satisfy me or my robots. Happily, though, my feeding urge found itself aligned with a project to discover the patterns of contribution in the Mozilla community and to find a way to thank the contributors responsible.

So, while we're still working on the thank-you angle, allow me to introduce you to the Lizardfeeder. The LizardFeeder is a feed aggregator, whose source code is built atop Sam Ruby's Planet Venus. The LizardFeeder pulls together and archives activity streams from a wide variety of Mozilla community sources. Beyond the usual human-readable pages produced by a blog-gathering Planet, the LizardFeeder accumulates statistical and historical data meant for consumption and analysis by robots.

At present, the only robot navigating the LizardFeeder archives is an AJAX-ified user interface that animates the firehose as a near real-time or time-lapsed stream of events scrolling by.

This is just meant as a conversation starter, though. I'm hoping to gather feedback and find more sources, as well as to entice creative community members to come up with more sophisticated visualizations of this data.

So, take a look, check it out, and let me know what you think!

Archived Comments

dria
2009-01-06T01:26:18

Is there a list of what sources are already being read by the LizardFeeder anywhere? I scanned through the various links here but didn't see anything obvious :)
l.m.orchard
2009-01-06T01:37:15

Yeah, that part could use some improvement. There's a monster list here:

http://feeds.mozilla.com/sources.opml
Brian King
2009-01-06T10:11:46

Excellent stuff. I was going to ask about access to the list of sources for each category, but Dria beat me to it.
l.m.orchard
2009-01-06T15:26:50

Actually, it occurs to me that this config file might work as a more readable version of the list of sources:

https://svn.mozilla.org/projects/lizardfeeder/trunk/conf/config.ini-dist

There's also this, which is where most of the previous list came from:

https://svn.mozilla.org/projects/lizardfeeder/trunk/conf/hg-feeds.opml-dist
Austin King
2009-01-06T20:55:52

Thanks for the background surrounding lizard feeder. Great post!

+1 Dria and Brian

Maybe make the title of the link to the OPML more descriptive than just "Feeds" and/or link to it in the body of the UI too and write something around it to encourage other's visualizations.

Awesome work Les.
Sanjay Parekh
2009-02-04T20:38:04

Is the AJAX UI available anywhere for download? I'd like to hack it for another use altogether. Great visualization and great application. Good job.
Deen Seth.
2009-09-16T18:21:02

This is a very good idea. Can we accomplish the same result using Yahoo Pipe? Do you plan to add events from Bugzilla, and mailing list to the feed?

I am more interested in development activities. There aren't much in code category.

I am interested in analyzing development related events.

OPML reading lists in FeedMagick2

2007-10-17T07:22:47+00:00

For anyone who's interested: I've been hacking a little bit on FeedMagick2 again, with the latest addition being an OPML reading list feed blender.

What's an OPML reading list? Basically, it's the same as as OPML export of a feed reader's subscription list - only rather than doing a one-time import into another program, the OPML is itself treated as a live feed. A feed reader that supports OPML reading lists will continually check the list for updates and sync RSS/Atom feed subscriptions with its contents, maybe in a special sub-folder.

Here's a quick demo:

The itch I mean to eventually scratch is to replace the front page of decafbad.com with a live updating aggregation of the stuff I create and capture daily on the web. It'll be basically a self-assembling tumblelog pulled from many different services across the web. It'll also replace the footer of accumulated crud I've got on this very blog - which I thought was a good idea at one point, but now consider NeatLikeDigitalWatches.

With that in mind, the next thing I plan to develop is an hAtom module or XSL transform. This will turn the blended feed into an XHTML page. Maybe someday, hAtomic will launch, and I'll have a nice pretty style for the page too. Some time after that, I might work up a module that stows away dated historical archives of the feed and pages. I have further plans and ideas, but I'm trying to focus on the itchy spots first so that I might actually get something done in this round of serial enthusiasm.

Say hello to FeedMagick2

2007-04-30T03:06:32+00:00

Yeah, things have been basically silent around here thanks to post-work brain fryage and a general lack of things to say. Really, everyone else around the blogosphere seems to be covering things satisfactorily. However, I have been idly working on a new project over the past few weeks, namely a total rewrite and redesign of my format-ignorant feed filtering and munging kit dubbed FeedMagick.

You can find a demo installation of FeedMagick2 here and find it ready for checkout from SVN over here. It's basically just a step away from being a proof of concept, but I'm hoping to get around to fleshing out docs and battening down the hatches with tests. In any case, if my serial enthusiasm holds out, this thing could eventually subsume everything else I've done with feeds.

For now, peek at some of these highlights:

Master Personal Feed - One big feed blended from 10 other personal metadata feeds pulled from various Web-2.0-ish sites.
Feed to JSON via Magpie - Get feed data parsed by way of Magpie into JSON data structures
Flickr Favorites Feed - Feed of photos marked as favorites by a Flickr user, pulled via the API
jbox.com scraper - Pipeline composed of HTML Tidy and XSL to scrape jbox.com to build an RSS feed of new items for sale.

Beyond practical examples, there are some things under the hood that seem keen to me. Apropos of my pipes-via-web ramblings back in February, I'm trying out a few different approaches to pipelining feed content through processor modules. My original FeedMagick relied on feeding URLs to URLs as parameters. That, unfortunately, can be mighty cumbersome and inefficient. So, FeedMagick2 explores a few more approaches:

The first and obvious approach is to chain them together in a single script. So, I've got objects instances that pass content from one to the next. The thing is, the pipe works in reverse: The driver script asks the last module in the pipe for content, which then asks the one before it for content, and so on. At any point along the way, modules can cache the output of previous modules, and refrain from calling up the chain.
The second way to chain pipelines together is just like the first FeedMagick: Some pipelines start with fetching a URL. That can be an original feed, or a URL leading to the output of an antecedent pipeline. And, oh yeah, most pipelines are run via parameterized URLs, so there's that bit of handy recursion.
The third way to chain pipelines together is with HTTP POST: A pipeline can accept feed data via the request body of an HTTP POST, thus allowing antecedent pipelines (or even cURL scripts) to push data into the pipeline rather than getting pulled via URL. This is kind of like my years-old jiggery pokery with pipelines via XML-RPC, only much much simpler.

I'm also poking around at making all of the above available at the command line via PHP-CLI, and I'm having gratuitous fun exploring PEAR to roll my own stripped-down web framework. I still hate PHP, but I'm at least finding ways to entertain myself while I'm holding my nose. Of course, I find weird things entertaining.

And, as a side note, the only reason I'm using PHP is because I'd like to play around with the idea of the de facto WordPress installation requirements standard. That is: If you can run WordPress, you can run this. In reality, I don't think I'm there, but I'm hoping to get close. For one, I'm refusing to play with anything older than PHP 5.

Anyway, play with it, tell me what you think and give me a reason to keep hacking at it. :)

Archived Comments

jamesv
2007-04-30T11:39:15

Aw man, now I've got to port all the code you wrote while you were here over to this new hotness ;) I really like (and appreciate) the single script approach.

Is caching at a module level done automatically, or is that something I need to flag in my original call? Some installs of the original code base are now aggregating large sections of a pool of around 400 feeds, and eeking out even minor performances gains would be just lovely.
dRAUPP
2007-04-30T13:40:16

hawt.
l.m.orchard
2007-04-30T14:43:54

@jamesv: Take a look at the source to this big-ish pipeline, all the way at the end:

http://decafbad.com/2007/04/FeedMagick2/pipelines/masterfeed

You can basically slap a Cacher module at the tail-end or even middle of a long string of modules, and it'll cache the results of everything before it. I've got a cache lifetime set in the conf/config.php, and you can also set the lifetime in the Cacher parameters. There can multiple Cacher's per pipeline too.

This thing might not quite yet be even as stable / in working order as the original FeedMagick, but it might be worth poking at for you. :)
kellan
2007-04-30T16:02:31

For one, I’m refusing to play with anything older than PHP 5.

That must be nice.

And curse you, this looks interesting, now I've got to find time to look at it.
John Manoogian III (jm3)
2007-04-30T17:53:08

addendum to README:
Installation
cp conf/config.php-dist conf/config.php chmod a+w logs - RewriteBase /~lorchard/FeedMagick2 + RewriteBase /FeedMagick2
l.m.orchard
2007-04-30T17:58:49

@jm3: Ah! Good catch. I really need to eventually installer-ify that kind of stuff.
megalar
2007-08-05T12:25:34
Yesterday my host upgraded to php5 so I ran over to your svn dump and installed feedmagick2. After editing the $baseurl and rewritebase i tested it and it worked like a charm, so I took a nap. Upon waking I was gonna go play with it and see what hacks I could get going with it but something was fubar. Warning: fopen(/home/megalar/www/html/feedmagick/logs/feedmagick2-debug-20070805.log) [function.fopen]: failed to open stream: Permission denied in /usr/share/php/Log/file.php on line 216 are the errors I get as you can see @ my site. If it hadn't worked to begin with I would think module problems on my server. To my knowledge, and my host's knowledge, nothing has changed since the upgrade so I'm wondering if it is some sort of bug or a server problem that waited a few hours to reveal itself. The latter doesn't really make much sense to me but I can't rule it out since I'm not r00t on the box and am not 100% certain my host didn't bork something while I slept.
```
              anyhoo, your thoughts?
```

Ficlets enhanced author feed, an XSL scraper hack

2007-04-05T05:37:00+00:00

I've been trying to get myself serious about writing and even set up a personal slush pile for my output. Then, I found Ficlets, and spewed a few quick starter stories there. And then... I stopped. I'm hoping to pick it up again very soon, but I guess that's the nature of my serial enthusiasm—it doesn't just apply to hacking.

So, here's something ironic: I just spent a few hours tonight throwing together a hack for Ficlets. See, Ficlets runs on original stories, comments, ratings, and sequels and prequels to stories. You can get an Atom feed of an author's stories and a feed of comments—but it seems like there's no way to get notified of prequels and sequels, which are a very gratifying part of the whole shebang.

With that in mind, check out this RSS feed:

http://decafbad.com/2005/12/FeedMagick/www-bin/ficlets.php?author=l_m_orchard

That's a blend of all my stories, comments on my stories, as well as prequels and sequels found for my stories. Subscribing to that feed will give me updates whenever there's anything new in all the above. It's thrown together using a semi-crazy mix of my FeedMagick package for caching, and some XSL for scraping. If you'd like a feed like this of your own, just replace l_m_orchard for your own author name in the author parameter.

Please note, however, that this little service is hosted on my site and may go away at any time for any reason.

In case you're interested in what's under the hood, here's the quick and dirty XSL that's behind it:

http://decafbad.com/2007/04/ficlets.xsl

This thing's made possible because the Ficlets feeds are XML, and so are the XHTML pages happily infested with microformats. If they ever go invalid, this scraper breaks. But, that's the nature of scrapers, and it works for now. Oh, and although they provide Atom, this feed is RSS 2.0. Why? Because it was easier that way. I might put some more effort into an Atom feed, but my itch has so far been scratched.

Archived Comments

megalar
2007-04-05T09:31:48

I had already decided I was gonna grab your FeedMagick source and try to do something with it but this just gives me more incentive as a Ficlets lover. I recently read your thoughts on pipes and feel much the same way => meh to GUI, especially a buggy one. It seems much more satisfying to just tell php what I want done and watch it go rather than spend 20 minutes fiddling with a silly applet.
l.m.orchard
2007-04-05T14:04:06

@meglar: Well, just to warn ya - FeedMagick needs a lot of work. :) I've got ideas for it, but have yet to get back around to it.
Kevin Lawver
2007-04-05T14:59:37

That may be the coolest thing ever. Good job!
l.m.orchard
2007-04-05T15:14:00

@Kevin: One thing that might not be cool is that this XSL hits Ficlets.com once for each story in the feed to pick up the additional info, but I try to cache the feed on my end for about 20 minutes per author... so hopefully it doesn't put any more stress on your site than I might have done manually in opening all my stories in tabs to check for comments and prequels/sequels. :)
Kevin Lawver
2007-04-05T15:44:25

That's fine. I think we can handle it. We'll get prequels/sequels added to either the author feed or their own feed in the near future.
l.m.orchard
2007-04-05T21:05:26

@Kevin: Yeah, I think the bare author feed is a good thing to leave alone. But, a personal author feed with comments and prequels/sequels and possibly notes would be excellent! You know, basically, notifications of significant events that'd bring me back to the site.

I'm also thinking it might be neat to gently spider through prequels and sequels on stories to map them out and track progress, but that's just a future hackery idea.
Roger Costello
2007-04-06T13:54:55

Is there any way for me to view the documents you are mashing up to generate the resulting RSS document? I am particularly interested in seeing how Microformats are being mashed up. Thanks!
l.m.orchard
2007-04-06T14:25:44

Roger: Sure... Really, the only things I'm mashing up are the author Atom feed, like mine here:

http://ficlets.com/feeds/author/lmorchard

And then, I chase down each story in the feed, like this one:

http://ficlets.com/stories/1763

You can view source on a story page and see all the information sprinkled in there - look for 'abbr' tags and class names like 'pubdate' and 'hentry'. The pages are also valid XML, so they can be manipulated right in XSL.
Bill
2009-01-28T00:36:02

I'm sure that many Ficlets fans are sad that Ficlets.com is no more. We are in the process of constructing Blurbtree.com a site that might be helpful for them. It's not a replacement for Ficlets, but it's a clean start and we'll build the site based on the feedback we get from the online community. Please give it a try and let us know what we can do to make it fit your needs. You can also get more information at our Blurbtree FAQ.

Blog posting delegation and third-party auth

2007-03-05T22:13:45+00:00

Here's something I've been meaning to post about, brought back to mind from Kim Cameron's post on "Wrong-headed impersonation":

I wish that blog posting interfaces (ie. MetaWeblog API and Atom Publishing Protocol) offered a way to delegate blog posting to a 3rd party app (desktop or web) in such a way as to avoid providing one's login details (i.e. user name and password). For instance, consider both Flickr's and Upcoming's 3rd party token-based authentication / authorization schemes.

In particular, I'm looking at things like del.icio.us' own Daily Blog Post and others. These can be used to auto-post content to one's blog generated elsewhere - but at the price of sharing login details. Granted, you can mostly trust these 3rd parties not to do anything nasty with your credentials, but it would be nice not to have to.

I figure that something RESTful like extending HTTP authentication (ala Atom Authentication) with a token scheme could be interesting, and possibly fit nicely into APP itself. It could probably be retrofit into the MetaWeblog API by specifying a per-app user name and password. I can imagine a WordPress admin plugin that issues approved authentication tokens to restrict the categories and other activities allowed by 3rd party apps.

Just something I'm thinking about, as more services may or may not grow into delegated blog posting.

Archived Comments

fluffy
2007-03-05T22:40:26

Wouldn't openID be a better candidate for this?
l.m.orchard
2007-03-05T22:48:24

Not really, unless I don't entirely understand OpenID.

OpenID offers way to authenticate yourself. However, what I want is an easy way to delegate your access to 3rd party apps, without those apps being able to authenticate as you or performs actions outside of the permissions you approve. I don't really want Flickr Uploader, Ecto, MarsEdit to be able to use OpenID to login as me with carte blanche.

With a token-based system like Flickr has, I can approve delegation to various apps and revoke their access at any time - without ever releasing my login details.
George Hotelling
2007-03-06T17:51:55

At first I had this great idea to build a blog posting proxy service that could grant access to your blog to different services by giving them different usernames and passwords and you could revoke them at any time and it would be great and everyone would be posting to everyone's blog all the time.

Then I realized I had put on complicator gloves and that it would be way easier to just add new users to my blog without a proxy service.
l.m.orchard
2007-03-06T18:04:57

Yeah, I figured making multiple accounts is a way to go when you've got your own install of something like WordPress. It kind of gets stymied, though, if you want the same thing for a hosted service like WordPress.com, Vox, TypePad, or LiveJournal where one account = one blog.
Ian McKellar
2007-03-12T17:43:49

It seems to me that the ideal solution to this is to have your blog pull new posts from an RSS/Atom feed that's hosted by the service that is generating your content. Feedburner already has a half-assed implementation of this where they'll pull your delicious bookmarks and flickr photos into your burned feed.
l.m.orchard
2007-03-12T17:58:35

@ian: Hmm, yeah, that probably does end up being the least onerous solution - just offer an aggregated feed of one's output. I do like the idea of auto-posting to a blog, though, for the sake of something sorta like daily backups to a system I control.
Ian McKellar
2007-03-12T18:44:46

@l.m.orchard: you don't need to just aggregate you can also import posts into your own store. We could come up with a cleverly confusing buzzword acronym like Atom Pull Publishing. You just periodically poll a feed and import the items. Actually, this is kind of what Google Reader is doing. They've even got a namespace in their atom for expressing the original id and the source feed. Perhaps I should write a drupal module.
l.m.orchard
2007-03-12T19:01:45

@ian: Hmm, yeah, that does sound like the least complicated way to get outside content pulled into a blog without releasing the blog's login details. I know I've seen / used some WordPress plugins that do basically that, though I've not been happy with any of them yet. I should re-look into why that's been the case
Ian McKellar
2007-03-12T19:26:16

@l.m.orchard: in my experience the problems are with the formatting of the content (do blank lines turn into paragraphs, etc) and fragile post IDs (the upgraded wordpress and spammed all my friends on livejournal or planet problem). I'm not sure how to fix either of these problems but I think they're both problems Atom is attempting to solve...

Thoughts on Pipes on the Web - Part II

2007-02-15T08:49:56+00:00

In the last post, I expressed concern for my own well-being over a lack of head-over-heels love for Yahoo! Pipes. On the surface, I think it's because it's not a freshly discovered concept for me, and also probably because I'm tired and a bit hacked out right now. But, I think there's a bit more to it.

First of all, I'm not impressed by GUI builders for most things. Don't get me wrong: That Pipes GUI is pretty sweet and quite an impressive implementation — but as a rule, such things don't quite get my geek on. So, there's that.

Another thing is that I've done something kinda like Yahoo! Pipes, called FeedMagick — only, it doesn't have a GUI and I mostly abandoned it after releasing the code and having used it for a project at my old job. It's one of those serial enthusiasms that I've figured I'd circle back to eventually. I still think it's a pretty cool idea.

All kudos to the Yahoo! Pipes team though: Unlike me, they've actually got a living and breathing project — which trumps a paged-to-disk serial enthusiasm most days.

So what else is really curbing my enthusiasm? Well, Yahoo! Pipes looks like a pretty self-contained pipes engine — data goes in one end, frobnication happens in the middle, data comes out the other side.

But, what I like about the notion of the URL-line is that you can take one URL and supply it as a parameter to another URL — making messy pipelined URLs while building a crazy web-wide distributed execution environment powered by HTTP and REST. This is the kernel of the notion that I think really excites me about pipe on the web — I just haven't had a chance to do much with it lately.

Depending on the perspective, true pipes on the web — that is, URLs fetching URLs — look to me like functional programming ala Lisp. Consider an idempotent GET request as a pure function call with no side-effects. Then, consider a GET request that accepts a URL as a parameter — it's a nested function call: the outer GET must make an inner GET to fetch the parameter-supplied URL. Give the inner URL another URL as a parameter, and you've got yet another nested function call.

But, if you like, ignore the theoretical benefits of functional programming — flip the nested function calls inside out and you've got a pipe. And, since you're using HTTP GET, you can get all the benefits of HTTP — like caching of execution results and a web full of distributed processing nodes, among other things.

I haven't explored Yahoo! Pipes deeply enough yet, so maybe I'm missing the features that pipe authors can used to call on other distributed pipe elements out on the web at large. But, I think that's what ultimately gets me psyched about pipes the web and hasn't yet for Yahoo! Pipes.

And, of course, I think I'm just a little hacked out and tired of whizbang new stuff right now. :)

Archived Comments

John Evans
2007-02-15T13:59:01

The thing that struck me about Pipes is that it seemed rather limiting to someone who can code but too geeky to someone who can't. Maybe I am wrong but I wonder that once the hype dies down who exactly the audience will be long term.
Stuart Robinson
2007-02-15T19:35:41

Don't be so apologetic! Conceptually it's cool, but the UI quickly becomes frustrating for anyone with text-based coding experience. I guess this is the web's visual basic and we are all going to be more interested in it's perl.

Thoughts on Pipes on the Web

2007-02-15T08:09:56+00:00

By way of my meme tracker, I found something like 15 entries linking to Yahoo! Pipes from my feed subscriptions. That's crazy — I've usually never seen more than 3-5 feeds linking to anyone thing, maybe as many as 8 if it's something really hot. I read about it over morning coffee and aggregator skimming; I heard about it on TWiT on the bus to work; I was told all about it in the hallway once I got to work. Yahoo! Pipes has been hard to miss.

With that in mind, it seems weird to me that I didn't get as excited about the release of Pipes as everyone else in my local blogosphere seems to have. Pipes has all the ingredients of something that should really set me off. Pipes, filters, mashups, RSS, Atom, RDF — (singing) these are a few of my favorite things! So, what gives? I'm genuinely worried that I'm sick, depressed, losing it, or otherwise replaced by a pod alien. Possibly all the above.

I knew about Pipes as an internal project at Yahoo! quite awhile ago — possibly even in my first week or two after having joined. I'm fuzzy on the timing here, but it's not important. Someone mentioned it to me at some point, I poked at it a bit from the internal network, and thought it was neat / keen. My main impression at the time was, "Cool, someone's doing FeedMagick, but done right and with a slick visual layout tool."

Then, I got really busy with del.icio.us and wandered off. So, the release wasn't big surprising news to me — my reaction was something like, "Huh, so they did a public release of that? Nice." Clearly I need more caffeine.

The thing is that the idea of pipes on the web is not itself a new concept to me. I think my XmlRpcFilteringPipes seems kinda silly in retrospect, and I'm all about the Power of the URL-line nowadays — but suffice it to say that I've been thinking about pipes on the web since as long ago as 2002 or so.

So, while I'm all jaded and apparently incapable of feeling the wonder, this seems to be one of the biggest revelations to people playing with Yahoo! Pipes:

Pipes? On the web? BRILLIANT!

So, it's not you, it's me. I think I've got a serious deficit of enthusiasm for what's really a pretty cool thing. I want to think a bit more about why that is.

Archived Comments

Kenneth
2007-02-15T14:06:37

http://yakubovich.blogspot.com/2007/01/visipipe-visual-version-of-core-utils.html

Might have been buried in all the buzz about Yahoo, but this sounds like a neat idea.

Kenneth Writer
Jeff Daly
2007-02-16T01:08:54

You're not alone. I was excited when I learned about Pipes, but ultimately I was let down by the lack of parsing tools. Pipes is seriously limited without any XPath/regular expression parsing capabilities. Fortunately, there is a post on XPath in the feature request page.

content sniffing sucks

2006-11-24T06:28:11+00:00

If you’re using FF2.0 go here and you’ll see why.
No, I don’t want to subscribe to a PHP template used to generate Atom feeds, thank you very much.

Source: snellspace.com » Blog Archive » Content Sniffing Sucks

I know this is just taunting the Happy Fun Ball I said I was done taunting , but there's a false-positive for ya. :)

MyDreamApp disappointments and Hijack

2006-10-26T16:56:30+00:00

After five grueling rounds, the MDA winners have been announced. Let the development begin!

Source: digg - MyDreamApp, 3 Winners Announced!

Spoiler alert - the winners are Cookbook, Portal, and Atmosphere. Pretty disappointing, since my faves were Hijack, Whistler, and Desktop Wars. Of course, I didn't do anything other than vote and occasionally forum-lurk.

Which, matter of fact, is why Hijack was my top favorite of all. I've gotten a lot of read-only coverage of the web from my news aggregator, but I've never ever been able to get into web-distributed forum groups. Some have RSS feeds, but most don't and I never remember to revisit them. I love the power of the web, and I love distributed apps - but sometimes I miss Usenet.

Seems like there's a need for Atom-ized comments - that is, a data format for forum aggregators to read and an API for reply submission with spam resistance. Hijack featured forum HTML scraping as a proposed feature - but what if you could get adoption for some simple plugins for Wordpress, phpBB, Vanilla, etal? co.mments seemed promising, but I've just not gotten into it much. I need to do some poking around to see who is already working on this.

Archived Comments

Dan Dickinson
2006-10-26T18:20:16

It's funny; a week ago, something set me off on writing a "Why do we not have forum aggregators?!" post, and then a bit of googling led me to Hijack, and then the massive disappointment last night that it lost.

And then this morning, I was just thinking that it'd be easily doable in Atom.

I need to stop thinking in other people's blog posts.

(If you do get going on this, let me know.)
Aristotle Pagaltzis
2006-10-27T09:45:34

This should be a pretty trivial job of stringing together the APP and Atom Threading (RFC 4685).

(Hmm, where did your AJAX preview button go? I hope the Markdown links are interpreted properly…)

Use del.icio.us to build & share Reading Lists?

2006-01-19T20:08:37+00:00

As far as I know, the most popular link managment tool is del.icio.us, a tool I love for its power and simplicity. del.icio.us allow you to export all your links in RSS which is cool. So, I wrote a quick and dirty PHP script that converts this RSS export to an OPML list (see at the end of this post).

Source: vrypan|net|log » del.icio.us as feed manager

I haven't had much chance to dig into Dave Winer's new newsRiver tool for the OPML Editor, but I've at least installed it and nodded at it. It's not much more than the aggregator I used to use in Radio—but the big difference is the promise of fresh new development.

And yeah, I know, I groused about OPML versus XOXO not more than a few months ago. But, although I still would prefer cool things being done in XOXO, Reading Lists (in OPML) would appear to be one way to get the stratified River of News that I'm looking for. That is, a Reading List can be a news strata in an aggregator.

But, beyond my prioritized folders in NetNewsWire, a Reading List in OPML can be serialized and shared and subscribed. Yeah, aggregators can already import and export OPML subscription lists—but I've never seen this feature automated by subscription until now. This is the difference between an MP3 download and a podcast.

And, if you check out the above-quoted blog post, these could even be collaboratively maintained by way of del.icio.us and a simple RSS-to-OPML gateway service. I've got to say, this could be one of the niftiest bits of mojo to hit syndication and aggregation tech in awhile.

Look mom, I'm on the podcast!

2006-01-07T16:24:02+00:00

I just caught a brief mention of Hacking RSS and Atom on Show #45 of the Mike Tech Show.

Thanks for the plug, Mike! I hope you enjoy the book!

Sharing attention while reading feeds

2006-01-03T14:45:01+00:00

Instead of reading their individual selections of RSS feeds privately, everyone should be encouraged to publish those aggregated feeds on the Web. ... the simple act of publishing those aggregations then makes them available to others, and thus makes them amenable to network effects in a way that they never can be if they're kept private.

Source: » It's time to bury RSS | Software as services | ZDNet.com

My current two responses to the above:

I love my popular links summary feed.
Attention.xml is starting to make sense to me.

Putting these two together could be a very powerful tool for bringing network effects to feed reading.

First off, one of the most useful feeds I have is my private Popular Links feed. Touting my book again, I showcase the script behind this thing in Chapter 15. I've had this running for well over a year, and it's always my first stop in the feed reader. Almost without fail, this tends to surface what's buzzworthy.

Basically, this script scans all the current entries of all my subscribed feeds for unique hyperlink URLs found in descriptions and summaries. It collates all entries by these links, then sorts by the number of entries under each link. A threshold is applied, filtering for links pointed to by 3 or more entries.

At the end, I see a new feed entry displaying the most linked-to things of the moment. Think of this as a kind of real-time PageRank. How's that for network effects?

I think this is sort of how Memeorandum works, only this is constrained to the feeds in my subscription list. I've been considering making this script a full-on service: Upload an OPML export from your aggregator, get your own Popular Links feed. I've got all the parts laying around but I haven't yet had time to put them together—but if it sounds like something useful, and possibly worth clicking a Paypal donation button, let me know!

Secondly: Attention.xml makes a lot of sense with regard to the above-quoted article. When I first heard about Attention.xml, I merely cocked my head at it and made a confused sound. This was before I caught the microformats bug, and before I realized that I started reinventing it a bit in my own ramblings.

Basically, Attention.xml is a feedroll enriched with data about the entries you've lately read from each feed. It's in an XHTML-based format which—albeit ugly in my opinion and in need of more elegant microformat influence—is indeed viewable in a browser. In a sense, this format is an auto-blog of my feed consumption. I was looking for hacks for my NetNewsWire in AppleScript, when I found this Attention.xml generator. Seeing the output of that, it all clicked.

Combining Attention.xml with my Popular Links algorithm could be a very powerful thing, methinks. Rather than waiting for my friends to tip over the laziness point to blogging about something, I could digest their shared Attention.xml files and collate the links they've merely read. In this way, I could build an AttentionRank for various things, and cause the cream to rise to the top in my feed reader.

I'm pretty sure that I'm playing catch-up here, but this all suddenly seems hot to me. And not to mention, it seems neat that I have all the pieces laying around to build it. The only bad thing is that I just don't have the time to spare—I've already spent too much time writing this blog post!

Anyway, maybe you'll hear more from me about this soon.

Archived Comments

Daniel Nicolas
2006-01-03T15:39:45

I think you could get away with charging $5 for a service like that. Maybe even $10 if it was all web-based with cool ajaxyness flowing out of it.
Anne Zelenka
2006-01-03T19:26:21

That Ch. 15 script looks very useful, though I haven't had a chance to try it yet. The attention.xml thing... interesting but don't know how it would work with three-panel newsreaders like Bloglines. I'm not necessarily interested in a post just because I skimmed the latest articles in a certain feed. I didn't know before I clicked on the feed name whether I'd be interested in the latest stuff. Much of the time I'm not interested.

What I'd like to see is some smart recommendation engine that watches how I read, how much time I spend on a certain article display, whether I click through the links it has, and even provides a "boring" button like some other commenter suggested so it could do some Bayesian filtering. That's getting far beyond what you've described here but what you're describing is a start on it.

Another somewhat different issue is that Memeorandum works by limiting itself to a certain very popular and generic set of issues. Once you have one Memeorandum for politics or tech do you need another? Might individual quirks of attention (like the fact that you are interested in Detroit and I am interested in Maui) make sharing individual attention.xmls less useful? I've never really gotten the social sharing bug though so maybe I'm just showing a lack of imagination.

But just because Memeorandum already exists doesn't mean there isn't a real opportunity. While it's easy to find the hot conversations on politics and tech in the blogosphere, other domains are poorly aggregated and filtered, if at all. I follow mom blogs and feminist blogs and nothing like Memeorandum or digg or reddit exists for those yet there are regularly hot topics that ripple through them. What you describe could be used to build something for those and other domains on the web but is there room for everyone to do it individually? Or will people gravitate to one or a few sites that do a good job of highlighting popular and important conversations in certain well-defined domains?

These are just my raw thoughts on it... your ideas are inspiring.
Phil Wilson
2006-01-04T16:38:10

Presumably your subscription list needs to be of a certain size before you can start extracting interesting URLs or you'll just have 20 URLs mentioned twice and 50 URLs mentioned once.

If you were to just read the short head of your popular items list, would this actually be encouraging people to be infovores rather than normal information consumers (where you read or at least skim all of the items in your aggregator)? i.e. the data only starts working for you when you collect enough feeds. Gosh, you could have your own little long-tail on your desktop ;)
Bill Seitz
2006-01-06T16:45:16

have you look at at TailRank?

http://tailrank.com/

FeedMagick gains an RSS-to-JSON filter

2005-12-19T03:01:51+00:00

So, I put some more infrastructure behind FeedMagick and lifted a RSS-to-JSON idea from John Resig to transform from MagpieRSS parsing to JSON output.

Here's a spiffy JSON-enriched demo. It was surprisingly easy, all told.

I'm not sure how much time I'll have to really put into this project, but this is the first decent effort I've put into a URL-line suite in a long time. I'm trying to throw in some bits to make it easy to build, document, and use these feed processing commands.