0xDECAFBAD - Tag: xsl

i (heart) ficlets, microformats, and feed scrapers

2007-04-05T06:00:05+00:00

I've really got to reiterate / call out a few points from that last entry:

I ❤ making feed scrapers, although they're sometimes a pain.
I ❤ Microformats, for making scrapers easy.
I ❤ Ficlets for using Microformats and making it easy for me to use their site as an API using just XSL and HTTP GET.

I'd love to see more sites like Ficlets.

Archived Comments

Rube
2007-05-07T21:45:17

Ficlets == best software name EVER (at least if you speak German)

Ficlets enhanced author feed, an XSL scraper hack

2007-04-05T05:37:00+00:00

I've been trying to get myself serious about writing and even set up a personal slush pile for my output. Then, I found Ficlets, and spewed a few quick starter stories there. And then... I stopped. I'm hoping to pick it up again very soon, but I guess that's the nature of my serial enthusiasm—it doesn't just apply to hacking.

So, here's something ironic: I just spent a few hours tonight throwing together a hack for Ficlets. See, Ficlets runs on original stories, comments, ratings, and sequels and prequels to stories. You can get an Atom feed of an author's stories and a feed of comments—but it seems like there's no way to get notified of prequels and sequels, which are a very gratifying part of the whole shebang.

With that in mind, check out this RSS feed:

http://decafbad.com/2005/12/FeedMagick/www-bin/ficlets.php?author=l_m_orchard

That's a blend of all my stories, comments on my stories, as well as prequels and sequels found for my stories. Subscribing to that feed will give me updates whenever there's anything new in all the above. It's thrown together using a semi-crazy mix of my FeedMagick package for caching, and some XSL for scraping. If you'd like a feed like this of your own, just replace l_m_orchard for your own author name in the author parameter.

Please note, however, that this little service is hosted on my site and may go away at any time for any reason.

In case you're interested in what's under the hood, here's the quick and dirty XSL that's behind it:

http://decafbad.com/2007/04/ficlets.xsl

This thing's made possible because the Ficlets feeds are XML, and so are the XHTML pages happily infested with microformats. If they ever go invalid, this scraper breaks. But, that's the nature of scrapers, and it works for now. Oh, and although they provide Atom, this feed is RSS 2.0. Why? Because it was easier that way. I might put some more effort into an Atom feed, but my itch has so far been scratched.

Archived Comments

megalar
2007-04-05T09:31:48

I had already decided I was gonna grab your FeedMagick source and try to do something with it but this just gives me more incentive as a Ficlets lover. I recently read your thoughts on pipes and feel much the same way => meh to GUI, especially a buggy one. It seems much more satisfying to just tell php what I want done and watch it go rather than spend 20 minutes fiddling with a silly applet.
l.m.orchard
2007-04-05T14:04:06

@meglar: Well, just to warn ya - FeedMagick needs a lot of work. :) I've got ideas for it, but have yet to get back around to it.
Kevin Lawver
2007-04-05T14:59:37

That may be the coolest thing ever. Good job!
l.m.orchard
2007-04-05T15:14:00

@Kevin: One thing that might not be cool is that this XSL hits Ficlets.com once for each story in the feed to pick up the additional info, but I try to cache the feed on my end for about 20 minutes per author... so hopefully it doesn't put any more stress on your site than I might have done manually in opening all my stories in tabs to check for comments and prequels/sequels. :)
Kevin Lawver
2007-04-05T15:44:25

That's fine. I think we can handle it. We'll get prequels/sequels added to either the author feed or their own feed in the near future.
l.m.orchard
2007-04-05T21:05:26

@Kevin: Yeah, I think the bare author feed is a good thing to leave alone. But, a personal author feed with comments and prequels/sequels and possibly notes would be excellent! You know, basically, notifications of significant events that'd bring me back to the site.

I'm also thinking it might be neat to gently spider through prequels and sequels on stories to map them out and track progress, but that's just a future hackery idea.
Roger Costello
2007-04-06T13:54:55

Is there any way for me to view the documents you are mashing up to generate the resulting RSS document? I am particularly interested in seeing how Microformats are being mashed up. Thanks!
l.m.orchard
2007-04-06T14:25:44

Roger: Sure... Really, the only things I'm mashing up are the author Atom feed, like mine here:

http://ficlets.com/feeds/author/lmorchard

And then, I chase down each story in the feed, like this one:

http://ficlets.com/stories/1763

You can view source on a story page and see all the information sprinkled in there - look for 'abbr' tags and class names like 'pubdate' and 'hentry'. The pages are also valid XML, so they can be manipulated right in XSL.
Bill
2009-01-28T00:36:02

I'm sure that many Ficlets fans are sad that Ficlets.com is no more. We are in the process of constructing Blurbtree.com a site that might be helpful for them. It's not a replacement for Ficlets, but it's a clean start and we'll build the site based on the feedback we get from the online community. Please give it a try and let us know what we can do to make it fit your needs. You can also get more information at our Blurbtree FAQ.

content sniffing sucks

2006-11-24T06:28:11+00:00

If you’re using FF2.0 go here and you’ll see why.
No, I don’t want to subscribe to a PHP template used to generate Atom feeds, thank you very much.

Source: snellspace.com » Blog Archive » Content Sniffing Sucks

I know this is just taunting the Happy Fun Ball I said I was done taunting , but there's a false-positive for ya. :)

XoxoOutliner and further outline addressing adventures

2006-11-15T08:07:12+00:00

Revised the addressing code a bit, adding a few new kinds of addresses and getting ready to support sub-outline updates. That is, fetch a sub-branch of an outline and then later post a change to that sub-branch using the same address. Needs more thought - ie. what happens if things move between fetch and update? - but here are a few more samples:

First is a straight linear index counting down from the top of the outline:
- http://decafbad.com/2006/11/XoxoOutliner/outlines/README;index:4?format=xoxo
Second is a navigation of outline structure, alternating numbers and letters:
- http://decafbad.com/2006/11/XoxoOutliner/outlines/README;level:3c4?format=xoxo

That's all for now. In my next round of enthusiasm, I may try stealing Tom Morris' Opath idea...

Archived Comments

Vincent D Murphy
2006-11-18T20:52:07

I think (and said as much on Tom Morris' site) that a fragment identifier would be a better solution, in which case Opath would be a fragment identifier syntax for OPML and XOXO. At least it would be the best solution from a REST/web architecture point of view..
l.m.orchard
2006-11-18T21:40:41

One reason I didn't use the #identifier URI syntax for suboutlines is because some gymnastics need to be done to get the hash through to the server from a browser. Otherwise, it gets treated as an in-page anchor. The semicolon syntax seems to work well for a set of path-segment parameters, and follows the standard (if I've read it correctly).

In either case, it works for me, and should be just fine in a REST context - the suboutline syntax here should always identify a single parent outline node as a resource, and will eventually work for GET / PUT / POST / DELETE.

Now I just need to implement a solution for the Lost Update Problem.
Michael Poremba
2008-09-18T23:45:56

Wondering if you ever completed your online outliner? Check out dynamiclist.com, a functioning but incomlete project I launched back in 2001. The editor is rich and works well. Been thinking of reviving now that all major browsers support the contentEditable tag.

XoxoOutliner and suboutline addressing

2006-11-13T09:34:02+00:00

Here's a feature I just hacked together for XoxoOutliner and plan to refine further:

Not entirely sure that this is how I want this to work, but these three URLs demonstrate the ability to address and fetch subsets of outlines. I'm hoping this will be a basis for selective transclusion in other outlines, or maybe even in a sidebar of a blog. (Which, depending on the blog software, might be built from outlines anyway.)

firefox, rss, xsl - from anger to apathy

2006-11-07T21:46:35+00:00

Having skimmed through this Mozilla newsgroup thread, I'm perturbed anew about the new Firefox 2.0 misfeature with regard to RSS feeds and client-side XSL. My original grief was that it cost me a significant bit of pointless debugging time even discovering that it was a brand new gotcha. But, the "we know what's good for you" sentiment is clear - I feel like posting any newsgroup responses would be just more time wasted. At least shoving an arbitrary 512 bytes into the feed fixes the bug. (This seems to compress well: "AAAAAAAAAAAA ... AAAAAAAAAARGH!")

Brilliant: I'll just add that to the body of obscure CSS hacks and bug workarounds that'll someday make me give up web development for raising sheep. Thanks a million, guys.

Archived Comments

Mad William Flint
2006-11-07T22:00:34

Some days man, I'm not sure raising sheep would be such a bad idea.

30 years behind the keyboard is a long time.

Especially at 37.
l.m.orchard
2006-11-07T22:38:55

I'm about 6 years behind you, so let me know if you hit the sheep herding phase before me, and let me know how it treats ya. :)
Joe Grossberg
2006-11-07T23:13:23

Not to be pedantic, but don't you think there's a lot of random shit a shepherd needs to memorize too?

And you get to work indoors, avoid anthrax and keep your hands free of ovine feces.

I agree that the overriding of RSS styling is wack, but maybe you should go on a sheep-herding vacation if for no other reason than to realize this is a speck on a mote on a flea's arse on the giant ewe of web development. :)
Robert Sayre
2006-11-08T00:19:42

les, your feedback seems to fall into the "IT IS OBJECTIVELY WRONG" bucket. It does fail to account for the behavior of Safari 2 and IE 7, and doesn't account for authors who would like their stylesheet to show only in older browsers without RSS support. Why do you think it's good for the Web for Firefox to diverge from the new de-facto standard? Seems like pissing in the wind to me.

BTW, I can't say framing the decision as some collective character flaw of the Mozilla project is particularly productive.
l.m.orchard
2006-11-08T01:26:32

Joe: Sheep farming has always been my frustrated-programmer joke. It would be something different to memorize - not necessarily easier. But, hmm, I never thought about anthrax. I suppose that would be a job hazard to consider, beyond simple wasted time.

Robert: Admittedly, I'm frustrated. This change breaks things I've invested hours into across several jobs, and into which I've been investing hours again. At least I've been paid for those hours, and I'm glad I don't have to go back and explain / fix the stuff now broken at previous jobs.

From my perspective, I'm seeing all the latest generation browsers now breaking something I thought was standard - but about that I guess was mistaken. At least, it "smelled" like a fairly clean quasi-standard approach to me. I really don't have standards all memorized or even well studied.

But, in the new defacto standard, I get to stuff 512 random bytes into a feed to get the old behavior back. That certainly doesn't "smell" like a clean standard to me, even if IE 7 does it and Firefox 2 now follows. It still doesn't work in Safari. It seems a defacto standard to me in the same way that exploiting CSS parser errors to filter by browser is a defacto standard. Yet another weird edge case to memorize.

From my perspective, the defacto standard has been what FeedBurner and others have been doing with client-side XSL for feeds. And there've been some exciting things starting there - so there's some personal disappointment invoked. And now, all those feeds are broken in that respect. Retroactive wind-blown urine, at this point, I guess.

Maybe all this was really just a neat geeky trick best now forgotten, and the decision really will be the right one in the end. Nonetheless, I don't like it. So, perhaps not "IT IS OBJECTIVELY WRONG". But, from my perspective, it was an unexpected and unwanted surprise.

As for being productive, the decision seems made and others have already registered the same or better objections than I would just rehash. And I can't see with what collective character flaw I've painted the team - I'm perturbed, but I certainly don't think everyone involved is a villain. My apologies if my frustration vents that way. It's just that taking over feed styling from content providers strikes me as "we can do user experience better", and the explanations I've read don't seem to disagree with that notion.

But, I disagree and I don't like it. So, fine, from here on out I use the 512-byte-hack if I want custom XSL styling for feeds. Another trick in the bag. That's pretty much my last word on the subject, unless the capability disappears entirely.

All I have left to say is that I hope the client-side XSL-styled World of Warcraft home page never switches to an Atom or RSS format - whether or not using a non-HTML format for a front page was a good idea in the first place. I thought it was, but that could be my whole problem. :)
CraigM
2006-11-08T02:42:06

My "frustrated with technology" catch-phrase has always been to become a Buddhist Monk. That complete detatchment of technology seems to be quite the fantasy of all coders everywhere. Maybe that's saying something.
Robert Sayre
2006-11-08T03:07:32

By "collective character flaw", I meant that I've read you and other web authors writing "they think they know better than us". That's not how the decision was made. And to be honest, it bothers me somewhat to read a rant on the subject that only focuses on Firefox. To me, it seems like taking advantage of an open product development process because you know we will listen, while other vendors won't. A rant that claims "Firefox and IE and Safari suck" would be different, because then it is an opinion on a behavior. Criticism of browser behavior is welcome, criticism of people based on tech religion is not so welcome.

I understand that FeedBurner and others have a business interest here, but you have to understand that at least some of the things they do directly conflict with capabilities that are beneficial to browser users (that we can't deliver if it only works with half of the feeds out there). We want our users to have a coherent feed experience, with browser-specific preferences and minimal click-through screens. To that end, the screen that supplants the XSL was termed the "feed preview". It should show what's in the feed, and default to user-specified feed preferences. If the feed doesn't contain ads for aggregators and feed services (and wrong instructions), a preview that shows that stuff deceives our users. Not in an evil way, but in a "bad usability" way. When combined with authors who want a browser feed screen with a stylesheet for older browsers, and Safari/IE7 behavior, the scales tilted away from using the XSL.

I understand that people disagree here. But the fact that all browser vendors have done something that doesn't align with the way they used to do it should tell you that the issue is more complicated than "author's intent", for better or worse.
l.m.orchard
2006-11-08T04:38:57

Robert: You have a great point there - I do in fact think that Firefox, IE, and Safari all suck with this current change in feed handling. And yeah, the only reason I ranted about Firefox is because I think it's the only one of the three there's much chance in affecting. It's also the only browser I use, so it's the first one where the feed handling change really sunk in for me. I really haven't cared about it until just now.

Of course, I've been nowhere near participating in the newsgroups or code checkins or anything whatsoever related to actually contributing to Firefox - so this is all blog-chair quarterbacking from a syndication feeds enthusiast.

As for the "they think they know better than us" issue, I think it can be found in a newsgroup response like Mike Shaver's:

... one significant motivation was that very many uses of XSLT-on-RSS was to create a page that said "this is not for you browsers, go to the main site instead", which is not what we wanted the user's experience to be. Distinguishing those cases from the "righteous use of XSLT for dual presentation" ones is a publishable result, I submit.

To this, the most vocal respondents say, "But I really want my users to experience my righteous XSLT!" Even if it has flaming GIFs, dancing hamsters, and cat pictures. It's righteous, damn it. (Yeah, the quoted "righteous" is a little off-putting when you actually think the XSLT thing is kind of keen.)

Why not apply the new-and-improved styling by default and bow out when there's a processing instruction? Then, spread the word - "Get rid of your cruddy not-for-you XSLT!" I'm sure you'd get lots of converts grateful to stop screwing with their feeds, which would leave the outlier nutters like me and FeedBurner to keep happily shooting themselves in the feet. Instead, I see, "style 'em all and let hacks sort 'em out," which totally rubs me the wrong way.

At this point, I'm really just mildly disappointed, and feel like I've gotten a downgrade in an otherwise all-around great new browser. I'm not actually looking to farm sheep over this. There is a hack-around available for IE7 and FF2, so at least there's that.

I guess what would make me feel like I'd gotten a decent consolation prize would be if that Feed Preview screen were significantly richer and easier on the eyes than the other browsers' offerings. I'm assuming that's projected for the future. Maybe what I need is to do a checkout and dig around.
Mad William Flint
2006-11-08T04:41:06

CraigM: You may want to reconsider. I've known WAY too many buddhist monk programmers in my day.
Jason Clark
2006-11-08T13:19:11

conscious Robert: Why do you keep seeing people writing "they think they know better than us"? Because Mozilla has chosen to ignore an Author's preferences when displaying a feed. Period. You can explain it all you want, your reasoning may even be right- but the bottom line is, Mozilla made a conscious decision to ignore styling instructions included by the feed's author. The only reasons I can think of to do this is to be perverse (which I doubt), or because you think you know better (and from here on out, I use the collective you for Mozilla, because I'm tired of typing Mozilla).

You claim it's better for users. I say you think you know better what is good for your users than the feed author does. This is the same feed author that created the feed's content, and the stylesheet, and decided to present the feed to users with the stylesheet. I'm sorry, I just don't buy it.

Another non-starter is the IE7/Safari argument. I don't care what those browsers do; I chose Firefox because it decided to lead, not follow, when it comes to standards. Whether you believe that displaying RSS feeds with stylesheets is a standards issue or not, you can't deny it was a de-facto standard until recently.

And what about those authors who only want their stylesheets applied in older browsers that aren't as enlightened as IE7? Not that I've seen a public outcry from these authors in any of the forums discussing this "feature", but let's assume they must exist. They were smart enought to add the stylesheet to their feed, why not allow them to also add to their feed an indication that a browser's built in styling may trump the linked stylesheet (perhaps a PI)? Wouldn't that allow both types of authors to get what they want without wasting bandwidth in 1/2 kb increments of sheer hack?

I've read the arguments, here, on the buglist, and in the newsgroups. To me, and I suspect, to many others, Mozilla (or those who made this decision within Mozilla) has acted as though they think they know better than web site authors. Explanations such as yours appear to me as an explanation of why you think that, not a refutation that you don't.
Jason Clark
2006-11-08T13:21:50

Regarding the above: Sorry about the extra "conscious" at the beginning.... I was typing my comment in a text editor, and used the comment form to check the spelling of that word. Forgot to delete it when I pasted in my comment. It is not a poorly-formed attack on Robert :)
Tim
2006-11-27T04:16:54

I, as a user, don't think feeds should be styled. Feeds are used in ways where styles are not easy to apply, would cause theming chaos, and just look good when all in the same style.

That said, I heard Firefox 2.0 has an ugly default style, which, I guess, should be improved. I like IE7s feed view. Oh, and the arbitrary 512 byte switch is odd - reminds me of Internet Explorer's Friendly HTTP errors... Which weren't that friendly.

Hm, I might not be thinking in the same way a regular user is thinking...