syndication

I'm a little shocked to see that we haven't talked about Yahoo Pipes around these parts. It's an awfully cool service, and one that we use around the office for various non-mission-critical functions — getting larger photos from a Flickr stream for display on the monitors in the lobby, for one thing.

For those unaware, Pipes lets you mix, translate, annotate and otherwise alchemically modify feeds of all sorts. Want to add a (possibly) relevant photo to each entry on your blog? Or only get posts from a particular author on a group blog? Or get filtered Craiglist real estate listings translated into German and sent to your mobile phone? Pipes can do all of this, and does so via a terminally-slick GUI.

But recently a friend tipped me off to Pipes' most significant capability. The folks at Yahoo! have unveiled the Holy Grail of feed-manipulation components: a JSON-powered interface to external web services. If Pipes doesn't offer some functionality you need, you can now write a web service that does and connect it to the system.

Their example code is in Java, but it didn't take much effort for me to get one working in PHP. Just install the JSON PECL Package and use code like the following:

Partial-text RSS feeds are a pet peeve of mine. I'm not alone: I've read about Dave Winer and Steve Rubel's dislike of the practice. I'm sure there are a lot of other RSS users who are similarly irked by it.

So, after having a post-workout algorithmic epiphany (it's the best time for them), I started work on a little project to fix this annoyance — and ended up quite pleased with the result. You might find it useful, too: it's a little script that creates full-text RSS feeds from partial feeds. Just enter the URL of a partial feed in the box below and hit submit. You'll be directed to a URL that will (hopefully) provide a full-text version of the feed you specified.

I've been through a few different versions of the algorithm, but this one seems to be fairly universal and stable. It won't work for every partial-text feed, but it seems to work for a lot of them. I'm sure it could be better, which tempts me to open source the algorithm and invite people to improve upon it. But I won't — not yet, anyway.

I'm sensitive to the pressures that make bloggers use partial text feeds — some of my friends depend on selling advertising to support their sites. Unfortunately, RSS simply isn't respected by marketers and their clients. Offering a full text feed means fewer page views, which means less revenue — I've been told this bluntly by a friend who wanted to offer full text, did so, then noticed his revenues were shrinking. It's hard to fault him for returning to partial-text feeds.