scripting

Sometimes I wonder why I ever bothered to compile wget for OS X. I keep finding clever new things that curl can do — even some that make me wonder why I bother with Perl and Ruby's Mechanize module/gem.

For instance: today I found myself needing to export a friend's Typepad blog. He's got a lot of content, and I wanted to pull it down on a speedy connection and then gzip it before pulling it onto my laptop over my sluggish DSL (Typepad unhelpfully delivers its exported content in an uncompressed non-ML format).

Typepad provides a URL for the export, but it only works when you've authenticated through their website. Curl makes working around this pretty simple, though, as I found out from this Ask Metafilter thread. Here's how it works:

curl -k -d "__mode=redir&next=http://www.typepad.com/t/app&username=user%27s+name&password=their%20password" -c cookie.txt https://www.typepad.com/t/app

You simply use -k to turn off SSL requirements, -d to pass a series of key/value form pairs (properly url-encoded) to the URL to which the form posts, and -c to output the returned cookie to a file.

Then you pass in the session-containing cookie file, allowing you to stay authenticated on subsequent requests:

curl -b cookie.txt -o complete-export.txt "http://www.typepad.com/t/app/weblog/post?__mode=export&blog_id=YOUR_BLOG_ID"

Naturally, you'll want to replace the URL in that last request with the one corresponding to your own export URL.

It all worked perfectly — well, up until the point at which I discovered that Typepad exports seem to inevitably fail once they top 100 megs. Seriously guys: file compression. Try it, you'll like it.

To borrow a phrase from the execrable band Linkin Park, I'm breaking the habit... tonight! Or I was planning to, anyway. The habit in question? Perl.

I was powerless even as it strengthened its hold on me. It's not that I love the syntax — different operators for strings and scalars? A vast collection of frequently-unintuitive file test flags? No thanks. And the procedure for passing parameters to functions is frankly inexcusable.

But it's installed everywhere, and it's got bindings for everything. Yes, it's slow and unglamorous, but Perl reliably gets the job done.

Still, the cool kids were leaving it behind just as I was discovering the full, terrifying power of a scripting language and a well-stocked package manager. I tried to follow those cool kids, but a flirtation with Python went nowhere. I hated tying whitespace to logic, and when I realized how cumbersome its regex syntax was I threw in the towel.

So now I'm giving Ruby a shot, and I'm cautiously optimistic. I worked through the downright-awesome Try Ruby interactive tutorial and liked what I saw. Getting the language installed on my OS X and Linux machines was a snap, and Gems broke in ways that didn't seem outrageously worse than CPAN. Still, tonight hasn't been encouraging.

How can you have a scripting language with Flickr API bindings that are this bad? Of the three Ruby projects that claim to talk to Flickr, one is a partial implementation that is now outdated and broken; another got folded into a larger project that's never actually released anything; and the third is completely undocumented and doesn't work unless you authenticate with Flickr — a step that's complex and should be completely unnecessary for many operations.