Several days ago I began work on Turpentine, a Twitter client written in Ruby that currently runs as a CLI. After I got things working to my satisfaction, I decided to add OAuth — partly because the whole project is meant to give me exposure to a variety of stuff I’m not familiar with, and partly because Twitter will remove basic auth at some point in the future.

After going through the Ruby on Rails example, I got stuck at the post-authorization point — the example assumes the user will be in a browser, which obviously doesn’t work for me at the command prompt.

I tried to figure out the solution using the sample code and the OAuth gem it uses, but it was beyond my level. (Though I did learn that all the OAuth information is passed around using query strings.) At this point I decided I’d give up on trying to write the code myself, and switched over to the twitter_oauth gem. The sample code was tremendously helpful, and needed only a little modification to handle my scenario.

---

Because I’m working purely in Ruby in the moment, I’ve kept things simple by using YAML to manage login information.

Step one is to load the config.

CONFIG_FILE = 'config.yaml'
CONFIG = YAML::load(File.read(CONFIG_FILE))

The relevant part of the config looks like this:

oauth:
    consumer_key:
    consumer_secret:
    request_token: 
    request_secret:

I’ve registered my application for Oauth, so I’ve filled in the consumer_key and consumer_secret fields. (Not shown because the secret must be kept secret.)

Note: if you’re going to use the command prompt like I am, specify that your application is a client, not a browser.

Back to code. I begin by loading the values:

consumer_key = CONFIG['oauth']['consumer_key']
consumer_secret = CONFIG['oauth']['consumer_secret']
request_token = CONFIG['oauth']['request_token']
request_secret = CONFIG['oauth']['request_secret']

Check if the application has been authorized (i.e. check if the request_ fields have any information):

if request_token.nil? and request_secret.nil?
  # not yet authorized
else
  # authorized
end

Authorization is the most code-heavy part of the process.

if request_token.nil? and request_secret.nil?
  client = TwitterOAuth::Client.new(
       :consumer_key => consumer_key,
       :consumer_secret => consumer_secret
       )
  request_token = client.request_token

  …

Here, we give twitter_oauth the application’s key and secret and let it work out the address that we’ll use to authorize Twitter access.

  …

  puts "Please open the following address in your browser to authorize this application:"
  puts "#{request_token.authorize_url}\n"

  puts "Hit enter when you have completed authorization."
  STDIN.gets

  …

The user is then prompted to open the address, which will ask if they really do want to authorize the application:

Finally, we put the user’s new request token and secret into the configuration file:

  …

  access_token = client.authorize(
      request_token.token,
      request_token.secret
  )

  File.open(CONFIG_FILE, 'w') do |out|
    CONFIG['oauth']['request_token'] = access_token.token
    CONFIG['oauth']['request_secret'] = access_token.secret
    YAML::dump(CONFIG, out)
  end
else
…

Things are a lot simpler if we’re already registered:

…
else
  client = TwitterOAuth::Client.new(
    :consumer_key => consumer_key,
    :consumer_secret => consumer_secret,
    :token => request_token,
    :secret => request_secret
  )
end

Now that we have a functional login, we can use the gem’s built-in functionality to do useful things like get the friends timeline:

puts client.friends_timeline

And proceed as if everything used basic auth.

Due to the news that last.fm may have given the RIAA user data, I’ve thrown together a Ruby script that will dump out a user’s listening history. Having exported my own data, I’m now free to cancel my last.fm account while keeping my play logs. Of course, if you’re not interested in jettisoning your last.fm account, this still serves as a convenient backup method.

This is no major loss to me, as I’ve been scrobbling solely from my phone for the past eight months or so, which turns out to be a very haphazard process — there’s no guarantee that anything will scrobble, or that all the tracks will scrobble. Even beyond that, I’ve only ever used the service for a personal music history. So even though I’m removing the possibility of my play history being recorded, in practice not much is changing.

Jason Scott:

Don’t trust the Cloud to safekeep this stuff. Hell yeah, use the Cloud, blow whatever you want into the Cloud. The Internet’s a big copy machine, as they say. Blow copies into the Cloud. But please:

• Don’t blow anything into the Cloud that you don’t have a personal copy of.
• Insult, berate and make fun of any company that offers you something like a “sharing” site that makes you push stuff in that you can’t make copies out of or which you can’t export stuff out of. They will burble about technology issues. They are fucking lying. They might go off further about business models. They are fucking stupid. Make fun of these people, and their shitty little Cloud Cities running on low-grade cooking fat and dreams. They will die and they will take your stuff into the hole. Don’t let them.
• Recognize a Cloud when you see it. Are you paying for these services? No? You are a sucker. You are giving people stuff for free. I pay for Vimeo and I pay for Flickr and a couple other things. This makes me a customer. Neither of these places get my only copy of anything.

In that spirit, I’m going to demonstrate two ways to export your delicious bookmarks into an XML file for safety. As noted previously, this is really a pretty simple thing to do —  you just need to call the URL api.del.icio.us/v1/posts/all and pass in basic auth. For more information on this and everything else in the delicious API, please refer to the documentation.

cURL

cURL is definitely the easiest way to do this. Many web APIs will readily interact with cURL, and delicious’ is especially simple due to the use of basic auth (meaning there’s no need to do a login dance) and the usage of URLs rather than arguments.

curl --user 'username':'password' -o ~/Desktop/delicious.xml 'https://api.del.icio.us/v1/posts/all'

We can turn this into a simple little Bash script:

#!/bin/bash
user='username'
password='password'

curl --user $user:$password -o ~/Desktop/delicious.xml 'https://api.del.icio.us/v1/posts/all'

Ruby

The Ruby form is a little more verbose, but still quite simple. Please note that the following code has largely been borrowed from the nice example here, which unfortunately does not specify a license.

#!/usr/bin/ruby

require 'net/http'
require 'net/https'

user = 'user'
password = 'password'

http = Net::HTTP.new('api.del.icio.us', 443)
http.use_ssl = true
xml = http.start { |http|
  req = Net::HTTP::Get.new('/v1/posts/all', {'User-Agent' => 'Delicious backup'})
  req.basic_auth(user, password)
  http.request(req).body
}

dumpfile = File.new('delicious.xml', 'w')
dumpfile.puts xml
dumpfile.close

Automated backups

Now that you’ve got a script to make an XML backup, why not have it automatically run so you don’t have to remember?

Here’s my crontab, which automatically runs the backup every day at noon:

00 12 * * * /Users/stilist/deliciousbackup.sh

Where deliciousbackup.sh is the Bash script noted above.

Ideas for further development

1. These examples download the entire bookmark collection regardless of whether anything has changed. Checking /posts/update could save on bandwidth.
2. It’s sort of assumed that the service will always be around, and the API call will always succeed. Error handling would be useful.
3. Only a single day’s archive is kept. This means that if something goes wrong with the online copy — accidentally deleting tags, account vandalism, or anything like that — you’ll have a maximum of twenty-four hours to fix it before the backup is overwritten.

Finally

The code here is provided under the MIT license. For a copy, drop by the git repository.

And if you code up any of the improvements noted in the previous section, I’d love to know!