Sunday, 13 November 2011

Google Reader API - with Java

I have a morning commute routine that used to consist of trying to get a seat if possible, or at the very least a place where I could lean against something - both with an aim to being able to read while on the move.

Following that, I'd load either read a book, or load up FeedlerPro on my iPhone - its an RSS reader, and together with Google Reader is the main way I track various tech related blogs and news sources.

One problem I have is that the 3G signal I get on my commute varies from pretty good, to none at all.

I decided that I needed some way of downloading all available articles to my phone via WiFi before leaving the house - but how?

Looking around I found a number of options, but the one I went for in the end was Instapaper. The main reason I chose Instapaper was that it had pretty uniformly positive reviews, and it accepted article URLs via browser bookmarks, email or a simple API - its the latter capability that sold it for me.

So, the next step was how to:
  • Find out what unread articles were on Google Reader
  • Retrieve said unread articles
  • Mark retrieved articles as Read
  • Send these articles to Instapaper
As Google doesn't (yet) have an official API for Google Reader trying to get information was tricky - using judicious use of Google I've managed to stitch all the bits together - hopefully it'll be of help to others...

Below are the various steps need to log in, retrieve unread article URLs and how to mark article URLs as read. I've extracted (and modified where necessary) only the relevant bits of code for each task - the final app I wrote is structured differently, and has more code (involving logging, exception handling etc) which I've stripped out here.

There are 2 utility methods I use - toString and getAuthorizedConnection - these can both be found at the end of the article
  • Logging into Google Reader
In order to access Google Reader (and other Google APIs) you need an "authorization string", and a "token". This data token is only valid for a relatively short period and will need to be refreshed if you're doing long running tasks - this isn't generally necessary for Google Reader related calls.

You need to hit a Login URL with your Google credentials and retrieve an authorization string. You then use this string to retrieve a token, from a separate URL.

Login - get the authorization string and token

// login, and get the authorization string
String email = "";
String password "";
final String AUTHENTICATION_URL = "https://www.google.com/accounts/ClientLogin";
URL loginURL = new URL(AUTHENTICATION_URL + "?service=reader&Email;=" + email + "&Passwd;=" + password);
String sid = loginBuffer.readLine().split("=")[1];       // not needed
String lsid = loginBuffer.readLine().split("=")[1];      // not needed
String authorization = loginBuffer.readLine().split("=")[1]; 

// now get the token
URL tokenURL = new URL(http://www.google.com/reader/api/0/token);
URLConnection tokenURLConnection = tokenURL.openConnection();
tokenURLConnection.setRequestProperty("Authorization", "GoogleLogin auth=" + auth);
tokenURLConnection.connect();
String token = toString(tokenURLConnection.getInputStream());

Retrieve unread items
 
// USER_ID below can be found by going to Google Reader, viewing the source and searching for 
// USER_ID - this 20 character string is your unique Google identifier
private Map getUnreadItems(final int numberOfItems) 
             throws IOException, SAXException, XPathExpressionException, ParserConfigurationException {
    final String READING_LIST_URL = 
             "http://www.google.com/reader/atom/user/%s/state/com.google/reading-list?n=%s&xt;=user/-/state/com.google/read";
    final String readingListURL = String.format(READING_LIST_URL, USER_ID, numberOfItems);
    URLConnection connection = getAuthorizedConnection(readingListURL, false);
    InputStream inputStream = connection.getInputStream();

    // parse the retrieved RSS XML, and map article ID to article URL
    Map tagIDToURLs = parseRSSData(inputStream);

    inputStream.close();
    return tagIDToURLs;
}

private Map parseRSSData(InputStream xmlInputstream) 
             throws XPathExpressionException, ParserConfigurationException, IOException, SAXException {
    Map itemTagToURL = new LinkedHashMap();
    XPath xpath = XPathFactory.newInstance().newXPath();
    xpath.setNamespaceContext(new DefaultNamespaceContext());
    InputSource inputSource = new InputSource(xmlInputstream);
    NodeList nodes = (NodeList) xpath.evaluate("/ns:feed/ns:entry", inputSource, XPathConstants.NODESET);
    for (int i = 0; i < nodes.getLength(); i++) {
        Node node = nodes.item(i);
        String stream = xpath.evaluate("ns:source/@gr:stream-id", node);
        String id = xpath.evaluate("ns:id", node);
        String url = xpath.evaluate("ns:link[@rel='alternate']/@href", node);
        itemTagToURL.put(id, url);
    }
    return itemTagToURL;}

Mark items as read

// the value for T below is the token you retrieved during the login process above
private String postItem(String url, Map arguments) throws IOException {
    URLConnection queryURLConnection = getAuthorizedConnection(url, true);
    // Post the data item
    if (arguments != null && !arguments.isEmpty()) {
        OutputStream outputStream = queryURLConnection.getOutputStream();
               StringBuilder builder = new StringBuilder();
        for (String key : arguments.keySet()) {
            builder.append(URLEncoder.encode(key, "UTF-8")).
                append("=").
                append(URLEncoder.encode(arguments.get(key), "UTF-8")).
                append("&");
        }
       
        builder.append(URLEncoder.encode("T", "UTF-8")).
            append("=").
            append(URLEncoder.encode(token, "UTF-8"));
        
        outputStream.write(builder.toString().getBytes("UTF-8"));
        outputStream.close();
    }
    return toString(queryURLConnection.getInputStream());
}

Utility methods

private String toString(InputStream inputStream) throws IOException {
     String string;
        StringBuilder outputBuilder = new StringBuilder();
        if (inputStream != null) {
        BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
        while (null != (string = reader.readLine())) {
            outputBuilder.append(string).append('n');
        }
    }
    return outputBuilder.toString();
}

// authorization is the authorization string retrieved during the login process above
private URLConnection getAuthorizedConnection(String url, boolean doOutput) throws IOException {
        URL queryURL = new URL(url);
        URLConnection queryURLConnection = queryURL.openConnection();
        queryURLConnection.setDoOutput(doOutput);
        queryURLConnection.setRequestProperty("Authorization", "GoogleLogin auth=" + authorization);
        queryURLConnection.connect();
        return queryURLConnection;
}

And that's it - hopefully all of this will be useful to you if you're trying to do something similar.

2 comments:

  1. The java codes you have provided is of best result and much effective for the readers and thanks for sharing it.
    java training institutes in marathahalli bangalore

    ReplyDelete
  2. Good post. Speaking about online technologies. I suggest to keep your paperwork stored online. Read about electronic data room and everything will be clear.

    ReplyDelete