Sandy Power Outage Timeline - Application Overview

I've been interested in power outage data for a while. I create the @Stormpocalypse twitter account, which has been posting power outage updates for the past 6 or so months. When Sandy arrived, the Twitter feed became overwhelming and incomprehensibile:
@ckundo is @stormpocalypse really right? I’ve only seen the numbers go up, never down.
— Greg Borenstein (@atduskgreg) October 31, 2012
@stormpocalypse Are # of people out of power going up or down? Hard to tell / can't remember #s from Monday. :( @yonkersoem40
— The Symposium (@TheSymposiumNY) October 31, 2012
This is how I decided to build stormpox.com. It is a timeline of power outages in the Greater New York area, updated every 15 minutes. I've made the source code public. I've also created the sandy rubygem that encapsulates the feed logic into an easy to use library.



Here is a technical overview of the application:

The application pulls from obscured API endpoint that feeds ConEd's stormcenter app. ConEd's version of the app makes 2 ajax calls, one to an xml file that gets regenerated every 15 minutes and shows the timestamp of the report json feed. The second call gets the json via a url with that timestamp. The resulting report json gets parsed in the app.

I'm consuming the same endpoints for stormpox.com and consuming the response object in a ruby app.

I'm running a rake task that gets called via cron job. The crontab looks like this:

*/15 * * * * /bin/bash -l -c 'cd /path/to/coned_charts/ && rake'

The rake task simple creates a new instance of the Coned class and populates the spreadsheet.

In the Rakefile:

task :default do
Coned.new.populate_worksheet
end

and in coned.rb, the request to the ConEd is made:

class Coned
BOROS = ["Bronx" "Brooklyn", "Manhattan", "Queens", "Staten Island", "Westchester"]

def initialize
@response = HTTParty.get(coned_url_with_directory, :format => :json).parsed_response
@report = Report.new
end

def populate_worksheet
@report.update_worksheet(worksheet_data)
end

private

def worksheet_data
current_time = Time.now.strftime("%m/%d/%Y %H:%M:%S")
[current_time] + areas.collect { |area| area["custs_out"] || ""}
end

def areas
@response["file_data"]["curr_custs_aff"]["areas"][0]["areas"]
end

def coned_url_with_directory
base_uri = "http://apps.coned.com"

directory = HTTParty.get(
"#{base_uri}/stormcenter_external/stormcenter_externaldata/data/interval_generation_data/metadata.xml",
:format => :xml).parsed_response['root']['directory']
url = "#{base_uri}/stormcenter_external/stormcenter_externaldata/data/interval_generation_data/#{directory}/report.js"
end
end

The rake task fetches and parses the feed and posts it to a google spreadsheet via the Google Drive API, using the google_drive rubygem. (see report.rb)

stormpox.com itself is running as a static site on heroku with an embedded google timeline chart that auto-updates when the spreadsheet changes. The google spreadsheet contains the entire history of outages since Monday.