Nest thermostat data access using R

UPDATE 12/03/13: Since Nest Labs’ move to an official API, the code below no longer works. Still, a good example of a webservice connection…

Gets “energy_latest” (10 days of usage) data from Nest Labs:

getNestData <- function(username,password) {
    loginURL <- c("")
    loginParams <- c("username" = username, "password" = password)
    loginResponse <- fromJSON(postForm(loginURL,.params=loginParams))
    loginResponseObjects <- list("urls","access_token","userid","expires_in","user")
    #note: the following function returns "urls" for the name of the transport_url, since getting 
    #only the first element in each response object (in this case, think of urls as "url secure")
    sapply(loginResponseObjects,function(x) {
        respObj <- unlist(loginResponse[[x]][[1]],use.names=F)
    #url to get system status
    statusURL <- paste(urls,'/v2/mobile/',user,sep="")
    #authorization token string
    authToken <- paste("Basic",access_token)
    #timestamp in milliseconds -- to pass with data request
    msTimestamp <- as.integer(Sys.time()) * 1000
    #RCurl info for User-Agent
    sessInfo <- sessionInfo()
    RCurlVersion <- sessInfo$otherPkgs$RCurl$Version
    userAgent <- paste("RCurl",RCurlVersion)
    #status request is just a call to the transport url
    statusHeaders <- c("Host" = urls, "Authorization" = authToken, "X-nl-user-id" = userid, "User-Agent" = userAgent, "Accept-Language" = "en-us")
    status <- fromJSON(getURL(statusURL,httpheader = statusHeaders)) #, verbose = T
    #body for energy_latest 
    deviceID <- as.character(status[['device']][[1]]['serial_number'])
    elKey <- paste("energy_latest",deviceID,sep=".")
    elPostfield <- toJSON(list("keys"=list(list("key" = elKey))))
    energyURL <- paste(urls,'/v2/subscribe',sep="")
    #headers for energy call
    energyHeaders <- c(statusHeaders,c("X-nl-client-timestamp" = msTimestamp, "X-nl-protocol-version" = "1", "Content-Type" = "application/json")) #
    #get the energy data  
    energy <- fromJSON(postForm(energyURL,.opts = list(httpheader = energyHeaders, postfields = elPostfield), style='post')) #, verbose = TRUE
Posted in data access, R | Tagged

Getting names of objects in an R workspace by type

To identify data frames in a workspace, and iterate over them by name, use Filter() with a few other functions:

> foo <- data.frame(matrix(1:10,ncol=2))
> rainbow <- c("red","orange","yellow","green","blue","indigo","violet")
> hello <- function(x) cat("Hello, ",x,"!\n",sep="")
> dfNames <- names(Filter(function(x) x=="data.frame",
+ sapply(objects(),function(x) class(get(x)))))
> dfNames
[1] "foo"

Here, objects() lists objects in the workspace, class() identifies their classes, and sapply() looks at each object in turn, returning a character vector to pass to Filter(). Importantly, get() must be used in class(), otherwise only the names of the objects are evaluated, and class() will always return “character”.

Each data frame can then be called by name in a loop:

> for( i in seq(along=dfNames) ) {
+ curr.df <- get(dfNames[i])
+ ## do things with the df
+ }

Note: not done here, but storing names in a list will allow for use of the “apply” family of functions, avoiding the need for a loop.

Posted in data munging, R | Tagged

Map of the United States including Alaska and Hawaii with R

Alaska and Hawaii represented in a typical state-level choropleth map — i.e. below the Southwestern border states. Turns-out it’s not that straightforward to do. After some tinkering…

+            c(1,0,1,3,3,0),
+            c(1,2,1,3,3,2)),
+          heights=c(.8, 0, .3),
+          widths=c(0, 1, 0, 0, 1, 2))
par(mar=rep(0, 4))
par(oma=c(8,rep(0, 3)))
map("world2Hires", "USA:Alaska")
par(mar=rep(0, 4))
par(mar=rep(0, 4))
map("world2Hires", "Hawaii")

…the result:
Seems as though the AK and HI portions of the layout cannot overlap the 50 contiguous states without obscuring them. There is also the problem of the difference of scale (and possibly projection) between the boundary datasets used. With more effort on the layout, and some edited boundary data, it does seem possible to create a presentable map of this kind without using non-core graphics packages.

Posted in mapping, packages, R | Tagged , , , ,

Data collection quiz question

Q: If a large working fan is blowing on an air quality sensor, will the data be biased?

Posted in data collection | Tagged , , , ,

Converting timestamps in PostgreSQL (again)

Revisited converting timestamps to UTC from different timezones. One way to do this is a variation on the PostgreSQL documentation on date/time functions, where epoch is used as a common unit:

select timestamp without time zone 'epoch' + (select extract(epoch from timestamp without time zone '2006-07-03 10:20:00' at time zone 'US/Eastern')) * interval '1 second' as timestamp_zulu;

Gets the job done, but the following is cleaner:

select (select '2012-08-01 03:00:00'::timestamp with time zone EDT) at time zone 'zulu';

And for insert:

insert into wq_obs (wq_timestamp) values ((select (select '2012-08-01 03:00:00'::timestamp with time zone EDT) at time zone 'zulu'));
Posted in PostgreSQL | Tagged , ,


Tursiops truncatus

Pic of a Bottlenose dolphin (Tursiops truncatus) breaching during cooperative hunting activity; May River, Beaufort County, South Carolina. Amazing.

Posted in coastal ecology, marine biology, marine ecology | Tagged , , , , , ,

From a long to a wide R data frame, with array values

From a “long” data frame with columns containing the names of measured properties and corresponding vectors of values, attempting to make a “wide” data frame with a column for each measured property. Appears as though the cast() function of the reshape package should do this, but I can only get it to return the lengths of the vectors of measured values without giving me an error. Realized I can do this with R’s base functionality:

Starting with a data frame named “crabs”…

> crabs
   loc aphia_id    var        val
1 a123       43  abund          3
2 a123       43 cwidth    1, 6, 3
3   b1       43  abund          4
4   b1       43 cwidth 4, 3, 2, 5
5 a123      456  abund          1
6 a123      456 cwidth          4
7 a123      203  abund          1
8 a123      203 cwidth          3

> abund <- crabs[crabs$var==”abund”,] > abund$abund <- abund$val > abund$val <- NULL > abund$var <- NULL > cw <- crabs[crabs$var==”cwidth”,] > cw$cwidth <- cw$val > cw$val <- NULL > cw$var <- NULL > crabs.wide <- merge(abund,cw,by=c(“loc”,”aphia_id”),all.x=T,all.y=T) > crabs.wide
   loc aphia_id abund     cwidth
1 a123      203    1             3
2 a123       43     3      1, 6, 3
3 a123      456    1             4
4   b1       43      4   4, 3, 2, 5

Note: R is using an attractive but misleading display for crabs and crabs.wide here when called from the prompt by name. The cwidth column is shown with comma-separated values. Use of the str() function verifies that they are indeed numeric vectors.

Guessing the plyr package must address this kind of reshape, as well.

Posted in data munging, R | Tagged , , , , ,