Plotting truncated times from zoo time series - r

Let's say I have a data frame with lots of values under these headers:
df <- data.frame(c("Tid", "Value"))
#Tid.format = %Y-%m-%d %H:%M
Then I turn that data frame over to zoo, because I want to handle it as a time series:
library("zoo")
df <- zoo(df$Value, df$Tid)
Now I want to produce a smooth scatter plot over which time of day each measurement was taken (i.e. discard date information and only keep time) which supposedly should be done something like this: https://stat.ethz.ch/pipermail/r-help/2009-March/191302.html
But it seems the time() function doesn't produce any time at all; instead it just produces a number sequence. Whatever I do from that link, I can't get a scatter plot of values over an average day. The data.frame code that actually does work (without using zoo time series) looks like this (i.e. extracting the hour from the time and converting it to numeric):
smoothScatter(data.frame(as.numeric(format(df$Tid,"%H")),df$Value)
Another thing I want to do is produce a density plot of how many measurements I have per hour. I have plotted on hours using a regular data.frame with no problems, so the data I have is fine. But when I try to do it using zoo then I either get errors or I get the wrong results when trying what I have found through Google.
I did manage to get something plotted through this line:
plot(density(as.numeric(trunc(time(df),"01:00:00"))))
But it is not correct. It seems again that it is just producing a sequence from 1 to 217, where I wanted it to be truncating any date information and just keep the time rounded off to hours.
I am able to plot this:
plot(density(df))
Which produces a density plot of the Values. But I want a density plot over how many values were recorded per hour of the day.
So, if someone could please help me sort this out, that would be great. In short, what I want to do is:
1) smoothScatter(x-axis: time of day (0-24), y-axis: value)
2) plot(density(x-axis: time of day (0-24)))
EDIT:
library("zoo")
df <- data.frame(Tid=strptime(c("2011-01-14 12:00:00","2011-01-31 07:00:00","2011-02-05 09:36:00","2011-02-27 10:19:00"),"%Y-%m-%d %H:%M"),Values=c(50,52,51,52))
df <- zoo(df$Values,df$Tid)
summary(df)
df.hr <- aggregate(df, trunc(df, "hours"), mean)
summary(df.hr)
png("temp.png")
plot(df.hr)
dev.off()
This code is some actual values that I have. I would have expected the plot of "df.hr" to be an hourly average, but instead I get some weird new index that is not time at all...

There are three problems with the aggregate statement in the question:
We wish to truncate the times not df.
trunc.POSIXt unfortunately returns a POSIXlt result so it needs to be converted back to POSIXct
It seems you did not intend to truncate to the hour in the first place but wanted to extract the hours.
To address the first two points the aggregate statement needs to be changed to:
tt <- as.POSIXct(trunc(time(df), "hours"))
aggregate(df, tt, mean)
but to address the last point it needs to be changed entirely to
tt <- as.POSIXlt(time(df))$hour
aggregate(df, tt, mean)

Related

How Can I Create a Distribution Visualization for an Average Day?

In R, my dataframe ("sampledata") looks like this:
The timestamp column is POSIXct, format: "2018-10-01 00:03:23"
The state column is Factor w/ 3 levels "AVAILABLE", "MUST_NOT_RUN", "MUST_RUN"
There are 6 unique device_id. The timestamps for each device are not the same, meaning data was not always collected at the same minute for each device. In some cases, there are multiple records per minute for the same device.
I want to transform the data into a visualization that shows distribution of "state" across a "typical" day. Ideally, something like this:
I've tried to count each occurrence of "state" grouped by timestamp minutes but failed (Error: can't sum factors). I've been trying to use ggplot and geom_area for the visualization, but believe I need to restructure my data before it will work. Very new to R (obviously). Happy to read any tutorials or links provided as background and appreciate any help you can provide. Thanks!
Other information that may/may not be helpful:
There are a handful of columns in the dataframe not shown.
223,446 entries between 10/2/18 - 11/8/18.
You can take the hours from the timestamps and then compute proportions of your states by hour:
library(ggplot2)
library(plyr)
#get hours from timestamp
obj$hour <- as.POSIXlt(obj$timestamp)$hour
#get average state proportions per hour
plot_obj <- ddply(obj,.(hour), #take data.frame "obj" and group by "hour"
function(x) with(x,
data.frame(100*table(state)/length(state))))
ggplot(plot_obj, aes(x=hour,y=Freq,fill=state)) +
geom_area()

Force ggplot scales to start on e.g. 1st of year, 1st of month etc

I'm looking for a way to force the date labels on a ggplot to start at a (seemingly) logical time. I've had the problem a number of times but my current problem is I want the breaks to be on the 01/01/yyyy
My data is a large dataset with POSIXct Date column, data to plot in Flow column and a number of site names in the Site column.
library(ggplot2)
library(scales)
ggplot(AllFlowData, aes(x=Date, y = Flow, colour = Site))+geom_line()+
scale_x_datetime(date_breaks = "1 year", expand =c(0,0),labels=date_format("%Y"))
I can force the breaks to be every year and they appear okay without the labels=date_format("%Y") (starting on 01/01 each year) but if I include labels=date_format("%Y") (as there is 10 years of data so gets a bit messy) the date labels move to ~November, and 1989 is the first label even though my data starts on the 01/01/1990.
I have had this problem numerous times in the past on different time steps, such as wanting to force it to the 1st of the month or daily times to be at midnight instead during the day. Is there a generic way to do this?
I have looked at create specific date range in ggplot2 ( scale_x_date), but I do not want to have to hard code my breaks as I have a fair few plots to do with different date ranges.
Thanks
If the dates come to you in a vector like:
dates <- seq.Date(as.Date("2001-03-04"), as.Date("2001-11-04"), by="day")
## "2001-03-04" "2001-03-05" "2001-03-06" ... "2001-11-03" "2001-11-04"
use pretty.Dates() to make a best guess about the end points.
range(pretty(dates))
## "2001-01-01" "2002-01-01"
Then pass this range to ggplot.
However, I recommend coord_cartesian() instead of scale_x_date(). Typically I want to crop the graphic bounds, instead of flat-out exclude the values entirely (which can mess up things like a loess summary).

Weekly time series plot in R

I am trying to create a plot of weekly data. Though this is not the exact problem I am having it illustrates it well. Basically imagine you want to make a plot of 1,2,....,7 for for 7 weeks from Jan 1 2015. So basically my plot should just be a line that trends upward but instead I get 7 different lines. I tried the code (and some other to no avail). Help would be greatly appreciated.
startDate = "2015-01-01"
endDate = "2015-02-19"
y=c(1,2,3,4,5,6,7)
tsy=ts(y,start=as.Date(startDate),end=as.Date(endDate))
plot(tsy)
You are plotting both the time and y together as individual plots.
Instead use:
plot(y)
lines(y)
Also, create a date column based on the specifics you gave which will be a time series. From here you can add the date on the x-axis to easily see how your variable changes over time.
To make your life easier I think your first step should be to create a (xts) time series object (install/load the xts-package), then it is a piece of cake to plot, subset or do whatever you like with the series.
Build your vector of dates as a sequence with start/end date:
seq( as.Date("2011-07-01"), by=1, len=7)
and your data vector: 1:7
a one-liner builds and plots the above time series object:
plot(as.xts(1:7,order.by=seq( as.Date("2011-07-01"), by=1, len=7)))

How to produce a scatter plot of dates vs magnitudes in R?

This is what i have done so far but its wrong.
earthquakes<- c(6.6,6.8,8.4)
dates <- (13/02/2001 ,28/02/2001,23/06/2001)
plot(earthquakes,dates)
I have only started learning R. Please help.
earthquakes<- c(6.6,6.8,8.4)
dates <- as.Date(c("13/02/2001", "28/02/2001", "23/06/2001"), format="%d/%m/%Y")
plot(dates, earthquakes)
You had a few issues:
Dates should be in quotes (otherwise R will think you're trying to do arithmetic (i.e. 13 divided by 02 divied by 2001)
To convert dates to actual date objects, use as.Date, pass a vector of dates (this is the c(... part), and then specify the format that they are in so that R knows what to do with the strings
you had x and y swapped
Note, the as.Date step is not strictly necessary, but if you don't do that, then the x axis of the plot will plot every item equidistant, irrespective of how far apart the dates actually are in time.

Line up ts or zoo timeseries of different frequencies at "midperiod" on X axis

I need to plot a number of time series of different frequencies in R, and I need them to have the points centered on a period instead of starting at the beginning of each period. Here is an illustration of what I'm running into:
test1 <- ts(rnorm(24), start=2004, freq=12)
test2 <- ts(rnorm(2), start=2004, freq=1)
plot(test1, type='l')
lines(test2, col='red')
I'd like the red line to essentially be shifted forward 6 months, to the middle oaf each year. I've spent a little time with the R documentation for "ts" and haven't figured out how to do this -- any suggestions?
How about changing the time-series start?
test2 <- ts(rnorm(2), start=2004.5, freq=1)
I agree with #haggai_e that shifting the 'start' parameter makes sense, but if you already have a ts-object then the code to use those values would be:
lines(ts(test2, start=2004.5, freq=frequency(test2)) )
ts-objects are really just numeric vectors with attributes. You recover those attributes with start, end and frequency. The end is actually calculated on the fly from(length/frequency -1 ) of the vector added to start.

Resources