I have accelerometer data Log, the data contain (accX, accY, accZ,timestamp)
The data look like
I am so confused how to processing it in R.
I have two questions:
Is there any library for handle this data? I want plot it like time
series data and analysis it.
How to process the timestamp? because the timestamp is not every
second but milisecond.
Please anyone can give me some light?
Thank you in advance
To answer your two specific questions:
1) There are many packages for time series analysis. See here: http://cran.r-project.org/web/views/TimeSeries.html
2) There are many ways to process the time stamp data. #bjoseph gives you some very good advice in his response. The lubridate package (http://cran.r-project.org/web/packages/lubridate/index.html) is very good at handling time data with somewhat more sensible functions than the POSIX set. ggplot2 (http://ggplot2.org/) plots time series data quite sensibly as well.
The classes POSIXlt and POSIXct exist in R for manipulating calendar times and dates. See more information here: http://stat.ethz.ch/R-manual/R-devel/library/base/html/as.POSIXlt.html
In your specific case you need to may need to modify your system time to include milliseconds:
test <- "2014-07-09 15:03:33:252"
test
[1]"2014-07-09 15:03:33:252"
options("digits.secs"=6)
Sys.time()
[1] "2014-07-23 11:16:32.480932 EDT"
as.POSIXlt(test)
[1] "2014-07-09 15:03:33 EDT"
test
[1] "2014-07-09 15:03:33:252"
time.test <- as.POSIXlt(test)
class(time.test)
[1] "POSIXlt" "POSIXt"
time.test
[1] "2014-07-09 15:03:33 EDT"
To apply this to the entire column of your data.table you can run:
dt$timecolumn <- as.POSIXlt(dt$timecolumn)
where time column is the name of the column with the times in it.
If you need help importing the data from excel, a good guide:
> library(gdata) # load gdata package
> help(read.xls) # documentation
> mydata = read.xls("mydata.xls") # read from first sheet
Related
I have a column "DateTime".
Example value: 2016-12-05-16.25.54.875000
When I import this, R reads it as a Factor.
Now, when I sort the dataset by decreasing "DateTime", the maximum DateTime is 23 June 2017. When I use DateTime = as.POSIXct(DateTime), it changes to 22 June 2017. How is this happening?
P.S. I am running this R script in Power BI.
So some comments first. When you read strings in R, unless you specify otherwise they are imported as factors. You can use the Option
Trying what #Disco Superfly has suggested works if you define the data as a string in R
> a <- "2016-12-05-16.25.54.875000"
> as.POSIXct(a, format="%Y-%m-%d-%H.%M.%S")
[1] "2016-12-05 16:25:54 CET"
> as.POSIXct(a)
[1] "2016-12-05 CET"
Is not clear what you are saying about the fact that the data is being changed. Can you give a reproducible example?
To summarize, if your Dates are strings that what other have already suggested works perfectly. I suppose you are trying to do more than what you are explaining and therefore I don't understand what you are saying exactly.
I'm trying to plot the created time of twitters. I can extract the created time with the following code:
tweets <- searchTwitter('weather', n=100,lang='en')
t <- twListToDF(tweets)
s <- t[, c("created")]
The time format I get is something like: 2017-02-25 18:52:06 UTC
Trying to plot it with plot(s) provides just a list of dots. I'm not sure if it is due to the date/time format.
I want to create a barchart that each bar represents the count of tweets on an hourly period. The x axis would represent time and the y axis the number of tweets.
Any ideas?
One approach that I've used uses the 'lubridate' package, which is available on CRAN.
library(lubridate)
date(now())
hour(now())
You would replace now() with your vector s. If your s vector has the class POSIXct, I believe that this will work. There might be alternative solutions that involve the lubridate R package, too. I hope that this helps.
I am currently trying to determine the time and date on the observations in my dataset.
The date/timestamp is as follows:
1458024601.18659
1458024660.818
The observation are recorded ever minute.
I am trying to convert the above date/time stamp into something for understandable/ interpretable.
Could you please help me with this issue.
Many thanks.
Looks like seconds, but seconds starting from when? Typically, 1970-01-01:
> x = 1458024601.18659
> as.POSIXct(x, origin="1970-01-01")
[1] "2016-03-15 06:50:01 GMT"
So if you are expecting that timestamp to be that time, we've got the origin right.
If you are expecting a date in 1946, then origin="1900-01-01" is probably what you want.
Since, according to your most recent post, the data is stored as a factor class, some further manipulations are required.
To convert the factor column into the required numeric class, this modification of #Spacedman's answer should work:
as.POSIXct(as.numeric(as.character(all_prices$timestamp)), origin="1970-01-01")
Your solution is perfect, except that i have another issue :(
I tried to run this code on the data.frame that i have. Unfortunately, i keep getting this following error after running the code.
dates <- as.POSIXct(all_prices$timestamp, origin="2016-03-15")
Error in as.POSIXlt.character(as.character(x), ...) :
character string is not in a standard unambiguous format
Data "all_prices" is a data.frame.
class(all_prices)
[1] "data.frame"
data "all_prices$timestamp" is a factor.
class(all_prices$timestamp)
[1] "factor"
I am completely new to R and I am learning how to program in R to get historical stock index data. I am planning to build an daily update code to update historic index data. I use a data environment call "indexData" to store the data as a xts. But unfortunately, the rbind or merge does not support data environments. They only support objects. I am wondering whether there are any workarounds or packages that I can used to solve this. My code is the following:
indexData<-new.env()
startDate<-"2013-11-02"
getSymbols(Symbols=indexList,src='yahoo',from=startDate,to="2013-11-12",env=indexData)
startDate<-(end(indexData$FTSE)+1)
NewIndexData<-new.env()
getSymbols(Symbols=indexList,src='yahoo',from=startDate,env=NewIndexData)
rbind(indexData,NewIndexData) #it does not work for data environments
Much appreciate for any suggestions!
If you use auto.assign=FALSE, you get explicit variables -- and those you can extend as shown below.
First we get two IBM price series:
R> IBM <- getSymbols("IBM",from="2013-01-01",to="2013-12-01",auto.assign=FALSE)
R> IBM2 <- getSymbols("IBM",from="2013-12-02",to="2013-12-28",auto.assign=FALSE)
Then we check their dates:
R> c(start(IBM), end(IBM))
[1] "2013-01-02" "2013-11-29"
R> c(start(IBM2), end(IBM2))
[1] "2013-12-02" "2013-12-27"
Finally, we merge them and check the dates of the combined series:
R> allIBM <- merge(IBM, IBM2)
R> c(start(allIBM), end(allIBM))
[1] "2013-01-02" "2013-12-27"
R>
You can approximate this by pulling distinct symbols out of the two environments but I think that is harder for you as someone beginning in R. So my recommendation would be to not work with environments -- see about lists instead.
I have a set of data that need to be plotted (1M rows) with R. The time column (column 1) is in hh:mm:ss.000 format. I would like to plot the graph in a time range, say from 08:05:00 to 09:00:00. How do I do it? I have searched the web and couldn't find a way to set the xlim properly.
Here's a short example of the data. Column 1 is time, Column 2, 3, 4.. will be on y axis.
07:51:19.553,10.785,0.000,0.392,1.512,1.527,1.553,1.560,2.838
08:05:00.661,-1.555,0.000,0.041,0.310,0.314,0.321,0.327,1.474
08:06:58.250,30.781,0.000,0.093,0.156,0.160,0.168,0.173,1.411
08:30:02.506,-0.002,0.000,0.052,0.120,0.123,0.132,0.137,1.361
09:05:00.997,-1.802,0.000,0.032,0.078,0.080,0.087,0.090,1.258
10:05:00.661,-1.555,0.000,0.041,0.310,0.314,0.321,0.327,1.474
Thanks in advance for your help.
You really want to use a proper time series class such as zoo or xts
Subsetting, plotting, ... then come for free. Start with the excellent zoo documentation before maybe switching to xts for even better performance and subsetting.
Now, one million rows is too many as you end up with more data than pixels -- but at least this will give you a chance to summarize your data.
Here is a quick illustration:
> options(digits.sec=3) ## important: turn on milli-sec via print()
> library(xts)
Loading required package: zoo
> X <- xts(cumsum(rnorm(100)), order.by=Sys.time()+cumsum(runif(100)/10))
> plot(X)
To change character vector to "date & time" object, POSIXlt(ct) object, function strptime() will come handy. Here's a short example how it's done.
dtm <- strptime(c("1.1.2010 11:35"), format = "%d.%m.%Y %H:%M", tz = "CET")