I am having a bit of difficulty properly extracting timestamps from an xts object and putting them into another matrix. Basically, I have an xts object with a timestamp column in a YYYY-MM-DD HH:MM:SS.SSS format, and I want to extract specific times (in order) and put them into a column in another matrix in the exact same format as they are in the xts object. For example, let's say I have an xts object with a timestamp column given as:
For example, let's say the timestamp column for a matrix called mat is given as follows:
2000-01-01 09:05:02.333
2000-01-01 09:06:03.212
2000-01-01 09:06:04.764
2000-01-01 09:07:02.211
Now let's say I want to take the 2nd and 4th times and put them into another matrix (which I'll call mat2), then ideally it should come out like this:
Time
---------------------------
2000-01-01 09:06:03.212
2000-01-01 09:07:02.211
Now, I know that by using the index() function on an xts object you can get the timestamp for that object at a particular index value. However, when I try to do this by writing (for example) mat2[i,"Time"] <- index(mat[i]), then rather than putting the date/time value from mat into mat2 it instead puts a number into the matrix, not a time, and I'm not sure why that happens. Is there a way to copy the timestamp of an xts object and put it into take two different time values from one matrix and put them into two separate columns in a different matrix?
Related
I am having trouble with my R assignment I am working on this semester.
Here is the part that I am tasked with doing that I am confused about:
iv. Download 3 month TBill rate from Fred for the same sample period 01/01/1993 to 12/31/2013.
Useful Hints: You may have to chop the data to match the sample period.
v. Construct a matrix of return series combining Stock, S&P500, and TBill for the sample period.
Useful Hints:
Note that the rownames for the TBill may not match with the other two return series, as the dates do not match, although the month and year matches
You have to construct the row names for each of the series as Year – Month format (e.g. 1993-01) or delete the rownames from T-bill before you can combine all three series into one Return matrix.
You have to convert the Return matrix to a dataframe before you use the lm() function.
I tried this below like I have used getSymbols before for SPY and AAPL but it pulls an entire data set rather than the specific date range. How can I chop the data so it fits the desired date range?
getSymbols('TB3MS', src = 'FRED', from = "1993-01-01", to = "2013-12-31")
Next, how would I go about constructing the matrix of return series combining all of the stocks? Can anyone point me in the right direction?
Filtering an xts object: see examples in the xts documentation ?xts.
# filter 1993 until 2013
TB3MS["1993/2013"]
But these dates are of, because tbills are at the first day of the month, the stock dates are the last day of the month. With the coredata you can extract the tbill data and stick it into the other timeseries if the rows match.
Taking the data example from your previous question, you could do something like this (and I'm creating more steps than needed, you could combine a few statements into one):
# create monthly returns of the spy data and give the column a better name than monthly.returns
spy_returns <- monthlyReturn(SPY)
colnames(spy_returns) <- "SPY_returns"
# filter the tbill data
TB3MS_1993_2013 <- TB3MS["1993/2013"]
# add tbill data to spy data
spy_returns$TB3MS <- coredata(TB3MS_1993_2013)
Merging xts objects can just be done with merge. They will be merged on the dates.
merge(spy_returns, aapl_returns) would combine these two. If you have a lot of tickers, use Reduce (check help and SO on how to use Reduce with merge) but better would be to use the tidyquant package if allowed.
I have a simple(and long) Panda DF with Datetime Index and Prices, I am trying to create a new Column ['ewm_12'] which pick 11 previous rows every 120 minutes appends current price and calculates the ewm
I am trying to do it in a vectorized fashion as the DF is long, using the following code:
dftemp['ewm_12'] = dftemp.loc[dftemp.index::120][-11:].append(dftemp.loc[(dftemp.index)])[dfs].ewm(min_periods=12,span=12, adjust = True).mean()[-1:][0]
TypeError: Cannot convert input [DatetimeIndex(['2018-01-01 22:00:00+00:00', '2018-01-01 22:01:00+00:00','2019-01-18 22:00:00+00:00'],dtype='datetime64[ns, Europe/London]', length=394561, freq=None)] of type to Timestamp
This seem very strange as if i pick just one row dftemp.index it returns a Timestamp but when I ask it to iterate over the whole df.index it says it cannot convert the Datetimeindex (which is a collection of Timestamps), I can do it with a for loop but will take several minutes and I am sure there must be a way if someone knows pls help
Dear colleagues I have the following dataset:
Time1 Signal1 Time2 Signal2 Time3 Signal 3
2018-05-06 17:41:44 Value 1 2018-05-06 17:32:39 Value 1 2018-05-07 00:06:00 .....
Time X columns are in POSIXct format, Because the time of the signals is different I am trying to make a custom resampling and I am trying to extract the timestamp of each signal.
I need to storage the time of each signal, putting this values in one vector and short this vector in ascending order.
I have try to:
NewTime<-sort(dataset[,c(1,3,5)])
Error: Can't use matrix or array for column indexing
Also with:
NewTime<-sort(unlist(Time_Trend[, c(1,3,5)]))
But with the last time I loose the date format, is there any way of doing this procedure without loosing the POSIXct format apart that having the vector in messy format.
Finally I have tried with this:
NewTime<-cbind(data$X1,data$X3, data$X5)
actualTime<-as.POSIXct(actualTime, origin="2018-05-06 07:50:32") #lowest value
But it returns me a vector with year date 2066. Anyone that has done this before?
If we want to order based on multiple columns
dataset[do.call(order, dataset[,c(1,3,5)]),]
If we are looking for creating a vector of datetime variables and then do the sort
sort(do.call(`c`, dataset[c(1, 3, 5)]))
I have the following problem. I have a XTS containing a date column and several valuations, which should be ranked (biggest = best rank). So my original XTS is test:
> str(index(Test))
Date[1:235], format: "1995-01-31" "1995-02-28" "1995-03-31" "1995-04-28" "1995-05-31" "1995-06-30" "1995-07-31" ...
Now, my rankValuations function:
rankValuations<-function(ValuationXTS){
#Ranks the xts object asset valuations
#ValuationXTS is a xts time series with asset valuations
#Returns an xts object with ranks (asset with the greatest valuation receives 1)
#The ties parameter results in an ascending ranking. If two share the same rank, the first in the matrix gets the first rank
ranked<-as.xts(t(apply(-ValuationXTS,1,rank, ties.method="first",na.last=TRUE)))
}
After running this my index format has changed to POSIX:
> Test<-rankValuations(Test)
> str(index(Test))
POSIXct[1:235], format: "1995-01-31" "1995-02-28" "1995-03-31" "1995-04-28" "1995-05-31" "1995-06-30" "1995-07-31" ...
And this is a big problem because in the POSIX I have now a timezone. If using later on merge.xts it never matches since the POSIX dates are 1 day prior than in the to be merged with XTS which has a Date index. So how can I stop the rank method of changing Date to POSIX?
I've been trying to do a time series on my dataframe, and I need to strip times from my csv. This is what I've got:
campbell <-read.csv("campbell.csv")
campbell$date = strptime(campbell$date, "%m/%d")
campbell.ts <- xts(campbell[,-1],order.by=campbell[,1])
First, what I'm trying to do is just get xts to strip the dates as "xx/xx" meaning just the month and day. I have no year for my data. When I try that second line of code and call upon the date column, it converts it to "2013-xx-xx." These months and days have no year associated with them, and I can't figure out how to get rid of the 2013. (The csv file I'm calling on has the dates in the format "9/30,10/1...etc.)
Secondly, once I try and make a time series (the third line), I am unsure what the "order.by" command is calling on. What am I indexing?
Any help??
Thanks!
For strptime, you need to provide the full date, i.e. day, month and year. In case, any of these is not provided, current ones are assumed from the system's time and appended to the incomplete date. So, if you want to retain your date format as you have read it, first make a copy of that and store in a temporary variable and then use strptime over campbell$date to convert into R readable date format. Since, year is not a concern to you, you need not bother about it even though it is automatically appended by strptime.
campbell <-read.csv("campbell.csv")
date <- campbell$date
campbell$date <- strptime(campbell$date, "%m/%d")
Secondly, what you are doing by 'the third line' (xts(campbell[,-1],order.by=campbell[,1])) command is that, your are telling to order all the data of campbell except the first column (campbell[,-1]) according to the index provided by the time data in the first column of campbell (campbell[,1]). So, it would only work given the date is in the first column.
After ordering the data according to time-series, you can replace back the campbell$date column with date to get back the date format you wanted (although here, first you have to order date also like shown below)
date <- xts(date, order.by=campbell[,1]) # assuming campbell$date is campbell[,1]
campbell.ts <- xts(campbell[,-1], order.by=campbell[,1])
campbell.ts <- cbind(date, campbell.ts)
format(as.Date(campbell$dat, "%m/%d/%Y"), "%m/%d")