I have multiple symbols in a portfolio but when running a blotter trading strategy the end equity only updates for the last symbol that has been run. When looking into how the equity updates each transaction it seems that when a new symbol is introduced the equity goes back to its original value set at the beginning (1mil).
This is how the portfolio is getting updated for a symbol month by month:
updatePortf(portfolioName,Symbols=symbolName, Dates=currentDate)
updateAcct(accountName,Dates=currentDate)
updateEndEq(accountName, currentDate)
Why is this happening?
Hope my question makes sense and thank you in advance
This is a good question. if you look at applyStrategy you'll see that each symbol in the loop is run in isolation -- independently. You may want to check out applyStrategy.rebalancing which does a nested loop of the form:
for(i in 2:length(pindex)){
#the proper endpoints for each symbol will vary, so we need to get
#them separately, and subset each one
for (symbol in symbols){
#sret<-ret[[portfolio]]
This means it loops over a section of timestamps, and then for each symbol, which is what you want when you want some interactions between symbols (applyStrategy simply does a for symbols, then an inner by timestamp loop, so you'll never get interactions going on) in terms of equity.
When I first starting using quantstrat I initially had the same frustration. My solution was to modify applyStrategy.rebalancing do become a (slower) double loop, for each time stamp, then an inner loop across each symbol.
Yes, this means you can't directly compute portfolio PL accurately in quantstrat. So things like opening positions that are position of current portfolio equity can't be done directly. (But you can modify the code to do it if you want).
Why does quantstrat behave this way by default? The authors will give you good reasons. In short, my view is that (after some brief discussions with the authors), if a signal has predictive power, and gives you an edge in a strategy, it'll work regardless of how you combine it with other symbols later. quantstrat is about identifying whether signals are good or not in relation to mktdata you pass to it.
Logically, if a signal is good on a per symbol level, then it will likely do ok on a portfolio level too (if not better, with smoother portfolio PL). quantstrat's current approach will give you a reasonable approximation to what portfolio PL will look like, but not in a true "compounding return" sense. To do that, you'd want to scale your positions according to the current portfolio PL (which isn't possible in applyStrategy as noted above). This simplification of running a strategy per symbol only also makes simulations much, much faster. Note that you can introduce still interactions with other symbols in applyStrategy by adding additional columns to the symbol data that relate to other symbols though (e.g. in pairs trading, etc).
At the end of the day backtest results are always simplifications of trading in reality, so there isn't a big motivation to get "super" accurate backtest results that project profit/trading revenue very accurately.
Related
I'm trying to compute ( MACD - signal ) / signal of prices of Russel 1000 (which is an index of the 1000 US large cap stocks). I keep getting this error message and simply couldn't figure out why :
Error in EMA(c(49.85, 48.98, 48.6, 49.15, 48.85, 50.1, 50.85, 51.63, 53.5, :n = 360 is outside valid range: [1, 198]
I'm still relatively new in R although I'm proficient in Python. I suppose I could've used "try" to just work around this error, but I do want to understand at least what the cause of it is.
Without further ado, this is the code :
N<-1000
DF_t<- data.frame(ticker=rep("", N), macd=rep(NA,N),stringsAsFactors=FALSE)
stock<-test[['Ticker']]
i<-0
for (val in stock){dfpx=bdh(c(val), c("px_last"),start.date=as.Date("2018-1-
01"),end.date=as.Date("2019-12-30"))
macd<- MACD( dfpx[,"px_last"], 60, 360, 45, maType="EMA")
num<-dim(macd)[1]
ma<-(macd[num,][1]-macd[num,][2])/macd[num,][2]
i=i+1
DF_t[i,]<-list(val,ma)
}
For your information,bdh() is a Bloomberg command to fetch historic data.dfpx[] is a dataframe.MACD() is a function that takes a time series of prices and outputs a matrix,where the first column are the MACD values and the second column are the signal values.
Thank you very much! Any advice would be really appreciated. Btw, the code works with a small sample of a few stocks but it will cause the error message when I try to apply it to the universe of one thousand stocks. In addition, the number of data points is about 500, which should be large enough for my setup of the parameters to compute MACD.
Q : "...and error handling"
If I may add a grain of salt onto this, the error-prevention is way better than any ex-post error-handling.
For this, there is a cheap, constant O(1) in both [TIME]- and [SPACE]-Domains step, that principally prevents any such error-related crashes :
Just prepend to the instantiated the vector of TimeSERIES data with that many constant and process-invariant value cells, that make it to the maximum depth of any vector-processing, and any such error or exception is principally avoided :
processing-invariant value, in most cases, is the first know value, to be repeated that many times, as needed back, in the direction of time towards older ( not present ) bars ( yes, not relying on NaN-s and how NaN-s might get us in troubles in methods, that are sensitive to missing data, which was described above. Q.E.D. )
For those who are interested, I have found the cause of the error: some stocks have missing prices. It's simple like that. For instance, Dow US Equity has only about 180 daily prices (for whatever reason) over the past one and a half years, which definitely can't be used to compute a moving average of 360 days.
I basically ran small samples till I eventually pinpointed what caused the error message. Generally speaking, unless you are trying to extract data of above 6,000 stocks or so and you are querying say 50 fields, you are Okay. A rule of thumb for daily usage limit of Bloomberg is said to be around 500,000 for a school console. A PhD colleague working in a trading firm also told me professional consoles of Bloomberg are more forgiving.
This is more of a methodological (rather than a programming) issue, yet it feels SO is the right place for it. Following the ups and downs after Yahoo changed its defaults in May 2017 for fetching daily data (discussed on https://github.com/joshuaulrich/quantmod/issues/174, http://blog.fosstrading.com/2017/06/yahoo-finance-alternatives.html and also on SO Why Open,High,Low prices are wrong when using quantmod?) I am probably not the only one not 100% certain which data to use in a backtesting procedure and whether quantmod getSymbols.yahoo and adjustOHLC still provide the relevant data for quality backtesting.
Quantmod 0.4.11 also includes AlphaVantage as (adjusted stock) data provider, but I am not familiar with their reliability.
How to prepare the (stock and index) data obtained from getSymbols calls? Which data ((stock & dividends) adjusted or unadjusted) should be used? Which transformations do you use? The adjustOHLC function also contains a bug, as it is not split adjusted (easily seen on AAPL by calling
getSymbols(AAPL)
chart_Series(adjustOHLC(AAPL))
and observing a jump in 2014.
You should always use adjusted prices. Most of the time when data provider doesn't have adjusted prices then usually provider's close prices are adjusted. There is no point doing backtests on a raw close prices data. I've once made a mistake by downloading close prices instead of adjusted and at the end of backtesting, my strategy told me that among all S&P composites Master Card was the worst performer. After looking at the MA chart it was obvious why.
Beacuse of a split on January 22, 2014 my data had a single return over -90%! In conclusion raw close data for backtesting might give you utterly false results.
How to deal with splits
Divide every price before a split by split ratio. For example Master Card had 1:10 split ratio so you should divide every price before 21.01.2014 by 10. It's very easy to find splits in a data, you just have to look for returns around or below -50%.
Dividends
Subtract from every price before dividend day dividend amount. To find dividends days you need dividends calendar, it's impossible to find them by yourself.
How do I backtest over specific dates, for example 2008::2010 in Quanstrat?
I want to load symbols from 2001::2017, but i only want to back test over a subset of dates. (rather than reload the symbols every time for specific date ranges)
There is no built-in way to do this in quantstrat. In fact, there is a comment at the beginning of the apply* functions that says:
#TODO add Date subsetting
(patches welcome)
There are a number of possible ways to do this with the existing code though.
Probably the simplest way is to load all your market data into an environment, and then subset your market data into the .GlobalEnv before each call to applyStrategy.
Indicators and signals should use vectorized functions, and should take (at most) seconds to apply to the entire series. So the simplest thing is probably to run applyIndicators and applySignals manually over the entire series, and then call applyRules with just the subset you want.
You could also add a signal function that does understand subsets. This signal function would be last in the strategy specification, and would filter all your other signals to 0 outside of your preferred date range.
I have some COT data that I want to plot under the main price window as an indicator. The COT data are external data, i.e. independent of the prices. So one can not write it like a traditional indicator calculated from the prices. Since I have all the data needed, I don't need to do any calculation. I only need to convert the date and time so that it aligns with the price chart. I will figure out how to do it later. Now, if we ignore the alignment, what I want to ask is how could I plot the data under the price chart? Thanks!
Alternative A:
Use the MT4-GUI tools and plot the data programmatically right into the MT4.Graph or using the screen-layout-plane of GUI-objects, independent of the underlying live-[TimeDOMAIN,PriceDOMAIN]-graphing, both using Expert Advisor-type of MQL4-code. We use this approach most often for all the tasks, that would normally land as a Custom Indicator-type of MQL4-code, as the New-MQL4.56789 code-execution engine has reduced the achievable performance for all ( yes, ALL ) Custom Indicator code-units' execution into a single, thus both RealTime-sensitive and potentially blocking, thread.
Using this alternative, you retain the full freedom of the code-design and may benefit a lot from pre-computing & pre-setting the GUI-objects inside OnInit(){...} section, before entering the trading-loop. This also minimises the latency costs associated with a need to update the GUI-scene from inside an OnTick(){...} event-loop.
Alternative B:
One may also opt to do a similar job using an independent Script-type of MQL4-code unit, as the COT data are weekly announced and thus static per-se.
Launching Script is a step, that can happen whenever feasible and this implementation model may also enjoy some ex-post modification tools, that could be run from another Expert Advisor or another Script MQL4-code, for the sake for some ex-post live-GUI-scene modification/maintenance.
Alternative C:
If one indeed insists to do so, the GUI-composition might be assembled inside a rather special-purpose live-calculated Custom Indicator-type of MQL4-code.
This approach but has to carefully deploy the GUI-composition into the Custom Indicator OnInit(){...} section and avoid any risk of blocking a flow of execution inside the above said critical section of OnCalculate(){...}.
Buffer-mapped, register-based Custom Indicator data & graphing tools are rather rigid for more advanced purposes, that do not strictly follow the hard-wired logic of a code, responding just to a stream of MarketEvent-s, which may, but need not, happen at once, but is being arranged by a sort of mini-batches, so as to process the whole depth of the DataStore in a segmented ( thus less-blocking ) processing approach.
Building the GUI-scene inside the OnInit() section of the Custom Indicator, one may still benefit from distributed processing, if external data source is to be read and/or any similar type inter-platform communications ( be it for a messaging or a signalling purpose ).
My choice would be the [A]
Mapping { Date, Time } onto a MQL4-datetime is trivial, MQL4 used to use since its beginning datetime as int seconds elapsed since 1970-01-01,00:00:00.000 - so simple, so easy.
declare the indicator buffer:
double ExtBufferCOT[];
assign indexes of buffers
SetIndexStyle( 0, DRAW_LINE );
SetIndexBuffer( 0, ExtBufferCOT );
in the OnCalculate() function - make sure it is time to check the levels again ( I think you do not need to update them every tick, right? Maybe once a day or once a week) and then read the file that you have ( we do not have example of file so senseless to describe how to do that here ), convert elements of the file, using StrToTime() and StrToDouble()
the last step - get last N lines from your file, and map them to the indicator buffers:
double value;
datetime time; // - your values from file are here
int shift = iBarShift( _Symbol, 0, time );
ExtBufferCOT[shift] = value; /* probably need to fill buffer
of next candles too
if your chart timeframe
is smaller then frequency
of observations in the file
*/
I am currently attempting to implement a trading idea that I have been playing around with. It consists of 50+ securities and has a strategy very similar to this one. (Current package I am using is quantmod).
http://www.r-bloggers.com/backtesting-a-simple-stock-trading-strategy/
For those who aren't interested in clicking, it is a strategy that will look at the pass X days( in his case 200 ) and enter a position depending on the peak reached in the stock. I understand how to do this strategy for my idea, but I cannot grasp how to aggregate my data into one summary.
Is there a way I can consolidate the summary for all the positions I have entered into one larger portfolio summary and chart that against the S&P 500?
Any advice on where I can find resources or being lead to the information. I have looked at portfolio analysis package for R and I do not believe that will be much help to me.
Thank you in advance.
Edit: In the link, at the bottom, there are 3 indexes that are FTSE, N225, DJIA. Could i combine those 3 summaries to show the same output as below, BUT combined
FTSE:
Me Index
Cumulative Return 3.56248582 3.8404476
Annual Return 0.05667121 0.0589431
Annualized Sharpe Ratio 0.45907768 0.3298633
Win % 0.53216374 0.5239884
Annualized Volatility 0.12344579 0.1786895
Maximum Drawdown -0.39653398 -0.5256991
Max Length Drawdown 1633.00000 2960.0000
Could I get that same output but for the 3 securities data combined? Is there a effective way of doing that. Thank you so much. Happy holidays
It's a little unclear to me what you mean by "combine" in this case. If you want a single column representing the combined returns from all three exchanges as if they were a single unified market, that's really tricky, because the exchanges trade in different currencies (British pounds; U.S. dollars, Japanese Yen, etc.). The underlying analysis would have to be modified substantially to take into account fluctuating daily foreign exchange rates.
I suspect that this is NOT want you want. Rather, you are simply asking how to take three sequential two-column outputs and turn them into a single parallel six-column output.
If that is indeed what you want, then you need to rewrite the testStrategy() function shown near the bottom of the link. As it's currently written, that function takes three inputs: an index name myStock (with allowed values of FTSE, DJIA, or N225), and two integer values, nHold and nHigh. You would need to change it so that it instead accepts five inputs; e.g., myStockA, myStockB and myStockC, plus the two integer values already mentioned. Then each of the lines currently referring to myStock would have to be replicated three times. Finally, the two cbind() lines that you see at the bottom would have to be modified so that instead of merging the data together into only two columns, you include all six.
For a good intro tutorial on how to write and modify your own R functions, please see this. To understand how to use the cbind() function, which you will have to call with six rather than two inputs, please see this.