I have a few questions regarding parameter shifts and curve fitting I'm hoping I can get help with. Let me preface by saying I am trying to set up a multisystem search for a single symbol.
1. When I set up the Parameter Shift test (using variable equity terms instead of fixed), I am noticing that when I run a strategy from the multisystem separately using the same dates and prices that were used for the search, I'm not seeing bars which correspond to my cutoffs. 50% for 1S. 25% for 2S, 12.5 for 3S. For example, one of the 2S bars will be less than 25% of the 0 bar (I have the Up and Down Required box checked in Trade Settings). This is especially common the higher I go in Strategy #. I'm sure there is some "silly" reason for this, but I'm not seeing it.
2. I understand that due to the "arranging" of strategies within a multisystem that the Parameter shifts will experience a drop off from what I see from each strategy, but why is it so drastic? For instance, the search result has 75 trades (70 win 5 loss) and the Performance tab shows say 67 trades (63 win 4 loss) for the multisystem and yet the parameter shift chart for the multisystem is drastically worse.
3. Should I be preoccupied with parameter shifts if I am using 24 years worth of data? I would think that it wouldn't be as much of an issue as if I were using 5 years. How many years of data (10, 15, 20 etc) should one be really concerned with robustness IF at 24 years one does not need to be THAT concerned?
4. Is preoccupation with seeing a nice Avg Ann Return Parameter Shift chart the best way to insure robustness?
5. If a multisystem search reaches the goals set for the only two scoring factors in the Multisystem tab, will continuing the search seek to improve on the strategies "score" in the Strategy tab? Optimizing towards a higher percentage of the Parameter Shift tests for instance?
6. Ultimately, I would like the Multisystem search to "push" towards both Avg Annual Return and Minimizing changes in Parameter shifts EQUALLY. Is it possible to "push" for a nicer Parameter Shift chart via the Multisystem tab? If not, what settings would you recommend to have the search "push" toward higher returns and robustness equally? I understand that I will be sacrificing Avg Ann Return for Robustness, but I would like to better control the amount each is "pushed" for by the search. If you recommend other techniques for guarding against curve fitting, such as Sharpe, please provide examples of values to consider. I've seen in the manual where it states that a "high number" is preferred for the Sharpe ratio, but without a range, the term "high number" doesn't really help.
I know there is a lot here. I really appreciate any guidance/insight offered.