WFA

Let us know your likes and dislikes, or what you'd like to see next. We listen!

WFA

Postby taowave » Wed Aug 27, 2008 11:33 pm

Hi Pete,

I know you havent produced any videos for version 4,but I hope you come out with an illustrative one for Dynamic Strategies in One Click.I have a decent grasp on it,but not nearly what it should be.

TIA,

Allan
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Postby Overload » Thu Aug 28, 2008 9:36 am

That's a good idea, although it's a pretty heavy topic and I'll have to think pretty hard about how best to explain it. Interestingly, the idea of a Walk-Forward Analysis has been around for at least 15 or so years. But it's only in the past few that computers (and software) have been advanced enough to do a good job of running them. Hopefully more books will become available on the subject, but right now it's pretty slim.

Probably what will happen is that the initial V4 release will contain just the basic training videos. But over time, we'll expand the library into some advanced features such as WFAs. Thanks for the suggestion.

Pete
Overload
 
Posts: 2246
Joined: Wed Nov 30, 2005 12:14 pm

Postby taowave » Fri Aug 29, 2008 12:46 pm

Hi Pete,
If you dont mind,I am going to ask some questions portaining to Dynamic strategies.I pretty much understand the One Click process,so most of my questions will be on the setup of the DS.

I get that the dynamic systems is a walk foward approach.Jans post of the importance of walking foward as opposed to optimising static backtests got me started.

For the OC setting,I chose 2 for Multi System # of stategies,and 5 for the max # of positions open per strategy.I am assuming that means my portfolio will me a max of 10 stocks at any one time.In the evaluation process for scoring/ portfolio sizes,I selected 5 for Strategy and 10 for portfolio.Am I correct that if one selects to search for 2 strategies with a max of 5 open positions per stategy,it would be logical to base the scoring of the OC with a portfolio size of 5 for the strategys and 10 for the portfolio size of the combined multi system??

Moving along to DS..Lets assume my evaluation period is set to 1 year,from 01/01/2007-12/31/2007.

My confusion concerns New strategy frequency and Evaluation period.I think I am correct that if one has an evaluation period of one year,New strategy frequency would obviously have to be set for less than 1 year as in daily,weekly , monthly or some custom period.Per the manual;

New Strategy Frequency: This setting identifies how frequently new strategies are to be put in place for trading. For example, if Yearly is chosen, new strategies will be implemented yearly throughout the evaluation period specified.


So if I select a custom period of 90 days,every 90 days a new strategy may be selected depending upon strategy replacement selection.I am good here,but not quite sure on how and when the strategies are searched for to be selected.

From the manual..

Since the final results are based on the period in which the Multi-System would be used, and not on the period in which the Multi-System is determined, the Evaluation Period is adjusted by StrataSearch accordingly. For example, if the Evaluation Period is January 2004 through December 2004, the initial evaluation for a 30-day rolling period will be Dec 1, 2003 to Dec 31, 2003. During this time period, the Multi-System is determined. It is then used in January 2004, and the ending results are, in essence, walk-forward results.




I am guesing the evaluation period settings determine the answer to my question.As in the example,if I select a 30 day rolling period,the first "test" period would be the 30 days prior to my 1 year evaluation period, i.e. 12/1/06 -12/31/2006.If the rolling days selected is 60,would it be the 60 days prior the evaluation period.If I had selected 60 days for my New Strategy,is there a preferred setting that I should have for the rolling period??Also,if I am looking at a 30 day rolling window with a 60 day New strategy,does SS run 2 monthly "tests" do determine the New strategies for 60 day periods,or just one 30 day test? I am confused here,and I am not 100% sure what setting actually selects that initial test period.In the manual you have a 30 day rolling window as well as a monthly New Strategy selected.Obvioulsy both are 30 days,so its not easy to figure out

When I select cumulative evaluation period,I notice that days is greyed out.So how many days is used for the inital test period? S is that determined by New Strategy frequency(same question as above)?
If the New strategy is every 90 days,does the cumulative test period days increase by 90 days as we move foward in time??

I have more questions,but I will stop for now as this will hopefully strengthen my understanding.Sorry about the essay.I think a step by step example would go a long way in the manual,and it would be helpful if you illustrated each option in selection of settings.

Thanks,

Allan





Overload wrote:That's a good idea, although it's a pretty heavy topic and I'll have to think pretty hard about how best to explain it. Interestingly, the idea of a Walk-Forward Analysis has been around for at least 15 or so years. But it's only in the past few that computers (and software) have been advanced enough to do a good job of running them. Hopefully more books will become available on the subject, but right now it's pretty slim.

Probably what will happen is that the initial V4 release will contain just the basic training videos. But over time, we'll expand the library into some advanced features such as WFAs. Thanks for the suggestion.

Pete
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Postby Overload » Fri Aug 29, 2008 3:56 pm

All good questions. I'll do my best to cover them:

Hi Pete,
If you dont mind,I am going to ask some questions portaining to Dynamic strategies.I pretty much understand the One Click process,so most of my questions will be on the setup of the DS.

I get that the dynamic systems is a walk foward approach.Jans post of the importance of walking foward as opposed to optimising static backtests got me started.

For the OC setting,I chose 2 for Multi System # of stategies,and 5 for the max # of positions open per strategy.I am assuming that means my portfolio will me a max of 10 stocks at any one time.In the evaluation process for scoring/ portfolio sizes,I selected 5 for Strategy and 10 for portfolio.Am I correct that if one selects to search for 2 strategies with a max of 5 open positions per stategy,it would be logical to base the scoring of the OC with a portfolio size of 5 for the strategys and 10 for the portfolio size of the combined multi system??

Yes, you're correct on all points.

Moving along to DS..Lets assume my evaluation period is set to 1 year,from 01/01/2007-12/31/2007.

My confusion concerns New strategy frequency and Evaluation period.I think I am correct that if one has an evaluation period of one year,New strategy frequency would obviously have to be set for less than 1 year as in daily,weekly , monthly or some custom period.Per the manual;

Yes, correct.

So if I select a custom period of 90 days,every 90 days a new strategy may be selected depending upon strategy replacement selection.I am good here,but not quite sure on how and when the strategies are searched for to be selected.

I am guesing the evaluation period settings determine the answer to my question.As in the example,if I select a 30 day rolling period,the first "test" period would be the 30 days prior to my 1 year evaluation period, i.e. 12/1/06 -12/31/2006.If the rolling days selected is 60,would it be the 60 days prior the evaluation period.

Yes.

If I had selected 60 days for my New Strategy,is there a preferred setting that I should have for the rolling period??

No, not that I'm aware of. On one hand, a short period will provide greater weight to the most recent data. But on the other hand, a longer period will add additional back testing depth. So, like a great deal of technical analysis, this is up to the trader.

Also,if I am looking at a 30 day rolling window with a 60 day New strategy,does SS run 2 monthly "tests" do determine the New strategies for 60 day periods,or just one 30 day test?

It will run only one test for each period, as defined in the New Strategy Frequency setting. So if that is set for Monthly, it will search for a new system each month. If it is set for Year, it will search for a new system each year. The DTS Evaluation Period settings only identify what data is used when searching for those trading systems. But it doesn't identify how frequently they're implemented.

I am confused here,and I am not 100% sure what setting actually selects that initial test period.In the manual you have a 30 day rolling window as well as a monthly New Strategy selected.Obvioulsy both are 30 days,so its not easy to figure out

Actually, I think it's a 90 day Rolling Period. Basically, it means that the back test will search for new trading systems on a Monthly basis, and will always use the most recent 90 days of data when performing this search.

When I select cumulative evaluation period,I notice that days is greyed out.So how many days is used for the inital test period? S is that determined by New Strategy frequency(same question as above)?

Yes, it depends on the New Strategy Frequency. If it's Monthly, it will begin with the most recent month. If it's Yearly, it will begin with the most recent year. There probably could have been a separate setting for this, but since we're already suffering from "setting overload", it didn't seem worth it at the time.

If the New strategy is every 90 days,does the cumulative test period days increase by 90 days as we move foward in time??

Yes, that's exactly right.

I have more questions,but I will stop for now as this will hopefully strengthen my understanding.Sorry about the essay.I think a step by step example would go a long way in the manual,and it would be helpful if you illustrated each option in selection of settings.

I agree. As WFA's are becoming more popular, some better documentation on this would be helpful.

On a side note, the only difference between a Walk-Forward Analysis (WFA), and the more generic Dynamic Trading Strategies (DTS) has to do with the trading rules used. In a WFA, only the parameter sets of a system will change from one period to the next. In a DTS, you can search for entirely new trading strategies from one period to the next.

Pete
Overload
 
Posts: 2246
Joined: Wed Nov 30, 2005 12:14 pm

Postby taowave » Fri Aug 29, 2008 7:42 pm

Hey Pete,

Thanks for the help.I am going to play with DS settings and get a bit more comfortable.I have a funny feeling I may be more comfortable with the WFA feature,and wasnt fully aware of the differences until now.I see they are substantially different.

As a semi discretionary trader it is easy to see how to apply WFA in ones trading.I am still scratching my head on Dynamic systems.I guess its still the discretionary trader in me not feeling completely at ease with different systems and parameters for each new period.

Am I correct that if one wants to run a long/short WFA,only Dynamic systems could do that??

Do you(or anyone else) use the dynamic systems in your trading??

Allan


More questions on the way,




Overload wrote:All good questions. I'll do my best to cover them:

Hi Pete,
If you dont mind,I am going to ask some questions portaining to Dynamic strategies.I pretty much understand the One Click process,so most of my questions will be on the setup of the DS.

I get that the dynamic systems is a walk foward approach.Jans post of the importance of walking foward as opposed to optimising static backtests got me started.

For the OC setting,I chose 2 for Multi System # of stategies,and 5 for the max # of positions open per strategy.I am assuming that means my portfolio will me a max of 10 stocks at any one time.In the evaluation process for scoring/ portfolio sizes,I selected 5 for Strategy and 10 for portfolio.Am I correct that if one selects to search for 2 strategies with a max of 5 open positions per stategy,it would be logical to base the scoring of the OC with a portfolio size of 5 for the strategys and 10 for the portfolio size of the combined multi system??

Yes, you're correct on all points.

Moving along to DS..Lets assume my evaluation period is set to 1 year,from 01/01/2007-12/31/2007.

My confusion concerns New strategy frequency and Evaluation period.I think I am correct that if one has an evaluation period of one year,New strategy frequency would obviously have to be set for less than 1 year as in daily,weekly , monthly or some custom period.Per the manual;

Yes, correct.

So if I select a custom period of 90 days,every 90 days a new strategy may be selected depending upon strategy replacement selection.I am good here,but not quite sure on how and when the strategies are searched for to be selected.

I am guesing the evaluation period settings determine the answer to my question.As in the example,if I select a 30 day rolling period,the first "test" period would be the 30 days prior to my 1 year evaluation period, i.e. 12/1/06 -12/31/2006.If the rolling days selected is 60,would it be the 60 days prior the evaluation period.

Yes.

If I had selected 60 days for my New Strategy,is there a preferred setting that I should have for the rolling period??

No, not that I'm aware of. On one hand, a short period will provide greater weight to the most recent data. But on the other hand, a longer period will add additional back testing depth. So, like a great deal of technical analysis, this is up to the trader.

Also,if I am looking at a 30 day rolling window with a 60 day New strategy,does SS run 2 monthly "tests" do determine the New strategies for 60 day periods,or just one 30 day test?

It will run only one test for each period, as defined in the New Strategy Frequency setting. So if that is set for Monthly, it will search for a new system each month. If it is set for Year, it will search for a new system each year. The DTS Evaluation Period settings only identify what data is used when searching for those trading systems. But it doesn't identify how frequently they're implemented.

I am confused here,and I am not 100% sure what setting actually selects that initial test period.In the manual you have a 30 day rolling window as well as a monthly New Strategy selected.Obvioulsy both are 30 days,so its not easy to figure out

Actually, I think it's a 90 day Rolling Period. Basically, it means that the back test will search for new trading systems on a Monthly basis, and will always use the most recent 90 days of data when performing this search.

When I select cumulative evaluation period,I notice that days is greyed out.So how many days is used for the inital test period? S is that determined by New Strategy frequency(same question as above)?

Yes, it depends on the New Strategy Frequency. If it's Monthly, it will begin with the most recent month. If it's Yearly, it will begin with the most recent year. There probably could have been a separate setting for this, but since we're already suffering from "setting overload", it didn't seem worth it at the time.

If the New strategy is every 90 days,does the cumulative test period days increase by 90 days as we move foward in time??

Yes, that's exactly right.

I have more questions,but I will stop for now as this will hopefully strengthen my understanding.Sorry about the essay.I think a step by step example would go a long way in the manual,and it would be helpful if you illustrated each option in selection of settings.

I agree. As WFA's are becoming more popular, some better documentation on this would be helpful.

On a side note, the only difference between a Walk-Forward Analysis (WFA), and the more generic Dynamic Trading Strategies (DTS) has to do with the trading rules used. In a WFA, only the parameter sets of a system will change from one period to the next. In a DTS, you can search for entirely new trading strategies from one period to the next.

Pete
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: WFA

Postby taowave » Thu Mar 15, 2012 12:45 pm

Hi Pete,
Revisiting an old topic..

If I am testing one simple strategy with only one optimzable parameter,is there a way to maximise the settings for speed??

I understand the need for the "Search time initial period" and the "Search time normal periods" when searching for different strategies and walking foward,but "how much time" should I select for a simple one parameter walk foward?? As an example,if I select an hour,but the optimisation is completed in 5 minutes for the first IS,will SS move on to the OOS backtest?I am guessing it would move right along.

With that said,I am running WFA's in SS and it appears that it takes a bit longer than in another program I use.Its not an apples to apples comparison as you present much more detailed output/charts and at every interval.If I was only interested in the final performance,could I speed things up?

Allan
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: WFA

Postby Overload » Thu Mar 15, 2012 2:56 pm

The first thing you can do is turn off the automatic run of the Detailed Analysis at each interval. On the OneClick Setup, navigate to the Results tab and uncheck the "Automatically Run Detailed Analysis on Latest Multi-System" box.

As for the time devoted to each interval, that is something that you can adjust as you go along. The Statistics section of the Detailed Analysis tab is a helpful tool for this. It will tell you how many combinations you can expect at each interval. So you can then run an interval or two and see how long each combination takes. Then plug the estimated seconds per combination into the Statistics and it will give you information on how many combinations will be run per minute etc. Then, as you adjust the Search Time, it will also adjust the statistics and give you an estimate of the percentage of your combinations that will be run.

Ideally, you'll want the Est. Sampling Per Period to be about 110% to 120% if you want to test all possible combinations. The WFAs are not exhaustive but are instead based on time, so there is the possibility that some combinations will run twice. It's for this reason that you want to be over 100%.

In any case, after you've run a couple intervals and adjusted your settings, you can then restart the WFA to have it continue with the new and optimized settings.

Pete
Overload
 
Posts: 2246
Joined: Wed Nov 30, 2005 12:14 pm

Re: WFA

Postby taowave » Thu Mar 15, 2012 4:43 pm

Any tips on the "Search time initial period"??

I have another question..

Lets say i am optimising moving average length from 1-100 x1 step.

That woud be 100 combinations

If I am running an optimisation on the SP 100 index would that imply 100 x 100 "calculations?

By looking at the "statistics" that doesnt appear to be the case

thanks,

Allan
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: WFA

Postby Overload » Thu Mar 15, 2012 5:33 pm

For you situation, the Initial Period should be the same as the Normal Period.

If your Trading Rule has 100 possible steps as you mentioned, then there are also 100 possible combinations if you are running against 1 sector. The number of symbols in the sector isn't relevant to this calculation.

Pete
Overload
 
Posts: 2246
Joined: Wed Nov 30, 2005 12:14 pm

Re: WFA

Postby taowave » Fri Mar 16, 2012 10:25 pm

Hi Pete,
I have been running WFA on single stratagies with the settings you reccomended(over 100%) and was cross checking the results vs another program I use.I became somewhat concerned as I was getting markedly different results between the programs using the same settings and portfolios.In one of the WFA's,the annual return from SS was -8.3% vs +6.2% from the other program..

After several more tests,I decided to increase the time periods by a factor of 2.5 which brought the sampling per period up to 309%.I then ran the WFA again,and the WFA annual return went from -8.3% to +6.0%.That came very close to the returns generated from my other program,but left me with significant concerns with SS walk foward process.

I dont think it is the most prudent approach to have the user ballpark what the time period should be in hopes of obtaining an exhaustive Walk Foward Analysis.I would really like to see an option to select an exhaustive WFA approach,leaving out the aproximations for the time period.The fact that my returns went from negative 8.3% with 123% vs positive 6.0 with 309% is a bit scary.Keep in mind,if I wasnt cross checking my results vs another program,the desparity in returns would have gone unnoticed,and I would have simply discarded the stratagies.

On a realted note,when I increased my sampling period up to 309%,my estimated time to completion went from 20 minutes to 50 minutes,and I still dont know how many permutaions SS may have skipped over.Compare that with a 4 minute exhaustive search for my other program,and you can hopefully see my concern.As you know,SS is my favorite program,and I rely on it quite heavily.If I am not using it correctly,please let me know..

Thanks,

allan


Overload wrote:The first thing you can do is turn off the automatic run of the Detailed Analysis at each interval. On the OneClick Setup, navigate to the Results tab and uncheck the "Automatically Run Detailed Analysis on Latest Multi-System" box.

As for the time devoted to each interval, that is something that you can adjust as you go along. The Statistics section of the Detailed Analysis tab is a helpful tool for this. It will tell you how many combinations you can expect at each interval. So you can then run an interval or two and see how long each combination takes. Then plug the estimated seconds per combination into the Statistics and it will give you information on how many combinations will be run per minute etc. Then, as you adjust the Search Time, it will also adjust the statistics and give you an estimate of the percentage of your combinations that will be run.

Ideally, you'll want the Est. Sampling Per Period to be about 110% to 120% if you want to test all possible combinations. The WFAs are not exhaustive but are instead based on time, so there is the possibility that some combinations will run twice. It's for this reason that you want to be over 100%.

In any case, after you've run a couple intervals and adjusted your settings, you can then restart the WFA to have it continue with the new and optimized settings.

Pete
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: WFA

Postby Overload » Fri Mar 16, 2012 11:16 pm

First, have you confirmed that your estimated seconds per combination is correct? If not, then your Estimated Sampling Period Period is off. As I suggested, you need to get 120% real coverage if you want a near-exhaustive sampling. If you are requiring over 300%, I'm guessing your estimated seconds per combination is incorrect.

But rather than look at this limited sampling as a weakness, I consider it a strength. Sometimes I will intentionally prevent a complete sampling because I want to minimize overfitting. If you have an exhaustive search every time, then your results may be overly precise. However, if you bypass 5% of your sampling at each interval, you get a helpful indication of how robust your trading system is. Does your other program offer this?

As for the "exhaustive" search of the other program, that may be an incorrect statement. While some programs offer a sequential test of parameters, that sequential approach makes it virtually impossible to use against a large number of parameters sets. Doing so would take a lifetime. However, most optimization programs that do allow WFAs against large parameter sets use their own algorithms to drill down to the "best" combinations. You have little control over what is considered "best", and you bypass the majority of combinations for the sake of speed.

StrataSearch is unique in that it allows you to run a WFA against large numbers of paremeters without the bias of our own internal algorithms. One of the drawbacks to this is, of course, speed. But that choice is up to you. If you want speed, by all means use the other program. But if you want flexibility and non-biased results, you'll need to accept that it will take longer to run. Or you'll need to get a faster machine.

Pete
Overload
 
Posts: 2246
Joined: Wed Nov 30, 2005 12:14 pm

Re: WFA

Postby taowave » Sat Mar 17, 2012 12:07 am

Pete,I am lost..Perhaps a simple step by step example detailing the manner in which you use it would go a long way..
I naively thought all walk fowards should be equal..
Could you explain why my returns shift form -8% to +6%


Allan

Overload wrote:First, have you confirmed that your estimated seconds per combination is correct? If not, then your Estimated Sampling Period Period is off. As I suggested, you need to get 120% real coverage if you want a near-exhaustive sampling. If you are requiring over 300%, I'm guessing your estimated seconds per combination is incorrect.

But rather than look at this limited sampling as a weakness, I consider it a strength. Sometimes I will intentionally prevent a complete sampling because I want to minimize overfitting. If you have an exhaustive search every time, then your results may be overly precise. However, if you bypass 5% of your sampling at each interval, you get a helpful indication of how robust your trading system is. Does your other program offer this?

As for the "exhaustive" search of the other program, that may be an incorrect statement. While some programs offer a sequential test of parameters, that sequential approach makes it virtually impossible to use against a large number of parameters sets. Doing so would take a lifetime. However, most optimization programs that do allow WFAs against large parameter sets use their own algorithms to drill down to the "best" combinations. You have little control over what is considered "best", and you bypass the majority of combinations for the sake of speed.

StrataSearch is unique in that it allows you to run a WFA against large numbers of paremeters without the bias of our own internal algorithms. One of the drawbacks to this is, of course, speed. But that choice is up to you. If you want speed, by all means use the other program. But if you want flexibility and non-biased results, you'll need to accept that it will take longer to run. Or you'll need to get a faster machine.

Pete
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: WFA

Postby Overload » Sat Mar 17, 2012 9:03 am

It might be good if you sent me a copy of your WFA. Then I can use your exact numbers in a more detailed explanation. You can send me a copy by opening the File > Export Database menu in StrataSearch and entering a valid Filename. Then click the Filter button to the right of the OneClick Setups, and check the box for this particular WFA. After click OK on the Export Database window, the export file will be created and you can send me a copy at support@stratasearch.com.

Thanks,
Pete
Overload
 
Posts: 2246
Joined: Wed Nov 30, 2005 12:14 pm

Re: WFA

Postby taowave » Sat Mar 17, 2012 10:53 am

Appreciate it Pete..
As you know I am a discretionary trader who uses SS to lend a little science to the art.

Currently,it appears I know just enough to be dangerous:)

Allan

Overload wrote:It might be good if you sent me a copy of your WFA. Then I can use your exact numbers in a more detailed explanation. You can send me a copy by opening the File > Export Database menu in StrataSearch and entering a valid Filename. Then click the Filter button to the right of the OneClick Setups, and check the box for this particular WFA. After click OK on the Export Database window, the export file will be created and you can send me a copy at support@stratasearch.com.

Thanks,
Pete
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: WFA

Postby Overload » Sun Mar 18, 2012 10:51 am

Thanks, I got your WFA and was able to look at what you're doing a little closer. And I have several thoughts:

1) First, you may find it difficult to match the results of a WFA from one program to another. It is not that the same parameters sets will not be run, but rather that there are many settings regarding how WFAs are handled at each interval and you would need to be very sure both programs are handling everything identically. If even one setting is different, you will be comparing apples to oranges. This includes things like how the "best" parameter set is chosen, the number of days in the interval, the number of days in the evaluation period, whether positions are exited at the end of the interval, etc. Even further, you would need to be sure the results themselves are identical, including use of portfolio, removing best/worst trades, fixed versus variable equity evaluations, spread and slippage, etc. This isn't to say a WFA can't be compared from one program to another. But doing so is a significantly more complicated task that simply comparing a static system from one platform to another.

2) I noticed in your WFA that you have both phases of optimization turned on. This is incorrect based on what you are trying to do. The first phase is basically redundant, since there is no need to optimize variables when you only have one variable. But the second phase is most definitely problematic since you are potentially inserting new trading rules into your strategies at each interval. I don't think that's what you want to do. So you should turn off Optimization altogether.

3) If your WFA is truly exhaustive, you should be able to run it countless times and get the exact same result every time. Since that didn't apparently happen with you, the only answer is that you weren't letting it run long enough at each interval. StrataSearch does not sequentially test each parameter set, but instead uses its common approach of testing random combinations for a specified time limit. While I agree this is less efficient if you only have one or two variables, I've found it is pretty rare in StrataSearch that you only have one or two variables. Most systems I've found have 5 or 10 or even 15, and it would take a lifetime to test every possible combination. In this case, the random sampling becomes very valuable, but the time you spend on each interval becomes a critical factor. In my opinion, it is as important as any one of the variables.

To better explain my technique of partial sampling, consider both the Monte Carlo Simulations and the Parameter Shift Testing. Both of these techniques are methods of tweaking the algorithms slightly with the intention of discovering how robust the trading system actually is. A partial sampling in a WFA is a similar concept. If you can only get the 2nd best, or 3rd best parameter set in some of your intervals, would your system still be profitable? If not, then your system may not be very robust and could fail moving forward. Again, it is similar to a Parameter Shift Test. If you are using a moving average of 19 days instead of 20, does your system fall apart? If so, that system lacks robustness and could very well fail moving forward.

Pete
Overload
 
Posts: 2246
Joined: Wed Nov 30, 2005 12:14 pm

Next

Return to Comments and Suggestions

cron