Thoughts on: "One Click" Strategy parameters

StrataSearch will find what you tell it to, but what should that be? Will the same criteria work across different Sectors? Different time periods? Here we discuss the ins and outs of OneClick Searches.

Thoughts on: "One Click" Strategy parameters

Postby TomHam » Mon Mar 23, 2009 9:55 pm

After setting up the Sector / Family of securities and the appropriate time period(s) to make the strategy runs in (previous post), the next concept to tackle is what to use for settings in the Stratagy Performance tab. This is a "can of worms" because it all boils down to what your objectives are; and everyones WILL be different. But let me throw a few ideas out there, and maybe we can get other thoughts as well.

In general, I'm thinking most of us will start with Ave. Annual Return, and give it a high value ranking (perhaps arounf 50%).
Next, DrawDowns have got to be relatively high on the important list (25% or so). (after all, if your drawdown is close to your return, will you stay will the strategy ?; likely not.)
Third on the "likely list" I'm thinking would be # of Trades. This setting again is a personal perference, but an important one since it effects the number of trades and likely the % time in the market, etc.

From here on out, it is more subjective, and should vary with what the objectives are of the search. These are my personal favorites:
I give a "Monte Carlo Ave. Return Comparison" (i.e. Ave. Ann Return divided by Monte Carlo Ave. Return) a small value. This ratio would penalize a Ave. Ann. return that was significantly above / below the Monte Carlo Return. My limited experience has shown that if the Ave. Annual Return is very much above the Monte Carlo Return, the strategy will likely become unstable in the near future.
I also include "Percent of Perfect" and give it a small ranking. I am surprised on how effective % Perfect is in weeding out some poor strategies. Well done StrataSearch Team !

Now, we progress to adding a few parameters for "Anti Curve Fitting". Again, these are heavily based on personal strategy needs and would vary quite a lot. Here's a few that I usually add:
Kurtosis, Alt. Data MDD, Alt. Data CAR and Parameter Shift Test.
The StrataSearch manual gives good examples of settings to start off with on these. I usually use an Alternate Data period in my strategy runs, just for the purpose of minimizing curve fitting problems and to "keep it real".

One question that I have for Steve and / or Pete: Can you "over anti-curve fit" ? (i.e. add sooooo many anti-curve fitting parameters that you degrade, in some way, the results or robustness of the chosen strategy ?)

One last technique that I use is to let the search run for 100,000 runs or so, then check the exclusion results via the Setups, One Click SetUps, Update, Stratgy Performance tab. You can see how many stratagies have been eliminated. It's not a bad idea to stop a search and modify parameters or tweak them, then Clear and re-start the search.

I would sure like to here the thoughts of other users on what they feel, or have found to be effective for paramters & weights, especially for detecting strategies that perform well in the future.
Cheers and Good Trading ! .......... Tom .......... :)
TomHam
 
Posts: 18
Joined: Sun Jan 18, 2009 1:41 pm
Location: near Detroit, Michigan

Re: Thoughts on: "One Click" Strategy parameters

Postby Overload » Tue Mar 24, 2009 11:59 am

I'm going to start with your assumption that the Average Annual Return and Drawdowns should be the foundation of all search criteria. While that's certainly one way to do it, there are other criteria that can create a good foundation as well. More specifically, the equity-based Average Annual Returns are portfolio-based, meaning an evaluation of the Monte Carlo returns is required to evaluate "all trades". But there are ways to evaluate "all trades" without having to make that extra step.

Along those same lines, a high Average Annual Return of an individual Strategy isn't always what's best for the creation of a Multi-System. Clearly a high Average Annual Return is a good thing, but if it gets the high return primarily by being in the market a large percentage of the time, then that won't ulimately be beneficial to the Multi-System. Such a Strategy ultimately "steals" the portfolio, when instead it should be sharing it. Thus, an ideal Strategy in a Multi-System (in my opinion) is one that's in the market a small percentage of the time, but makes larger profits while it's in there. In other words, large daily profits over a short period of time rather than small daily profits over a long period of time.

One suggestion is DailyROR. This performance value identifies the Daily Rate of Return on a net basis, so it includes spread, slippage and commission adjustments. In addition, one can reference this value from the Combination Results Listing, which is based on "All Trades" rather than a portfolio selection only.

Another option is Average Trade Return, also from the Combination Results Listing. One potential problem with many systems is that the Average Trade Return is very small. This creates a rather risky system, since even the smallest additional slippage can wipe out the profits of the system. Speaking of this, I think it's always a good idea to search for systems based on a "worst case scenario" of spread and slippage. It's worth back testing the worst case scenario prior to trading a system anyway, and if you wouldn't trade a system that had an exceptionally poor worst case scenario, then you might as well include those parameters in your search.

As for drawdowns, those are definitely worth including as part of the foundation. But there are other methods of doing this as well. For example, the $C_WorstTrade value lets you know evalute the worst loss incurred by the system. If your risk tolerance won't let you have a trade with a 75% loss, then this is a value you might want to keep an eye on. Of course, unless you have an intraday stop and don't hold a position overnight, such a loss is possible anyway, but evaluating this value at least provides some additional protection in your search criteria.

Another area you didn't mention is consistency over time. This is extremely important to me. A couple values I like to keep an eye on are $F_WorstMonthU, which gives you the worst performance month in your back test. And also $F_NumWinMonthsU/$F_NumActiveMonths, which provides the percentage of winning months in which the system is active.

As for the possibility of "over anti-curve fit", yes, I do believe that's possible. I've often said that there is an inverse relationship between the expected annual returns and the curve-fit protection that's added. Or, put another way, a lower risk system will likely yield lower returns. So each trader needs to evaluate how much risk they're willing to accept, and look for a system that's appropriate for their needs.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Re: Thoughts on: "One Click" Strategy parameters

Postby TomHam » Tue Mar 24, 2009 1:02 pm

Thanks Pete. Great ideas to evaluate, I appreciate the alternatives. That's the neat thing about StrataSearch; I feel like I've walked into a college chem lab for the first time. :) So many ideas and concepts to try, and now the tools to explore them. I'm keep'in a "To Do List".
Take Care & Good Trading. ........... Tom ..........
TomHam
 
Posts: 18
Joined: Sun Jan 18, 2009 1:41 pm
Location: near Detroit, Michigan

Re: Thoughts on: "One Click" Strategy parameters

Postby Dacamic » Thu Mar 26, 2009 3:12 pm

It's questionable whether I should respond in this thread to the subject question because I don't use OneClick in textbook fashion. But, since throwing away the textbook can be beneficial, I'll jump into this discussion anyway.

For several reasons (some of which are probably illogical) I have not settled upon any one group of settings for Strategy Performance. While such, ummmm, flexibility has its drawbacks, it does allow me to select settings based upon the task at hand. Regardless of minstrel-like wanderings that my searches may resemble, Average Annual Return is a constant in my Evaluation Fields (in fact, I have used that as the only field in a search or two). Other fields frequently used include the following:

    Average Days Held (Combination Listing);
    Average Percent Profitable (Monte Carlo report in Fixed Trade Equity);
    Daily ROR (Combination Listing);
    Number of Trades (Combination Listing); and,
    Average Drawdown (Monte Carlo report in Fixed Trade Equity).
Evaluations regarding consistency and robustness remain as substantially manual tasks in my strategy development laboratory; thus, I do not use any Evaluations Fields for those tests.
Steve
Dacamic
 
Posts: 457
Joined: Wed Nov 30, 2005 12:40 pm

Re: Thoughts on: "One Click" Strategy parameters

Postby TomHam » Sat Mar 28, 2009 2:52 pm

One idea that I forgot to bring up, but has the potential of being important, and I would appreciate thoughts on:

Is it normally a good pratice to use the same parameters and weighting to evaluate the Multi-System Performance that you used to evaluate the Stratagy Performance ?

I have been doing that, and it appears to be successful, but I have a feeling that using the same criteria to evaluate a Multi-Strategy as I use for a Single Strategy, may be missing a important point in selecting a robust Multi-Strategy System. (i.e. I would think a "big picture" portfolio evalaution should be different than a single element of that portfolio.)

Thoughts & Comments ? ........... Tom ............
TomHam
 
Posts: 18
Joined: Sun Jan 18, 2009 1:41 pm
Location: near Detroit, Michigan

Re: Thoughts on: "One Click" Strategy parameters

Postby Dacamic » Sun Mar 29, 2009 5:04 pm

Tom,

In my opinion, it is not important to use the same evaluation criteria for both strategies and multi-systems. It indeed can be preferable to not do so.

As one example, I do not use the scoring mechanism at the strategy level; thus, "weighting" settings under Strategy Performance are not applicable to my OneClick searches. Along those same lines, I often use OneClick for the sole purpose of finding a single strategy, and so will use one set of criteria to filter strategies (Strategy Performance) and another set to rank them (Multi-System Performance).

To borrow a contemporary phrase, I stumble into unintended consequences when I use more than a handful of Evaluation Fields in Multi-System Performance. There might be users who have nicely mastered using a large number of criteria in their scoring. My lack of talent is in this regard prompts me to to push the bulk of the evaluation work to the strategy level.
Steve
Dacamic
 
Posts: 457
Joined: Wed Nov 30, 2005 12:40 pm


Return to OneClick Searches and Scoring