Optimization Beyond One Click

StrataSearch will find what you tell it to, but what should that be? Will the same criteria work across different Sectors? Different time periods? Here we discuss the ins and outs of OneClick Searches.

Optimization Beyond One Click

Postby JLG » Thu Nov 01, 2007 1:47 pm

I would be interested in suggestions for further optimization beyond exhaustion of gains from One Click with default settings.

Potential areas for further optimization appear to include:

(1) parameter shifts;
(2) order types on entry and exit;
(3) dynamic walk forward optimization;
(4) changing general defaults for numbers of strategies and alternate data analysis; and
(5) changing otimization defaults by running optimzation after a smaller number of combinations and increasing the number of parameter and trade rule cominations to optimize and the number of combinations.

Does anyone have any thoughts on which to pursue, in what steps and order?
JLG
JLG
 
Posts: 100
Joined: Thu Oct 25, 2007 3:32 pm

Postby Overload » Thu Nov 01, 2007 2:00 pm

That may depend on your objective. Do you wish to do additional optimization to increase your risk/reward ratio, or to build more robust systems? Obviously both are important, but the order of optimization areas depends on your primary objective.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Postby JLG » Thu Nov 01, 2007 2:34 pm

I feel comfortable with the statistical robustness of these systems. I view curve fitting risk as an unknown due to lack of experience with the output from Stratsearch. Of course better reward is good as is reduction of risk. This would normally be my first optimization objective followed by testing against alternate data sets for curve fitting.
JLG
JLG
 
Posts: 100
Joined: Thu Oct 25, 2007 3:32 pm

Postby Overload » Thu Nov 01, 2007 7:32 pm

Okay, with that in mind, I'll focus on the risk/reward:

(1) parameter shifts;

This is strictly a robustness test, but will not directly lead to a better risk/reward.

(2) order types on entry and exit;

I'm a fan of intraday exits, since the thought of having to wait till the next morning to exit with a stop is too great a risk for me. So on the AutoSearch Setup "Generic AutoSearch Optimization", I'll usually navigate to the Supporting Exit tab and check "Good-Till-Cancelled Stop" to be set for "Always". Intraday entries may have some benefit, but normally only for short holding periods. If you plan on holding for more than 10-15 days, the benefit from an intraday entry is typically not large (in my experience).

(3) dynamic walk forward optimization;

I like to test this, but this is done normally to verify the robustness of a system. In some cases, I suspect you will get higher performance out of your system by periodically adjusting the parameters, but this depends on the system.

(4) changing general defaults for numbers of strategies and alternate data analysis; and

Depending on the size of the sector, I've found 5 strategies to be sufficient to fill a portfolio in most cases. A system can become less manageable when using many more strategies than that. However, I know numerous users that have a dozen or more strategies in their multi-system.

(5) changing otimization defaults by running optimzation after a smaller number of combinations and increasing the number of parameter and trade rule cominations to optimize and the number of combinations.

The default settings for optimization have always worked well for me. However, I have had success adjusting the numbers of trading rules according to the size of the sector being evaluated. In short, larger sectors allow for larger numbers of trading rules, while smaller sectors require fewer trading rules. This has to do partially with statistical significance, but I believe that getting the most possible trading rules into your system (while still maintaining statistical significance) is beneficial to a system.

All of this is just from my own opinions and experience. People use StrataSearch in many different ways, so I'm sure other ideas and options are just as valid.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Postby JLG » Fri Nov 02, 2007 12:39 pm

Thanks for your comments. Most seem to pertain to ways to reduce risk and ascertain robustness. Other than dynamic parameter adjustment any ideas on ways to add to the upside would also be appreciated.

I do agree that intraday opimization is a significant opportunity beyond Stratasearch's current capability. I would assume this is on the product plan along with real time signals.
JLG
JLG
 
Posts: 100
Joined: Thu Oct 25, 2007 3:32 pm

Postby JLG » Fri Nov 02, 2007 1:30 pm

I have read the optimization settings help a few times and I am surprised a bit that sticking with the default settings (except trade rules) generally works best as a search matures. The help talks about the trade off of random search versus optimization. It seems you would want to focus on random early in the process and move toward optimization as the number of runs grow. If random search has been exhausted with say 100,000 runs, it would seem that adjusting these settings to focus much more on optimization would be productive. But I guess the software must work in a way to make this not the case.
JLG
JLG
 
Posts: 100
Joined: Thu Oct 25, 2007 3:32 pm

Postby Overload » Fri Nov 02, 2007 1:33 pm

The only other thing I might suggest for optimizing toward a higher risk/reward would be to make adjustments to the scoring algorithm itself. The Strategy Performance tab is primarily to filter out not only the poor performing strategies, but to avoid the curve-fitted ones as well. But the Multi-System Performance tab is what identifies which strategies are kept in your final system. So that is the algorithm you may want to play with.

Many of the default Multi-System Performance setups include a variety of numbers. But if your interest is strictly risk/reward, you may want to set up a single Evaluation Field that uses something like:

$F_AvgAnnReturn / $F_MaxDrawDPctU

Or, if you're interested in evaluating "All Trades" instead of just the portfolio-based trades, you might try:

$F_MCAvgAnnReturn / $F_MCAvgDrawdown

Your filtering will not change from what it is now, since that takes place at the individual strategy level. But by making changes to the Multi-System Performance tab, you'll now be optimizing which of the (already approved) strategies should be kept in your final system.

Also, yes, a realtime version is in the plans, but quite a ways away still.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Postby Overload » Fri Nov 02, 2007 1:43 pm

Regarding the optimization, you should be aware that optimization only takes place on strategies that have been identified since the last random search phase began. Thus, there is no re-optimization of strategies that have already been optimized and selected into your multi-system.

I can see that there are different ways to look at this. One could certainly set it up so that you have just one random search phase followed by just one optimization phase. But we have it set up the way that it is so that users can start seeing results more quickly. While this may not be the most time-efficient approach for the reasons you mention, it can be helpful to monitor the overall progression of a search rather than to have to wait for the completion of the random phase to see how it is really doing.

Regardless, the configurations are there, so you can set it up however you wish.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Postby JLG » Fri Nov 02, 2007 2:50 pm

Very helpful. I was not aware that changes to the optimization settings effected only strategies new to the search phase.

Is there a way to force reoptimization with Rerun Combination or Create Optimization Strategy, Parameter Shifted Strategy, or Optimization Trading Rule?
JLG
JLG
 
Posts: 100
Joined: Thu Oct 25, 2007 3:32 pm

Postby Overload » Fri Nov 02, 2007 4:03 pm

Code: Select all
Is there a way to force reoptimization with Rerun Combination or Create Optimization Strategy, Parameter Shifted Strategy, or Optimization Trading Rule?

No, there is no additional optimization that can be done within the OneClick Search itself. You can, however, use whatever "external" techniques you like, such as "Create Optimization Strategy". If you find a better performing parameter set or other alteration, you can then upgrade your OneClick's multi-system with that enhancement as desired.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm


Return to OneClick Searches and Scoring

cron