Systems Results

Let us know your likes and dislikes, or what you'd like to see next. We listen!

Systems Results

Postby mguerreiro » Sat Mar 20, 2010 11:41 am

Hello,

How powerful is Strata? what are the best systems you have seen? I think it could be interesting to understand how far you can get with SS to create a post where people can post the returns they have found on systems, of course, without posting the formula. I was thinking of something where you can post the picture of the Performance page from the detailed analysis and the chart of returns/drawdowns/position count being optional.

So far, the best I have found is a portfolio of 20, with max Unr. DD of 26% and a Compounded return of 39% per year, tested over 25 years on the Russell 2000. Port. Size: 20. I am quite happy with the system and was wondering how far have other people gotten with SS.
Thank you,

Marco
system T.gif
System
system T.gif (66.89 KiB) Viewed 6296 times
mguerreiro
 
Posts: 34
Joined: Wed Mar 17, 2010 5:48 pm

Re: Systems Results

Postby Overload » Sat Mar 20, 2010 4:47 pm

A number of other users have tried to get others to post their results over the years, but the idea has never really caught on. I think it's partially because StrataSearch is used in so many different ways. Not only might users decide to use their own sectors and evaluation periods, but many users define their own set of trading rules as well. Even further, many users consider statistics outside of the Performance Report, such as Alternate Data Analyses, Parameter Shift Tests and Monte Carlo numbers. It's actually pretty rare that I see 2 users running StrataSearch exactly the same way. For this reason, it makes it difficult for one user to compare numbers with another. The approaches might be fundamentally different, forcing an apples to oranges comparison.

Regardless, the numbers from your report look pretty good. My biggest hesitation would probably be the holding periods of over a year. That's a pretty lengthy commitment.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Re: Systems Results

Postby taowave » Mon Mar 22, 2010 2:48 pm

Hi Marco,
Are you running backtests on the current list of stocks in the Russel 2000 to get those results??

I noticed your avg days held was pretty large.
Keep in mind that you are going to have MASSIVE survivorship bias..

Also,how does your system generate annual returns of 13123%,while only compounding at 39.52%??The power of compounding cant be that great..can it?


mguerreiro wrote:Hello,

How powerful is Strata? what are the best systems you have seen? I think it could be interesting to understand how far you can get with SS to create a post where people can post the returns they have found on systems, of course, without posting the formula. I was thinking of something where you can post the picture of the Performance page from the detailed analysis and the chart of returns/drawdowns/position count being optional.

So far, the best I have found is a portfolio of 20, with max Unr. DD of 26% and a Compounded return of 39% per year, tested over 25 years on the Russell 2000. Port. Size: 20. I am quite happy with the system and was wondering how far have other people gotten with SS.
Thank you,

Marco
system T.gif
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: Systems Results

Postby mguerreiro » Mon Mar 22, 2010 7:12 pm

Hi, yes, its on the russell 2000.

Well, the sample is over 25 years, so not small, I am fine with the low holding periods, the low drawdown is because th system is quite consistent and cuts losses quite quickly., The average annlual return is crazy because it seems to be calculated based on the initial capital, which is much lower than the capital over the years, as it compounds, then it creates the distortion.
mguerreiro
 
Posts: 34
Joined: Wed Mar 17, 2010 5:48 pm

Re: Systems Results

Postby wisestocktrader » Tue Mar 23, 2010 2:35 am

Hi mguerreiro

Try splitting you stocks in half like randomly select 50% of the stocks and backtest on that. If you get very different results i would be worried.
The reason i am saying that is because your kurtosis value is of the charts.
wisestocktrader
 
Posts: 13
Joined: Thu Feb 18, 2010 2:14 am

Re: Systems Results

Postby mguerreiro » Tue Mar 23, 2010 6:03 am

Hi, thanks for that, unfortunately, the entry signals are not that many, when I look at the total entry signals for the 25 year period, only 600 trades happen. I have looked at all the trades, and the average return is 112%, and the average holding period 287 days, therefore quite in line with the test results.59% profitable. I think this makes it relatively consistent, even if the number of signals is not that high.

The results are influenced by 5 trades of over 1500% return. When I take out the best 10 trades, the compounded return goes down to 33%, with Unr DD of 23%. still, good to me.

What I like is the low drawdowns of the system and relatively consistent returns, which makes it possible to even leverage the system a bit.

What kind of results do you get with your systems?
mguerreiro
 
Posts: 34
Joined: Wed Mar 17, 2010 5:48 pm

Re: Systems Results

Postby taowave » Tue Mar 23, 2010 1:10 pm

Pete,is it normal to have a Monte Carlo avg annual return that is apx 30x greater than the avg annual return?That seems a bit odd...

MG,how much did you outperfrom the CURRENT stocks that are in the Russel 2000?
That should be your benchmark as it somewhat "neutralises" the survivorship bias..
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: Systems Results

Postby mguerreiro » Tue Mar 23, 2010 1:24 pm

Since 1986, Rus2k I think returned somewhere around 7-8%, so it did outperform the index quite well.

Isnt the AVG. ANN return calculated off the initial starting base? otherwise I can not understand how it is so high.
mguerreiro
 
Posts: 34
Joined: Wed Mar 17, 2010 5:48 pm

Re: Systems Results

Postby taowave » Tue Mar 23, 2010 1:54 pm

MG,there is a huge difference between the Rus2k returning 7-8% per year and what the CURRENT stocks in your R2k portfolio/sector has returned.
As you have the sector containing all the current stocks in the index,calculate the return on that sector for the last 25 years and compare it to the R2k.You will see what I mean.I have asked Pete to have a performance stat on the sector we trade to compare apples to apples.The current benchmark method is very misleading.

I dont look at the R2k,but when i look at the SPY vs the current stocks in the index,the current stocks that make up the S&P 500 outperform the SPY by apx 600 bps per annum.I would think that outperformance would be greater on the R2k.The database we use does not account for survivorship bias.

As for the returns,I did a very quick check yesterday,and the componded return and annual return look correct.

I have no idea why the Monte carlo avg annula return should be 30x greater than the avg annual return.Pete will know...



mguerreiro wrote:Since 1986, Rus2k I think returned somewhere around 7-8%, so it did outperform the index quite well.

Isnt the AVG. ANN return calculated off the initial starting base? otherwise I can not understand how it is so high.
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: Systems Results

Postby wisestocktrader » Tue Mar 23, 2010 8:05 pm

mguerreiro when you start trading the system please give us an update after about 6 months as to how it went :).
wisestocktrader
 
Posts: 13
Joined: Thu Feb 18, 2010 2:14 am

Re: Systems Results

Postby Dacamic » Thu Mar 25, 2010 10:58 am

taowave wrote:I dont look at the R2k, but when i look at the SPY vs the current stocks in the index, the current stocks that make up the S&P 500 outperform the SPY by apx 600 bps per annum. I would think that outperformance would be greater on the R2k. The database we use does not account for survivorship bias.

I tend to agree with Allan's concerns in this regard. Stocks climbing into the NASDAQ 100, S&P 500 or Russell 2000 might have growth patterns that look like rockets moving into orbit ... and those growth rates eventually taper off after orbit is achieved. When back testing over very long periods, system selection would be naturally biased toward finding stocks in their "pre-orbital" phase. I do not know whether the Russell 2000 has a performance bias different than that of the S&P 500; nonetheless, it seems reasonable to assume that one exists.

I have no idea why the Monte carlo avg annula return should be 30x greater than the avg annual return. Pete will know ...

The subject stat jumped out at me, too. For the most part, though, it is just a distortion created by a linear calculation being extended over a long period of time. The difference isn't nearly so dramatic when the Monte Carlo average annual return is converted into a compounded annual return (~60%).
Steve
Dacamic
 
Posts: 457
Joined: Wed Nov 30, 2005 12:40 pm

Re: Systems Results

Postby taowave » Thu Mar 25, 2010 4:56 pm

Steve,I still am confused why the Avg anuual return and the Ang annual Monte carlo return should vary by a factor of 30x..

It seems a bit high.If I am reading you correctly,you are saying that 30x factor equates to a 60% compounded annual return differential?

Thats huge,is it not?

T


I have no idea why the Monte carlo avg annula return should be 30x greater than the avg annual return. Pete will know ...

The subject stat jumped out at me, too. For the most part, though, it is just a distortion created by a linear calculation being extended over a long period of time. The difference isn't nearly so dramatic when the Monte Carlo average annual return is converted into a compounded annual return (~60%).[/quote]
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: Systems Results

Postby Dacamic » Fri Mar 26, 2010 11:23 am

Allan,

Compounded annual return is ~40% for a portfolio size of 20, while in Monte Carlo it is ~60%.

In this particular case, the difference seems consistent with the high values of Standard Deviation by Trade and Kurtosis (as mentioned by Wisestocktrader). So, although the range does seem large, it strikes me as plausible within the Monte Carlo report.
Steve
Dacamic
 
Posts: 457
Joined: Wed Nov 30, 2005 12:40 pm

Re: Systems Results

Postby taowave » Fri Mar 26, 2010 11:37 am

I was a bit too lazy to break it down to annual compounded returns,but 40 versus 60 seems to be acceptable...

The magic of compounding!!


Dacamic wrote:Allan,

Compounded annual return is ~40% for a portfolio size of 20, while in Monte Carlo it is ~60%.

In this particular case, the difference seems consistent with the high values of Standard Deviation by Trade and Kurtosis (as mentioned by Wisestocktrader). So, although the range does seem large, it strikes me as plausible within the Monte Carlo report.
taowave
 
Posts: 584
Joined: Sat Dec 02, 2006 12:39 pm

Re: Systems Results

Postby ovtrading » Tue Mar 30, 2010 2:51 pm

This is an interesting topic which I'd like to see more of; thanks for getting it started.

I agree with the comments about survivorship bias. Also, for the types of things I trade (futures contracts), a profit factor of 7.5:1 screams out curve fitting to me. It seems like the robust systems that hold up over time can't manage more than 3:1 or so. It may be different for stocks; I'd be interested to hear other experiences here.
ovtrading
 
Posts: 63
Joined: Fri Jul 18, 2008 11:37 am


Return to Comments and Suggestions

cron