Any way to get a Monte Carlo Average for the Sharpe ratio?

Questions directed to our Support Team. Responses can be expected within 24 hours.

Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Kevin_in_GA » Sun Sep 23, 2012 6:20 pm

I use both the MC Av Annual return and the Sharpe ratio to help select good systems. The problem is that the Sharpe ratio is only for the specific combination of stocks selected during the backtest, and what would be more helpful would be to see the MC Av Sharpe ratio instead.

TIA,

Kevin
Kevin_in_GA
 
Posts: 171
Joined: Wed Aug 04, 2010 10:14 am

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Overload » Mon Sep 24, 2012 8:47 am

It isn't currently possible to get the Sharpe Ratio as part of the Monte Carlo result, but I will think further about whether this can be done in the future. It could be problematic since the Sharpe Ratio uses the monthly Standard Deviation in the denominator, but the Monte Carlo Simulations only track sequences of trades and do not track time.

In any case, the Sharpe Ratio is available in both the Alternate Data Analyses and the Parameter Shift Analyses. So the use of either of those might be helpful if you're looking to see the Sharpe Ratio performance on alternate scenarios.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Kevin_in_GA » Sat Dec 29, 2012 8:56 pm

Well, I defaulted to using

$F_MCAvgAnnReturn / $F_MCAvgDrawdown

as a measure of risk and return. Not the Sharpe per se, but it does help to guide system selection (especially in WFAs). In some ways this is more important since it gives you a sense of drawdown as risk (more meaningful to traders).

I was thinking that it might be more informative to graphically present the MC analyses as a histogram rather than as is currently presented. This would allow for an immediate visual gauge of the results, particularly the spread. It would also be nice to have vertical lines drawn at 1, 2, and 3 SDs.
Kevin_in_GA
 
Posts: 171
Joined: Wed Aug 04, 2010 10:14 am

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Kevin_in_GA » Sun Dec 30, 2012 6:13 pm

To expand on this (after some further thought) ... could you make it so that the individual trade combination results that comprise the MC averages can be exported into .csv format? I would like to see the overall distribution of the MC studies, especially for the annual return and average drawdown. If these could be exported to Excel I could do further analysis on that file. Basically I'd like to know for a given MC result what are the average and 1,2, and 3 SD from the mean.

Thanks,

Kevin
Kevin_in_GA
 
Posts: 171
Joined: Wed Aug 04, 2010 10:14 am

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Kevin_in_GA » Mon Jan 07, 2013 7:03 pm

Pete:

I want to follow up on this since I think it would be a great addition to SS, and hopefully would not require a lot of work to implement. If we could export the various MC averages in a .csv then it would be easy to look at the mean and SD for the MC analysies (I specifically would like to look at the average annual returns and average drawdown).

Thanks,

Kevin
Kevin_in_GA
 
Posts: 171
Joined: Wed Aug 04, 2010 10:14 am

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Overload » Tue Jan 08, 2013 11:52 am

I understand what you're looking for, but I think part of my hesitation comes from the risk of over-analyzing data that is only very hypothetical to begin with. While a Monte Carlo simulation has its benefits, the data is not real, and in most cases the scenarios could never happen. Digging too deep into that data may have its own share of risks, suggesting results that are unrealistically skewed.

Just as a refresher to those that maybe aren't familiar with the Monte Carlo, it starts with the pool of all possible trades, regardless of portfolio. Then it makes a random selection from that pool of all possible trades based on the number of trades actually selected from a particular portfolio size. So, for example, suppose a Portfolio Size of 5 held 20 positions during its back test, and there were a total of 100 possible positions without the portfolio selection. The Monte Carlo simulation will select 20 positions at random from the entire pool of 100. It will then do this 1000 times to create a large sampling of possibilities.

While this is a helpful tool for evaluating all possible positions, it also uses scenarios that are impossible. For example, it is possible to base a complete scenario on positions that were in actuality entered in just one month, or even one day. Positions that would actually have been entered at the end of the back test, might be entered at the beginning during a Monte Carlo. So again while I think it's a useful tool for examining a variety of scenarios, it's also important to remember the unrealistic aspects of its scenarios. And, even further, all of this makes me hesitant to pursue detailed metrics of those numbers when their foundation is already a little questionable. Your request is on the list, and I will consider it further, but I still have my doubts.

I understand that you want a better evaluation of all possible trades versus just those selected by the portfolio, so here are at least a couple approaches that I use:

1) In the Detailed Analysis Charts tab, select the View of: "Trade Returns - All Versus Portfolio". In the background, this chart shows the trade returns of all possible positions regardless of portfolio. But overlayed on top of that are the trade returns of the positions selected by the portfolio. By examining the distribution, you can get a good indication of whether the portfolio selection is including the best trades that the system has to offer.

2) I'm a big fan of looking at "All Trades", although this can only be done in the Trades Report of the Detailed Analysis. But first, open the Setups > Trade Settings menu, Detailed Analysis tab, and check the box titled "All Trades (Trades Report Only)". Then rerun your Detailed Analysis. When you then enter the Trades Report area of the Detailed Analysis, you can change the Portfolio Size to "[All]". What I find most helpful are the Summary By Year and Summary By Month tabs. In addition to items like Average Trade Percent, Number of Trades and Percent Profitable, you can see other important items like Best/Worst Trades and Consecutive Wins/Losses. But most importantly, you can see these values broken out into year and month. And it includes all trades, not just the portfolio selection. It's a great way of evaluating "all trades" in a system, and it is based on real data.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Kevin_in_GA » Tue Jan 08, 2013 1:23 pm

1) In the Detailed Analysis Charts tab, select the View of: "Trade Returns - All Versus Portfolio". In the background, this chart shows the trade returns of all possible positions regardless of portfolio. But overlayed on top of that are the trade returns of the positions selected by the portfolio. By examining the distribution, you can get a good indication of whether the portfolio selection is including the best trades that the system has to offer.


OK, that is basically what I am looking for on the trade returns. To be honest, I never knew what that tab was for! I had thought (incorrectly) that a Monte Carlo took the various combinations of trades factoring in their time sequence, but that is apparently not the case.

I am finding that in many variations of a key short term system I am developing, the Monte Carlo analyses are usually twice the portfolio return - often 160+% per year, versus 70-75% based on a RSI(2) ascending ranking. Knowing that I can't control the order of trades for a given day like the ranking would require, the Monte Carlo approach seemed to be the best way to randomize enties (note - I also use "random(1,100)" as the ranking, which helps but only gives you one tiny slice of the possible results).

Is there a way to run 1000 iterations of the system using "random(1,100)" as the rank? Could you use something like "random(1,@1rand)" with @1rand varying from 1 to 1000? That would give me what I am looking for.
Kevin_in_GA
 
Posts: 171
Joined: Wed Aug 04, 2010 10:14 am

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Overload » Tue Jan 08, 2013 3:20 pm

I am finding that in many variations of a key short term system I am developing, the Monte Carlo analyses are usually twice the portfolio return - often 160+% per year, versus 70-75% based on a RSI(2) ascending ranking.

It's important to identify the reasons why the Monte Carlo is so different than the portfolio selection. Sometimes the reason validates your strategy and sometimes it invalidates it. I looked at one strategy that had roughly 1000 trades spread over the last 5 years. The Monte Carlo was twice the portfolio selection's annual return, suggesting the strategy might do better than expected. But that was not the case. Upon further inspection, I discovered that just one year (2009) generated 80% of all trades in the 5-year back test, and most of those trades were highly positive. But the other 4 years were mediocre, with far fewer trades. The Monte Carlo was therefore skewed, by spreading the trades of 2009 equally across the Monte Carlo simulations, when that would not happen in real trading.

In another strategy, however, the Monte Carlo again was better than the portfolio selection, suggesting the strategy could do better. But in this case, the trades were distributed pretty evenly over time. A look at the "Trade Returns Sorted - All Versus Portfolio" showed that the portfolio selection was on the low end of what appeared to be available. Likewise, the All Trades view of the Trades Report suggested better portfolio selections could be made on a regular basis over time. By tweaking the Rank Selection, I was able to improve the portfolio selection. But even without that, random chance might have made this strategy do better than expected by simply pulling a more random selection of trades.

Is there a way to run 1000 iterations of the system using "random(1,100)" as the rank? Could you use something like "random(1,@1rand)" with @1rand varying from 1 to 1000? That would give me what I am looking for.

Yes, you can definitely do that. On the Strategy Setup, use the Rank Selection tab. You can then use a variable to do something like: "random(1,@1rand)". Keep in mind that the portfolio selection won't be purely random, depending on the trades available to be entered on each day. Naturally, if there are 100 buy signals on a particular day, the random selection will have a lot to choose from. But if there is just 1 buy signal on a particular day, that trade will be included in every single iteration you run. This creates a bit of a skew, since that one trade will be more heavily weighted in your results than the 100 that might occur on a different day. But this is just one of the challenges of finding that balance between reality and slight variations of it.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Kevin_in_GA » Tue Jan 08, 2013 3:35 pm

Overload wrote:
Is there a way to run 1000 iterations of the system using "random(1,100)" as the rank? Could you use something like "random(1,@1rand)" with @1rand varying from 1 to 1000? That would give me what I am looking for.

Yes, you can definitely do that. On the Strategy Setup, use the Rank Selection tab. You can then use a variable to do something like: "random(1,@1rand)". Keep in mind that the portfolio selection won't be purely random, depending on the trades available to be entered on each day. Naturally, if there are 100 buy signals on a particular day, the random selection will have a lot to choose from. But if there is just 1 buy signal on a particular day, that trade will be included in every single iteration you run. This creates a bit of a skew, since that one trade will be more heavily weighted in your results than the 100 that might occur on a different day. But this is just one of the challenges of finding that balance between reality and slight variations of it.

Pete


But the actual trades over the period would have those "single trade" days as the only option, so that would probably be OK. And the impact of that trade (which you would have to have done since there was no other option) would be normalized against the other options since all would have them as part of the results.

Thanks. While I can't really use this in a WFA it will definitely help me in looking at options going forward.
Kevin_in_GA
 
Posts: 171
Joined: Wed Aug 04, 2010 10:14 am

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby mandelmus » Fri Jan 11, 2013 4:14 pm

Kevin_in_GA wrote:I was thinking that it might be more informative to graphically present the MC analyses as a histogram rather than as is currently presented. This would allow for an immediate visual gauge of the results, particularly the spread. It would also be nice to have vertical lines drawn at 1, 2, and 3 SDs.


+1 vote on that

With regard to that, take a look at what TradeSim offers for Monte Carlo analysis ... http://www.compuvision.com.au/BatchSweep.htm ... read their PDF entitled, "AN8-Monte_Carlo_Sweep_and_Optimization_Procedures.pdf" for plenty of 3-D visuals (image attached).

Monte Carlo Sweep and Optimization.jpg
Monte Carlo Sweep and Optimization
Monte Carlo Sweep and Optimization.jpg (65.11 KiB) Viewed 2419 times



... on the other hand, gummy-stuff's (Peter Ponzo) thoughts on "Monte Carlo predictions" ... http://www.financialwebring.org/gummy-stuff/Monte-predictions.htm
Last edited by mandelmus on Fri Jan 11, 2013 4:27 pm, edited 2 times in total.
mandelmus
 
Posts: 141
Joined: Fri Dec 23, 2011 1:51 pm

Re: Any way to get a Monte Carlo Average for the Sharpe ratio?

Postby Kevin_in_GA » Fri Jan 11, 2013 4:19 pm

Nice. here's what I got from Excel. I looked at two different exit criteria, and used "random(1,@1rand)" for the rank, varying @1rand from 10 to 1010 to get 1000 iterations. For these, the Monte Carlo numbers were much larger, typically 160% to 190% annual return. I think I trust this approach a little more.
Attachments
connors comparison.jpg
connors comparison.jpg (153.13 KiB) Viewed 2415 times
Kevin_in_GA
 
Posts: 171
Joined: Wed Aug 04, 2010 10:14 am


Return to Technical Support

cron