Jaded wrote: ... I think that rejecting strategies that are too far outside of the Monte Carlo returns would be beneficial in producing strategies that are more likely to happen going forward ...

Over the years I have increasingly relied more on Monte Carlo returns and less on portfolio returns. In fact, I don't currently use the latter as a filter in my One-Click searches.

In the early days of StrataSearch, we didn't have the option to perform automated searches using statistics from the Detailed Analysis. Instead, the Monte Carlo statistics were available only by manually running a Detailed Analysis for each strategy we wanted to review at that level. In that era, I did use portfolio-based returns as a metric for filtering in high-volume searches ... because that was what was available.

As you noted, though, selecting strategies based upon portfolio returns seems more susceptible to false positives than using Monte Carlo returns. After OneClick was added to StrataSearch, which is when Monte Carlo statistics became accessible in high-volume searches, I eventually decided to avoid the potential pitfalls of relying on portfolio returns by not using them as a filter.

An advantage of your idea (comparing portfolio returns to Monte Carlo returns) is that it will allow portfolio-based statistics to be more reliable as filters. In other words if the portfolio returns are "right", then the other portfolio statistics might be "right", too. I need to think about this some more; nonetheless, it's an interesting idea.