Problem when running multiple instances

StrataSearch has many powerful features, with many techniques and approaches to explore. Here we discuss the things we've done, and the things we'd like to do.

Problem when running multiple instances

Postby rjay » Sat Aug 17, 2013 9:40 am

I have 5 instances of StrataSearch running at a time, all from their own data folders. Five seems to be the limit my CPU can take, although as I have 32gig RAM I have plenty of spare RAM.

The problem is that on a daily basis, at least one of the instances is usually found in a hung state. In every case it seems to have hung when it is refreshing the Multi-System combination, and it always seems to be on the last of my four Alternate Data sets (or after completion perhaps).

This morning however I awoke to find all 5 instances, had hung overnight, at different times (hours apart) and the last one had an error in the log of "Memory Allocation (malloc): returned null".

Just wondered if anyone else had this issue - is 5 instances excessive ? The CPU is certainly not pegged at 100% and I can't see how I have memory issues (though SS is a 32-bit program and I'm on Win7 64bit). Also, I currently have the processor usage set to 8 cores for every instance - should I change it to, say, 2 cores per instance ?
rjay
 
Posts: 116
Joined: Wed Jul 26, 2006 6:51 am

Re: Problem when running multiple instances

Postby Overload » Sat Aug 17, 2013 3:21 pm

Any individual instance can get a memory error if the combination of sector size and evaluation period is too large. For a variety of reasons, StrataSearch does not protect against this. But beyond that, every computer system is unique, so experimentation is probably the best option.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Re: Problem when running multiple instances

Postby rjay » Sun Aug 18, 2013 11:20 am

Almost all of the hung instances do not have the memory allocation error (or indeed any error) in the log. But are you saying that it is still likely to be memory related ? When I view Task Manager I see that each SS instance is using about 1.5gig of RAM so if my available RAM is 32gig I don't see where the memory bottleneck is.
rjay
 
Posts: 116
Joined: Wed Jul 26, 2006 6:51 am

Re: Problem when running multiple instances

Postby Overload » Mon Aug 19, 2013 8:26 am

But are you saying that it is still likely to be memory related ?

No, I did not say it was likely. But it is one possibility among many.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Re: Problem when running multiple instances

Postby Jaded » Mon Nov 11, 2013 12:45 pm

I came across this as well when I was running a OneClick on a single symbol (SPY) from 1995 - 2013. I was using trading rules which evaluated the whole market, but I found a quick solution. It seems that when StrataSearch instances use over 1.5GB of RAM, they are much more prone to a trading rule sending the process over the 32bit memory limit (per process) edge. To resolve this for myself, I lowered the number of threads that SS was allowed to use from 12 threads to 2 threads and memory usage dropped from 1.5GB per process down to about 300-500MB per process and I have not had a malloc error since.

Since I am only evaluating a single symbol right now, more threads doesn't really help, but SS allocated memory for all of them and might even have a leak somewhere in the HadAlert formula ( or coincidence ). If you are running a system against larger sectors, you'll obviously want to use as many threads a possible, so try to eliminate those trading rules that you might notice use a lot of RAM. It's hard to pinpoint them down, but hopefully this helps.
Jaded
 
Posts: 151
Joined: Mon May 14, 2007 6:49 pm


Return to General Discussion

cron