Importing intraday ASCII Data

Let us know your likes and dislikes, or what you'd like to see next. We listen!

Importing intraday ASCII Data

Postby redbank199 » Sun Nov 26, 2006 2:04 am

As a new trial user, I understand SS is strictly an EOD software and therefore
does not allow importing intraday (ticks and minutes) ASCII data.
However, I think SS can be changed easily to support importing intraday
data by generating fictitious dates for these records.

For example, if the tick file has 500 records of 1 tick transaction and the user wants
to generate 5-tick bars, SS could just convert every 5 tick records
into one "EOD" record and assign a fictitious date to it. Similar approach can
be used to convert minute data files into "EOD" files.

This new feature will expand SS's user base to those people who trade
in real time and need a tool to develop/test systems in offline mode with
large historical data.
redbank199
 
Posts: 29
Joined: Sat Nov 25, 2006 11:41 pm

Postby redbank199 » Sun Nov 26, 2006 10:13 am

One limitation I see with this fictitious dates approach is strategies
would not be able to use different intraday files together, e. g.,
tick files and minute files, or even eod data, since these fictitious
dates would not match. So this is just a quick work around solution
before SS can really support intraday data either in realtime, or via batch
ASCII files.

Do you have any plans to support intraday data ? If not, can this
workaround be implemented in a minor release ?
I would love to have this feature. I am using eSignal right now, but
it is limited to 10 days intraday data which forces me to test my
strategies 10 days at a time after I have saved batches of 10 day data.
redbank199
 
Posts: 29
Joined: Sat Nov 25, 2006 11:41 pm

Postby Overload » Sun Nov 26, 2006 2:11 pm

Thanks for the suggestion. There have actually been a number of people that have suggested this, and in fact tried it. But it's not an approach we support because there are numerous problems that can arise. In particular, any annualized numbers created by evaluating non-EOD bars would have little meaning. This is because the number of bars making up "one year" needs to be different depending on time period of the bar. That is not currently the case. In addition, one of the great benefits of StrataSearch running EOD is that it can perform so much sector-based evaluation. This would not be the case when running such strategies intra-day. In fact, since such numbers couldn't easily be created on a real-time basis, the sector-based trading rules would largely need to be removed for intra-day processing. There are other issues as well, but I won't get into all of them here.

Yes, we absolutely have plans for an intraday version. But it won't be soon, as there is still a fair amount to be done with the EOD version. StrataSearch is on the leading-edge of automated search capabilities. There are no other systems that even come close in this regard. But because we are doing things that have never before been done, there are many issues and ideas to explore, investigate and resolve. And we want to make sure we've investigated and fine-tuned this automated search approach thoroughly before we head down the path of intraday. It'll happen, but only when we've done all that we can with EOD. And there's still a lot more we can do.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Postby redbank199 » Sun Nov 26, 2006 5:48 pm

Pete,

You are certainly right that there are problems with this workaround
approach. But I thought people taking this approach would know that
certain numbers such as annualized returns do not mean anything
here. However, it still beats manual system testing and can benefit
from some of SS's great abilities !

Looks like I will just have to write some Perl scripts myself to convert
the intraday data and load it into SS and see what happens.

Thanks.
redbank199
 
Posts: 29
Joined: Sat Nov 25, 2006 11:41 pm

Postby Overload » Mon Nov 27, 2006 10:12 am

I understand why you wish to get intraday data into StrataSearch, but such partial support using fictitious dates and meaningless APR numbers would not be an impression we would want to leave our users with. For this reason, it's just not something we can implement at this time.

The users that have tried this thus far have done exactly what you describe... converting the data prior to importing. I've never actually tried it. And while it's not something we can provide support for, I'd be curious how it goes. If you decide to try it, please let us know what types of issues you run into.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Postby redbank199 » Mon Nov 27, 2006 7:27 pm

On my way to my "regular" job today, it occurred to me that I actually do not
need to dust off my Perl programming book to do this because Microsoft's
Excel can automatically generate dates via fill data series feature. For people
who do not know how to do that in Excel, just search
"About filling in data based on adjacent cells" in Excel's online Help.
So just import the data into Excel, generate the dates, export it to an
ASCII file and there you have it, no programming is required, although
it should be straightforward to do that in Perl.

I believe using fictitious dates should not break SS since it really should
not care and can not tell if my dates are real or not. However, certain
output information such as annualized returns as you pointed out before
would be meaningless or should be interpreted in a different way.
The other caveat is data with fictitious dates should not be used with
data with real dates.

I definitely will try this and will provde update on this.
redbank199
 
Posts: 29
Joined: Sat Nov 25, 2006 11:41 pm

Postby redbank199 » Tue Dec 12, 2006 9:26 am

As expected, I was able to import some intraday data with fictitious dates
and played with it without breaking SS. So it "works" with the caveat that
some report information is not meaningful.
redbank199
 
Posts: 29
Joined: Sat Nov 25, 2006 11:41 pm

Re: Importing intraday ASCII Data

Postby mandelmus » Fri Feb 17, 2012 5:08 am

Has anyone had good results using this method?

I imagine you wouldn't have to use every intraday tick bar, minute bar, etc; at least with TradeStation you can create a trading session (e.g., 11am to 1pm CST) and download data from only that session. I may try this soon.

Does SS care if trade data is reported for a Saturday or Sunday?
mandelmus
 
Posts: 141
Joined: Fri Dec 23, 2011 1:51 pm

Re: Importing intraday ASCII Data

Postby Overload » Fri Feb 17, 2012 9:57 am

StrataSearch does not have a problem with Saturdays and Sundays as trading days. However, I'm not aware of anyone who has had much success trying to use StrataSearch for intraday searches in this manner.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm

Re: Importing intraday ASCII Data

Postby rjay » Thu Jun 13, 2013 10:44 am

I see a post from Pete in 2006 (in this thread) saying that Intraday support was a possibility one day ... 7 years later are we any closer :?
rjay
 
Posts: 116
Joined: Wed Jul 26, 2006 6:51 am

Re: Importing intraday ASCII Data

Postby Overload » Thu Jun 13, 2013 6:41 pm

Sorry, but there are no announcements on upcoming releases at this time.

Pete
Overload
 
Posts: 2248
Joined: Wed Nov 30, 2005 12:14 pm


Return to Comments and Suggestions