OHLC Bar

I think I have missed one important element in my EPL, the OHLC Bar. That will most probably useful to mimic candle stick chart with some intervals.

• https://code.google.com/p/algo-trader/wiki/AlgoTraderDocumentation#Market_Data
• https://groups.google.com/forum/?fromgroups#!searchin/algo-trader/ohlc
• https://groups.google.com/forum/?fromgroups=#!searchin/algo-trader/ohlc/algo-trader/vZ5UBeLMkYE/enccRWIrJ2IJ
• https://groups.google.com/forum/?fromgroups=#!searchin/algo-trader/ohlc/algo-trader/JwLFM69vdGU/2vSmbNiiFIAJ
• https://groups.google.com/forum/?fromgroups=#!searchin/algo-trader/ohlc/algo-trader/8NvhNQzh9x0/uuz-H6Eq5KsJ

select * from Tick.std:groupwin(security.id).custom:ohlcbar(dateTime, last); 

select
        first(open) as open,
        max(high) as high,
        min(low) as low,
        last(close) as close
from
        Bar.win:length_batch(2);

 select
        first(price) as open,
        max(price) as high,
        min(price) as low,
        last(price) as close
from
        OHLCTick.win:time_batch(1 min)
group by
        ticker;

insert into Bar
select first as open, last as last, min as low, max as high,
* from Tick.std:groupwin(security.id).custom:ohlcbar(dateTime, last);

Smooth RSI

I was able to get RSI values from below EPL.

insert into RSI
select talib("rsi", currentValueDouble, 14) as value
from Tick(security.symbol = "GOOG");

But the series of values were not smooth. I wanna try to get smooth RSI line chart. Perhaps I can take the SMA as input for RSI, like this:

insert into SMA
select talib("sma", currentValueDouble, 5) as value
from Tick(security.symbol = "GOOG");

insert into RSI
select talib("rsi", sma.value, 14) as value
from pattern [every timer:interval(2 minutes) -> sma=SMA];

Or the other way? Making a smooth SMA line chart from RSI values.

insert into RSI
select talib("rsi", currentValueDouble, 14) as value
from Tick(security.symbol = "GOOG");

insert into SMA
select talib("sma", rsi.value, 5) as value
from pattern [every timer:interval(2 minutes) -> rsi=RSI];

Hacking TA-Lib

TA-Lib source code is available from http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-msvc.zip

I opened the com.tictactec.ta.lib.Core class. The RSI method signature looks like this:

   public RetCode rsi( int startIdx,
      int endIdx,
      double inReal[],
      int optInTimePeriod,
      MInteger outBegIdx,
      MInteger outNBElement,
      double outReal[] )

I think I made a mistake in double inReal[]. I was supplying single latest stock quote. Instead, I think I'd better supply an array of current and past stock quote in specific interval.

But when I supplied the array as a parameter, AlgoTrader can't compile the EPL. I asked this in AlgoTrader forum https://groups.google.com/forum/?fromgroups=#!topic/algo-trader/PtaMF04ay6Q

Below is the reply from Andy:

You should only provide the scalar value (tick.currentValueDouble). The GenericTALibFunction will collect all values and provide them as an Array to the talib function. Have a look at the JavaDoc of GenericTALibFunction.

Open Source Trading Platforms List

http://www.ta-lib.org/hdr_dev.html
A few of the products using the open source TA-Lib. If there we charting features in these products, I could probably hack into the code to see how TA-Lib get called.

http://www.ta-lib.org/hdr_lnk.html

http://www.traderslaboratory.com/forums/tools-trade/11086-open-source-trading-platforms-master-list.html

http://algotradingindia.blogspot.in/2012/05/open-source-trading-platforms-list.html

TA-LIB sucks

Well, I started writing my first strategy in Esper EPL. Not so difficult to reach a "Hello World" happy flow.

But too little documentation for TA-LIB http://www.ta-lib.org . No idea how to customize the parameters of MACD, RSI, STOCH etc to represent 2-minutes candle stick chart.

I think I better source other ways to generate Technical Analysis indicators. Let me hack into AIOTrade's http://aiotrade.com/ code to find out if I can do full customization on those Technical Analysis indicators.

Oh ya, that also bring the hints that I am going to use AIOTrade as my front end for AlgoTrader. Cool?

Esper Complex Event Processing

I have skimmed through Esper CEP official documentation quickly http://esper.codehaus.org/esper-4.6.0/doc/reference/en-US/pdf/esper_reference.pdf .

The EPL syntax is quite similar to SQL, with plenty extension. I think I'm ready to use EPL. Now I can understand the existing EPL used in AlgoTrader.

To develop strategies using EPL, I need to know TA-LIB http://ta-lib.org/ . I couldn't find many documentation about TA-LIB. Below are some simple documents:

http://www.ta-lib.org/function.html
http://qtstalker.sourceforge.net/talib.html
http://tadoc.org/
http://www.heatonresearch.com/content/technical-analysis-library-ta-lib-tutorial-java

AlgoTrader, I'm ready

By following the guide http://code.google.com/p/algo-trader/wiki/AlgoTraderQuickStartGuide , I got AlgoTrader binary run successfully. Though in Linux environment, I got to touch up the shell script, which is duplicated from Windows batch script.

I downloaded the source code according to the svn link in http://code.google.com/p/algo-trader/source/checkout . I made use of git-svn, so I can have the history in my local machine. In Eclipse, I installed the E-Git and J-Git plugins.

With the official documentation http://code.google.com/p/algo-trader/wiki/AlgoTraderDocumentation , I got my development environment setup. I also got the AlgoTrader "launchers" run in Eclipse. I spent some time skimmed through all the source code. Beautiful code, impressive!!

When I wanted to collect data (every 5 seconds) through Interactive Broker API, I got "duplicate record" error. Initially I thought Esper statement was propagating datetime with wrong precision. Later after I searched the word "duplicate" in https://groups.google.com/group/algo-trader , I found the thread "Duplicate entry error when persisting tick to DB". Andy Flury's replied "tick.setDateTime(date) in handlePersistTick sets the rounded Time".

Finally, I collected data every 5 seconds by doing the following modifications in com.algoTrader.service.MarketDataServiceImpl class's handlePersistTick(Tick tick) method:

            // get the current Date rounded to MINUTES
            //Date date = DateUtils.round(DateUtil.getCurrentEPTime(), Calendar.MINUTE);
            //tick.setDateTime(date);
            tick.setDateTime(DateUtil.getCurrentEPTime());


What's next? I guessed I got to spend sometime reading Esper Complex Event Processing's documentation at http://esper.codehaus.org/esper/documentation/documentation.html . Esper CEP is the heart of AlgoTrader, the soul of strategies building.

Paying for real-time market data

At Interactive Brokers, I got to pay USD 10 per month for real-time market data (US Securities & Commodities Non-Professional Bundle). Therefore, I must make full use of the money I'm paying for.

I decided to build my own historical database using IB's feeds. According to here http://www.interactivebrokers.com/en/p.php?f=marketData

All customers initially receive 100 concurrent lines of real-time market data (which can be displayed in TWS or via the API) and always have a minimum of 100 lines of data.

I think I can make use of at least 80 concurrent lines of real-time market data to build my historical database via the API. Whereas I can leave the remaining 20 concurrent lines of real-time market data for TWS.

Get ready for Algo Trader

I got my Interactive Broker new account approved. Yet pending for funding.

I'm about to start exploring algo-trader. I have read through the official wiki pages from https://code.google.com/p/algo-trader/w/list . These documentations gave me good foundation to play with the code.

Right before I start to run my first algo-trader strategy, I got further hints from a first time user in AlgoTrader google groups. https://groups.google.com/forum/?fromgroups#!topic/algo-trader/DLNCbxSNznw

... A very minor issue, but MarketDataStarter.launch tries to use the database AlgoTraderLight, but the included sql file, algotrader.sql, sets up the database with the name AlgoTrader.

I then ran into a slightly more frustrating issue that caused RMI to hang for a minute or so and then give an error about an ObjID already in use. It turned out the problem was that my hostname wasn't set in /etc/hosts, which screwed up RMI. Maybe it would be worth mentioning in the Wiki that Linux users should try pinging their hostname and make sure that it's set to a local IP address.

I was then able to able to run the Periodic strategy and place trades on IB. However, I noticed that transactions and position updates were not being recorded in the database. I needed to add "ib-trades" to the MODULES field in the BASE record of the strategy table. Perhaps it should be set by default or mentioned in the Wiki....