This is the another post of the series: How to build your own algotrading platform. Before running any live algotrading system, it is a good practice to backtest that means run a simulation our algorithms. For Forex data, I am using GainCapital. Their data are in the form of ticks.

pupuzifecose.web.fc2.com | Tag: csv

For a free source it is good enough. I used to use Oanda's historical data service but it seems that they moved it to a premium product. Make sure that you use GainCapital's data only for experimentation. For any other kind of paid historical data ETFs, stocks, options stc , I am using eoddata. Let's download data for a week and experiment a little bit.

The link to the data is http: These are data for one week for one currency pair. You can imagine the amount of data you need to process for all currencies for the last five years hint: But don't worry, we are going optimize this. For now, let's open the file and inspect. As you can understade each line has a timestamp and the how much was the price to buy or sell.

Formats downloaded by other services are pretty similar.

There are many ways to load these data into Python but the most preferable when it comes to data slicing and manipulating is using Pandas.

We can always use the csv library to load data and it might be faster but we need to do some optimizations and processing first that as you will see it is pretty easy with pandas. Another great tool to load TONS of GBs pretty efficiently and very fast is using Bcolz , covered in a much later post or you can read a preview if you have signed up in the newsletter.

Manipulating data using Pandas The data we downloaded are in ticks. Unless we are building an UHFT ultra high frequency trading algorithm, it is much more efficient memory, storage and processing-wise to "group" these ticks into seconds or minutes or hours depending on your strategy. This will make our download scale down from 25MB to just 35KB which translate to HUGE performance and memory benefits.

This is called OHLC Open High Low Close bar for every 15 minutes. Not only you have all the information you need but now it is extremely fast to load it.

You just need to save the data:. We can write a simple momentum algorithm that checks if there was a huge movement the last 15 minutes and if that was the case, let's buy.

We will dive into this in a later post.

You can see the code as always on github. Coming up next, building a backtesting system from scratch! If you have more feedback, ping me at jonromero or signup to the newsletter.

finance - Where can I find open data on historical forex rates for financial reporting purposes? - Open Data Stack Exchange

This is an engineering tutorial on how to build an algotrading platform for experimentation and FUN. Any suggestions here are not financial advices. Enjoy at your own risk. Twitter LinkedIn Github Bitbucket. There are four things that we need to take into consideration when we do our backtesting: The quality of the data How to load them efficiently How to built our backtesting system Try to have our backtesting and our live system share as much code as we can Today, we are going to focus on 1 and 2.

First we need to unzip the file Posted Thu 03 December