MoneyTec 2.0 - Online Forex Traders & Brokers Social Networking Community » Trading Talk » Career Trader

[dstock]

On the reliability of backtesting...and system development...

I've been on these forums for sometime now. I've learned a ton and thank everyone for their input over the years.

My basic conclusion has been that an automated static model/system won't work in the long run. Only a dynamic model/cta will. This is what makes Renaissance and DE Shaw powerhouses. In firms such as these it is the junior people who trade the models developed while the senior economist/strategist/traders look for big trades...ie the current yen situation,icelandic krona, turkish lira, etc.

I've long questioned the reliability of backtesting in programs. Programs such as metatrader or tradestation are great but I've made systems in both that work great in a backtest but when tested live never produce great results. Another problem is the programs have the gift of hindsight so signals will look great when backtested but when tried live often failed. This is for complete automation. All do provide great signal however.

I believe raw statistics are the best tool available. Of course most base trades off of probability of being right given this ma cross, this rsi level, etc. I think all are useful.

But I would argue that experienced traders enter largely based on price action. I don't think making money in a technical market such as fx is impossible, but would argue most fail due to lack of discipline and patience. I developed all these amazing expert advisors that made 100000000% a year but when tried live, are mediocre at best. We've seen people such as soul trader and others sell systems. Systems are on ebay. People here post demo systems. I have thousands on my system, and I've tried just about all of them out No I haven't paid for them, but we all know that decompiling anything in MT4 is easier than subscripting an 8th polynomial-curve fitting equation into a dynamic matrix. (Think MATLAB)

Most produce abismal results. Certainly they have good signals. And many like ST walk though how to properly use it. Again, I'm speaking for complete automation.

Off my soapbox, I believe one must basically make their own backtesting software for accurate model building. Languages such as Perl,PHP make this very doable, and programs like MATLAB make the capabilities of MT4 look moldy cheesecake.

So I ask, those that have traded fx for 3 years or more, and continue to make money, do think backtesting in MT4 or another program produces accurate results? Recommendations? Theres no better indicator then price action? Thoughts on cta development in MATLAB? PERL,PHP?

If you are sitting in New York or London right now and work for a bank or fund, toss me a PM. I'd be happy to speak privately. We may even grab a pint together.

And I'd be happy to hear how everyone is doing, and basically if its a discretionary system based off of indicators or not. I will not ask for details. PM's are welcome.

Cheers.

[4ncurnce]

I would agree that most if not all systems out there for sale are for sale for a reason and that is that the developer could not make the money they envisioned. that said the system may be good for a sideways or a trending market and they all certainly do have limitations.

Backtesting really is curve fitting in my opinion. I does serve some purpose but when you start fine tuning stops, entries etc. then you are fitting your system to the past. the real test is going forward with money but hypothetical is better in my opinion.

As for managed FX I suggest people look for registered entities like a CTA in the US and I am sure there are similar registrations in the UK, Australia etc. This at least ensures that there has been some auditing of a track record etc.

I think with a system or CTA the bottom line is you need to control the risk but have a mechanism in place to let a good trade run, this is where I have seen the best impact on a trading account. (was fortunate to see it last year in my fx account)

Regards,
Peter S