| |
| Subject: | Evaluating a Trading System |
| Posted by: | Henry Carstens ( C2 Score: 144 ) |
 |
| |
|
| When: | 1/31/09 (10:56) | |
| Systems: | |
|
|
|
|
| | Evaluating a Trading System
What before how: The goal of evaluating a trading system is to forecast its future performance.
With that in mind, here are three simple criteria for quickly evaluating a trading system. A system which passes all of these criteria is more likely to be successful over the long term than one that does not:
1. Review the backtest history and make sure it spans several market regimes and was consistently profitable across them all. With the amount of market data available today, there is no excuse for not having this information.
2. Calculate the t score of the system to see how statistically significant the results are. Generally, try to get 100 observations and a t score greater than 1.6. Higher is better. Over 3 is very good. Be wary of results less than 2 and walk away from anything less than 1.6. Make sure a reasonable slippage and commission have been removed, especially from high frequency systems.
t = sqrt(number of trades) * avg trade / std deviation of avg trade
3. Calculate the optimal f$ value. Optimal f$ is important because it tells you the maximum leverage (in contracts per $) the system can be traded w/o greatly increasing the odds of risk of ruin.
f = (((1 + win loss ratio) * prob winning) - 1) / win loss ratio
f$ = largest losing trade / f
[Note: Max drawdown can be substituted for largest losing trade as every traders ability to withstand drawdowns is suspect.]
Having passed the above tests, control charts and monte carlo simulation can be used to further investigate a system.
--h
A System Forecasting Tool is available here:
< http://www.verticalsolutions.com/tools/system_ranking.html>;
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Kevin Davey ( C2 Score: 1000) |
|
| |
|
| When: | 1/31/09 (11:31) | |
| Systems: | 3 Pair Forex, Congress ES, ETF Basket, Gen 2 - Mini S&P Only, Gen 3 - Mini S&P Only, KJ Mini Action, KJ Mini SP, KJ Price Action, Relative ETF, SFE, Trender, Gen 1 - Mini S&P Only (Closed), KJ Test 1 (closed), Master Mini S&P, Mighty Mini S&P, Mini N&R, No Trades Since Jan 08, Notable Nasdaq Mini |
|
|
|
|
| | In response to post by Henry Carstens of 1/31/09 (10:56) Evaluating a Trading System
What before how: The goal of evaluating a trading system is to forecast its future performance....
See entire
Good points, with a couple of points:
1. The only backtest that is worth anything is a walk forward backtest. Most backtests I've seen presented here or anywhere are optimized backtests, generated with the benefit of hindsight. Those are pretty much useless. If it looks too good to be true, it probably is.
2. If you try trading a system with optimal f, you are almost guaranteed to lose eventually, since your largest drawdown is almost always yet to come. Money management should be trading not to lose everything, without worrying about potential gains. This implies a much lower leverage the optimal f. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Username ( C2 Score: 232) |
|
| |
|
| When: | 1/31/09 (12:27) | |
| Systems: | (26227726), (52547431), (22698817), (22808053), (26281366), (26352273), (31603478), (45779563), (46518146), (47450233), (54347539), (64359398) |
|
|
|
|
| | In response to post by Kevin Davey of 1/31/09 (11:31) Good points, with a couple of points:
1. The only backtest that is worth anything is a walk forward backtest. Most backtests I've seen presented here or anywhere are optimized backtests, generated with the benefit of hindsight. Those are pretty much useless. If it looks too good to be true, it probably is....
See entire
I agree with your comments, backtests are fairly irrelevant in my opinion, walk forward is what counts, I'm staggered by how many people ask for backtest results as if that is going to prove the system works, you can pretty much make a backtest show whatever someone wants to see.
Re T score I would have no problem with that being added to the stats on the front system page.
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Rick Haines ( C2 Score: 988) |
|
| |
|
| When: | 1/31/09 (12:31) | |
| Systems: | RPH, RPH2, t2107 atm xlt5, Conservative Growth, Diversified ETF Timer, Extreme Trader, rydex 500, rydex qqqq, self directed 401 timer, SMA, Target 50 stopped trading 8-28-07, Target 60, Ultimate Trader, UT2, Wave Rider stopped trading on 4/25/08 |
|
|
|
|
| | In response to post by Henry Carstens of 1/31/09 (10:56) Evaluating a Trading System
What before how: The goal of evaluating a trading system is to forecast its future performance....
See entire
Henry this post is a great help to everybody.
Thanks
Rick Haines |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Index ( C2 Score: 976) |
|
| |
|
| When: | 1/31/09 (13:46) | |
| Systems: | |
|
|
|
|
| | In response to post by Henry Carstens of 1/31/09 (10:56) Evaluating a Trading System
What before how: The goal of evaluating a trading system is to forecast its future performance....
See entire
The fact you are only using this to redirect people to your site pretty much dashes your credibility/objectivity. As your site says
"This strategy is available for licensing by funds and money managers. Contact Henry Carstens for more information. "
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Karl A |
|
| |
|
| When: | 1/31/09 (17:40) | |
|
|
|
|
| | In response to post by Index of 1/31/09 (13:46)
The fact you are only using this to redirect people to your site pretty much dashes your credibility/objectivity. As your site says
...
See entire
The fact you are only using this to redirect people to your site pretty much dashes your credibility/objectivity.
I don't have any problems for a vendor to point to his related web site. However, for people who have not visited his Rembrandt system on C2 beforehand it was not clear that this is his own site. So, a better disclosure would have been in order. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 1/31/09 (19:44) | |
| Systems: | |
|
|
|
|
| | In response to post by Henry Carstens of 1/31/09 (10:56) Evaluating a Trading System
What before how: The goal of evaluating a trading system is to forecast its future performance....
See entire
I'd just laugh at a 1 week old vendor coming out and saying this stuff... just to get them to his site, which probably houses the backtests that he does not believe in.
Buyer beware. A backtest is useful as long as it is more than 2 years old with more than 30 trades. Normal population, and sufficient data over different market periods. Ideally 5 years or more. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 1/31/09 (19:46) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 1/31/09 (19:44) I'd just laugh at a 1 week old vendor coming out and saying this stuff... just to get them to his site, which probably houses the backtests that he does not believe in....
See entire
PTQQS has not changed since August of 2007. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Index ( C2 Score: 976) |
|
| |
|
| When: | 1/31/09 (20:56) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 1/31/09 (19:44) I'd just laugh at a 1 week old vendor coming out and saying this stuff... just to get them to his site, which probably houses the backtests that he does not believe in....
See entire
well, the purpose of C2 is running it LIVE under the lights, where one cannot say "gee, I meant to take that trade, but..." or many other excuses used by people on their private website.
Backtests mean zip. Live tracked results with metrics mean everything.
I thought it odd when a stranger walks in, telling everyone "use this smell-o-meter to see whether a system is any good..." and not bothering to tell people that "by the way, the authoritative site I am pointing to is mine..."
Not a particularly good start.... |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Kevin Davey ( C2 Score: 1000) |
|
| |
|
| When: | 2/01/09 (10:14) | |
| Systems: | 3 Pair Forex, Congress ES, ETF Basket, Gen 2 - Mini S&P Only, Gen 3 - Mini S&P Only, KJ Mini Action, KJ Mini SP, KJ Price Action, Relative ETF, SFE, Trender, Gen 1 - Mini S&P Only (Closed), KJ Test 1 (closed), Master Mini S&P, Mighty Mini S&P, Mini N&R, No Trades Since Jan 08, Notable Nasdaq Mini |
|
|
|
|
| | In response to post by Financial Scientist of 1/31/09 (19:44) I'd just laugh at a 1 week old vendor coming out and saying this stuff... just to get them to his site, which probably houses the backtests that he does not believe in....
See entire
In my opinion, the only way conventional backtests (ie, non-walk forward backtests) can help you is if they make you say "wow" after looking at them. In that case, they are probably curve fitted or otherwise optimized.
Good looking backtests are one of the easiest things in the world to create. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Bob Svan ( C2 Score: 968) |
|
| |
|
| When: | 2/01/09 (10:41) | |
| Systems: | |
|
|
|
|
| | In response to post by Henry Carstens of 1/31/09 (10:56) Evaluating a Trading System
What before how: The goal of evaluating a trading system is to forecast its future performance....
See entire
Brilliant, professional work.
Thank you, Henry.
I wish your knowledge be a part of Collective2.
---------------------------------
To: Index, Beau, Kevin:
Are we not on Collective2, guys?
Grab C2 forward-walking historical data, change the word "backtest" in Henry's text to "C2 data" and start to play with C2 systems.
Good luck to all,
Bob
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Kevin Davey ( C2 Score: 1000) |
|
| |
|
| When: | 2/01/09 (11:07) | |
| Systems: | 3 Pair Forex, Congress ES, ETF Basket, Gen 2 - Mini S&P Only, Gen 3 - Mini S&P Only, KJ Mini Action, KJ Mini SP, KJ Price Action, Relative ETF, SFE, Trender, Gen 1 - Mini S&P Only (Closed), KJ Test 1 (closed), Master Mini S&P, Mighty Mini S&P, Mini N&R, No Trades Since Jan 08, Notable Nasdaq Mini |
|
|
|
|
| | In response to post by Bob Svan of 2/01/09 (10:41) Brilliant, professional work.
Thank you, Henry.
I wish your knowledge be a part of Collective2.
---------------------------------
...
See entire
Henry's original post extolled the virtues of backtesting. I am merely pointing out shortcomings of backtesting, which is why C2 is so important for weeding out the good from the bad.
Your advice for using C2 data is definitely right on. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Karl A |
|
| |
|
| When: | 2/01/09 (13:55) | |
|
|
|
|
| | In response to post by Bob Svan of 2/01/09 (10:41) Brilliant, professional work.
Thank you, Henry.
I wish your knowledge be a part of Collective2.
---------------------------------
...
See entire
Thank you Bob Svan for bringing some clarity to this discussion.
And this comes from a guy who has an excellent track record of his systems as opposed to most of the other guys participating in this discussion. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Kevin Davey ( C2 Score: 1000) |
|
| |
|
| When: | 2/01/09 (14:14) | |
| Systems: | 3 Pair Forex, Congress ES, ETF Basket, Gen 2 - Mini S&P Only, Gen 3 - Mini S&P Only, KJ Mini Action, KJ Mini SP, KJ Price Action, Relative ETF, SFE, Trender, Gen 1 - Mini S&P Only (Closed), KJ Test 1 (closed), Master Mini S&P, Mighty Mini S&P, Mini N&R, No Trades Since Jan 08, Notable Nasdaq Mini |
|
|
|
|
| | In response to post by Karl A of 2/01/09 (13:55) Thank you Bob Svan for bringing some clarity to this discussion.
And this comes from a guy who has an excellent...
See entire
True. Don't listen to anything I say. I have a terrible track record:
C2 Systems:
5 of 8 are profitable (3 with >40% annual return)
2 of 8 are losers (both <5% annual loss)
1 of 8 is big loser
Real Time, Real Money Futures Contest (google "world cup futures challenge"):
2005 World Cup of Futures 148% return (second place)
2006 World Cup of Futures 107% return (first place)
2007 World Cup of Futures 112% return (second place)
Please check your facts before throwing stones at me. If what I have to say is so meaningless to you, just put me on "ignore" - you'd be the first...
Kevin |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Kevin Davey ( C2 Score: 1000) |
|
| |
|
| When: | 2/01/09 (14:55) | |
| Systems: | 3 Pair Forex, Congress ES, ETF Basket, Gen 2 - Mini S&P Only, Gen 3 - Mini S&P Only, KJ Mini Action, KJ Mini SP, KJ Price Action, Relative ETF, SFE, Trender, Gen 1 - Mini S&P Only (Closed), KJ Test 1 (closed), Master Mini S&P, Mighty Mini S&P, Mini N&R, No Trades Since Jan 08, Notable Nasdaq Mini |
|
|
|
|
| | In response to post by Kevin Davey of 2/01/09 (14:14) True. Don't listen to anything I say. I have a terrible track record:
C2 Systems:
5 of 8 are profitable (3 with >40% annual return)
...
See entire
The point is, please don't dismiss anyone's view based on a good, bad or non-existent track record. I judge comments on the quality of the post.
Some of the best advice I've gotten was from author/trader/hedge fund manager Victor Neiderhoffer, and we all know he crashed and burned twice. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/01/09 (16:21) | |
| Systems: | |
|
|
|
|
| | In response to post by Karl A of 2/01/09 (13:55) Thank you Bob Svan for bringing some clarity to this discussion.
And this comes from a guy who has an excellent...
See entire
Of course, PTQQS can't be outperforming by 8400 basis points since March 20th, 07..... |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Karl A |
|
| |
|
| When: | 2/01/09 (17:55) | |
|
|
|
|
| | In response to post by Kevin Davey of 2/01/09 (14:14) True. Don't listen to anything I say. I have a terrible track record:
C2 Systems:
5 of 8 are profitable (3 with >40% annual return)
...
See entire
Kevin, why are you so offended? I did not single you out, remember I said "most". In fact I have two of your systems on my Analyst page, they show promise, they just don't have a long enough track record.
Now another question and I hope you answer it here on this forum: Do you consider Beau Wolinsky's present system a good system? If one had started trading it at the height of the equity curve in June 2008, that is about one and a quarter year after incepetion - and is not this about the time a conservative person would wait before jumping in - and had started with the recommende capital of $100,000, that person would be down now about 50% and he would need a 100% return just to break even, a return the system has never produced so far. If I had to choose between his system and a new system which has a credible back testing history over many years with a maximum drawdown of 9% I think the choice would be easy. Am I missing something here?
I would be interested in your answer because you seem to be an experienced trader. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Kevin Davey ( C2 Score: 1000) |
|
| |
|
| When: | 2/01/09 (19:28) | |
| Systems: | 3 Pair Forex, Congress ES, ETF Basket, Gen 2 - Mini S&P Only, Gen 3 - Mini S&P Only, KJ Mini Action, KJ Mini SP, KJ Price Action, Relative ETF, SFE, Trender, Gen 1 - Mini S&P Only (Closed), KJ Test 1 (closed), Master Mini S&P, Mighty Mini S&P, Mini N&R, No Trades Since Jan 08, Notable Nasdaq Mini |
|
|
|
|
| | In response to post by Karl A of 2/01/09 (17:55) Kevin, why are you so offended? I did not single you out, remember I said "most". In fact I have two of your systems on my Analyst page, they show promise, they just don't have a long enough track record....
See entire
I get offended when, in the midst of a meaningful discussion (which are not all that plentiful on C2, by the way), someone decides to effectively end it by saying "ignore most of the posters, because they have what I think are poor systems." I get even more offended when I happen to be the most frequent poster in the thread where this ambush occurs.
As far as evaluation of any particular system goes, I would have to look at a system as part of the portfolio of systems I trade, and ask myself "would this fit in with my other systems?" A system with 20% annual return and 30% max DD could fit my objectives (especially if it reduced overall portfolio risk and drawdown), and then again it may not. For example, would I ever trade a coin flip system with a negative expectancy? I sure would, if it was traded in the right kind of portfolio. (So, you're probably saying "thanks for the non-answer," but sometimes the answer is more complicated than it appears).
So, you may ask, how can I say this and still "review" systems on My Analyst page? Simple, the systems I criticize on that page (which are not all the systems I review there) should not be traded by anyone, anywhere, anytime. I am trying to warn people about the systems that may look good at first glance, but are really just ticking time bombs. Since right now most of the top voted comments are mine, I think I am successful in this.
Finally, if I had to choose between a system with history on C2 and one that had no C2 history but had a backtest, I'd choose the C2 system, since at least I knew what I was getting, and could make an informed decision on whether the system met my objectives. Relying on traditional backtests is not for experienced traders. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Username ( C2 Score: 232) |
|
| |
|
| When: | 2/01/09 (23:13) | |
| Systems: | (26227726), (52547431), (22698817), (22808053), (26281366), (26352273), (31603478), (45779563), (46518146), (47450233), (54347539), (64359398) |
|
|
|
|
| | In response to post by Karl A of 2/01/09 (13:55) Thank you Bob Svan for bringing some clarity to this discussion.
And this comes from a guy who has an excellent...
See entire
Right, because people who are successful in life never have any failure first do they? So you should always only take advice from people without any failed C2 systems to their name. Good luck.
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Kevin Davey ( C2 Score: 1000) |
|
| |
|
| When: | 2/01/09 (23:20) | |
| Systems: | 3 Pair Forex, Congress ES, ETF Basket, Gen 2 - Mini S&P Only, Gen 3 - Mini S&P Only, KJ Mini Action, KJ Mini SP, KJ Price Action, Relative ETF, SFE, Trender, Gen 1 - Mini S&P Only (Closed), KJ Test 1 (closed), Master Mini S&P, Mighty Mini S&P, Mini N&R, No Trades Since Jan 08, Notable Nasdaq Mini |
|
|
|
|
| | In response to post by Username of 2/01/09 (23:13) Right, because people who are successful in life never have any failure first do they? So you should always only take advice from people without any failed C2 systems to their name. Good luck.
Well said, much better than my attempt! |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Trader Trade ( C2 Score: 926) |
|
| |
|
| When: | 2/01/09 (23:29) | |
| Systems: | |
|
|
|
|
| | In response to post by Karl A of 2/01/09 (13:55) Thank you Bob Svan for bringing some clarity to this discussion.
And this comes from a guy who has an excellent...
See entire
Here, how about I use my other login, now I haven't got any failed systems to my name so my advice has suddenly become much more worthy. Wow I feel like a better trader using this login. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | James Nelson ( C2 Score: 982) |
|
| |
|
| When: | 2/02/09 (9:40) | |
| Systems: | |
|
|
|
|
| | In response to post by Kevin Davey of 2/01/09 (19:28) I get offended when, in the midst of a meaningful discussion (which are not all that plentiful on C2, by the way), someone decides to effectively end it by saying "ignore most of the posters, because they have what I think are poor systems." I get even more offended when I happen to be the most frequent poster in the thread where this ambush occurs....
See entire
I'm a bit late to the thread, but I have to side with Kevin. His comments related to over-leverage and adding to losers are solid principles. I never really understood how investing/trading could be equaled to gambling until I read through Kevin's Analyst comments and related systems. Keep up the good work! |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Index ( C2 Score: 976) |
|
| |
|
| When: | 2/02/09 (12:00) | |
| Systems: | |
|
|
|
|
| | In response to post by Trader Trade of 2/01/09 (23:29) Here, how about I use my other login, now I haven't got any failed systems to my name so my advice has suddenly become much more worthy. Wow I feel like a better trader using this login.
I fail to see how Bob Svan's "Brilliant professional work" comment carries much weight, regardless of his momentary C2 performance. Since that vendor has no reliable C2 track record yet, then such claims remain unproven. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Schmitty # |
|
| |
|
| When: | 2/02/09 (21:48) | |
|
|
|
|
| | In response to post by Financial Scientist of 1/31/09 (19:46) PTQQS has not changed since August of 2007.
And has been losing money since April of 2008. Steadily. Down 30% since then.
Sorry, couldn't resist.
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Steve Auger ( C2 Score: 979) |
|
| |
|
| When: | 2/04/09 (16:13) | |
| Systems: | |
|
|
|
|
| | In response to post by Schmitty # of 2/02/09 (21:48) And has been losing money since April of 2008. Steadily. Down 30% since then.
Sorry, couldn't resist.
C2 does provide a valuable service in that walk forward verified stats are generated. However, the problem of optimization remains even with walk forward. If a system performs poorly going forward then it is discarded, either by the supplier or his potential customers. Then another system is started and same thing.
One cannot judge systems based purely on track record alone, no matter how good it appears to be. One needs to get inside the system provider's head (if that is possible) and understand his money management, risk management principles and you have to get to know him/her as well as possible over the internet to develop a trust in their abilities.
And now I redirect you to my website.
http://www.stockmarketstudent.com
Steve |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/05/09 (21:49) | |
| Systems: | |
|
|
|
|
| | In response to post by Karl A of 2/01/09 (17:55) Kevin, why are you so offended? I did not single you out, remember I said "most". In fact I have two of your systems on my Analyst page, they show promise, they just don't have a long enough track record....
See entire
of $100,000, that person would be down now about 50% and he would need a 100% return just to break even
Steve you're an idiot. DD is 33%. I did not go back to 100. It just shows you have no idea what you are talking about. I'm also up 44% from inception, and I was only down 10% in 2008.
Please refrain from making more idiotic comments that show you have no idea what you are talking about.
It's obvious to me why you are ignored for such idiocy. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/05/09 (21:52) | |
| Systems: | |
|
|
|
|
| | In response to post by James Nelson of 2/02/09 (9:40) I'm a bit late to the thread, but I have to side with Kevin. His comments related to over-leverage and adding to losers...
See entire
Anyone could and should read my analyst page for more in depth details about this topic of overleveraging and curve fitting. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/05/09 (21:54) | |
| Systems: | |
|
|
|
|
| | In response to post by Schmitty # of 2/02/09 (21:48) And has been losing money since April of 2008. Steadily. Down 30% since then.
Sorry, couldn't resist.
I'm not denying that. But for a moron like Steve to say you lost half is a misrepresentation, but I'll dismiss it as his calculation error, and a sign that he would like to have a system just like that. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/05/09 (22:01) | |
| Systems: | |
|
|
|
|
| | In response to post by Karl A of 2/01/09 (17:55) Kevin, why are you so offended? I did not single you out, remember I said "most". In fact I have two of your systems on my Analyst page, they show promise, they just don't have a long enough track record....
See entire
Sorry, strikeout Steve, and replace those comments with Karl.
Do some math before you claim I lost 50%, because to start from 100 to get to 200 is a 100% return, and back to 130 up to 144 is only a 33% drawdown, compared to the S&P's drawdown of 60%. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/05/09 (22:02) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 2/05/09 (22:01) Sorry, strikeout Steve, and replace those comments with Karl.
Do some math before you claim I lost 50%, because...
See entire
Oh, I should add, there's a glitch in the c2 rating's system. When I cross above 150 I'll be back into the 900 range. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/05/09 (22:16) | |
| Systems: | |
|
|
|
|
| | In response to post by Kevin Davey of 2/01/09 (10:14) In my opinion, the only way conventional backtests (ie, non-walk forward backtests) can help you is if they make you say "wow" after looking at them. In that case, they are probably curve fitted or otherwise optimized....
See entire
I'm not so sure, Kevin. I have 2 of the top 3 most profitable market order scripts, 40% of the top 10 most profitable limit order scripts ever written on WL.
These are all at least one year old, and when I released them I did enough testing and research to see they would rise to the top. Give it 1 or 2 more quarters and these might be the top 2 and top 4 respectively. Not sure if I'll ever beat the top limit order script, but it's exposure means it doesn't take trades very often http://wl4.wealth-lab.com/cgi-bin/WealthLab.DLL/getpage?page=Top25APR.htm
It's merely a warm up. But goes back to the question of why I published in the first place. I wanted credibility moreso than these mediocre trading systems.
I'm starting to look toward the APR/DD ratio now. I think that's a filter some people have used, but it needs to be in combination with risk measures like the Sharpe APD and profit factor. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Aspire Trader ( C2 Score: 975) |
|
| |
|
| When: | 2/06/09 (10:03) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 2/05/09 (22:16) I'm not so sure, Kevin. I have 2 of the top 3 most profitable market order scripts, 40% of the top 10 most profitable limit order scripts ever written on WL....
See entire
idiot...
moron...
He's back!
And with his full range of adolescent attack words ready to fire!
Watch out or you could be next as the C2 school yard bully takes aim.
Stay tuned, as I'm sure there is more to come as he's likely just getting warmed up...
:-) |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | C V ( C2 Score: 65) |
|
| |
|
| When: | 2/06/09 (10:27) | |
| Systems: | |
|
|
|
|
| | In response to post by Aspire Trader of 2/06/09 (10:04)
idiot...
moron...
He's back!
And with his full range of adolescent attack words ready to fire!
...
See entire
And amusingly enough those personal attacks were fired at wrong individual ... and belatedly realized.
Collateral damage !! |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/06/09 (12:51) | |
| Systems: | |
|
|
|
|
| | In response to post by Aspire Trader of 2/06/09 (10:03)
idiot...
moron...
He's back!
And with his full range of adolescent attack words ready to fire!
...
See entire
I don't like misinformed people making false statements about my systems. Idiot and moron are very apt descriptions for the difference between losing half and losing a third. Also, he missed that going from 100 to 200 is a 100% return, that I fully anticipate recovering in the future. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/06/09 (12:53) | |
| Systems: | |
|
|
|
|
| | In response to post by Schmitty # of 2/02/09 (21:48) And has been losing money since April of 2008. Steadily. Down 30% since then.
Sorry, couldn't resist.
losing money since April of 2008 as the market lost close to 55% of its value since that time. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | C V ( C2 Score: 65) |
|
| |
|
| When: | 2/06/09 (13:22) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 2/06/09 (12:51) I don't like misinformed people making false statements about my systems. Idiot and moron are very apt descriptions for...
See entire
Agreed.
But you were equally ignorant and misinformed to call Steve idiot and moron for something he did not say.
So, based on your aforementioned rules, Steve has every right to call you a moron and idiot ?
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Aspire Trader ( C2 Score: 975) |
|
| |
|
| When: | 2/06/09 (15:32) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 2/06/09 (12:51) I don't like misinformed people making false statements about my systems. Idiot and moron are very apt descriptions for...
See entire
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Karl A |
|
| |
|
| When: | 2/06/09 (16:06) | |
|
|
|
|
| | In response to post by Financial Scientist of 2/05/09 (22:01) Sorry, strikeout Steve, and replace those comments with Karl.
Do some math before you claim I lost 50%, because...
See entire
My mistake. I wrongly assumed, by not looking carefully at your system, that the position size remaines the same for each trade so that a new subscriber would start with $100,000 no matter a what point in time he would subscribe since I was just analyzing such a system, and BTW such a system in my opinion would be easier to evaluate by a new subscriber. So congratulation, you only had a drawdown of 33% since last April. BTW, you never reached $200,000, just look at the equity curve.
The point is, while other people made valuable contribution in this thread, you only wanted to promote your system. In any event, the language you use is inexcuseable in this forum. This is probably the reason why 30% of the people here have you on the ignore list. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Username ( C2 Score: 232) |
|
| |
|
| When: | 2/06/09 (16:21) | |
| Systems: | (26227726), (52547431), (22698817), (22808053), (26281366), (26352273), (31603478), (45779563), (46518146), (47450233), (54347539), (64359398) |
|
|
|
|
| | In response to post by Financial Scientist of 2/06/09 (12:53) losing money since April of 2008 as the market lost close to 55% of its value since that time.
I always thought you were going for absolute returns. Were your objectives just to outperform the market like a long-only mutual fund manager? |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Karl A |
|
| |
|
| When: | 2/06/09 (16:35) | |
|
|
|
|
| | In response to post by Karl A of 2/06/09 (16:06) My mistake. I wrongly assumed, by not looking carefully at your system, that the position size remaines the same for eac...
See entire
To clarify, I assumed that position sizing would be based on each month starting at $100,000. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/06/09 (17:57) | |
| Systems: | |
|
|
|
|
| | In response to post by Username of 2/06/09 (16:21) I always thought you were going for absolute returns. Were your objectives just to outperform the market like a long-only mutual fund manager?
Absolute, I'd like. For now, relative performance is still good. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | User Removed ( C2 Score: 217) |
|
| |
|
| When: | 2/08/09 (0:27) | |
| Systems: | |
|
|
|
|
| | In response to post by Henry Carstens of 1/31/09 (10:56) Evaluating a Trading System
What before how: The goal of evaluating a trading system is to forecast its future performance....
See entire
"t = sqrt(number of trades) * avg trade / std deviation of avg trade"
This is known as a system quality number, popularized by Van Tharp in his recent book "Definitive Guide to Position Sizing". He recommends using sqrt(100) instead of sqrt(number of trades) when the number of trades exceeds 100. This is to compensate for the overly optimistic scores for the systems with large number of trades.
For example, consider systems A and B. Both have the same "avg trade / std deviation of avg trade" ratio. System A had 100 trades, while system B had 10000 trades. The score calculated with the original formula would imply that system B is 10 times better than system A, which seems like a gross miscalculation.
The adjustment by using sqrt(100) is not ideal, either. It punishes the systems with a longer track record.
I am sure the correct (in a statistical sense) multiplier exists, but I don't have enough academic background to derive it rigorously. My guess is that it would be around sqrt(sqrt(number of trades)). That is, the final score would be:
t = sqrt(sqrt(number of trades)) * avg trade / std deviation of avg trade
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Henry Carstens ( C2 Score: 144) |
|
| |
|
| When: | 2/08/09 (15:22) | |
| Systems: | |
|
|
|
|
| | In response to post by User Removed of 2/08/09 (0:27) "t = sqrt(number of trades) * avg trade / std deviation of avg trade"
This is known as a system quality number, popularized by Van Tharp in his recent book "Definitive Guide to Position Sizing". He recommends using sqrt(100) instead of sqrt(number of trades) when the number of trades exceeds 100. This is to compensate for the overly optimistic scores for the systems with large number of trades....
See entire
"t = sqrt(number of trades) * avg trade / std deviation of avg trade"
Another approach would be to combine the t score with optimal f and some additional metrics to get multiple views into a system's past behavior. A single metric is not likely to be enough to generate a reliable forecast of the future behavior of the system.
--h
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Bob Svan ( C2 Score: 968) |
|
| |
|
| When: | 2/09/09 (4:29) | |
| Systems: | |
|
|
|
|
| | In response to post by User Removed of 2/08/09 (0:27) "t = sqrt(number of trades) * avg trade / std deviation of avg trade"
This is known as a system quality number, popularized by Van Tharp in his recent book "Definitive Guide to Position Sizing". He recommends using sqrt(100) instead of sqrt(number of trades) when the number of trades exceeds 100. This is to compensate for the overly optimistic scores for the systems with large number of trades....
See entire
If there are two sets of systems (less than 100 and more than 100 trades), I suggest to compare each set separately. Then you have only one formula for each set.
( For me it is simpler, because I do not believe anything with less than 100 trades :-) )
Is there some recommendation for minimum of "number of trades"? |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/09/09 (14:32) | |
| Systems: | |
|
|
|
|
| | In response to post by Bob Svan of 2/09/09 (4:29) If there are two sets of systems (less than 100 and more than 100 trades), I suggest to compare each set separately. Then you have only one formula for each set....
See entire
30 trades, and longer than a 2 year backtest or record. 30 is all that is required in a normal system. And, I might add, each day of the system is actually a datapoint. That is, a 2 year system has about 500 bits of data, and even if there's only 30 trades, as long as there's 30 or more, you have a normal population if you want to just go by trades. Really, each day is either buy sell hold or do nothing. So time is a big factor.
I wouldn't trust a 1000 trade system with only 30 days of data, I'd be the opposite. 1000 days of data with 30 trades is enough, assuming a win percentage at 70% and above. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | C V ( C2 Score: 65) |
|
| |
|
| When: | 2/09/09 (14:55) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 2/09/09 (14:32) 30 trades, and longer than a 2 year backtest or record. 30 is all that is required in a normal system. And, I might add, each day of the system is actually a datapoint. That is, a 2 year system has about 500 bits of data, and even if there's only 30 trades, as long as there's 30 or more, you have a normal population if you want to just go by trades. Really, each day is either buy sell hold or do nothing. So time is a big factor....
See entire
>> And, I might add, each day of the system is actually a datapoint.
An interesting way to look at it, Beau.
I wonder if this concept can be extended further - each day is considered a trade. After all, at the end of day, the system chooses not to exit open positions, thus implying that it expects to perform better on the next day.
A positive change in equity is considered a winning "trade", and vice-versa. If system statistics are computed based on this method, I wonder how they would compare against those that are computed on trade-by-trade basis.
|
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Kevin Davey ( C2 Score: 1000) |
|
| |
|
| When: | 2/09/09 (15:30) | |
| Systems: | 3 Pair Forex, Congress ES, ETF Basket, Gen 2 - Mini S&P Only, Gen 3 - Mini S&P Only, KJ Mini Action, KJ Mini SP, KJ Price Action, Relative ETF, SFE, Trender, Gen 1 - Mini S&P Only (Closed), KJ Test 1 (closed), Master Mini S&P, Mighty Mini S&P, Mini N&R, No Trades Since Jan 08, Notable Nasdaq Mini |
|
|
|
|
| | In response to post by Financial Scientist of 2/09/09 (14:32) 30 trades, and longer than a 2 year backtest or record. 30 is all that is required in a normal system. And, I might add, each day of the system is actually a datapoint. That is, a 2 year system has about 500 bits of data, and even if there's only 30 trades, as long as there's 30 or more, you have a normal population if you want to just go by trades. Really, each day is either buy sell hold or do nothing. So time is a big factor....
See entire
When I develop strategies, I personally aim for 5-10 years of walkforward history, with 100+ trades. Not always possible, though... |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/11/09 (18:06) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 2/01/09 (16:21) Of course, PTQQS can't be outperforming by 8400 basis points since March 20th, 07.....
Add about 1400 basis points now to about 9,891 basis points of outperformance since 3/20/2007 on the spx. |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/11/09 (18:17) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 2/05/09 (22:02) Oh, I should add, there's a glitch in the c2 rating's system. When I cross above 150 I'll be back into the 900 range.
Oooh, yeah, what was that? Go from 450 back to 963 which is really how I should be ranked? What now? |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Pegasi Capital ( C2 Score: 977) |
|
| |
|
| When: | 2/13/09 (2:41) | |
| Systems: | |
|
|
|
|
| | In response to post by Financial Scientist of 2/11/09 (18:17) Oooh, yeah, what was that? Go from 450 back to 963 which is really how I should be ranked? What now?
Perhaps a formula which tracks the psychological makeup of the System Developer going forward, which would give warning signs before they self destruct and cause subscribers' lose loads as has happened before on C2....That could be something! |
|
| | |
|
| |
|
| |
| Subject: | Evaluating a Trading System |
| Posted by: | Financial Scientist ( C2 Score: 770) |
|
| |
|
| When: | 2/13/09 (11:41) | |
| Systems: | |
|
|
|
|
| | In response to post by Pegasi Capital of 2/13/09 (2:41) Perhaps a formula which tracks the psychological makeup of the System Developer going forward, which would give warning ...
See entire
Maybe, but everyone was turned away near the peak. |
|
| | |
|
| |
|
|
