Noxa indicators for Neuroshell

[Noxa]

Thanks for this info, it is very valuable for a new users. Meanwhile I repeated the test on GOLD5 and also achieved very good hit - 80%. So for this level of noise NOXA outperformed MACD (30%) a lot !!! Than it confirms my statistic from post 94 that with high noise it performed well. Last result S=2.5, N=80 ==> -15db (calculation based on maks amplitude)

Any special reason that you don't want to repeat test using multiplication of random numbers ??? It's much easier than create the time series. Noise from GOLD5 is a gauss noise, multiplication of random numbers will not have a gauss distribution and will be more similar to market noise I think but I will check it ?

For curiosity. How did you measure S/N ratio of GOLD5. Just checking others .csv to find amplitude or you have a tool ??

Krzysztof

>> So for this level of noise NOXA outperformed MACD (30%) a lot !!! Than it confirms my statistic from post 94 that with high noise it performed well
The logic tells me that it should be better with less noise... We'll see...

>> Any special reason that you don't want to repeat test using multiplication of random numbers ??? It's much easier than create the time series. Noise from GOLD5 is a gauss noise, multiplication of random numbers will not have a gauss distribution and will be more similar to market noise I think but I will check it ?

I am now going through your data... The only way to avoid comparing oranges and apples is to use the same data... Send me your data if you want... I have no objections to my own tests but I won't try to duplicate your tests again. No other reasons.

Noxa

 

[Krzysiaczek99]

-9db

Hi,

Than I made a test for -9db noise again generating .csv file. I used similar settings
like suggested by NOXA in post 105

Results: MACD 76,9%/4.41 NOXA 34,3/0.37

NOXA, if you can please comment on the reason of such poor results.

Chart and csv is attached.

Krzysztof

 

[Noxa]

Noxa CSSA -9dB, 100% profitable trades

Hi,

Than I made a test for -9db noise again generating .csv file. I used similar settings
like suggested by NOXA in post 105

Results: MACD 76,9%/4.41 NOXA 34,3/0.37

NOXA, if you can please comment on the reason of such poor results.

Chart and csv is attached.

Krzysztof

Krzysztof,

I ran several models with your data and always got better than 85% profitable trades out-of-sample. Attached is a chart showing 100% profitable trades.
Blue cycle = cycle which troughs generate long entries
Red cycle = cycle which peaks generate short entries

Your problem is that you applied the same settings I posted for GOLD5. You did that last time already! This is a recipe for failure; you need to adapt your settings to new data. If I were to use these settings I would have get 87% profitable trades but the system had a tendency to over fit meaning that if I had let it go through the all optimization process, it would have produced bad results.

Here is what I did:
- I narrowed the range for m-histories to help the GA a bit,
- I periodically halted the optimization (at each new epoch) to avoid over-fitting.

Over-fitting occurs when an optimizer learns the noise in the data. One simple way to avoid it is to periodically halt the optimization. The optimization is then continued from the point at which it was halted. Nothing fancy here! Just a way to give you an access to interim results so that you can make an educated guess on when to stop the optimization!

To wrap up: the GA had not trouble finding good results with CSSA given that you make sure the system does not over-fit. This rule applies to any indicator.

Noxa

 

[Krzysiaczek99]

Hi,

I was suspecting that this under performance is because of over fitting but I didn't change the parameters on purpouse because:

1) the signal is the same
2)the settings are the same

what is changed is a noise level, in both examples noise had a constant but different levels.

in GOLD5 you adjusted parameters to obtain the best fit and this model stopped to work immediately for -9db when noise even decreased. Than you adjusted parameters again to -9db to get the best fit. Is this procedure is not called 'curve fitting' ??What will happen with NOXA model when noise level will change during out of sample period ?? Because from this it looks that performance will go down immediatelly. I will try to create this type of signal tommorow than we will see i think.

So your procedure of optimization for market conditions is simple adjust parameters, optimize,
break optimization from time to time to see that model is making money out of sample, if equity
curve is rising than model is OK ?? Is that correct ??

- I periodically halted the optimization (at each new epoch) to avoid over-fitting.

How you know when epoch is finished in NS ??

Krzysztof

 

[Noxa]

Hi,

I was suspecting that this under performance is because of over fitting but I didn't change the parameters on purpouse because:

1) the signal is the same
2)the settings are the same

what is changed is a noise level, in both examples noise had a constant but different levels.

in GOLD5 you adjusted parameters to obtain the best fit and this model stopped to work immediately for -9db when noise even decreased. Than you adjusted parameters again to -9db to get the best fit. Is this procedure is not called 'curve fitting' ??What will happen with NOXA model when noise level will change during out of sample period ?? Because from this it looks that performance will go down immediatelly. I will try to create this type of signal tommorow than we will see i think.

So your procedure of optimization for market conditions is simple adjust parameters, optimize,
break optimization from time to time to see that model is making money out of sample, if equity
curve is rising than model is OK ?? Is that correct ??

- I periodically halted the optimization (at each new epoch) to avoid over-fitting.

How you know when epoch is finished in NS ??

Krzysztof

>> in GOLD5 you adjusted parameters to obtain the best fit and this model stopped to work immediately for -9db when noise even decreased
Not true! When I don’t change the settings I get 77.8% profitable trades (see screen shots).

Genetic Algorithms learn and generalize on new data but do not adapt to changes in the data! I am surprised that you did not realize that before!!!! If data change you need to learn again this new data. This is not curve fitting, this is learning. The rationale behind CSSA is to learn persistent features that traders can exploit; CSSA infers cycles from the data and generalize on new data as Genetic Algorithms do.

Price tracking however (I think this is what you are talking about) tries to adapt to changes in data. A notorious drawback of that technique is lag. If you think that the only way to go is to adapt to changes in data, the best indicator would be price itself; any other indicator would make matter worse. So the best way forward would be to simply trade price action without indicator. And if you want to be consistent with your approach of using indicators that track price while adapting to it, you then should stop using Genetic Algorithms altogether.

>> So your procedure of optimization for market conditions is simple adjust parameters, optimize, break optimization from time to time to see that model is making money out of sample, if equity curve is rising than model is OK ?? Is that correct ??
It is not fully correct. The interim results can be many things that can help you assess the validity of your model and find market states. There are many things you can do; for example we look for multiple up and down trending segments in the equity lines and some willingness to reproduce that behavior out-of-sample...

>> How you know when epoch is finished in NS ??
This is one of them; simply watch the fitness clusters for the training and out-of-sample data sets. They will move in unison until they diverge.

Noxa

 

[Krzysiaczek99]

Mean Revert

Genetic Algorithms learn and generalize on new data but do not adapt to changes in the data! I am surprised that you did not realize that before!!!! If data change you need to learn again this new data. This is not curve fitting, this is learning. The rationale behind CSSA is to learn persistent features that traders can exploit; CSSA infers cycles from the data and generalize on new data as Genetic Algorithms do.

I know that it learns. But problem is that this noise is MEAN REVERT so it learns also mean revert noise thinking that it is a cycle and it should not do it, it should just extract the cycle signal. Anyway I will prepare a new test and we will see. This is a reason that NOXA made money on noise in my earlier posts because of MEAN REVERSION.

Krzysztof

 

[Krzysiaczek99]

epochs

>> How you know when epoch is finished in NS ??
This is one of them; simply watch the fitness clusters for the training and out-of-sample data sets. They will move in unison until they diverge.

Can you make a screenschots of it ??

Krzysztof

 

[Krzysiaczek99]

the same settings but different result

>> in GOLD5 you adjusted parameters to obtain the best fit and this model stopped to work immediately for -9db when noise even decreased
Not true! When I don’t change the settings I get 77.8% profitable trades (see screen shots).

And this is another major problem as for the same settings we got different results (post 122 and screenschots) You got 77%, I ve got 34% due to overfit. If we don't dedect early enough that the optimization goes in direction 'overfit' than we are in trouble so this dedection procedure is a key here !!!

Krzysztof

 

[Noxa]

Genetic Algorithms learn and generalize on new data but do not adapt to changes in the data! I am surprised that you did not realize that before!!!! If data change you need to learn again this new data. This is not curve fitting, this is learning. The rationale behind CSSA is to learn persistent features that traders can exploit; CSSA infers cycles from the data and generalize on new data as Genetic Algorithms do.

I know that it learns. But problem is that this noise is MEAN REVERT so it learns also mean revert noise thinking that it is a cycle and it should not do it, it should just extract the cycle signal. Anyway I will prepare a new test and we will see. This is a reason that NOXA made money on noise in my earlier posts because of MEAN REVERSION.

Krzysztof

If this is true with CSSA this can also be true with MACD. Correct me if I am wrong! So I don't see the point at continuing this endless comparison unless we prove the contrary. I clearly saw a one to one correspondence between the cycles CSSA produced and your signal. I doubt there is any correspondence between the signals MACD produced and your signal. From now on, please attach the detail of your signal (signal and noise in separate columns in a csv file) so that we can verify that. I would also like to see the rules from MACD, the full equity line and a zoom on the signals MACD produced.

Noxa

 

[Noxa]

About Mean Reverting Processes

Genetic Algorithms learn and generalize on new data but do not adapt to changes in the data! I am surprised that you did not realize that before!!!! If data change you need to learn again this new data. This is not curve fitting, this is learning. The rationale behind CSSA is to learn persistent features that traders can exploit; CSSA infers cycles from the data and generalize on new data as Genetic Algorithms do.

I know that it learns. But problem is that this noise is MEAN REVERT so it learns also mean revert noise thinking that it is a cycle and it should not do it, it should just extract the cycle signal. Anyway I will prepare a new test and we will see. This is a reason that NOXA made money on noise in my earlier posts because of MEAN REVERSION.

Krzysztof

Further to this post:
I thought I should describe what Mean Reverting Processes are (MRPs for short) for the readers of this thread. They are processes which have a tendency to return to their average level. For example, if the price of a barrel drops due to overproduction, it will eventually rise as producers reduce supply. MRP means that price changes are not fully independent of one another but rather are related.

What Krzysztof meant, I think, is that MRPs also apply to noise generators (his toy example); random number generators may exhibit deviations from the mean which can fool an indicator. But this is irrelevant when considering real market data! MRPs in noise are currently being traded from mean reversion rates; we actually need indicators that are able to see and exploit these reversions.

I think that CSSA is flexible enough to either exploit MRPs or become immune to it; it is likely that CSSA will see the mean reversions in the data and generate low variance cycles accordingly. This is then up to the user to discard them or use them to maximize profit. For example component #25 represents a low variance mean reversion (see print shot attached). For the toy example we chose to exclude this component from the search. In real market conditions however, we could keep similar components to learn from or build systems.

MACD however entirely relies on mean estimation; in other words MACD will be subjected to mean reversion with no possibility of discarding it when needed.

Noxa

 

[Krzysiaczek99]

MACD/NOXA overfit ratio

If this is true with CSSA this can also be true with MACD. Correct me if I am wrong! So I don't see the point at continuing this endless comparison unless we prove the contrary. I clearly saw a one to one correspondence between the cycles CSSA produced and your signal. I doubt there is any correspondence between the signals MACD produced and your signal. From now on, please attach the detail of your signal (signal and noise in separate columns in a csv file) so that we can verify that. I would also like to see the rules from MACD, the full equity line and a zoom on the signals MACD produced.

Noxa

Of course you are right but key here is MACD/NOXA overfit ratio - who better overfits than worse perform out of sample I think. I was going to do my test exacly as you said.

Krzysztof

 

[Krzysiaczek99]

mean revert, markets and my next test

What Krzysztof meant, I think, is that MRPs also apply to noise generators (his toy example); random number generators may exhibit deviations from the mean which can fool an indicator. But this is irrelevant when considering real market data! MRPs in noise are currently being traded from mean reversion rates; we actually need indicators that are able to see and exploit these reversions.

It's true but it's not a case here that CSSA was set in those cases to exploit mean reversion. CSSA suppouse to find the cycle than for example for gauss noise should not produce ANY buy/sell signals because of this cycle less signal. It produced. Also performance were growing when S/N ratio was falling so my suspicion is that instead or together with extracting cycle it learns from mean reverted noise and this is wrong also because it was not set for it.

Simple example for people who are real time FOREX traders. The hours between sessions, before US open, after London close, after US close this is a periods of low
S/N ratio or pure noise market. If in such conditions you will adjust CSSA to the market, you will find cycles which don't exist than you are in trouble.

My test will be simple. I will merge GOLD5(-15db) with-9db than S/N goes up, signal gets stronger, CSSA should keep equity curve rising. Settings for test are given
already by NOXA for GOLD5, out of sample period will start in GOLD5 and continue though -9db. If equity curve will collapse during out of sample with stronger signal (-9db) than it is obvious that the curve fitting occurs here as signal in GOLD5 and -9db is the same, just noise level different. For me it is a KEY TEST. I will do it tomorrow, if somebody want to comment this idea please do it.

And of course I will compare with MACD.

NOXA, can you answer/comment post 127 and 128 ??

Krzysztof

 

[Noxa]

No need to redo the test

What Krzysztof meant, I think, is that MRPs also apply to noise generators (his toy example); random number generators may exhibit deviations from the mean which can fool an indicator. But this is irrelevant when considering real market data! MRPs in noise are currently being traded from mean reversion rates; we actually need indicators that are able to see and exploit these reversions.

It's true but it's not a case here that CSSA was set in those cases to exploit mean reversion. CSSA suppouse to find the cycle than for example for gauss noise should not produce ANY buy/sell signals because of this cycle less signal. It produced. Also performance were growing when S/N ratio was falling so my suspicion is that instead or together with extracting cycle it learns from mean reverted noise and this is wrong also because it was not set for it.

Simple example for people who are real time FOREX traders. The hours between sessions, before US open, after London close, after US close this is a periods of low
S/N ratio or pure noise market. If in such conditions you will adjust CSSA to the market, you will find cycles which don't exist than you are in trouble.

My test will be simple. I will merge GOLD5(-15db) with-9db than S/N goes up, signal gets stronger, CSSA should keep equity curve rising. Settings for test are given
already by NOXA for GOLD5, out of sample period will start in GOLD5 and continue though -9db. If equity curve will collapse during out of sample with stronger signal (-9db) than it is obvious that the curve fitting occurs here as signal in GOLD5 and -9db is the same, just noise level different. For me it is a KEY TEST. I will do it tomorrow, if somebody want to comment this idea please do it.

And of course I will compare with MACD.

NOXA, can you answer/comment post 127 and 128 ??

Krzysztof

Krzysztof,

I think your -9dB file is already what you just described. So there is no need to redo the test. Have a look at the screen shot attached. As you can see, there are already two different noises; noise clearly changes when it enters the out-of-sample periods. Pure luck or illusion? I don't think so. If you are not sure, you can also have a look at the red curve (bottom chart) for confirmation; the Shannon Entropy which is a measure of noise (from our NEI package) clearly indicates a demarcation.

Could you please comment on that?

Noxa

 

[Krzysiaczek99]

-9db noise

I dont know. This noise was generated by multiplication of two random numbers in NS. I simply extracted csv from previous test. So it can be different type of noise than in GOLD5
where is a gauss noise. I didn't want to have a gauss PDF in this noise.

I hope you remember about post 127 and 128 and mysterious 'fitness clusters '. Are you connecting special tool during optimization ??

Krzysztof

 

[Noxa]

Krzysztof,

I think your -9dB file is already what you just described. So there is no need to redo the test. Have a look at the screen shot attached. As you can see, there are already two different noises; noise clearly changes when it enters the out-of-sample periods. Pure luck or illusion? I don't think so. If you are not sure, you can also have a look at the red curve (bottom chart) for confirmation; the Shannon Entropy which is a measure of noise (from our NEI package) clearly indicates a demarcation.

Could you please comment on that?

I dont know. This noise was generated by multiplication of two random numbers in NS. I simply extracted csv from previous test. So it can be different type of noise than in GOLD5
where is a gauss noise. I didn't want to have a gauss PDF in this noise.

I hope you remember about post 127 and 128 and mysterious 'fitness clusters '. Are you connecting special tool during optimization ??

Krzysztof

Noxa

I see!!!

Do you mind sending me the details that I would need to duplicate this noise in NeuroShell Trader? My skepticism is based on the near-perfect separation of the two different noise regions.

Noxa

ps. I have posted side by side both GOLD5 and -9dB charts (see the red Shannon Entropy lines at the bottom).

 

[Krzysiaczek99]

-9b chart

Hi,

Here is a chart, signal. noise= multiplication of 2 random numbers and signal+noise)
Based on values of maximums of random numbers 5 and 4 than maks is 20 than -9db.
I calculated in this way.

Sorry but I know nothing about Shannon Entrophy, I don't have this indicator so I can not discuss.

It seems that you are pretty selective in your answers, you didn't comment post 127 and 128 and answer the question if you connect special tool during optimization.

Krzysztof

 

[Krzysiaczek99]

attachements

here it is

 

[Krzysiaczek99]

FFT of -9db

Here is FFT of -9db in comparision with GOLD5. I generated it once more (9db_1) but it's the same. I love it !!!

Krzysztof

 

[Noxa]

Hi,

Here is a chart, signal. noise= multiplication of 2 random numbers and signal+noise)
Based on values of maximums of random numbers 5 and 4 than maks is 20 than -9db.
I calculated in this way.

Sorry but I know nothing about Shannon Entrophy, I don't have this indicator so I can not discuss.

It seems that you are pretty selective in your answers, you didn't comment post 127 and 128 and answer the question if you connect special tool during optimization.

Krzysztof

>> It seems that you are pretty selective in your answers, you didn't comment post 127 and 128 and answer the question if you connect special tool during optimization.
We need to sort this out!!! I want to make sure I understand your noise before anything else.

Please send a csv file with noise and signals in separate columns along with a screen shot of your formula. You know that we cannot open your intraday charts.

Noxa

 

[Krzysiaczek99]

curve fitting

As i promised I made a key test for me. In order to check my suspicion that curve fitting occurs in CSSA i made following test

First I fetched GOLD5 file (noise -15db) with the settings given by NOXA, the results were superb, PF for CSSA 605/80% for MACD 0.46/30%

Than I fetched optimized settings from GOLD5 for CSSA and MACD and used them as an input setting for file with noise -9db. as a signal behind it the same just masked by noise with different level i expected CSSA to perform even better because of higher S/N ratio !!! This situation is equal to changing market conditions (noise level) to actually better conditions - lower noise.

Than look what happened. The performance of MACD has grown but performance of CSSA has fallen immediatelly. MACD outperform CSSA by 50% during the same period.

So my conclusion is that even if settings are for cycle extraction CSSA makes curve fitting as well and this is not acceptable.

FIles are attached and charts

-9db -combined signal (GOLD15+ 9db_noise)
-9db_noise - just noise from NS

Other files you have

GOLD1 - gauss noise
GOLD15 - clean signal
GOLD5 = GOLD1+GOLD15

Krzysztof