Noxa indicators for Neuroshell

[Krzysiaczek99]

noise formula

screenshot, chart and formula is in post 137......posted already a few times

Krzysztof

 

[Noxa]

As i promised I made a key test for me. In order to check my suspicion that curve fitting occurs in CSSA i made following test

First I fetched GOLD5 file (noise -15db) with the settings given by NOXA, the results were superb, PF for CSSA 605/80% for MACD 0.46/30%

Than I fetched optimized settings from GOLD5 for CSSA and MACD and used them as an input setting for file with noise -9db. as a signal behind it the same just masked by noise with different level i expected CSSA to perform even better because of higher S/N ratio !!! This situation is equal to changing market conditions (noise level) to actually better conditions - lower noise.

Than look what happened. The performance of MACD has grown but performance of CSSA has fallen immediatelly. MACD outperform CSSA by 50% during the same period.

So my conclusion is that even if settings are for cycle extraction CSSA makes curve fitting as well and this is not acceptable.

FIles are attached and charts

-9db -combined signal (GOLD15+ 9db_noise)
-9db_noise - just noise from NS

Other files you have

GOLD1 - gauss noise
GOLD15 - clean signal
GOLD5 = GOLD1+GOLD15

Krzysztof

A first look at your graph makes me suspicious already: if I look at the MACD performance you refer to (right graph, second chart from the top, I see very high frequency signals which can't be related to the 20bar period of the signal. The entries that were generated by MACD have nothing to do with the signal. To me this invalidates the experiment upfront.

Anyway, I'll try to duplicate your results later; we'll see if we have an agreement...

For the time being I want to understand your -9dB signal. How do you calculate it again? I am not expert in noise so you might teach me. But if I take the peak to peak amplitudes of the noise and the signal from your file and apply the dB conversion formula 20*log10(amplitude ratio) I can't obtain the -9dB value. How do you do it?

Noxa

 

[Krzysiaczek99]

S/n

not 20 but 10log(2,5/(5x4))=-9db

2.5 amplitude of GOLD 15
5 and 4 maximum of random numbers = maks 20 (multiplication)

How you measured my GOLD5 than ?? you gave a S/N ratio for it.

MACD - I just checked and I don't think that there is something wrong in it.

Krzysztof

 

[Krzysiaczek99]

in detail

it was like this, I don't know if you can open my chart, seems not. MACD simple made more trades but it always does.

Point is that PF went down from 605 to 2.86 and this is curve fit !!!

Krzysztof

 

[Noxa]

not 20 but 10log(2,5/(5x4))=-9db

2.5 amplitude of GOLD 15
5 and 4 maximum of random numbers = maks 20 (multiplication)

How you measured my GOLD5 than ?? you gave a S/N ratio for it.

MACD - I just checked and I don't think that there is something wrong in it.

Krzysztof

Sorry but I think you are wrong! 10 is for powers, 20 is for amplitudes and with deal with amplitudes here therefore it is 20. Check this out:
Signal-to-noise ratio - Wikipedia, the free encyclopedia

So the signal is 20log(2,5/(5x4))=-18dB for your supposedly -9dB file. I have used 20 as well for a quick estimate of GOLD5. I might be wrong on the amplitudes though since I took the peak to peak amplitude for the noise which might not be correct with gaussian PDF; it is possible that the noise for GOLD5 is less to what I initially thought.

MACD generated entries that are not related to the sinewave signal; instead the entries look random. It is very likely that your system suffers from overfitting... The fact that MACD shows an uptrending equity line is certainly random luck. I'll double check...

Noxa

 

[Krzysiaczek99]

db and random

DB - see picture - John Ehlers 'Rocket sience for traders'. It was m source

Random MACD - I don't think so. See my previous mail about MACD/CSSA overfit rate.

Anyway point here is not MACD but CSSA. PF went down from 605 to 2.8, settings were yours. It's possible to repeat this by generating gauss noise with lower amplitude than
this one in GOLD1 and mix it with GOLD15. Than it will also say something.

krzysztof

 

[Noxa]

it was like this, I don't know if you can open my chart, seems not. MACD simple made more trades but it always does.

Point is that PF went down from 605 to 2.86 and this is curve fit !!!

Krzysztof

MACD:
Ratio Gross Profit Loss = 1.5
%Profitable trades = 56.8%
My conclusion is: Your system with MACD generated random entries. One experiment I'll try is to train it on noise only. If am able to generate similar profit with almost as much entries than they are bars, this will be a proof that your system produces noise.

CSSA:
Ratio Gross Profit Loss = 2.89
%Profitable trades = 69.5%
My conclusion is: very good results indeed! One way to explain such good results is that CSSA was successful in learning the sine waves therefore there is no curve fitting. One way to explain the drop in performance (if there is such drop) is that the noise during testing increased (to -18dB) instead of decreasing as initially thought. I'll do some research on the real noise values and I'll duplicate the experiments for confirmation.

Noxa

 

[Noxa]

DB - see picture - John Ehlers 'Rocket sience for traders'. It was m source

Random MACD - I don't think so. See my previous mail about MACD/CSSA overfit rate.

Anyway point here is not MACD but CSSA. PF went down from 605 to 2.8, settings were yours. It's possible to repeat this by generating gauss noise with lower amplitude than
this one in GOLD1 and mix it with GOLD15. Than it will also say something.

krzysztof

P is for power; John Ehler is talking about power in phone lines. In your case you deal with amplitudes. Therefore it is 20 times not 10 times! So, and this is final, the signal to noise ratio is -18dB, not -9dB.

You don't seem to know what you are doing!!! I have real concerns about the legitimacy of your experiments. I am also concerned about the legitimacy of what you might produce next. And since you compare CSSA with MACD which to me produced noise (need to be verified), I believe my concerns are justified. The only way to know is to duplicate your systems...

Noxa

 

[Krzysiaczek99]

to be exact

MACD:
Ratio Gross Profit Loss = 1.5
%Profitable trades = 56.8%
My conclusion is: Your system with MACD generated random entries. One experiment I'll try is to train it on noise only. If am able to generate similar profit with almost as much entries than they are bars, this will be a proof that your system produces noise.

CSSA:
Ratio Gross Profit Loss = 2.89
%Profitable trades = 69.5%
My conclusion is: very good results indeed! One way to explain such good results is that CSSA was successful in learning the sine waves therefore there is no curve fitting. One way to explain the drop in performance (if there is such drop) is that the noise during testing increased (to -18dB) instead of decreasing as initially thought. I'll do some research on the real noise values and I'll duplicate the experiments for confirmation.

Noxa

Just let's be exact OK....

MACD:
Ratio Gross Profit Loss went up from 0.46 to 1.5 === 3times up

CSSA:
Ratio Gross Profit Loss went down from 605 to 2.89 so 209 times down


Please duplicate my system or simple you can use my charts and post the results. I'm not going to produce much more as after almost 2 weeks I have a picture what's going on here....post 127 and 128 still not answered so for me this test is closed.

But at least you have seen how the BASIC FUNCIONALITY test should look like because I'm almost 100% sure it was never done. If it was done than post Test Instruction and Test Report document as perhaps you know that it is basic documentation of software production

Krzysztof

 

[Krzysiaczek99]

und das is alles !!

Life must go on....

 

[Noxa]

Just let's be exact OK....

MACD:
Ratio Gross Profit Loss went up from 0.46 to 1.5 === 3times up

CSSA:
Ratio Gross Profit Loss went down from 605 to 2.89 so 209 times down


Please duplicate my system or simple you can use my charts and post the results. I'm not going to produce much more as after almost 2 weeks I have a picture what's going on here....post 127 and 128 still not answered so for me this test is closed.

But at least you have seen how the BASIC FUNCIONALITY test should look like because I'm almost 100% sure it was never done. If it was done than post Test Instruction and Test Report document as perhaps you know that it is basic documentation of software production

Krzysztof

>> MACD:
>> Ratio Gross Profit Loss went up from 0.46 to 1.5 === 3times up
Sorry to say but a Ratio Gross Profit Loss of 0.46 to start with was already very questionable.

>> CSSA:
>> Ratio Gross Profit Loss went down from 605 to 2.89 so 209 times down
Where did you get the 605 value again? And I got myself a Ratio Gross Profit Loss of 9.5 in place of 2.89 you recently reported.

>> Please duplicate my system or simple you can use my charts and post the results. I'm not going to produce much more as after almost 2 weeks I have a picture what's going on here....post 127 and 128 still not answered so for me this test is closed.
Interesting. What do you think is going on? I told you that I wanted to sort out the noise issue before. We made one important discovery today as you reported a noise of -9dB instead of -18dB. That explains a lot about your test results.

>> But at least you have seen how the BASIC FUNCIONALITY test should look like because I'm almost 100% sure it was never done. If it was done than post Test Instruction and Test Report document as perhaps you know that it is basic documentation of software production
This is beside the point. I thought your intentions were about making money with CSSA not to tell us how to develop software.

For your future investigations with noise, you can use this high quality integer random number generator by H.W.Lewis, with "Bays-Durham shuffle" ;-)

Noxa

 

[Krzysiaczek99]

MACD change performance

Macd

 

[Krzysiaczek99]

CSSA performance change

CSSA here. I go sleep now, is 3 am. The test was to check behaviour of CSSA/MACD optimized in GOLD5 noise conditions and moved to -9db noise conditions === change of market noise conditions. And this is a results. Please open my charts for confirmation.

Krzysztof

 

[Krzysiaczek99]

reverse test

In order to have a clear picture I made a reverse test to the test from yesterday

So I used NOXA settings given for -9db in post 123 and run out of sample in GOLD5 to see reaction for market change (noise level change). Here are the results.

The purpouse of this test and test from yesterday is to find out if initial settings found
by eigenshow/optimizer/expirience of person who set this will hold out of sample.

In both cases those settings are given by NOXA......i just change the noise conditions where they run and compare the results

Krzysztof

 

[Krzysiaczek99]

MACD performance change

PF went up form 0.32 to 0.69 = 2times

 

[Krzysiaczek99]

CSSA performance change

PF went down form 11.6 to 4.72 so 2.5 times. NOXA still profitable, MACD still loser.

So in both tests, from yesterday and reverse test, behaviour is the same, PF of NOXA down, PF of MACD up regardless if we move to higher or lower noise conditions.

So I believe NOXA can comment this first.

Anyway, i have quite clear picture now, no need for more tests.

Krzysztof

 

[at120]

PF went down form 11.6 to 4.72 so 2.5 times. NOXA still profitable, MACD still loser.

So in both tests, from yesterday and reverse test, behaviour is the same, PF of NOXA down, PF of MACD up regardless if we move to higher or lower noise conditions.

So I believe NOXA can comment this first.

Anyway, i have quite clear picture now, no need for more tests.

Krzysztof

hi guys!

Nice and interessting discussion!

@Krzysiaczek99: So your final conclusion about Noxa CSSA is, you won't use it, do I got the point right?
What about the other program "Entropy Indicators (NEI)"

@Noxa: What's your comment about Krzysiaczek99?

Bye, AT

PS: Krzysiaczek99: sprichst du deutsch?
(= are you talking german)

 

[Krzysiaczek99]

comments

Hi At120,

No final conclusion yet becouse:

- settings for real out of sample test with market was never given (i doubt that we will ever get it)

- post 127 and 128 not answered and they are crucial in my opinion plus a question if NOXA using additional tools during optimization.

- last tests and falling PF regardless of moving to higher or lower noise conditions should be commented

However after two weeks of digging i have a picture how CSSA can act. Now at least We have reference settings given by noxa to my signals plus signals so everybody can play. I still didn't recover slower cycle, i didn't have time so maybe any of the readers will try and post the settings.


NEI indicators. I simple don't have them so can not say anything

Yes, i speak German a bit.I worked in Germany 3.5 years and in Switzerland almost 4 years. But at work i was using English.

Krzysztof

 

[Noxa]

PF went down form 11.6 to 4.72 so 2.5 times. NOXA still profitable, MACD still loser.

So in both tests, from yesterday and reverse test, behaviour is the same, PF of NOXA down, PF of MACD up regardless if we move to higher or lower noise conditions.

So I believe NOXA can comment this first.

Anyway, i have quite clear picture now, no need for more tests.

Krzysztof

Krzysztof,

There is some need for more testing and this is why:

What you have shown so far with MACD is:
No over fitting means there is improvement on the PF when noise is changed. The problem is that improvements were achieved with MACD whatever you switched from GOLD to -9dB or from -9dB to GOLD. If by simply changing noise there is a systematic improvement in the PF you might have found the Holy grail but I doubt it. So to me the conclusion is that what you reported is coincidental.

As for CSSA exhibiting a drop in performance after noise is changed it is very much expected; Recall that CSSA learns from data (noise included) in the training range and generalizes on new data without adapting to changes. This is the rationale behind CSSA; inferring persistent cycles that can be exploited. The fact is CSSA was able to learn the sine waves buried in noise without overfitting and was even able to generalize well on data for which the noise was modified (remained profitable). This is not curve fitting, this is learning. We are conducting testing to exemplify that.

Noxa

 

[Krzysiaczek99]

Krzysztof,

There is some need for more testing and this is why:

What you have shown so far with MACD is:
No over fitting means there is improvement on the PF when noise is changed. The problem is that improvements were achieved with MACD whatever you switched from GOLD to -9dB or from -9dB to GOLD. If by simply changing noise there is a systematic improvement in the PF you might have found the Holy grail but I doubt it. So to me the conclusion is that what you reported is coincidental.

As for CSSA exhibiting a drop in performance after noise is changed it is very much expected; Recall that CSSA learns from data (noise included) in the training range and generalizes on new data without adapting to changes. This is the rationale behind CSSA; inferring persistent cycles that can be exploited. The fact is CSSA was able to learn the sine waves buried in noise without overfitting and was even able to generalize well on data for which the noise was modified (remained profitable). This is not curve fitting, this is learning. We are conducting testing to exemplify that.

Noxa

First I agree MACD PF rise is coincidental and not important here, it's simple EMA strategy and it does not distinguish between noisy and not nosy markets.

Regarding CSSA. In my opinion it simple learns cycle and noise, this is the reason that i was calling this 'curve fitting' to noise. So when noise level changes it starts to under
perform because is not adapted to level of a new noise. So it doesnt extract cycle correctly, if it would extract correctly (clean signal which was the same in GOLD5 and -9db) PF would not fall at least in case when we move to lower noise conditions. The settings which were extracting cycle correctly in one case should extract cycle in another because it is the same cycle.

Than it is very risky business to use it if it is not able to filter out noise well. Does it has filtering mechanism build in ?? Which filters if any ?? On the other hand GroupStart and GroupDepth settings should filter noise i guess. So why PF falls ??

Krzysztof