Re: Statistics of cross-correlation

On Mon, 26 May 2008 01:46:52 -0700 (PDT), nico
<romano.nicola@xxxxxxxxx> wrote:

First of all thank you for your reply.


Always significant?

That won't happen for two "random time series" if it
is a real test and it is being applied appropriately.

Now, if your series both show a linear increase (decrease),
you will get some positive correlation, no matter the lag.
Is that what you are referring to?


I'm referring to the test in the first link that was given:
http://secamlocal.ex.ac.uk/people/staff/dbs202/cat/stats/corr.html

It says:
A simple method to test the null hypothesis that the product moment
correlation coefficient is zero can be obtained using Student's t-test
on the t statistic = r sqrt(N-2)/sqrt(1-r^2) where N is the number of
samples (Statistics, M. Spiegel, Schaum series).

Now... with N>1000 (as it is my case) you can easily see that even
very low values of correlation (r) would give a significant result
using this test.

Okay, I joined this late. And I usually avoid Time-series
because others deal with it better.

Someone should have pointed out that this is merely
the conventional test a correlation. It is *valid* when
the points are independent -- That is not the case for
your ordinary, un-pre-processed time series, a series
that has any "autocorrelation". One way to look at the
effect of autocorrelation is that it reduces the effective
degrees of freedom (sometimes, vastly).

By the way, I Googled for Ljung-Box test, and it is *not*
what you said. According to Wikipedia, it does not
"control for other correlations." Rather, it is an omnibus
test over a whole set of observed correlations, and
it rejects when the set "has something going on." That
might be one r that is really large, or it might be a set
of nearly-significant correlations.


---

Going back to my original question:

One thing that I did was to cross-correlate one of the signals with an
unrelated signal or a random signal and the ccf is flat. Could I just
take the sd of this "negative control" and use, say, 3 times this sd
as a significant value?

Do you think that this would be a valid test?


Probably not.

Getting valid tests for regression is potentially
tricky, when you don't know about the data. Which
I don't. Time series is *far* trickier.

I suspect that you need to show your data problem,
in full, to a statistician who has experience with time series.


--
Rich Ulrich

http://www.pitt.edu/~wpilib/index.html
.

Relevant Pages

  • Re: point me in the right direction for time series analysis?
    ... > I want to analyse each infant separately. ... For each baby two time series are generated: ... > a correlation between two time series variables which are both strongly ... > managed to get hold of a decent book on time series analysis, ...
    (sci.stat.consult)
  • Re: Hypothesis testing, two different rates
    ... The correlation between to time series can be computed. ... significance of a correlation coefficient using "standard procedures" ... In specific ARREST RATE is significantly related to OD rate in the ...
    (sci.stat.edu)
  • Re: A question about Covariance??
    ... I learn that people calculate the correlation of two Random Process to ... as in autocorrelation, autocovariance and autocrosscovariance. ... offsets of differing time series. ... with the cosine but well correlated at an offset of 1/4 period. ...
    (comp.dsp)
  • Re: time varying correlation
    ... > I am trying to find out relation b/w two variables, ... You are looking at time series. ... > with monthly or weekly data, there will be very few points and that may ... > not give a good estimate of correlation. ...
    (sci.stat.math)
Loading