Re: Rounding problem with corrcoef() ?

Greg Heath <heath@xxxxxxxxxxxxxxxx> wrote in message
<1186088305.613189.242970@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>...
On Aug 2, 3:32 pm, "Gautam Vallabha"
<gvall...@xxxxxxxxxxxxx> wrote:
"Pawel Zdziarski" <pawel.zdziar...@xxxxxxxxxx> wrote in
message

news:f8t5mr$pit$1@xxxxxxxxxxxxxxxxxxxxx





I am calculating the correlation coefficient of two
completely random samples, and repating this many
times
(say, 100k times). Then I look at the average of this
coefficient and get something around -0.0267. What is
the
problem? Obviously, the mean correlation coefficient
should
be 0. I have run this several times, getting this
negatively biased result each time.

========== MATLAB code I use for my simulation======

count = 100000;
for i=1:count
x = randn(2000,1);
y = randn(2000,1);
c=corrcoef(x,y);
res(i) = c(1,2);
end;
mean(res)
ans =
-0.0267
====================================================
what is going on??
(I also get biased means for autocorrelations of a
random
array of normally distributed numbers).

count = 10000;
for i=1:count
x = randn(2000,1);
y = randn(2000,1);
c=corrcoef(x,y);
res(i) = c(1,2);
end;>> mean(res)

ans =
1.0266e-004>> std(res)

ans =
0.0222

Is it possible that your array 'res' is larger than
100000 before you run
your loop, with values that are skewing your
calculation of the mean?

In addition to initializing res, note that

stderr = std(res)/sqrt(count)

Hope this helps.

Greg


I've encounted a similar problem using corr(), corrceof()
and when I implement a function for Pearson correlation in
Matlab. It seems that Matlab is not interpreting an
operation that results in an answer of zero to be zero.
For instance, I'm correlating two vectors of data:

x1 = 0.3302 0.3302 0.3302 0.3302
y1 = 1.5318 1.4268 1.2248 1.0988

the mean of x1 = 0.3302
the mean of y1 = 1.3205

Since all the values of x1 are the same, the correlation
should result in a divide by zero error since the numerator
and the denominator of the correlation equation equals to
zero. Howver, this is how Matlab interprets the numerator
of the correlation equation:

1.0e-014 *

-0.1665 -0.1665 -0.1610 0.4885


resulting in a correlation of -0.7624. Which doesn't make
sense. Seems that Matlab is avoiding an interpretation of
a result that equals zero and giving it a very small number
(1.0e-014 * -0.1665 for instance).

When I do the calculation of the correlation by hand in
Excel and also use Excel's Correl function I get a divide
by zero error. This should be the true answer unless I'm
mis-informed. I'll report this to Matlab.

Pierre


.

Relevant Pages

  • Re: Rounding problem with corrcoef() ?
    ... It seems that Matlab is not interpreting an ... operation that results in an answer of zero to be zero. ... Since all the values of x1 are the same, the correlation ... Use FORMAT LONG G or FORMAT HEX and compute x1 - ...
    (comp.soft-sys.matlab)
  • Re: basic problem!
    ... set the values in x that are less than the mean to zero. ... greater than 3 to some value, I'll pick inf here ... Try this in matlab ...
    (comp.soft-sys.matlab)
  • Re: Problem with random variable generation
    ... However, if you try to numerically determine the expected value of Z^2, the result is equal to j*2*E, which demonstrates that MATLAB is only capable of generating improper random variables. ... If you drew one value from a standard normal distribution, would you expect that value to be exactly zero, simply because the mean of its distribution is zero? ... But you aren't computing theoretical means, you are computing a Monte-Carlo approximation, and those MC approximations are random variables and have a mean and variance of their own. ... ans = 0.0056569>> ans = ...
    (comp.soft-sys.matlab)
  • Re: error: Math script code giving only zeros..
    ... I wouldn't trust the answer in either Matlab or Labview. ... the values Matlab gave for M were equal to zero. ...
    (comp.lang.labview)
  • Re: solving an algebraic equations
    ... suggest a numerical implementation in MATLAB to do this? ... funshould be reasonably close to zero. ...
    (comp.soft-sys.matlab)