3D reconstruction, fundamental matrix and projections
- From: Jayhawk <thomas.greenleaf@xxxxxxxxx>
- Date: Tue, 14 Oct 2008 11:55:14 -0700 (PDT)
I am trying to reconstruct a scene from 168 points in two images and
have run into a baffling problem with extracting the two projection
matrices from the fundamental matrix. I will try to explain the steps
I have taken, what was the result and what was my expectations, so
that perhaps someone can tell me where the problem is. A little matlab
code at the bottom.
I calculate F by the so called normalized 8 point algorithm, though I
currently do not normalize. The algorithm solves a set of homogenous
equations to find an F which satisfies x’^T*F*x = 0 where x’ is the
“right” image point and x is the “left” point. I skip the details for
now since I feel pretty sure F is correct.
After finding F I enforce the rank 2 constraint by [U S V]=svd(F) and
setting last singular value in S to zero and then building F as
F=U*S*V^T.
I evaluate F by using it to calculate epipolar lines for the
correspondences and I see that if I multiply a “left” point coordinate
I get a “right” epipolar line which perfectly runs through the “right”
point in the image. I also see that the lines in one image intersect
at a point where the other camera would be and that the lines in the
other image seem to intersect somewhere outside the image.
I calculate the left epipole as rightmost right singular vector
(column 3 in V) and the right epipole as the left singular vector
(right column in U). When I normalize them they match with the
intersection of epipolar lines in the image.
The perfect match between epipolar lines and image points tell me that
F is correct but I may be wrong? Could F have error and still perform
this task perfectly? F looks like this
F =
-0.000000040525147 -0.000001020866741 0.000768441690963
0.000001332540844 0.000000090247275 0.000295372558795
-0.000695894205940 -0.001505870289121 0.959622033288647
The next task is to get the two projection matrices P and P’. I define
P as the identity matrix concatenated with a zero column vector
[eye(3) zeros(3,1)].
P =
1 0 0 0
0 1 0 0
0 0 1 0
I define P’ as P’=[e_SM*F|e^T] where E_sm is e (left epipole)
transformed into a skew symmetric matrix like this E_SM = [0 –e3 e2;
e3 0 –e1;-e2 e1 0]; Here e1 means the first element in the column
vector e.
This gives
Pm =
-0.000220503729526 -0.000477154034793 0.304068136939364
-0.948471219653465
-0.000660035653084 -0.001428275305558 0.910174389014026
0.316862599984881
-0.000001251035737 0.000000237877547 -0.000523642803228
0.000661981789580
This is not what I would expect when comparing to P, or when
considering the project itself. The bottom row of the 3x3 sub matrix
is very small and the two first rows are more or less the same
direction, so the basis of P’ is generally quite deform. If I consider
the three first columns the basis vectors and the fourth the
translation, it certainly does not match up with a world view defined
by the first camera. The rotation og the basis axis and translation
makes no sence when compared to what should be expected from the
images. Some text images are known to have identical up vector and a
rotation around the up axis (Y) of 45 degrees and I get basis vectors
for P’ which has x and y coinsiding and z along y or something equally
odd. It does have rank 3 though, as it should have.
What kind of look should I expect for P’?
I do not know how to better test P’, so the next step is another
simple linear method to find X given x and x’, but that certainly
fails. The X found does not project into x and x' when myltiplied with
P and P'. Not even close. The code is in linearTriangulation which
follows this text.
It is very confusing this. F looks correct in every way but P’ does
not... even though it should be so very simple to get from F. Any step
by step examples of going from two images with correspondences though
F to P and P’ with actual numbers would be great. Any insight into if
F and P and P’ looks right would be welcome as well. Any pointers
actually because as far as I can see I do everything by the book
(multiple view geometry, Hartley&Zisserman) and still it fails.
-----
Find F from a set of correspondences
function F = DetermineF(Coor)
A = buildA (Coor);
[U D V]=svd(A);
F = zeros(3,3);
F(1,1:3)=V(1:3,end);
F(2,1:3)=V(4:6,end);
F(3,1:3)=V(7:9,end);
[U D V]=svd(F);
D(3,3)=0; %ensure rank 2
F=U*D*V’;
end
function A = buildA (Q)
numOfCorr = size(Q,2);
A = zeros(numOfCorr, 9);
for i= 1:numOfCorr
A(i,1:9) = buildRowA(Q(1:2,i),Q(3:4,i));
end
end
function R = buildRowA (l,r)
x1 = l(2);
y1 = l(1);
x2 = r(2);
y2 = r(1);
R = [x1*x2 y1*x2 x2 x1*y2 y1*y2 y2 x1 y1 1];
end
Find X from two projection matrices and x and x' which are the image
points.
function X = linearTriangulation(P,Pm,leftX,rightX)
x = leftX(1);
y = leftX(2);
xm = rightX(1);
ym = rightX(2);
p1T = P(1,:);
p2T = P(2,:);
p3T = P(3,:);
pm1T = Pm(1,:);
pm2T = Pm(2,:);
pm3T = Pm(3,:);
A = [x*p3T - p1T; y*p3T - p2T; xm*pm3T - pm1T; ym*pm3T - pm2T];
[U S V] = svd(A);
X = V(:,3);
.
- Follow-Ups:
- Prev by Date: Re: Rotating vectors in 3D
- Next by Date: Re: B-Spline Curve Fitting
- Previous by thread: visualisation with freeglut, opengl
- Next by thread: Re: 3D reconstruction, fundamental matrix and projections
- Index(es):
Relevant Pages
|
