Re: Obtaining 2d shape from perspective image

Hi all,

I am new to 3D geometry and have the following doubt. Is there any
perspective transformation math involved while obtaining the 3D
information from a 2D perspective projection image? I know that to
calculate the 2D projected image, we have to calculate each projected
point using the transformation matrix.

To get back the original 3D shape, do I need to convert the 2D
coordinates into world coordinates first? Or is the process totally
dependent on the reconstruction algorithm? How would you approach the
problem of obtaining the depth information of the image given a
perspective projection image of a surface?

The possibility of reconstructing a 3D shape out of a 2D projection
image depends on the amount of information available.

Lets consider the geometric pipeline running as:
local space -> world space -> view space -> projection space -> screen
space

If we make the trick of using homogeneous coordinates the
transformations between local, world, view and projection spaces are
(usually chosen) linear and thus described by matrices. If these
matrices are also non-singular, the transforms are invertible and you
can trivially change from one space to another, forward or backward.

The transform from the projection space to the screen space involves a
"downcast" from homogeneous coordinates to normal coordinates. This
transform is the "divide by w": given the projective coordinates
(x,y,z,w) the transformed point is (x/w, y/w, z/w, 1) which we take
equivalent to (x/w, y/w, z/w) on the screen space.

To invert this operation, that is, to get from screen coordinates to
homogeneous screen coordinates, you clearly need to multiply every
transformed point by the w it had before the division: (x, y, z) -> (x *
w, y * w, z * w, w) If you don't have the w's, the operation is
irreversible.

The "division by w" transform has a singularity when w = 0 (a division
by zero). Thus, actually you are unable to map points with w = 0 to the
screen space. A computer graphicist deals with this problem by clipping
away everything but what's inside the view volume (most conveniently
done in homogeneous coordinates just before the divide). Of course, for
this to work it must hold that all points in the view volume have w !=
0. This is easy: just avoid the approximity of the center of projection.
So, in general you can't take all the points with you from homogeneous
screen coordinates to the screen coordinates.

Here is a quick summary:
Given the necessary input of:
- screen coordinates (x, y, z) of each point
- the projective w component of each point
- inverse of the projection transform
you are able to reconstruct the 3D points in the view space. However,
during the "division by w" transform you have already lost access to
those points with w = 0, so the reconstructed 3D points are in general
only a subset of the original 3D points. Given the additional
information of inverses of view and world transformations you can get
further backwards in the process.

Now let us be realistic. In real life you do not carry around an extra
bag of w's on your pocket. An neither do you carry around the full
geometric description of the scene in the screen space. Rather, what you
probably have is a nice 2D image with each pixel assigned a depth, and
probably, a color. In this stage it is too late to start inverting the
process. The problem is impossible to solve because it does not have
enough information to make the right deductions in 3D space.

But you must have seen 3D scenes being constructed from 2D depth images
before. The keyword here is the plural "images": given an input set of
images taken from known positions and orientations, the information that
was lost during clipping and division by w can be deduced together
again, although always just approximately, .

Construction of a 3D scene from 2D depth projections is hard and goes by
the name "image based rendering" (IBR). Google should find you a lot of
material. IBR is hardly ever discussed in this newsgroup, which might
reflect the complexity of this problem.

--
Kalle Rutanen
http://kaba.hilvi.org
.

Relevant Pages

  • Re: Obtaining 2d shape from perspective image
    ... Iis a perspective projection of the object. ... transform is the "divide by w": ... To invert this operation, that is, to get from screen coordinates to ...
    (comp.graphics.algorithms)
  • Re: correlation of rotated image with perspective
    ... transform, but it's the distortion from perspective thats really ... perspective projection, orientation and distance from the landing pad ... as a single mser. ... baseline stereo from maximally stable extremum regions". ...
    (sci.image.processing)
  • Re: Normalization XYZ -> xyz
    ... transform. ... My second "suggestion" would be a projection onto a sphere in R^3, again having the same property is mapped to a single point on the sphere), the same holds for the first suggestion. ... is the plane through parallel to X,Z (a plane normal is given ... Again, k*X, k*Y, k*Z delivers the same chromaticities x',y',z' for any ...
    (sci.engr.color)
  • Re: Using a different type of projection matrix in Direct3D?
    ... projection transform with Direct3D apart from the standard canonical ... view transform that you get from the Matrix.PerspectiveFovLHmethod ... You should be able to use any projection ... Would this cause the rendering to flip out, ...
    (microsoft.public.win32.programmer.directx.graphics)
  • Re: correlation of rotated image with perspective
    ... perspective projection, orientation and distance from the landing pad ... I did try finding straight lines with a hough transform, ... resolution and how much calculating power you can spend. ... Make contour from the shape. ...
    (sci.image.processing)