Re: optimal set of basis vectors
- From: jg.campbell.ng@xxxxxxxxx
- Date: 21 Mar 2007 06:08:04 -0700
On Mar 20, 3:07 pm, "jindra" <jpa...@xxxxxxxxx> wrote:
Hi all,
I have a vector space L with an inner product. The vector space L has
a set of linearly independent basis vectors
B = {bi}, i=1, ..., N,
i.e., each element w of L can be represented as a linear combination
of N basis vectors w=li.bi, where li is i-th real coeficient of the
linear combination. I also have a set of vectors V = {vi} and a number
M < N.
I would like to compute an orthogonal projection of vi to L0, where L0
is a subspace of L and the dimension of L0 is M. The general idea is
that I would like "approximate" the vectors vi by vectors vi', so that
vi' are elements of L0 which has a lower dimension than L. Then the
vectors vi' can be represented as a linear combination of M basis
vectors of L0.
The problem is that I don't know how to compute basis of L0. For
example I can choose M basis vectors from B but I would like to choose
such basis vectors which will lead to the best approximation of the
set V in L0.
To describe some real application. Let's have a set of 3D vectors V =
{vi} in E3 (analogy of aforementioned L where N = 3). Project vi using
orthogonal projection on some plane (the plane is a subspace of E3,
i.e., M = 2) so that the projection is closest possible with respect
to V. For example, we can project vectors vi to xy plane which will
yield some error, or we can project vectors vi to xz plane which will
yield another error. I would like to find such plane which would yeild
the smallest error. The question is how to find such plane, i.e., its
basis vectors. Of course I'm looking for some dimension indepedent
solution, this was just one instance of my problem.
I ask here because I think that this must be some classical problem of
linear algebra and I don't want to develop some ad hoc solution.
It seems to me that you may be looking for principal components
analysis (aka Karhunen-Loeve transform); however, I may have
misinterpreted your statement of requirements. I have the impression
that the algorithm on page 2 of Dave Eberly's document may amount to
the same thing for a specific dimensionality.
I was looking for a decent coverage of PCA on the web but found none
that looked like it got quickly to the essence.
Maybe appendix A of http://www.jgcampbell.com/ip/pr.pdf would let you
quickly see if PCA answers your need. Incidentally I see that there is
a typo in eqn. A6; the lhs should be squared.
You can think of PCA as a mapping A: R^n -> R^m, m < n, which
minimises the loss of variance; in a data compression sense, you can
apply A to n-dimensional vectors to yield m-dimesnional vectors, then
apply truncated A^-1 (= A^t) to the m-dimensional vectors to retrieve
reconstructed n-dimensional vectors; the latter reconstruction will be
optimal in an LSE sense.
Geometrically, you can think of A as mapping to basis vectors
projection onto which which capture maximal variance.
Alternatively, you can think of basis vector 1 (a1) as an LSE fitting
line, a2 as an LSE fitting line that is orthogonal to a1, etc. (I
think that's what Dave's page 2 is doing --- but I didn't look
carefully).
The matrix A is obtained as the matrix of eigenvectors of the
covariance matrix, K, of the data set, K = E{ (x-mu)(x - mu)^t}.
Best regards,
Jon C.
.
- Follow-Ups:
- Re: optimal set of basis vectors
- From: Dave Eberly
- Re: optimal set of basis vectors
- References:
- optimal set of basis vectors
- From: jindra
- optimal set of basis vectors
- Prev by Date: INTELLIGENT SYSTEMS AND AGENTS 2007 - Deadline for submissions extended
- Next by Date: Re: Algos for Windows Media Player visualisations?
- Previous by thread: Re: optimal set of basis vectors
- Next by thread: Re: optimal set of basis vectors
- Index(es):
Relevant Pages
|
