Re: Volterra series

Thanks for the detailed reply, Robert!

See comments below.


robert bristow-johnson wrote:
Randy Yates wrote:
Andor wrote:
r b-j wrote:

the characterization of non-linear systems
is difficult. possibly more art than science (unless one were to
brute-force employ Volterra series to do it).

Can you (or anybody) expand on how one would actually go about
brute-force employing Volterra series to characterize a non-linear
black box? The little I've read about it I didn't comprehend.

...
I've also just wondered in general about these Volterra series for a
long time - never understood them either.

let me explain the philosophy a little and then the only "analysis"
method that i know (it's horribly numerically intensive):

first imagine a Time-Invariant, possibly non-linear, uniformly sampled
SISO system with finite memory (a finite delayed "forgetting time" from
which before any input has no effect on the current output). an FIR
filter would be an example but it is, of course, linear. so, IN
GENERAL, the output of this possibly linear or possibly non-linear
system would be:

y[n] = f( x[n], x[n-1], x[n-2], ... x[n-N+1], x[n-N] )

for some general function, f( ), and where N is the "order" and N+1 is
the number of states. the special case of this being linear would be
the simple FIR where

f( x[n], x[n-1], ... x[n-N] ) = h[0]*x[n] + h[1]*x[n-1] + ...
h[N]*x[n-N]

in that case you can think of the function f( ) as being an N+1
dimensional plane that passes through the origin. clear to this point?

Cool, an FIR filter is a hyperplane!

now, all Volterra really did is connect the concept of this function of
N+1 variables to the multivariable Taylor Series (or more specifically
McLauren Series) that we learned about in 3rd semester calculus:

f( x0, x1, ... xN ) = C + a0*x0 + a1*x1 + ... + aN*xN

+ a00*x0^2 + a01*x0*x1 + a02*x0*x2 + ... + a0N*x0*xN

+ a11*x1^2 + a12*x1*x2 + ... + a1N*x1*xN

+ a22*x2^2 + ... + a2N*x2*xN
...
+ aNN*xN^2

+ a000*x0^3 + a001*x0^2*x1 + a002*x0^2*x2 + ... +
a00N*x0^2*xN

+ terms with other triplets x0*x1*x2, etc.

+ terms with all possible combinations of 4 factors
x0*x1*x2*x3, etc.

This is just a technicality, but I don't think that f(...) is assumed
to be smooth (and therefore has a Taylor or McLaurin series). Smooth
functions are a very restrictive model for non-linearity - eg. a
piecewise linear function is not smooth. Probably we assume that f is
continuous, at which point it is arbitrarily well approximated by
polynomials (Weierstrass approximation theorem).



there are an infinite number of possible terms but the hope is (just
like with a finite power series) that we can add up a finite number of
these terms to quantitatively represent f( ). if you know the values
of all of the coefficients of any significance, you have modeled the
system.

does this make sense?

Yes.



now, the only way i know of to use this to try to model a black box is
to, in advance, decide on N and how many (and which) terms above you're
keeping. all of those higher power terms can be calculated from the
input and treated like just another sample in a big vector that is dot
producted against the vector of coefficients that you're trying to
figger out (just like the dot product in an FIR). then, to *get* those
coefficients, you hook up your Volterra "filter" to the same input as
to the Black Box and subtract the Black Box output (just like the
"desired" signal, d[n] in an LMS filter) from the Volterra output and
use that difference signal to slightly adjust the coefficients up or
down using the same gradient descent method that LMS adaptive filters
use. maybe, by next week, the coefficients will settle down to nice
stable values.

does that make sense?

Definitely so. It is really a regression problem: you have an n x (N+2)
data grid that you model as n equations of the form

y_k = f( x_{k,0}, x_{k,1}, ...., x_{k,N} ) + e_k, 1 <= k <= n,

where e_k is some possible iid measurement noise. The Volterra series
approach is to assume f() continuous and approximates f with a p-th
degree polynomial in N+1 variables - which has the nice property that
it is linear in the unknown coefficients, and thus can be solved using
the usual LS methods. The big drawback is the sheer number of
parameters involved, which grows exponentially with p (curse of
dimensionality).

Thanks, that has made everything much clearer!

Regards,
Andor

.

Relevant Pages

  • Re: questions raised by reading and thinking with possibly missing background
    ... One of the references I was reading stated that "a FIR filter would be 'linear phase' if its coefficients were symmetric about the middle coefficient." ... With an even number of coefficients, ... What implication does it have for the passband response? ...
    (comp.dsp)
  • Re: Volterra series
    ... brute-force employ Volterra series to do it). ... the output of this possibly linear or possibly non-linear ... of all of the coefficients of any significance, ...
    (comp.dsp)
  • Re: Tunable Filter /Time Varying Coefficents
    ... In contrast to an addaptive filter where some kind of algoritm calculates ... For example if a linear FM signal was expectied and one wanted a filter ... Consider what happens if you change all the coefficients at the same time ... In such a receiver, as the analysis time gets longer, the time resolution ...
    (comp.dsp)
  • Re: Trying to solve 2 homogenous quadratics
    ... Firstly, if F, G, or any linear combination of them, is definite then ... Step 1 - Express F in Determinant Form ... we can assume that one of the non-zero coefficients ...
    (sci.math)
  • Re: Gaussian elimination with guaranteed positive coefficients
    ... It seems that linear optimization is exactly ... If your functions W, etc, are linear, this is a classical linear ... coefficients must be positive values and the sum of all coefficients ... different light bulbs, and have only one light bulb on at a time. ...
    (sci.math.num-analysis)