Saturday, January 31, 2009

Gaussian Processes are ill-conditioned (Example)

Someone asked me "why did you say in the Gaussian processes tutorial that Gaussian processes are ill-conditioned?" Here is an answer in the form of a tiny exercise.

Take the simplest case where the function is just a constant.

y(x) = Y for all x. (Y is Gaussian distributed and not known a priori)

Make a GP with the appropriate covariance function.

Now

(1) think about the inference of the GP given data {x,t}. (Not very difficult is it?)

(2) see what happens when you use the standard GP matrix inversion formalism to solve the problem. What is the well-conditioned-ness of the matrix you must invert?

1 comment:

Jim said...

Dear Mr. McKay,

I really enjoyed the SEWTHA book (about half done now) and was going to get "the other book". However, I was informed that the URL was no longer valid.

Jim Riddell