Optimization & Learning

[benfrantzdale]

I've been looking into optimization algorithms recently. That is, given f(x), find x to minimize f(x) where x may be multidimensional (or even infinite-dimensional). It turns out that surprisingly many useful problems can be cast as an optimization problem. For example, solving the equation Ax=b (where A is a symmetric matrix) can be thought of as

Comments

[benfrantzdale]

It is (link added). Calculus of varioations does solve the problem beautifully for the simple case, but if you change the problem much (maybe as little as giving the marble rotational inertia?), I suspect you'll quickly have to resort to numerical methods.

[amoken]

Also, since it is so gracefully solvable, it makes a fabulous test problem for these algorithms, which are needed on more complicated problems. If they don't get that right, they clearly aren't working properly.

[amoken]

I find all this a particularly satisfying explanation of why practice works but why teachers are tremendously helpful, even when their advise is simple.

Absolutely. Especially since the space is not fully revealed to the poor little vector/algorithm/person (whatever your POV), and teachers often open your eyes to different parts of the space you're trying to explore. Even the dimensions may not be known at first!