Stanford's machine learning class on coursera is pretty good. What's surprising to me is that exercises take much more time than listening to lectures and taking after class quizes.

The programming environment used for the class is Octave. The most challenging part so far with writing code in Octave is coming up with what's called vectorization solutions. The idea of vectorization is that instead of writing loops, you compute values by vector and matrix operations.

Let's look at a simple example. Imagine that you have a vector of zeros and ones. You have to convert it to a vector where for each one we have a one and for each zero we have minus one. That came up as a component of the cost function computation in logistic regression.

An iterative solution in Scala might look like this:

v.map(x => if (x == 0) -1 else 1)

How would that look in Octave if we can't use loops and if statements? A function F that maps from {0, 1} to {-1, 1} actually looks like this:

F(x) = 2x - 1

Pretty cool. Now if our vector of zeros and ones is A, by using that function, we can compute the desired value by doing this:

D = 2 * A - 1

This is pretty much only a part of a typical vectorized computation in machine learning. Crafting those takes some time, but at the end there's a good a-ha moment and usually a quite short solution.

So far I went through the linear and logistic regression lectures. Both methods are widely practical and it's very easy to come up with usage example. Those methods are examples of supervised learning algorithms, since we train them on a data set and then once we've computed a model we just plug new examples into it and come up with results.

In practice, I doubt that anyone is going to implement their own gradient descent algorithms to come up with model coefficients. I'd expect that usually a library / package like Matlab / Octave / R will be used to train a model. Implementing the model itself is quite trivial and would be just a couple of lines of code.