Introduction

Welcome to our new blog

This page will be the research blog for the LocusLab group, Zico Kolter’s research group in the School of Computer Science at Carnegie Mellon University. Our research focuses mainly on new algorithmic methods and analysis in machine learning. Some our of recent work (though these topics are constantly changes) has considered adversarial robustness (specifically, provably robust deep learning), integrating complex modules like optimization solvers as differentiable layers within deep networks, analzying the convergence and generalization properties of machine learning.

As a bit of historical information, the “LocusLab” name stands for “Learning, Optimization, and Control, for Uncertain Systems”, but at this point it’s main relevance comes from the fact that it is the GitHub repositiory that contains code releases for our group.

Philosophy of the blog

There are a lot of great technical blogs for machine learning work out there (our own Machine Learning Department runs one), but there will be one slightly different aspect to our blog: all blog posts will be created and downloadable as self-contained Juypter notebooks, so that all code/figures/results/etc will be directly reproducible from the post itself. (Even this post can be downloaded as a notebook, though not a very interesting one). Of course, given that we do work on deep learning, there will be scenarios where training the full model to reproduce results will be too time-consuming to easily run on your own machine, so we’ll also supply pre-trained models that you could reproduce if you desired.

Our philosophy for building posts in this manner is simple: in many technical presentations, we’ve found that it can be difficult to translate the abstract ideas from papers (and even blog posts) into concrete code that you can use to evaluate and get a better intuiion for the ideas. As much as possible, we want to avoid this scenario, where the algorithms and math all make perfect sense when you read it, but when you fire up your favorite IDE, you have no idea how to actually get started. By making each blog post a self-contained collection of explanatory prose and code, we hope to bridge this common gap as much as possible. On the flipside, directly reading Jupyter notebooks is often a bit cumbersome, so all the pages are also rendered as static HTML that you can just read directly on the blog. We developed the tools for creating these posts for Zico’s Practical Data Science course as well as a recent tutorial on adversarial robustness.

Finally, we should note that the material on this blog does presume some background knowledge about machine learning and deep learning. Typically, this would be at the level of having completed a single advanced topic in the area. So if you’re completely new to Machine Learning, we’d recommend that you first check out some of the many online resources like Coursera or Udacity’s many courses on the subject. The code we present will all be written in Python, primarily using the PyTorch deep learning framework, plus NumPy and other standard Python libraries.

A simple illustration

If you’re curious to see how the combination of code/figures looks in these notebooks, here’s an simple example below. This is one of my (Zico’s) favorite examples of a simple dynamical system that produces an interesting picture. This is, of course, not really related to machine learning in any way, it’s just a fun example. If you want to actually see the blog in action, take a look at the next post.

P = np.array([[0, 0], [0.5, 1], [1, 0]])
X = np.zeros((5000,2))
for i in range(X.shape[0]-1):
    X[i+1] = 0.5*X[i] + 0.5*P[int(3*np.random.rand())]
plt.plot(X[:,0], X[:,1], 'b.');

Can you think of why it produces this figure?

We hope you enjoy the blog!