Wednesday, December 22, 2010

Using Amazon EC2 to speed up matlab optimisation

I run lots of optimisation programs in Matlab as part of my research. One major problem is that they can be very slow, especially if you have lots of variables. One solution is to use more computer hardware to run the procedure faster. The more the better. Amazon offer their EC2 service, which allows you basically to rent computers by the hour. So rather than running your Matlab software on one computer, you can rent a lot of computers say for a few hours or a day and get the optimisation run much, much quicker.

I received an academic research grant from Amazon (Thanks!) which consisted of $3500 credit for their AWS services (including EC2). Mathworks have published a "white paper" on how to use EC2 with Matlab, but it relies on having available licenses for the instances of Matlab running on the EC2 servers, and those servers being able to access your license server. Here at MACCS the license server has a limited number of licenses, and they are behind the university firewall, so there is no way for the EC2 instances to use them.

My solution was to use the Matlab compiler to compile the optimisation part of my work into a stand-alone component. Then, I will get my computer running matlab to connect to the computer(s) running on EC2, send them commands, and get the results. I chose to do this using a socket interface.

These tutorials, split into three parts, will explain the process I went through (mostly so that I can remember how to do it next time!):

  1. Writing a socket interface in Matlab to send / receive the commands
  2. Setting up an Ubuntu EC2 instance to run the compiled Matlab code
  3. Writing matlab software to communicate with the server.