Using Amazon EC2 to speed up matlab optimisation I: Writing a socket interface in Matlab to send / receive the commands

The aim in this tutorial (part 1 of 3) is to have a small program running in Matlab on your computer, which will send off requests to a program (compiled Matlab code) running on an Amazon EC2 server or servers.

I am using sockets to do the communication, and in Matlab I use the free msocket toolbox to do this (you will need to download it and add it to your matlab path). The benefit of this toolbox is that it allows you to send matlab variables between machines. In this case, I send a matlab structure containing the command, and the parameters.

The architecture I use is to have a single server running on every core of the target machine. The server will wait for a connection, once it receives one, it will run the desired program, and return the results.

The source code for the entire program can be found here:
socketserver.m. It will also require the program messagecodes.m


It specifies which port to listen on:


socket = mslisten(port);


Then there is a endless loop that waits for a connection

% Keep listening until a connection is received
sock = -1;
while sock == -1
sock = msaccept(socket,0.0000001);
drawnow;
end


Once a connection has been accepted, a confirmation is returned to the creator:

m.accepted = 1;
mssend(sock,m);


and another loop is started to wait to receive commands:

success = -1;
while success<0 [received,success] = msrecv(sock,0.0000001); drawnow; end

I use a "switch" command to execute the appropriate command (in this example,
there is only one, but there is no reason not to have multiple possible commands).

In this case, it is executing the "decompose" command (an optimisation program I have written).
After running, it sends back the result in the rv variable:


switch received.command
case {codes.decompose}
[time,vel,numsubmovements,method,algorithm] = deal(received.arguments{:});
[rv.best,rv.bestresult,rv.bestfitresult] = ...
decompose(time,vel,numsubmovements,method,algorithm);
mssend(sock,rv);


Once a client has finished running the program, it can close the socket, which is dealt
with by the server as follows:


case {codes.closesocket}
msclose(sock);
break;


The break causes it to leave the innermost while loop, and wait again for a new connection.

In order to use this code on Amazon EC2 (without a license server), it is necessary to first compile it. You will need to have a license for the Matlab compiler (available on the
computer doing the compiling, but not on the one running the final program). Note that you will
need to compile this on a machine similar to the one you are planning on running it on
(e.g., I compiled mine on a 64-bit ubuntu machine). I installed ubuntu as a Virtualbox
image as I don't have a "real" ubuntu machine available.

Then from inside matlab, it is as simple as to run:


mcc -m socket_server


and matlab will compile it for you. If this is the first time using mcc, you may
have to answer some questions. Part 3 described how to upload this server to EC2.

Now for the client. The client has to connect to the server:


sock = msconnect(address,port);


It then can send commands to the server:


m.command = codes.decompose;
m.arguments{1} = time;
m.arguments{2} = vel;
m.arguments{3} = numsubmovements;
m.arguments{4} = method;
m.arguments{5} = algorithm;
success = mssend(sock,m);


It then needs to wait for a result:

[thisrv,success] = msrecv(sock);


Then the return value can be used as desired.

Part 2 continues by explaining how to setup Amazon EC2 to run the server component.

Part 3 of the tutorial will describe an automated way to run many servers and collect the results.