nitrc-ce > NITRC-CE + MIT Starcluster?
Showing 1-3 of 3 posts
Sep 4, 2013 12:09 PM | Erik Ziegler
NITRC-CE + MIT Starcluster?
Hello!
I've been using the NITRC-CE for Compute Cloud AMI to process data on cc2.8xl instances. Everything has worked quite well and it's very impressive. The only issue I have now is that it's a bit tedious to repeatedly launch instances to process large datasets.
It turns out there is already a solution for this known as StarCluster (http://star.mit.edu/cluster/).
StarCluster has it's own "already setup" AMIs ready-to-go with stuff like Sun Grid Engine already configured. I would like to create a NITRC-CE AMI on top of one of the StarCluster AMIs so that I can spawn small clusters of instances more easily. Users could store data on an Elastic Band Storage block and the spawned instances can all grab data from there. I'll be using this mainly for long complicated Nipype pipelines.
I guess what I need is some instruction on how to install and copy the setup from the NITRC-CE AMI onto another Ubuntu image (i.e. the StarCluster AMI). It's easy enough to replicate the installed packages with apt-get, but I'd also like the VNC setup as well.
Any ideas?
I've been using the NITRC-CE for Compute Cloud AMI to process data on cc2.8xl instances. Everything has worked quite well and it's very impressive. The only issue I have now is that it's a bit tedious to repeatedly launch instances to process large datasets.
It turns out there is already a solution for this known as StarCluster (http://star.mit.edu/cluster/).
StarCluster has it's own "already setup" AMIs ready-to-go with stuff like Sun Grid Engine already configured. I would like to create a NITRC-CE AMI on top of one of the StarCluster AMIs so that I can spawn small clusters of instances more easily. Users could store data on an Elastic Band Storage block and the spawned instances can all grab data from there. I'll be using this mainly for long complicated Nipype pipelines.
I guess what I need is some instruction on how to install and copy the setup from the NITRC-CE AMI onto another Ubuntu image (i.e. the StarCluster AMI). It's easy enough to replicate the installed packages with apt-get, but I'd also like the VNC setup as well.
Any ideas?
Sep 8, 2013 01:09 PM | David Kennedy
RE: NITRC-CE + MIT Starcluster?
Thanks for the query. This is a topic we will be actively
trying to assist with in the near future. As we develop some of the
necessary instructions we will make them available.
The NITRC Team
The NITRC Team
Dec 2, 2014 01:12 PM | Nina Preuss
RE: NITRC-CE + MIT Starcluster?
Erik, we posted StarCluster in April. I hope you saw the news
bulletin that but if not, the link is here: (http://www.nitrc.org/frs/?group_id=664&r...)