Scoping out ARCCA11 September 2014
If you have read the previous posts you will see that they can be quite technical and a couple of readers have mentioned they would like to read about something more relevant to non-technical users. I hope to make this blog accessible to a wide audience and make people aware of some of the work we perform within ARCCA and how we can help researchers get the most from advanced computing.
ARCCA provides services to allow researchers access to supercomputing. A supercomputer can be thought of many desktop PCs connected together (the nodes of the supercomputer) and provide a larger resource which allows advanced computing to take place. For example, my background is weather modelling and performing the calculations on just one computer is not possible due to memory and performance issues (storing information on over 10 million data points can require access to a large amount of memory and require many calculations). Therefore imagine the atmosphere of the globe being divided into chunks so each node can hold a smaller amount of data points and therfore perform the calculation in parallel. This is what makes advanced computing useful to solve this type of problem to get the “high performance”.
High performance vs. high throughput
Weather modelling is a problem where we need to solve a performance issue, i.e. a forecast is required well in advance of the actual weather occurring. There is another type of problem where lots of similar jobs need running at the same time and therefore allow the researcher to have high throughput for their work. In the medical and bio-science field there is a requirement to analyse a large set of gene data which are independent of each other and therefore can be analysed in parallel. This is a “high-throughput” problem.
Order out of chaos
As you can imagine to allow all the work to happen on one system requires a system to schedule the work and therefore our system uses a scheduler to prioritise and allow work to be fairly run on the system (without the system being flooded with jobs and slowing everyone down). Scheduling the work is something that smaller clusters can overlook and therefore not get the most out of their systems. Maintaining the nodes and the scheduler is one of the most important tasks our team performs.
Through the looking glass
To perform the above work requires expertise in programming on supercomputers to take advantage of the many nodes, and also to maintain and install the hardware and software used by the work, this is where ARCCA can help. If you have a research idea or problem which requires lots of computation then please get in contact with us. From the previous blog post on Excel it can be a simple matter of exploring other software to perform the work on our systems, or if you need a substantial amount of computation we can help explore the different options available to you.