Durham University adds server power and expands storage to 1PB to support world-leading cosmology research
The world-leading Institute for Computational Cosmology (ICC), based at Durham University, has upgraded its server and storage cluster, COSMA4. The new cluster, now with 1.1 PetaByte of storage and 2992 cores, will support essential new research including two significant projects* into galaxy formation and the fate of gas outside of galaxies. Along with exciting the general public and helping people understand their place in the cosmos, the research helps raise the profile of science in general and serves as an important factor in motivating young people to become scientists.
The upgrade was carried out in partnership with data processing, management and storage integrator, OCF. The Institute for Computational Cosmology (ICC) is a leading international centre for research into the origin and evolution of the Universe. It addresses some of the most fundamental questions in science.
Originally launched in February 2011, the IT team within the ICC has expanded the storage capacity of COSMA4 from 612 TeraBytes (TB), which was already half full from current research projects, to 1.1 PetaBytes (PB). The new storage capacity is necessary because collectively, both new research projects will require 150-200 TBs of storage capacity. The new storage capacity will also support other, on-going research projects.
• The enhanced storage cluster will enable up to 50 researchers, working collaboratively, to access and review data simultaneously.
• Researchers will also be able to access data up to seven years old.
• Operating through a single management system, the IBM General Parallel File System (GPFS)™, the IT team at Durham can manage data effectively – making data available, moving some old data to tape archive or even deleting data if unrequired. IBM GPFS is a high-performance, extremely scalable clustered file system.
“Our researchers are sitting on a lot of data,” says Dr Lydia Heck, Senior Computer Manager
Institute for Computational Cosmology, Department of Physics, Durham University. “With hierarchical storage management and using the full capabilities of IBM GPFS, we can learn to manage data better. Some data can be shifted. Lots of data needs tidying up. Older projects might have to be archived or deleted. Storage is too expensive to use it poorly. IBM GPFS is helping to keep our costs down.”
Durham University has also updated its IBM System x® server cluster. It has expanded its core count by 288 to 2992 cores to support both new research projects and other, smaller research projects.
Heck adds: “Just one of our new research projects will consume the whole processing power of COSMA4 for half of a year. 288 extra cores are very welcome because we have other smaller projects, which can be serviced on the side.”
OCF is responsible for the design, delivery, rapid four-week install and management of the cluster. OCF is also providing training and system support services to Durham University on the new server and storage cluster. The server and storage cluster upgrade includes IBM System x iDataPlex® dx360 M3 server with 48 Intel X5650 6C processors and one IBM System Storage® DCS3700 with 180 2TB disks. This is one of the first deployments of IBM DCS3700 storage technology in the world. Additional capacity was also added to two existing IBM System Storage DS3512 storage arrays on site with 10 additional Expansion Arrays containing 120 2TB SATA HDDs.
The aim of the first project, Eagle, is to model galaxy formation in a region with a size today of 300 million light years across from a time soon after the Big Bang until the present day. This volume is large enough to contain thousands of galaxies as big as our own, as well as rarer structures such as clusters of galaxies. Having a large sample of different objects makes it possible to make meaningful comparisons between the properties of the model galaxies and real galaxies. Data from the simulation is saved at regular intervals through cosmic time allowing researchers to piece together not only what the model galaxies look like today, but how they were assembled. The Eagle simulation will generate hundreds of TeraBytes of data over a period of several months. This data is the legacy of the project and will be used for many research projects for years to come.
The second project, Millgas, will generate approximately another 100 TeraBytes of data. The focus of this project is to model the gas outside of galaxies, particularly in clusters and groups of galaxies. In the real Universe this gas can be observed in great detail using space-based x-ray telescopes. The properties of this intracluster gas are determined in part by outflows of matter and energy from galaxies as they form. The Millgas project will allow researchers to explore the interactions between galaxies and the gas around them.