Telephone: 0114 2572 200
Email: info@ocf.co.uk

OCF supercomputer speeds up research at the University of Exeter

Researchers from across the University of Exeter are benefitting from a new High Performance Computing (HPC) machine, called Isca. Existing departmental HPC resources within Life Sciences and Physics were coming to the end of life, so using funding from the University and a large grant from the Medical Research Council, the University acquired a new, central core HPC resource to support researchers University-wide across numerous disciplines.

The new system has already been contributing to research into the modelling and formation of stars and galaxies, using Computational Fluid Dynamics (CFD) within Engineering to understand how flooding affects bridges, as well as being used in the Medical School looking at genetic traits in diabetes using data from the UK Biobank. The HPC resource is now in use by more than 200 researchers across 30+ active research projects in Life Sciences, Engineering, Mathematics, Astrophysics, and Computing departments.

As part of the original tender, the University asked for options to provide temporary housing for the new HPC machine whilst work on a new data hall was being finished. High performance compute, storage and data analytics integrator, OCF, proposed a unique solution to house the new HPC machine in a Rapid Deployment Data Centre (RDDC) container solution from Stulz Technology Integration Limited (formerly TSI UK).

Nicknamed The Pod, this is a dedicated HPC-related containerised solution developed by Stulz Technology Integration, which was custom fabricated for the University by the data centre design and build specialists. OCF designed, integrated and configured the HPC machine and had the entire system delivered in its container to the University in July 2016.

“This was phase one of the new supercomputer, located on campus in the specialised container, where the machine ran for the first twelve months,” commented David Barker, Technical Architect at the University of Exeter. “We tested and used the system while it was housed in the temporary location to give us an understanding of what we used a lot of; this informed phase two of the project which was to expand the system with the help of OCF and move it to its final location in the new data centre on campus.”

In addition, OCF and Lenovo jointly worked on the design of the computer to support the differing needs of the life sciences and physics researchers, which required virtualised and traditional environments respectively. The new 6,000 core system is comprised of Lenovo NeXtScale servers with a number of NVIDIA GPU nodes, Intel Xeon Phi processors and additional high memory compute resources to cater to these needs,

Lenovo’s NeXtScale servers are connected through Mellanox EDR Infiniband to three GS7K parallel file system appliances from DDN Storage, providing just under one petabyte of capacity. OCF’s own Open Source HPC Software Stack, based on XCAT, runs on the system along with RDO OpenStack, NICE DCV and Adaptive Computing MOAB.

“As well as having the standard nodes, we also have various pieces of specialist kit which includes NVIDIA GPU nodes, Intel Xeon Phi nodes and OpenStack cloud nodes as well,” commented David. “We wanted to ensure that the new system caters for as wide a variety of research projects as possible, so the system reflects the diversity of the applications and requirements our users have.”

The impact it has had on research has been significant, with researchers seeing 2-3x speed up compared to the previous departmental clusters.

“We’ve seen in the last few years a real growth in interest in High Performance Computing from life sciences, particularly with the availability of new high-fidelity genome sequencers, which have heavy compute requirements, and that demand will keep going up,” comments David Acreman, Senior Research Fellow at the University of Exeter. “Isca is proving to be an incredibly valuable service to the whole university and is now proving indispensible to our research groups.”

Read More »

OCF achieves Elite Partner status with NVIDIA

OCF has successfully achieved Elite Partner status with NVIDIA® for Accelerated Computing, becoming only the second business partner in Northern Europe to achieve this level.

Awarded in recognition of OCF’s ability and competency to integrate a wide portfolio of NVIDIA’s Accelerated Computing products including TESLA® P100 and DGX-1™, the Elite Partner level is only awarded to partners that have the knowledge and skills to support the integration of GPUs, as well as the industry reach to support and attract the right companies and customers using accelerators.

“For customers using GPUs, or potential customers, earning this specialty ‘underwrites’ our service and gives them extra confidence that we possess the skills and knowledge to deliver the processing power to support their businesses,” says Steve Reynolds, Sales Director, OCF plc. “This award complements OCF’s portfolio of partner accreditations and demonstrates our commitment to the vendor.”

OCF has been a business partner with NVIDIA for over a decade and has designed, built, installed and supported a number of systems throughout the UK that include GPUs. Most recently, OCF designed, integrated and configured ‘Blue Crystal 4’, a High Performance Computing (HPC) system at the University of Bristol, which includes 32 nodes with 2 NVIDIA Tesla P100 GPUs accelerators each.

In addition, as a partner of IBM and NVIDIA via the OpenPOWER Foundation, OCF has supplied two IBM® Power Systems™ S822LC for HPC systems, codenamed ‘Minsky’, to Queen Mary University of London (QMUL).

The two systems, which pair a POWER8 CPU with 4 NVIDIA Tesla P100 GPU accelerators, are being used to aid world-leading scientific research projects as well as teaching, making QMUL one of the first universities in Britain to use these powerful deep learning machines. The university was also the first in Europe to deploy a NVIDIA DGX-1 system, described as the world’s first AI supercomputer in a box.

Read More »

OCF deliver new 600 Teraflop HPC machine for University of Bristol

For over a decade the University of Bristol has been contributing to world-leading and life changing scientific research using High Performance Computing (HPC), having invested over £16 million in HPC and research data storage. To continue meeting the needs of its researchers working with complex and large amounts of data, they will now benefit from a new HPC machine, named BlueCrystal 4 (BC4).

Designed, integrated and configured by the HPC, storage and data analytics integrator OCF, BC4 has more than 15,000 cores making it the largest UK University system by core count and a theoretical peak performance of 600 Teraflops.

Over 1,000 researchers in areas such as paleobiology, earth science, biochemistry, mathematics, physics, molecular modelling, life sciences, and aerospace engineering will be taking advantage of the new system. BC4 is already aiding research into new medicines and drug absorption by the human body.

“We have researchers looking at whole-planet modelling with the aim of trying to understand the earth’s climate, climate change and how that’s going to evolve, as well as others looking at rotary blade design for helicopters, the mutation of genes, the spread of disease and where diseases come from,” said Dr Christopher Woods, EPSRC Research Software Engineer Fellow, University of Bristol. “Early benchmarking is showing that the new system is three times faster than our previous cluster – research that used to take a month now takes a week, and what took a week now only takes a few hours. That’s a massive improvement that’ll be a great benefit to research at the University.”

BC4 uses Lenovo NeXtScale compute nodes, each comprising of two 14 core 2.4 GHz Intel Broadwell CPUs with 128 GiB of RAM. It also includes 32 nodes of two NVIDIA Pascal P100 GPUs plus one GPU login node, designed into the rack by Lenovo’s engineering team to meet the specific requirements of the University.

Connecting the cluster are several high-speed networks, the fastest of which is a two-level Intel Omni-Path Architecture network running at 100Gb/s. BC4’s storage is composed of one PetaByte of disk provided by DDN’s GS7k and IME systems running the parallel file system Spectrum Scale from IBM.

Effective benchmarking and optimisation, using the benchmarking capabilities of Lenovo’s HPC research centre in Stuttgart, the first of its kind, has ensured that BC4 is highly efficient in terms of physical footprint, while fully utilising the 30KW per rack energy limit. Lenovo’s commitment to third party integration has allowed the University to avoid vendor lock-in while permitting new hardware to be added easily between refresh cycles.

Dr Christopher Woods continues: “To help with the interactive use of the cluster, BC4 has a visualisation node equipped with NVIDIA Grid vGPUs so it helps our scientists to visualise the work they’re doing, so researchers can use the system even if they’ve not used an HPC machine before.”

Housed at VIRTUS’ LONDON4, the UK’s first shared data centre for research and education in Slough, BC4 is the first of the University’s supercomputers to be held at an independent facility. The system is directly connected to the Bristol campus via JISC’s high speed Janet network. Kelly Scott, account director, education at VIRTUS Data Centres said, “LONDON4 is specifically designed to have the capacity to host ultra high density infrastructure and high performance computing platforms, so an ideal environment for systems like BC4. The University of Bristol is the 22nd organisation to join the JISC Shared Data Centre in our facility, which enables institutions to collaborate and share infrastructure resources to drive real innovation that advances meaningful research.”

Currently numbering in the hundreds, applications running on the University’s previous cluster will be replicated onto the new system, which will allow researchers to create more applications and better scaling software. Applications are able to be moved directly onto BC4 without the need for re-engineering.

“We’re now in our tenth year of using HPC in our facility. We’ve endeavoured to make each phase of BlueCrystal bigger and better than the last, embracing new technology for the benefit of our users and researchers,” commented Caroline Gardiner, Academic Research Facilitator at the University of Bristol.

Simon Burbidge, Director of Advanced Computing comments: “It is with great excitement that I take on the role of Director of Advanced Computing at this time, and I look forward to enabling the University’s ambitious research programmes through the provision of the latest computational techniques and simulations.”

Due to be launched at an event on 24th May at the University of Bristol, BC4 will house over 1,000 system users, carried over from BlueCrystal Phase 3.

Read More »

Supporting scientific research at the Atomic Weapons Establishment

AWE benefiting from new end-to-end IBM Spectrum Scale and POWER8 systems

We are pleased to announce that we are supporting scientific research at the UK Atomic Weapons Establishment (AWE), with the design, testing and implementation of a new HPC, cluster and separate big data storage system.

AWE has been synonymous with science, engineering and technology excellence in support of the UK’s nuclear deterrent for more than 60 years. AWE, working to the required Ministry of Defence programme, provides and maintains warheads for the Trident nuclear deterrent.

The new HPC system is built on IBM’s POWER8 architecture and a separate parallel file system, called Cedar 3, built on IBM Spectrum Scale. In early benchmark testing, Cedar 3 is operating 10 times faster than the previous high-performance storage system at AWE. Both server and storage systems use IBM Spectrum Protect for data backup and recovery.

“Our work to maintain and support the Trident missile system is undertaken without actual nuclear testing, which has been the case ever since the UK became a signatory to the Comprehensive Nuclear Test Ban Treaty (CTBT); this creates extraordinary scientific and technical challenges – something we’re tackling head on with OCF,” comments Paul Tomlinson, HPC Operations at AWE. “We rely on cutting-edge science and computational methodologies to verify the safety and effectiveness of the warhead stockpile without conducting live testing. The new HPC system will be vital in this ongoing research.”

From the initial design and concept to manufacture and assembly, AWE works across the entire life cycle of warheads through the in-service support to decommissioning and disposal, ensuring the maximum safety and protecting national security at all times.

The central data storage, Cedar 3, will be in use for scientists across the AWE campus, with data replicated across the site.

“The work of AWE is of national importance and so its team of scientists need complete faith and trust in the HPC and big data systems in use behind the scenes, and the people deploying the technology,” says Julian Fielden, managing director, OCF. “Through our partnership with IBM, and the people, skills and expertise of our own team, we have been able to deliver a system which will enable AWE maintain its vital research,”

The new HPC system runs on a suite of IBM POWER8 processor-based Power systems servers running the IBM AIX V7.1 and Red Hat Enterprise Linux operating system. The HPC platform consists of IBM Power E880, IBM Power S824L, IBM Power S812L and IBM Power S822 servers to provide ample processing capability to support all of AWE’s computational needs and an IBM tape library device to back up computation data.

Cedar 3, AWE’s parallel file system storage, is an IBM Storwize storage system. IBM Spectrum Scale is in use to enable AWE to more easily manage data access amongst multiple servers.

About the Atomic Weapons Establishment (AWE)
The Atomic Weapons Establishment has been central to the defence of the United Kingdom for more than 60 years through its provision and maintenance of the warheads for the country’s nuclear deterrent. This encompasses the initial concept, assessment and design of the nuclear warheads, through component manufacture and assembly, in-service support, decommissioning and then disposal.

Around 4,500 staff are employed at the AWE sites together with over 2,000 contractors. The workforce consists of scientists, engineers, technicians, crafts-people and safety specialists, as well as business and administrative experts – many of whom are leaders in their field. The AWE sites and facilities are government owned but the UK Ministry of Defence (MOD) has a government-owned contractor-operated contract with AWE Management Limited (AWE ML) to manage the day-to-day operations and maintenance of the UK’s nuclear stockpile. AWE ML is formed of three shareholders – Lockheed Martin, Serco and Jacobs Engineering Group. For further information, visit: http://www.awe.co.uk

Read More »

eMedLab Shortlisted for UK Cloud Award

UK Cloud Awards Finalist logo

Congratulations eMedLab on being shortlisted for the UK Cloud Awards 2017

A solution designed and integrated by OCF has been shortlisted in the 2017 UK Cloud Awards in the ‘Best Public Sector Project’ category.

The MRC eMedLab consortium consists of University College London, Queen Mary University of London, London School of Hygiene & Tropical Medicine, the Francis Crick Institute, the Wellcome Trust Sanger Institute, the EMBL European Bioinformatics Institute and King’s College London and was funded by the Medical Research Council (£8.9M).

The vision of MRC eMedLab is to maximise the gains for patients and for medical research that will come from the explosion in human health data. To realise this potential, the consortium of seven prestigious biomedical research organisations need to accumulate medical and biological data on an unprecedented scale and complexity, to coordinate it, to store it safely and securely, and to make it readily available to interested researchers.

The partnership’s aim was to build a private cloud infrastructure for the delivery of significant computing capacity and storage to support the analysis of biomedical genomics, imaging and clinical data.  Initially, its main focus was on a range of diseases such as cancer, cardiovascular and rare diseases, subsequently broadening it out to include neurodegenerative and infectious diseases.

The MRC eMedLab system is a private cloud with significant data storage capacity and very fast internal networking designed specifically for the types of computing jobs used in biomedical research. The new high-performance and big data environment consists of:

  • Red Hat Enterprise Linux OpenStack Platform
  • Red Hat Satellite
  • Lenovo System x Flex system with 252 hypervisor nodes and Mellanox 10Gb network with a 40Gb/56Gb core
  • Five tiers of storage, managed by IBM Spectrum Scale (formerly GPFS), for cost effective data storage – scratch, Frequently Accessed Research Data, virtual clusters image storage, medium-term storage and previous versions backup.

The project has become a key infrastructure resource for the Medical Research Council (MRC), which has funded six of these projects. The success has been attributed to MRC eMedLab’s concept of partnership working where everybody is using one shared resource. This means not just sharing the HPC resource and sharing it efficiently, but also sharing the learning, the technology and the science at MRC eMedLab. Jacky Pallas, Director of Research Platforms, UCL, comments,From the beginning there was an excellent partnership between the MRC eMedLab operations team and the technical specialists at OCF, working together to solve the issues which inevitably arise when building and testing a novel compute and data storage system.”

In total, there are over 20 different projects running on the MRC eMedLab infrastructure which include:

  • The London School of Hygiene & Tropical Medicine is working on a project looking at population levels and the prevalence of HIV and TB, how the pathogen/bacteria evolve and the genetics of human resistance. This research is done in collaboration with researchers in Africa and Vietnam
  • Francis Crick Institute cancer based science – supporting a project run by Professor Charles Swanton investigating personalised immunotherapies against tumours
  • Great Ormond Street Hospital – collaboration on research on rare diseases in children
  • Linking genomics and brain imaging to better understand dementia
  • Studying rare mitochondrial diseases and understanding how stem cells function
  • Projects using the computing infrastructure use UK Biobank data to identify and improve treatments for cardiovascular diseases
  • Deep mining of cancer genomics data to understand how cancer tumours evolve
  • Analysing or looking at virus genome sequences to enable the modelling and monitoring of infectious flu type epidemic

The MRC eMedLab private cloud has shown that these new computing technologies can be used effectively to support research in the life sciences sector.

Professor Taane Clark, Professor of Genomics and Global Health, London School of Hygiene and Tropical Medicine comments, “The processing power of the MRC eMedLab computing resource has improved our ability to analyse human and pathogen genomic data, and is assisting us with providing insights into infectious disease genomics, especially in malaria host susceptibility, tuberculosis drug resistance and determining host-pathogen interactions.”

Read More »

Virtual HPC Clusters Enable Cancer, Cardio-Vascular and Rare Diseases Research OpenStack based Cloud enables cost-effective self-provisioned compute resources

eMedLab, a partnership of seven leading bioinformatics research and academic institutions, is using a new private cloud, HPC environment and big data system to support the efforts of hundreds of researchers studying cancers, cardio-vascular and rare diseases. Their research focuses on understanding the causes of these diseases and how a person’s genetics may influence their predisposition to the disease and potential treatment responses.

The new HPC cloud environment combines a Red Hat Enterprise Linux OpenStack Platform with Lenovo Flex System hardware to enable the creation of virtual HPC clusters bespoke to individual researchers’ requirements. The system has been designed, integrated and configured by OCF, an HPC, big data and predictive analytics provider, working closely with its partners Red Hat, Lenovo, Mellanox Technologies and in collaboration with eMedlab’s research technologists.

The High Performance Computing environment is being hosted at a shared data centre for education and research, offered by digital technologies charity Jisc. The data centre has the capacity, technological capability and flexibility to future-proof and support all of eMedLab’s HPC needs, with its ability to accommodate multiple and varied research projects concurrently in a highly collaborative environment. The ground-breaking facility is focused on the needs of the biomedical community and will revolutionise the way data sets are shared between leading scientific institutions internationally.

The eMedLab partnership was formed in 2014 with funding from the Medical Research Council. Original members University College London, Queen Mary University of London, London School of Hygiene & Tropical Medicine, the Francis Crick Institute, the Wellcome Trust Sanger Institute and the EMBL European Bioinformatics Institute have been joined recently by King’s College London.

“Bioinformatics is a very, very data intensive discipline,” says Jacky Pallas, Director of Research Platforms, University College London. “We want to study a lot of de-identified, anonymous human data. It’s not practical – from data transfer and data storage perspectives – to have scientists replicating the same datasets across their own, separate physical HPC resources, so we’re creating a single store for up to 6 Petabytes of data and a shared HPC environment within which researchers can build their own virtual clusters to support their work.”
The Red Hat Enterprise Linux OpenStack Platform, a highly scalable Infrastructure-as-a-Service [IaaS] solution, enables scientists to create and use virtual clusters bespoke to their needs, allowing them to select compute memory, processors, networking, storage and archiving policies, all orchestrated by a simple web-based user-Interface. Researchers will be able access up to 6,000 cores of processing power.

“We generate such large quantities of data that it can take weeks to transfer data from one site to another,” says Tim Cutts, Head of Scientific Computing, the Wellcome Trust Sanger Institute. “Data in eMedLab will stay in one secure place and researchers will be able to dynamically create their own virtual HPC cluster to run their software and algorithms to interrogate the data, choosing the number of cores, operating system and other attributes to create the ideal cluster for their research.”

Tim adds: “The Red Hat Enterprise Linux OpenStack Platform enables our researchers to do this rapidly and using open standards which can be shared with the community.”

Arif Ali, Technical Director of OCF says: “The private cloud HPC environment offers a flexible solution through which virtual clusters can be deployed for specific workloads. The multi-tenancy features of the Red Hat platform enable different institutions and research groups to securely co-exist on the same hardware, and share data when appropriate.”

“This is a tremendous and important win for Red Hat,” says Radhesh Balakrishnan, general manager, OpenStack, Red Hat. “eMedLab’s deployment of Red Hat Enterprise Linux OpenStack Platform into its HPC environment for this data intensive project further highlights our leadership in this space and ability to deliver a fully supported, stable, and reliable production-ready OpenStack solution.

Red Hat technology allows consortia such as eMedLab to use cutting edge self-service compute, storage, networking, and other new services as these are adopted as core OpenStack technologies, while still offering the world class service and support that Red Hat is renowned for. The use of Red Hat Enterprise Linux OpenStack Platform provides cutting edge technologies along with enterprise-grade support and services; leaving researchers to focus on the research and other medical challenges.”

“Mellanox end-to-end Ethernet solutions enable cloud infrastructures to optimize their performance and to accelerate big data analytics,” said Kevin Deierling, vice president of marketing at Mellanox Technologies. “Intelligent interconnect with offloading technologies, such as RDMA and cloud accelerations, is key for building the most efficient private and cloud environments. The collaboration between the organisations as part of this project demonstrates the power of the eco-systems to drive research and discovery forward.”
The new high-performance environment and big data environment consists of:
 Red Hat Enterprise Linux OpenStack Platform
 Red Hat Satellite
 Lenovo System x Flex system with 252 hypervisor nodes and Mellanox 10Gb network with a 40Gb/56Gb core
 Five tiers of storage, managed by IBM Spectrum Scale (formerly GPFS), for cost effective data storage – scratch, Frequently Accessed Research Data, virtual clusters image storage, medium-term storage and previous versions backup.

Read More »

BlueBEAR at the University of Birmingham

A team of researchers based at The University of Birmingham is working on ground-breaking research to create a proton Computed Tomography (CT) image that will help to facilitate treatment of cancer patients in the UK. Proton therapy targets tumours very precisely using a proton beam and can cause less damage to surrounding tissue than conventional radiotherapy – for this reason it can be beneficial treatment for children.

Generally reliant on X-rays to image the body’s composition and healthy tissue location before treatment, this research is hoping to simulate use of actual protons – not X-rays – to image the body – and in doing so improve accuracy of the final treatment. It forms part of a larger research project set up to build a device capable of delivering protons in this way in a clinical setting.

Working for the PRaVDA Consortium, a three-year project funded by the Wellcome Trust* and led by researchers at the University of Lincoln, the team of researchers are using The University of Birmingham’s centrally funded High Performance Computing (HPC) service, BlueBEAR, to simulate the use of protons for CT imaging. The team hopes to simulate 1000 million protons per image over the course of the project, and will do so 97 per cent faster than on a desktop computer. A test simulation of 180 million protons, which would usually take 5400 hours without the cluster, has already been delivered in 72 hours / 3-days.

The research team is tasked with proving the principle that a 10cm proton CT image, similar in size to a child’s head, can be created. In doing so, it will be the largest proton CT image ever created.

Dr Tony Price, PRaVDA research fellow, says, “The research will give us a better understanding of how a proton beam interacts with the human body, ultimately improving the accuracy of proton therapy. The HPC service at The University of Birmingham is essential for us to complete our research, as it gives us the necessary capacity to simulate and record the necessary number of histories to create an image. It took us only three days to run a simulation of 180 million protons which would usually take 5400 hours without the cluster.”

The BlueBEAR HPC service in use by the PRaVDA Consortium was designed, built and integrated in 2013 by HPC, data management, storage and analytics provider OCF. Due to the stability and reliability of the core service, researchers have invested in expanding this service with nodes purchased from their own grants and integrated into the core service on the understanding that these nodes will be available for general use when not required by the research group.

This has expanded the capacity of the centrally-funded service by 33 per cent, showing the confidence that the researchers have in the core service. The service is used by researchers from the whole range of research areas at the University, from the traditional HPC users in the STEM (Science, Technology, Engineering and Mathematics) disciplines to non-traditional HPC users such as Psychology and Theology.

Paul Hatton, HPC & Visualisation Specialist, IT Services, The University of Birmingham says, “The HPC service built by OCF has proven over the past two years to be of immense value to a multitude of researchers at the University. Instead of buying small workstations, researchers are using our central HPC service because it is easy for them to buy and add their own cores when required.

We work closely with OCF to encourage new users onto the service and provide a framework for users requesting capacity. The flexible, scalable and unobtrusive design of the high performance clusters has made it easy for us to scale up our HPC service according to the increase in demand.”

Technology

  • The server clusters uses Lenovo System x iDataPlex® with Intel Sandy Bridge processors. OCF has installed more high performance server clusters using the industry-leading Lenovo iDataPlex server than any other UK integrator.
  • The server clusters also uses IBM Tivoli Storage Manager for data back up and IBM GPFS software which enables more effective storage capacity expansion, enterprise wide, interdepartmental file sharing, commercial-grade reliability, cost-effective disaster recovery and business continuity.
  • The scheduling system on BlueBEAR is Adaptive Computing’s MOAB software, which enables the scheduling, managing, monitoring, and reporting of HPC workloads.
  • Use of Mellanox’s Virtual Protocol Interconnect (VPI) cards within the cluster design would make it easier for IT Services to redeploy nodes between the various components of the BEAR services depending on changing workloads should the demand arise.

 

Read More »

CLIMB: World’s largest single system dedicated to Microbial Bioinformatics research

World’s largest single system dedicated to Microbial Bioinformatics research

Bioinformatics researchers from across the UK and international collaborators will soon be able to benefit from a new private cloud HPC system to support their work on bacterial pathogens. The Cloud Infrastructure for Microbial Bioinformatics (CLIMB) project, a collaboration between the University of Birmingham, the University of Warwick, Cardiff University, and Swansea University, will create a free-to-use, world leading cyber infrastructure specifically designed for microbial bioinformatics research.

With over 7,500 vCPU cores of processing power, the CLIMB system represents the largest single system designed specifically for microbiologists in the world.

Dr. Thomas Connor, Senior Lecturer at the Cardiff School of Biosciences, Cardiff University, who designed the system with integration partner OCF says: “Bioinformatics research using the system is already helping to track viral and bacterial pathogens, develop new diagnostics, increase the understanding of bacterial resistance to antibiotics, and support many other related research projects. Using CLIMB we are able to overcome the difficulties we face when trying to perform our analyses on HPC infrastructure that is generally not suitable for microbial bioinformatics workloads.”

Powered by OpenStack cloud computing software, and provided by HPC, big data and predictive analytics provider OCF, one site’s system, at the University of Birmingham, is already in production running OpenStack Juno, and will soon be linked directly to the system at Cardiff. The final two sites are currently undergoing final testing by OCF before entering production this Autumn. CLIMB is already contributing to bioinformatics research both nationally and internationally, and research using the system has already been published in international scientific journals.

Simon Thompson, research computing specialist at the University of Birmingham, who built the initial CLIMB pilot system says “this is one of the most complex environments we’ve had to build, learning a lot of new technologies. Luckily we’ve been able to work with our contacts at IBM, Lenovo and OCF to help us build what is now quite a stable platform in Birmingham”.

To provide users with local high performance compute and storage, the fully open source OpenStack system is built using Lenovo System X servers connected to IBM Spectrum Scale storage connected through 56GB Infiniband – providing 500TB local storage at each of the four sites.

Using a set of standardised Virtual Machine images, the CLIMB system can spin up over 1,000 VMs at any one time enabling individuals and groups of researchers to take advantage of the system’s 78TBs of RAM. The system has been designed to provide large amounts of RAM in order to meet the challenge of processing large, rich, biological datasets and the system will be able to support the vast majority of the UK microbiology community.

“We are now able to rapidly take multiple bacterial samples and generate DNA sequence data for each sample in a computationally readable format. However, handling hundreds, or thousands of these samples simultaneously poses serious challenges for most HPC systems,” says Dr. Connor. “Datasets within microbiology are very different to traditional HPC research. Workloads are often either embarrassingly parallel or very high memory, and all require large amounts of high performance storage. CLIMB has been specifically designed to take this into account, to enable the microbial bioinformatics community within the UK to access facilities they are unlikely to have available locally.”

Dr. Connor continues: “One of the issues that is becoming increasingly faced by bioinformaticians is the ability to share bespoke software applications. What may work on one HPC system, won’t necessarily work on another – it’s often quicker to write a new application from scratch. We believe that by using containers/Virtual Machines this problem can be overcome, by creating a mechanism to share software and data within a single cloud environment. By enabling researchers to share their software and data in this way, we free up bioinformaticians to spend more time doing research, and less on installing software and downloading data from multiple, disparate data repositories.”

“The fully open source OpenStack system enables us to work with the team at CLIMB to help modify and improve the system,” comments Arif Ali, Technical Director at OCF. “CLIMB really is a leading edge, private cloud solution and using OpenStack means we can stay on top of the latest developments in the system as well as modify it to fit the needs of researchers. On top of this, the research community can also contribute to the system meaning that the solution can fit all of their research needs.”

OCF will continue to work with the institutes in CLIMB to manage the open source OpenStack system and is currently upgrading the systems installed at Warwick and Swansea to OpenStack Kilo.

Read More »

New HPC cluster benefitting University of Oxford

New HPC cluster benefitting University of Oxford

Researchers from across the University of Oxford will benefit from a new High Performance Computing system designed and integrated by OCF. The new, Advanced Research Computing (ARC) central HPC resource is supporting research across all four Divisions at the University; Mathematical, Physical and Life Sciences; Medical Sciences; Social Sciences; and Humanities.

With around 120 active users per month, the new HPC resource will support a broad range of research projects across the University. As well as computational chemistry, engineering, financial modeling, and data mining of ancient documents, the new cluster will be used in collaborative projects like the T2K experiment using the J-PARC accelerator in Tokai, Japan. Other research will include the Square Kilometer Array (SKA) project, and anthropologists using agent-based modeling to study religious groups. The new service will also be supporting the Networked Quantum Information Technologies Hub (NQIT), led by Oxford, envisaged to design new forms of computers that will accelerate discoveries in science engineering and medicine.

The new HPC cluster comprises of Lenovo NeXtScale servers with Intel Haswell CPUs connected by 40GB Infiniband to an existing Panasas storage system. The storage system was also upgraded by OCF to add 166TBs giving a total of 400TBs of capacity. Existing Intel Ivy Bridge and Sandy Bridge CPUs from the University of Oxford’s older machine are still running and will be merged with the new cluster.
20 NVIDIA Tesla K40 GPUs were also added at the request of NQIT, who co-invested in the new machine. This will also bring benefit to NVIDIA’s CUDA Centre of Excellence, which is also based at the University.

“After seven years of use, our old SGI-based cluster really had come to end of life, it was very power hungry, so we were able to put together a good business case to invest in a new HPC cluster,” said Dr Andrew Richards, Head of Advanced Research Computing at the University of Oxford. “W e can operate the new 5,000 core machine for almost exactly the same power requirements as our old 1,200 core machine.

The new cluster will not only support our researchers but will also be used in collaborative projects as well; we’re part of Science Engineering South, a consortium of five universities working on e-infrastructure particularly around HPC.

We also work with commercial companies who can buy time on the machine so the new cluster is supporting a whole host of different research across the region.”

Simple Linux Utility Resource Manager (SLURM) job scheduler manages the new HPC resource, which is able to support both the GPUs and the three generations of Intel CPUs within the cluster.

Julian Fielden, Managing Director at OCF comments: “W ith Oxford providing HPC not just to researchers within the University, but to local businesses and in collaborative projects, such as the T2K and NQIT projects, the SLURM scheduler really was the best option to ensure different service level agreements can be supported. If you look at the Top500 list of the World’s fastest supercomputers, they’re now starting to move to SLURM. The scheduler was specifically requested by the University to support GPUs and the heterogeneous estate of different CPUs, which the previous TORQUE scheduler couldn’t, so this forms quite an important part of the overall HPC facility.”

The University of Oxford will be officially unveiling the new cluster, named Arcus Phase B, on 14th April. Dr Richards continues: “As a central resource for the entire University, we really see ourselves as the first stepping stone into HPC. From PhD students upwards i.e. people that haven’t used HPC before – are who we really want to engage with. I don’t see our facility as just running a big machine; we’re here to help people do their research. That’s our value proposition and one that OCF has really helped us to achieve.”

Read More »

Imperial College London uses new Half PB storage to support research and e-learning

Imperial College London uses new Half PB storage to support research and e-learning

A new highly-scalable storage system is supporting staff, students and researchers across multiple departments at Imperial College London. Designed and implemented by OCF, the centrally-funded half PetaByte system enables secure, cost-effective storage of vast amounts of research data produced at the College, whilst underpinning a Panopto video capture system that records lectures from up to 100 separate theatres on a daily basis.

The new Linux-based, IBM GPFS storage system sits across two separate data centres. It replaces a legacy Microsoft solution that had grown on an ad-hoc basis and become increasingly difficult to grow and administer.

As well as long-term, cost-effective, secure storage requirements of research data, e-learning is a major area of focus, so the College needed a solution that was capable of storing captured and encoded video recordings of lectures, making them available for editing and viewing by potentially 20,000 students. Multiple lectures can be recorded in parallel on the system, then saved and encoded with a turnaround time of less than one day from raw video capture to encoded version. Uptime was also a key factor – the College wanted a solution that would remain online 24/7 to support students.

The new system, which comprises of 8x IBM x3650 M4 GPFS clustered storage and NAS servers, along with 4x IBM Storwize v3700 storage arrays provides increased uptime, greater resilience, and is more responsive, incorporating a number of disaster recovery features to protect against data loss. In the event of data loss or corruption, users can also recover previous data copies from up to 14 days themselves; saving the user time and reducing the burden on IT administrators significantly.

“We wanted a system that could grow and scale more easily and protect large data sets more effectively and efficiently,” said Steven Lawlor, ICT Datacentre Services Manager at Imperial College London. “Our old system no longer scaled and was becoming a bit of a beast. The new system will be easier to expand and more cost effective, in part because we can now move data to tape for long-term storage, a key requirement for certain types of research data. Uptime is one of the most important factors, we wanted a system that could also be maintained and expanded whilst still running.”

Julian Fielden, Managing Director of OCF comments: “We’re seeing a massive explosion of data growth. Academia is feeling the pressure of keeping up with storage demands, especially around research data. Imperial College London’s real challenge was around storing the lecture capture side. Research data can be broken up and separated, whereas its video capture system was one large system across the College. By developing a scalable storage solution based on GPFS they are now able to grow and expand storage in a simple and cost-effective manner.”

Imperial College London has already migrated all of its video content to GPFS. It is currently migrating research datasets from the Microsoft environment to the new Linux-based storage system.

Read More »

Recent Comments

    October 2017
    Monday Tuesday Wednesday Thursday Friday Saturday Sunday
    25th September 2017 26th September 2017 27th September 2017 28th September 2017 29th September 2017 30th September 2017 1st October 2017
    2nd October 2017 3rd October 2017 4th October 2017 5th October 2017 6th October 2017 7th October 2017 8th October 2017
    9th October 2017 10th October 2017 11th October 2017 12th October 2017 13th October 2017 14th October 2017 15th October 2017
    16th October 2017 17th October 2017 18th October 2017 19th October 2017 20th October 2017 21st October 2017 22nd October 2017
    23rd October 2017 24th October 2017 25th October 2017 26th October 2017 27th October 2017 28th October 2017 29th October 2017
    30th October 2017 31st October 2017 1st November 2017 2nd November 2017 3rd November 2017 4th November 2017 5th November 2017

    Contact Us

    HEAD OFFICE:
    OCF plc
    Unit 5 Rotunda, Business Centre,
    Thorncliffe Park, Chapeltown,
    Sheffield, S35 2PG

    Tel: +44 (0)114 257 2200
    Fax: +44 (0)114 257 0022
    E-Mail: info@ocf.co.uk

    SUPPORT DETAILS:
    OCF Hotline: 0845 702 3829
    E-Mail: support@ocf.co.uk
    Helpdesk: support.ocf.co.uk

    DARESBURY OFFICE:
    The Innovation Centre, Sci-Tech Daresbury,
    Keckwick Lane, Daresbury,
    Cheshire, WA4 4FS

    Tel: +44 (0)1925 607 360
    Fax: +44 (0)114 257 0022
    E-Mail: info@ocf.co.uk

    OCF plc is a company registered in England and Wales. Registered number 4132533. Registered office address: OCF plc, 5 Rotunda Business Centre, Thorncliffe Park, Chapeltown, Sheffield, S35 2PG

    Website Designed & Built by Grey Matter | web design sheffield