SeaWulf Wakes Up for the First Time

Friday, April 15, 2016
IACS Staff

The new SeaWulf cluster is now up and running having just been installed this week in the Computing Center at Stony Brook University. This cluster was purchased as a result of a $2M Major Research Instrumentation (MRI) grant from the National Science Foundation (NSF). The award was for $1.4M from NSF with a $300K match being applied internally and a $300K match being awarded from NYSTAR to purchase and operate, for at least four years, a highly-reconfigurable computer system. This highly-extensible system will seed an institutional approach to research computing that will provide great cost savings across SBU and will increase overall quality and functionality of the research cyber infrastructure. The system will deliver a roughly 50-fold increase in capability over the aged original SeaWulf cluster operated by the Applied Mathematics & Statistics Department.

The professional system administration staff are fully integrated with the campus Division of Information Technology (DoIT) and have experience with operating all key technology elements (clusters, Torque+Maui, GPFS, virtualized systems, IB, MPI, etc.). The vendor will provide substantial training for both staff and users in the first year of operations, and the cluster will be used for research as well as provide deep training opportunities for undergraduate and graduate students.

Some of the system's key features are:

  • 156 compute nodes from Penguin, each with two Intel Xeon E5-2683v3 CPUs
  • 8 Additional compute nodes with quad GPUs in each
  • 32 x Nvidia Tesla K80 Accelarators, offering 64x GK210 (K40) Cores (159,744 CUDA cores)
  • CPUs are codenamed “Haswell” and offer 14 cores each, and operate at a base speed of 2.0 Gigahertz. The new SeaWulf cluster has 4,592 of these cores in total and has a peak Double Precision (DP) FLOPS (FLoating-point Operations Per Second) of 240 Teraflops (240,000,000,000,000 FLOPS).
  • Each node has 128 Gigabytes of DDR4 Memory, configured as 8 memory modules, each operating at 2,133 Mega-Transfers, for a combined memory bandwidth per node of 63.5 Gigabytes per second per CPU.
  • The nodes are interconnected via a high-speed InfiniBand®(IB) network by Mellanox ® Technologies operating at 40 Gigabits per second, allowing transfer of ~5 Gigabytes of data each second. 
  • The Storage array is a DDN GPFS storage solution comprised of 180x 6 Terabyte nearline SAS disks IB-attached to two Network Shared Disk (NSD) servers, and 5x 1,600 Gigabyte Solid State Disks (SSD) acting as the metadata pool for the GPFS. This storage system can provide sustained 4k Random Read Input Output Operations per Second (IOPS) over 13,000, and can sustain sequential transfers at over 14 Gigabytes per second.

To watch a short clip of the new SeaWulf cluster being brought online for the first time, visit https://www.youtube.com/watch?v=eCI_P89g2Jc.