Skip to content

Storage Hardware Overview

The Devana HPC cluster is equipped with a multi-tiered storage architecture designed to balance speed, capacity, and resilience. This document provides an overview of the storage hardware available to users, detailing configurations for different storage tiers. If you are looking for filesystem overview please refer to Storage Overview.

Home Storage Configuration

The Home storage is intended for user-specific data, including configuration files, scripts, and small datasets. It offers a stable and backed-up environment for storing essential user files.

2x ThinkSystem SR630 V2(1U):

  • 2x 3rd Gen. Intel® Xeon® Scalable Processors Gold 6338: 2.00GHz/32-core/205W
  • 16x 16GB DDR4-3200MHz 8 ECC bits
  • 2x 480GB M.2 SATA SSD (HW RAID 1)
  • 2x InfiniBand ThinkSystem Mellanox ConnectX-6 HDR/200GbE, QSFP56 1-port, PCIe4
  • 1x Ethernet adapter ConnectX-6 10/25 Gb/s Dual-port SFP28, PCIe4 x16, MCX631432AN-ADAB
  • 2x ThinkSystem 430-16e SAS/SATA 12Gb HBA
  • 2x 1100W Power Supply
  • OS CentOS Linux 7

server_top

Fig. 1. Top view of ThinkSystem SR630 V2(1U) unit

1x DE6000H 4U60 Controller SAS:

  • 32GB controller cache
  • 100x 3,5" 8TB 7200RPM 256MB SAS 12Gb/s
  • RAID level 6
  • 8x 12 Gb SAS host ports
  • 1x 1 GbE port (UTP, RJ-45) per controller for out-of-band management

controller_back

Fig. 2. Back view of DE6000H 4U60 Controller SAS unit

Projects Storage Configuration

The Projects storage provides a shared environment for research groups to collaborate. This space is intended for storing shared datasets, project-specific files, and software used in research activities.

2x ThinkSystem SR630 V2(1U):

  • 2x 3rd Gen. Intel® Xeon® Scalable Processors Gold 6338: 2.00GHz/32-core/205W
  • 16x 16GB DDR4-3200MHz 8 ECC bits
  • 2x 480GB M.2 SATA SSD (HW RAID 1)
  • 2x InfiniBand ThinkSystem Mellanox ConnectX-6 HDR/200GbE, QSFP56 1-port, PCIe4
  • 1x Ethernet adapter ConnectX-6 10/25 Gb/s Dual-port SFP28, PCIe4 x16, MCX631432AN-ADAB
  • 2x ThinkSystem 430-16e SAS/SATA 12Gb HBA
  • 2x 1100W Power Supply
  • OS CentOS Linux 7

server_top

Fig. 3. Top view of ThinkSystem SR630 V2(1U) unit

1x DE4000F 2U24 SFF All Flash Storage Array:

  • 64GB controller cache
  • 24x 2,5" 15.36TB PCIe 4.0 x4/dual port x2
  • RAID level 6
  • 8x 12 Gb SAS host ports (Mini-SAS HD, SFF-8644) (4 ports per controller)
  • 8x 10/25 Gb iSCSI SFP28 host ports (DAC or SW fiber optics [LC]) (4 ports per controller)
  • 8x 8/16/32 Gb FC SFP+ host ports (SW fiber optics [LC]) (4 ports per controller
  • 1x 1 GbE port (UTP, RJ-45) per controller for out-of-band management

scratch_front

Fig. 4. Front view of DE4000F 2U24 unit

scratch_back

Fig. 5. Back view of DE4000F 2U24 unit

Scratch Storage Configuration

The Scratch storage is designed for temporary, high-performance access to large data files. It is optimized for active computations and is not backed up, meaning users should move critical results to more permanent storage after job completion.

2x ThinkSystem SR630 V2(1U):

  • 2x 3rd Gen. Intel® Xeon® Scalable Processors Gold 6338: 2.00GHz/32-core/205W
  • 16x 16GB DDR4-3200MHz 8 ECC bits
  • 2x 480GB M.2 SATA SSD (HW RAID 1)
  • 2x InfiniBand ThinkSystem Mellanox ConnectX-6 HDR/200GbE, QSFP56 1-port, PCIe4
  • 1x Ethernet adapter ConnectX-6 10/25 Gb/s Dual-port SFP28, PCIe4 x16, MCX631432AN-ADAB
  • 2x ThinkSystem 430-16e SAS/SATA 12Gb HBA
  • 2x 1100W Power Supply
  • OS CentOS Linux 7

server_top

Fig. 6. Top view of ThinkSystem SR630 V2(1U) unit

1x DE4000F 2U24 SFF All Flash Storage Array:

  • 64GB controller cache
  • 24x 2,5" 15.36TB PCIe 4.0 x4/dual port x2
  • RAID level 5
  • 8x 12 Gb SAS host ports (Mini-SAS HD, SFF-8644) (4 ports per controller)
  • 8x 10/25 Gb iSCSI SFP28 host ports (DAC or SW fiber optics [LC]) (4 ports per controller)
  • 8x 8/16/32 Gb FC SFP+ host ports (SW fiber optics [LC]) (4 ports per controller
  • 1x 1 GbE port (UTP, RJ-45) per controller for out-of-band management

scratch_front

Fig. 7. Front view of DE4000F 2U24 unit

scratch_back

Fig. 8. Back view of DE4000F 2U24 unit

Work Storage Configuration

The Work storage consists of local NVMe SSDs on each compute node, offering fast I/O operations for temporary files during job execution. This space is not persistent, and all data is removed after a job completes.

The capacity of the /work directory depends on the node number. Nodes numbered 001 - 049 and 141-146 have a 3.5TB /work capacity, while the remaining nodes have a 1.8TB capacity. There are no user or project quotas on the /work mountpoint.

The technical specifications of individual computing nodes can be found in the following table.

Feature n[001-048] n[049-140]
Processor 2x Intel Xeon Gold 6338 CPU @ 2.00 GHz ⁠
RAM 256 GB DDR4 RAM @ 3200 MHz ⁠
Disk 3.84 TB NVMe SSD @ 5.8 GB/s , 362 kIOPs 1.92 TB NVMe SSD @ 2.3 GB/s, 166 kIOPs
Network 100Gb/s HDR Infiniband ⁠
Created by: Marek Štekláč, Martin Blaško