Case Study: Evaluating the SSD7540 Suitability for Deployment in Immersion Liquid-Cooled Server Rack Applications
Deploying servers in physically challenging environments, such as regions prone to high temperatures and excessive dust, pose unique challenges for solution providers. Ensuring the longevity, reliability and optimal performance of a server platform in such an environment often calls for specialized computing hardware with custom-built cooling and/or filtration systems. Immersion Cooling technology has proven to be versatile and highly reliable solution for such applications.
Unlike standard rack servers, which rely on air cooling and may exposed to the operating environment, an immersion server’s components are fully submerged in an electrically non-conductive liquid within a sealed chassis. Each rack incorporates a fluid pump and filtration system to regulate internal temperatures and help isolate the server hardware from the outside world.
LiquidCool Solutions: LiquidCool Solutions, a leader in Immersion Cooling technology, was contracted to develop a server solution for large scale power generation company. The solution called for a rack-server platform capable of performing optimally in a hot, dusty environment, equipped with M.2-based NVMe RAID storage. Full liquid immersion would ensure optimal reliability and uptime by fully isolating the server hardware from the working environment. After exploring various NVMe RAID solution manufactures, HighPoint NVMe RAID AICs were selected for in-depth evaluation
HighPoint’s SSD7540: HighPoint’s class-leading SSD7540 RAID AIC was designed to maximize the storage performance and capacity potential of PCIe Gen4 M.2 media for a wide range of server and workstation applications. Designed to simplify integration and enhance serviceability, the AIC’s slim, single-width form factor can host up to 32TB of NVME storage, is compliant with industry-standard x86 computing platforms and tower/rackmount chassis, and can be easily installed into nearly any system with PCIe Gen4 or Gen5 x16 connectivity.
Proven x48 lane PCIe Switching technology enables the AIC to directly manage PCIe resources maximize the performance potential of each hosted M.2 SSD. A full x16 lanes of upstream bandwidth are allocated to the host connection, and x4 lanes of downstream bandwidth to each M.2 port. The end result is 28GB/s of real-world transfer performance, with enhanced signal integrity and minimal latency. This unique feature set made the SSD7540 an excellent candidate for the client storage needs. However, the AIC was developed for conventional air-cooled tower and rackmount chassis, and had not been qualified for platforms designed to operate in extreme environments.
The following case study explores how LiquidCool Solutions validated HighPoint's SSD7540 for the client’s unique solution. The AIC was extensively tested to determine its performance capabilities, reliability, and thermal efficiency in both air-cooled and liquid-cooled server environments.
Feasibility Analysis
Installation and Deployment: Deploying the SSD7540 in immersion-cooled environments involves practical considerations:
Unlike a standard rack server, which relies on conventional cooling fans or liquid-cooled heat exchangers, the computing hardware of an Immersion Cooling platform is fully submerged in dielectric fluid.
The unique properties of the dielectric fluid absorb heat directly from the components, enabling the server to maintain very stable operating conditions.
Although today’s server hardware, including PCIe devices, were specifically designed for platforms that utilize air-cooling, years of testing has revealed that most server components can operate very reliably in immersion servers without modification. However, some components may require minor modification to ensure compatibility with the server’s cooling fluid.
Suitability for a Liquid environment: To prepare the SSD7540 for qualification testing, technicians removed the full-length heat sink from PCB, which was designed for typical air-cooled environment and was determined to be unnecessary for the immersion server. They then removed the rubber padding used to fixate the heatsink to the board surface. Some rubber-based materials may break down when submerged in dielectric fluid; removing such material is key to avoid clogging with the chassis’ cooling mechanisms.
The prepared SSD7540 units were installed into a custom-designed riser with unique support brackets for integration into the immersion server chassis. A series of performance tests were initiated to simulate real-world workloads and verify the AIC’s endurance, reliability and suitability for a liquid operating environment.
The prepared SSD7540 units were installed into a custom-designed riser with unique support brackets for integration into the immersion server chassis.
The SSD7540’s robust construction, server-grade materials, and tidy board design proved to be a boon for a liquid environment. The photo above shows how the SSD7540 was positioned within the immersion chassis.
The AIC’s slim profile ensured there was ample space on all sides of the device, and did not impede fluid circulation within the chassis.
After the hardware had been properly mounted and installed, a series of performance tests were initiated to simulate real-world workloads and verify the AIC’s endurance, reliability and suitability for a liquid operating environment.
Performance Evaluation
The SSD7540 performance evaluation consisted of a series of performance benchmarks designed to simulate the client’s real-world application.
The AIC was tested with two platforms; a “baseline/control” configuration using a standard server that left the standard cooling hardware in place, and a full immersion system designed to match the customer’s projected hardware environment.
The results of each test session would then be compared and contrasted; both transfer and IOPS performance were analyzed, and the SSD7540 was closely examined to check for any damage or degradation.
Test Results
Baseline Performance: The baseline performance of the SSD7540 NVMe AIC Gen4 x16 RAID controller was evaluated under standard air-cooled conditions, using the AIC’s default cooling hardware. Tests were conducted with the SSDs set top operate independently and when configured into a RAID 0, 1 and 10 array. The performance metrics were measured in terms of transfer throughput and thermal performance.
Immersion Cooling Performance: The same metrics were observed while the SSD7540 was installed into the client’s full-immersion server build.
Results: The comparative analysis between the two cooling methods revealed negligible impact on performance. However, the thermal performance improved significantly: the full-immersion platform benefited from a 15% reduction over the baseline system, highlighting the superior cooling capability of LiquidCool Solution’s immersion technology.
Conclusion
LiquidCool Solutions’ full-immersion cooling system facilitates stable operating temperatures, minimizing temperature fluctuations that can affect performance and stability. NVMe media connected to the SSD7540 benefitted from the significantly improved thermal management and the AIC was able to maintain sustained high performance under load in the liquid-cooled environment.
LiquidCool Solutions determined the SSD7540 is capable of operating in a fully submerged server environment, and exhibited no signs of physical or performance degradation. The SSD7540 was incorporated into the finished solution and is now successfully operating in the field .
Deploying the SSD7540 in immersion liquid-cooled server rack applications offers significant advantages in thermal performance, stability, and hardware longevity, making it a viable solution for challenging environments. This case study demonstrates that immersion cooling is not only viable but also advantageous for high-performance computing platforms designed to operate in a high-dust, high-temperature environment.
It would be worth mentioning that the immersion configuration was operated without the SSSD7540’s default Hyper-Cooling hardware. This could be a cost reduction in an immersion version of the SSD7540 and similar RIAD adapters.
Appendix
Benchmark Test Settings: The appendix includes a series of tables and charts that compare/contrast air cooling and immersion cooling across different RAID configurations (RAID 0, 1, and 10).
Air cooling Baseline Performance Results: Legacy Mode (Single Disk), RAID 0, RAID 1, and RAID 10
The following data highlights the baseline performance of the SSD7540 under air cooling, establishing a benchmark for comparison with the immersion cooling results.
Legacy
RAID 0
RAID 1
RAID 10
Immersion Cooling Performance Results: Legacy Mode (single disk), RAID 0, RAID 1, and RAID 10
Detailed results show the performance metrics of the SSD7540 under various cooling conditions, documenting the LiquidCool Solution’s ability to maintain in optimal performance and thermal stability.