.png){kind=logo}
Why Your High-End Gen5 Platform Fails to Deliver 56GB/s Performance – What You Need to Know
October 2024, Fremont CA – Have you experienced subpar performance using Gen5 media with your high-end server or workstation platform? You are not alone – it has become a common frustration. Gen5 NVMe SSDs are costly, as is the system required to make use of them. The inability to surpass 50GB/s is disappointing, to put it mildly.
If you are currently experiencing this firsthand, do not despair -solutions are available. The following article shines a light on this widespread issue, and discusses some of the most common fixes and work-arounds.
BIOS Settings and Memory Allocation
Despite the availability of Gen5 media in today’s marketplace, most systems are still shipped with Gen4 storage. As such, many platforms are simply not optimized for Gen5 loadouts by default.
Make sure to consult your platform/motherboard’s user guide for PCIe Gen5 specific settings, as the ideal configuration varies from system to system. PCIe Gen5 connectivity may be limited to specific slots, or even certain types of PCIe devices.
Memory Frequency: Memory frequency settings can have a big impact on performance. 5600 Hz is recommended for Desktops while 4800Hz is ideal for Server and Workstations.
VT-d: The VT-d function associated with some Intel-based server and workstation motherboards may affect transfer performance. If you are experiencing sub-par performance, try disabling the VT-d setting.
NUMA: Although this does affect performance for all systems, some motherboards default NUMA settings may not be ideal for NVMe storage configurations. If the option is present, you may need to manually set the system to operate in NPS4 mode.
Power Management: A simple setting that is often overlooked. Many systems are now shipped optimized for Power Savings. However, this will often compromise performance. If the options are available, make sure to prioritize system performance over efficiency. We will cover this in greater detail in the following section.
System RAM: If you want to make the most of your PCIe Gen5 NVMe storage, you will want to install as much memory as your budget allows for. In addition, the type of memory used, and how this memory is configured can have a major impact on system performance. For example, dual-channel memory (two modules installed into two memory lanes)is often recommended for single CPU configurations as it can optimize bandwidth and improve transfer speeds.
While a dual-stick dual-channel setup is fine for most Desktop and Workstation platforms, some high-end systems may need as many as 10 modules allocated to a CPU to maximize performance.
Optimize Queue Depth and Thread Count for Accurate NVMe Benchmark Performance
If you’ve run performance benchmarks like CrystalDiskMark (CDM) and noticed that your Gen5 x16 storage seems to be underperforming, there’s often a simple explanation; the default test parameters are rarely optimized for NVMe devices. In all likelihood, you will have to make some adjustments manually. The most common are Queue Depth and Thread Count.
Queue Depth: Unlike SAS/SATA storage devices, NVMe media were engineered to execute a huge number of concurrent tasks. The Queue depth of and NVMe SSD (the number of I/O requests a device can handle at one time) is measured in the tens of thousands, compared to tens or hundreds for a SAS or SATA drive.
Thread Count: Additionally, NVMe’s multi-threaded architecture aligns with modern multi-core CPUs, enabling parallel I/O processing across multiple threads (A.K.A tasks or jobs). This approach eliminates bottlenecks inherent in single-threaded protocols like SATA, ensuring benchmarks reflect the true capabilities of NVMe storage. Properly configuring the number of threads (often represented as “workers” or “jobs”) is essential to unlock accurate performance measurements for NVMe storage.
HighPoint publishes benchmark reports and guidelines for their various NVMe products, including Rocket7600 and 1600 series Gen5 AICs and Adapters.
Green Features Limiting Gen5 Performance
High-end platforms, such as those built around AMD’s powerful WRX90 chipset, provide as many as seven PCIe 5.0 x16 slots. However, due to “green” energy saving features, true Gen5 connectivity is often limited to the first two slots (which are typically reserved for the system’s GPUs). In many cases, the remaining slots may only provide Gen4 bandwidth for most PCIe devices.
In some cases, a BIOS update or specific BIOS settings may be available to expands PCIe Gen5 connectivity. However, the most reliable solution is to opt for a PCIe Gen5 NVMe AIC or Adapter with PCIe Switching Technology, such as HighPoint’s Rocket 1600 Switch or Rocket 7600 RAID series. PCIe Switch ICs provide up to x48 lanes of internal bandwidth, which can be directly distributed by the AIC/Adapter as needed. Architecture based on this technology ensures that x4 lanes of Gen5 bandwidth will be made available to each hosted NVMe SSD. More information about PCIe Switching technology is available here.
HighPoint's Rocket 7600 and 1600 Series AICs & Adapters: Guaranteed Gen5 x16 NVMe Storage Performance
If you need true Gen5 Storage performance for your high-end workstation, you will want to consider a HighPoint Rocket 7600 Series AIC RAID controller or Rocket 1600 Series Switch AIC. The unique high-performance PCIe Gen5 Switching Architecture enables both series to deliver up to 64GB/s of transfer bandwidth, random performance up to 7.5 million IOPS, and 60GB/s of real-world transfer speed from a single PCIe slot. In addition, the solutions directly support up to 8 NVMe SSDs, and as many as 32 devices via backplane connectivity.
If you are experiencing lower than expected Gen5 performance, or need guidance with platform-specific settings (such as AMD’s TRX50 and WRX90 based platforms), HighPoint’s team of experts is ready to assist.
About HighPoint Technologies, Inc.
HighPoint Technologies stands at the forefront of storage innovation as the industry's -premier manufacturer of high-performance, high-density NVMe Switch and RAID AIC & Adapter solutions for off-the-shelf x86 AMD and Intel platforms. With a rich history spanning nearly three decades, our dedication to delivering innovative, reliable, and high-performance storage solutions has consistently set us ahead in the marketplace. HighPoint’s NVMe storage solutions are powered by industry-proven PCIe Switching technology, and are designed to address the dynamic requirements of AI/ML/LLM applications, Data Centers, Edge Servers, and high-performance workstations, enabling customers to keep pace with today’s rapidly evolving technology landscape.