Micron & GIGABYTE CXL Workload Demo

As technology advances, high-performance computing (HPC) and artificial intelligence (AI)-based services are increasingly integrated into everyday life. When discussing the enhancement of computational performance, attention is frequently directed toward CPUs and GPUs due to their significant processing capabilities. However, memory is equally essential to overall system performance, despite often receiving less recognition for its critical contributions.

Memory is where your computer temporarily stores the data it’s actively working on. The more memory you have—and the faster it is—the better your system can handle large, complex tasks. But here’s the problem: traditional memory technologies like DRAM (the kind found in most computers) are hitting limits. You can only fit so much of it on a motherboard, and high-capacity modules are expensive.

CXL is a new technology that enables computers to increase memory capacity by using PCIe connections, which are also used for devices like graphics cards and SSDs. While CXL memory may not match the speed of primary system memory, it can provide additional resources that support overall system performance.

This allows a system’s memory to be expanded beyond the physical limitations of the motherboard, providing a more cost-effective and adaptable solution.

Micron and GIGABYTE teamed up to run a series of real-world tests using GIGABYTE’s R284-A91-AAL3 CXL server, Micron’s CZ122 CXL memory expansion modules, DDR5 RDIMMs, and NVMe SSDs. The demo components are as follows:

We divided the tests into three categories, each highlighting a distinct benefit from CXL for clarity:
- CXL Memory Bandwidth Expansion
- CXL Memory Capacity Expansion
- CXL Cost Effectiveness

Before evaluating the benchmarks, it is essential to consider a significant challenge related to CXL: its comparatively lower performance relative to direct-attached memory. Unlike conventional DRAM, which interfaces directly with the CPU via dedicated memory channels, CXL memory operates across the PCIe interface. This indirect connectivity results in increased latency, causing the CPU to require more time to access data stored in CXL memory as opposed to DRAM.

To capitalize on the additional CXL memory, a technique known as Software-Based Weighted Interleaving has been employed. This method efficiently balances data allocation between DRAM and CXL memory. To evaluate its effectiveness, the team utilized Intel’s Memory Latency Checker (MLC)—a tool designed to assess memory bandwidth and latency across varying workloads—and conducted microbenchmark tests using different read/write patterns and memory distribution ratios between DRAM and CXL.

Each test used a weighted interleaving approach, where memory pages were split between DRAM and CXL based on user-defined weights. For example:

• A weight of 3:1 (DRAM:CXL) means 75% of memory traffic goes to DRAM, and 25% to CXL.

Imagine you’re driving from one city to another. There are four fast highway lanes (DRAM), but they’re getting crowded. Now you add a few slower side roads (CXL). Interleaving decides how to divide the traffic between them, so everything flows better. “Weighted” decides how much traffic takes the highway and how much traffic takes the side roads. It’s not just about speed—it’s about smart traffic control.

The weighted-interleaving feature, introduced in Linux kernel 6.9+, allows fine-grained control over memory allocation between DRAM and CXL memory. This enables optimized bandwidth utilization by assigning memory pages based on workload characteristics.

With this setup, data flow can be optimized even though DRAM and CXL have different latency and bandwidth.

The following section examines the effects of introducing CXL into practical workloads.

In applications that are intensive in memory usage, such as high-performance computing (HPC) and artificial intelligence (AI), increasing memory bandwidth can significantly enhance performance. This outcome was clearly observed in our results.

Performance across all four tested workloads increased by 22% to 33%, with the geometric mean of 28% increase across all HPC & AI workloads.

One advantage of CXL is its capacity to expand memory resources beyond the limitations of motherboard slots or the cost constraints of high-capacity DIMMs.

DuckDB, an analytical database engine, was used to test two benchmark suites: TPC-H and TPC-DS. TPC-H evaluates analytical queries on a simplified schema, while TPC-DS is a more complex benchmark designed to represent real-world retail database workloads with mixed query types.

Testing with DRAM+CXL+interleaving resulted in:
- 2.93x improvement on TPC-H
- 2.01x improvement on TPC-DS

These outcomes indicate that CXL may contribute to enhanced performance in decision support systems and big data applications. This shows that CXL isn’t just about speed, it’s about enabling bigger, more complex workloads that wouldn’t fit in memory otherwise.

Finally, it is important to consider cost efficiency.

We evaluated the performance of CXL memory by running the Deep Learning Recommendation Model (DLRM), which serves as a rigorous benchmark due to its high memory requirements and sensitivity to latency, particularly with large embedding tables. If CXL performs well under these demanding conditions, it suggests suitability for a wide range of workloads.

The observed performance impact was minimal:
- Approximately 2% degradation when 50% of memory was allocated via CXL
- Around 9% reduction when 67% of memory utilized CXL

In practical terms, this suggests organizations can significantly reduce expenses associated with purchasing large-capacity 128 GB RDIMMs—which may cost up to three times as much as 64 GB modules—while still maintaining most of the system performance. That’s a small trade-off for a big cost saving—especially at scale.

CXL is still a relatively new technology, but this demo proves it’s ready for real-world use especially when paired with smart memory interleaving methodology. Whether you’re building AI models, analyzing massive datasets, or running simulations, CXL offers:
- More memory bandwidth for faster performance
- More capacity to handle larger workloads
- Lower costs without major trade-offs

And with companies like Micron and GIGABYTE leading the charge, the future of computing is looking a lot more scalable—and a lot more efficient.

For any questions regarding the technology or products featured in our demonstration, please contact us.