AMD Instinct™ MI350 系列

將百萬兆級的技術導入主流高效能運算與人工智慧領域

AMD Instinct™ MI350 系列 GPU：極致效能、成本效率與全開放資源生態

AMD 於 2025 年 6 月正式推出 AMD Instinct™ MI350 系列 GPU，為資料中心運算帶來重大躍進，專為生成式 AI 與高效能運算工作負載而設計。此系列採用先進的第 4 代 AMD CDNA™ 架構，並以台積電 3 奈米製程打造，提供卓越的效能與能源效率，能以高速完成大型 AI 模型訓練、高速推論以及複雜的 HPC 工作負載，如科學模擬與資料處理。

MI350X 與 MI355X GPU 皆配備 288GB HBM3E 記憶體、高達 8TB/s 的頻寬，在 AI 運算效能上較前一代提升最高 4 倍，推論效能更達 35 倍的突破，奠定其在 AI 與 HPC 市場中強大的競爭地位。

AMD Instinct™ MI350 Series

The AMD Instinct™ MI350 Series GPUs, launched in June 2025, represent a significant leap forward in data center computing, designed to accelerate generative AI and high-performance computing (HPC) workloads. Built on the cutting-edge 4th Gen AMD CDNA™ architecture and fabricated using TSMC's 3nm process, these GPUs deliver exceptional performance and energy efficiency for training massive AI models, high-speed inference, and complex HPC tasks like scientific simulations and data processing. Featuring 288GB of HBM3E memory and up to 8TB/s bandwidth, the MI350X and MI355X GPUs offer up to 4x generational AI compute improvement and a remarkable 35x boost in inference performance, positioning them as formidable competitors in the AI and HPC markets.

MI355X GPU	MI350X GPU	Model	MI325X GPU
TSMC's N3P / TSMC's N6		Process Technology (XCD/IOD)	TSMC's 5N / TSMC's 6
AMD CDNA4		GPU Architecture	AMD CNDA3
256		GPU Compute Units	304
16,384		Stream Processors	19,456
185 Billion		Transistor Count	153 Billion
5.0 POPs	4.6 POPs	INT8/INT4	-*
10 POPs	9.2 POPs	INT8/INT4 (Sparsity)	-*
10 PF	9.2 PF	PF6/FP4	-*
5.0 PF	4.6 PF	FP8	2.6 PF
2.5 PF	2.3 PF	BF16	1.3 PF
288 GB HBM3E		Dedicated Memory Size	256 GB HBM3E
Up to 8.0 TB/sec		Memory Bandwidth	6 TB/s
PCIe Gen5 x16		Bus Interface	PCIe Gen5 x16
Passive, Active	Passive	Cooling	Passive, Active
1400W	1000W	Maximum TDP/TBP	1000W
		Virtualization Support	Up to 8 partitions

*Data types not supported

AMD Instinct™ MI300 系列規格

GPU & APUSpecifications

獨立加速器模組與資料中心級的加速處理單元

AMD Instinct™ MI300 系列，包括 MI300X & MI325X 加速器模組及MI300A ，旨在以精實、高效率的形式提升人工智慧和高效能運算能力；其中，AMD Instinct MI325X 加速器內建業界領先的 256GB 記憶體和 6 TB/s 的頻寬，以加速資料交換時間；用來驅動最大型的 AI 模型。

Instinct MI300X 提供強大的加速效能，於技嘉G系列伺服器中配備八組 GPU。AMD Instinct MI300 系列加速器旨在提升資料中心運算效率，應對預算和可持續性問題，並提供高度可編程的 GPU 軟體平台。它擁有針對生成式 AI 和高性能計算（HPC）設計的先進 GPU、高吞吐量的 AMD CDNA 3 GPU 計算單元（CUs）以及原生的硬體稀疏矩陣支援。MI300X 提供 304 個高吞吐量計算單元，用於 AI運算相關應用，包括新數據類型支援、照片和影片解碼；支援多種數據類型及最新的生成式 AI 模型。

AMD首款整合 CPU/GPU 的加速處理單元（APU）– Instinct MI300A – 旨在克服 CPU 和 GPU 之間通道不足的的效能瓶頸，減少管理數據的程式碼工作，以及每代 GPU 都需要重構和重新編譯程式碼的需求。這項技術已大規模部署於全球最快且最節能的超級電腦中，現在即可採用 GIGABYTE伺服器，針對需求進行適當規模的部署。

透過提升運算吞吐量並簡化程式設計和部署，AMD Instinct MI300 系列應對了資源、複雜性、速度和架構挑戰下日益增長的 AI 和高效能運算需求，技嘉伺服器與AMD Instinct MI300 系列已準備好讓您即刻部署人工智慧資料中心。

MI325X GPU	MI300X GPU	Model	MI300A APU
OAM module	Form Factor	APU SH5 socket
-	AMD ‘Zen 4’ CPU cores	24
304	GPU Compute Units	228
19,456	Stream Processors	14,592
163.4 TFLOPS	Peak FP64/FP32 Matrix*	122.6 TFLOPS
81.7/163.4 TFLOPS	Peak FP64/FP32 Vector*	61.3/122.6 TFLOPS
1307.4 TFLOPS	Peak FP16/BF16*	980.6 TFLOPS
2614.9 TFLOPS	Peak FP8*	1961.2 TFLOPS
256 GB HBM3E	192 GB HBM3	Dedicated Memory Size	128 GB HBM3
6.0 GHz	5.2 GHz	Memory Clock	5.2 GHz
6 TB/s	5.3 TB/s	Memory Bandwidth	5.3 TB/s
PCIe Gen5 x16	Bus Interface	PCIe Gen5 x16
8	Infinity Fabric™ Links	8
1000W	750W	Maximum TDP/TBP	550W / 760W (Peak)
Up to 8 partitions	Virtualization Support	Up to 3 partitions

Optimize Next Gen Innovation with AMD ROCm™ 7.0

The AMD ROCm™ 7.0 software stack is a key differentiator that enables high-performance AI and HPC development with minimal code changes. AMD Instinct™ MI350 Series GPUs are fully optimized for leading frameworks such as PyTorch, TensorFlow, JAX, ONNX Runtime, Triton, and vLLM, and offer Day 0 support for popular models through automatic kernel generation and continuous validation.
Expanded Hardware & Platform Support : ROCm 7 is fully compatible with AMD Instinct™ MI350 Series GPUs (including MXFP6/MXFP4) and extends development to select AMD Radeon™ GPUs and Windows environments, ensuring seamless performance across diverse hardware from cloud to edge. Advanced AI Features & Optimizations : ROCm 7 is targeting large-scale AI and LLM deployments with pre-optimized transformer kernels (OCP-FP8/MXFP8/MXFP6/MXFP4), integrated distributed inference via vLLM v1, llm-d, and SGLang, and enhanced "flash" attention and communication libraries for peak multi-GPU utilization. Optimized Performance : The ROCm 7 preview delivered up to 3.5x faster AI inference and 3x quicker training than ROCm 6 by leveraging lower-precision data types and advanced kernel fusion to maximize GPU efficiency and reduce memory and I/O load.^[1]	Enabling Developer Success : With the new ROCm Enterprise AI suite, it is now easier to fine-tune models on domain-specific data and deploy AI services in production, streamline install with a simple pip install rocm flow, and support advanced optimization features such as model quantization libraries to boost productivity and performance. Expanded Ecosystem & Community Collaboration : ROCm 7 deepens integration with leading AI and HPC models and frameworks, offering day-0 support for PyTorch, TensorFlow, JAX, and ONNX and more, while giving organizations flexibility in model selection with over 2 million pre-trained models. Its broad ecosystem and open-source collaboration ensures stability, compatibility, and readiness for future workloads.
^[1](MI300-080): Testing by AMD as of May 15, 2025, measuring the inference performance in tokens per second (TPS) of AMD ROCm 6.x software, vLLM 0.3.3 vs. AMD ROCm 7.0 preview version SW, vLLM 0.8.5 on a system with (8) AMD Instinct MI300X GPUs running Llama 3.1-70B (TP2), Qwen 72B (TP2), and Deepseek-R1 (FP16) models with batch sizes of 1-256 and sequence lengths of 128-204. Stated performance uplift is expressed as the average TPS over the (3) LLMs tested. Results may vary.

The AMD ROCm™ 7.0 software stack is a key differentiator that enables high-performance AI and HPC development with minimal code changes. AMD Instinct™ MI350 Series GPUs are fully optimized for leading frameworks such as PyTorch, TensorFlow, JAX, ONNX Runtime, Triton, and vLLM, and offer Day 0 support for popular models through automatic kernel generation and continuous validation.

Expanded Hardware & Platform Support : ROCm 7 is fully compatible with AMD Instinct™ MI350 Series GPUs (including MXFP6/MXFP4) and extends development to select AMD Radeon™ GPUs and Windows environments, ensuring seamless performance across diverse hardware from cloud to edge.
Advanced AI Features & Optimizations : ROCm 7 is targeting large-scale AI and LLM deployments with pre-optimized transformer kernels (OCP-FP8/MXFP8/MXFP6/MXFP4), integrated distributed inference via vLLM v1, llm-d, and SGLang, and enhanced "flash" attention and communication libraries for peak multi-GPU utilization.
Optimized Performance : The ROCm 7 preview delivered up to 3.5x faster AI inference and 3x quicker training than ROCm 6 by leveraging lower-precision data types and advanced kernel fusion to maximize GPU efficiency and reduce memory and I/O load.^[1]

Enabling Developer Success : With the new ROCm Enterprise AI suite, it is now easier to fine-tune models on domain-specific data and deploy AI services in production, streamline install with a simple pip install rocm flow, and support advanced optimization features such as model quantization libraries to boost productivity and performance.
Expanded Ecosystem & Community Collaboration : ROCm 7 deepens integration with leading AI and HPC models and frameworks, offering day-0 support for PyTorch, TensorFlow, JAX, and ONNX and more, while giving organizations flexibility in model selection with over 2 million pre-trained models. Its broad ecosystem and open-source collaboration ensures stability, compatibility, and readiness for future workloads.

^[1](MI300-080): Testing by AMD as of May 15, 2025, measuring the inference performance in tokens per second (TPS) of AMD ROCm 6.x software, vLLM 0.3.3 vs. AMD ROCm 7.0 preview version SW, vLLM 0.8.5 on a system with (8) AMD Instinct MI300X GPUs running Llama 3.1-70B (TP2), Qwen 72B (TP2), and Deepseek-R1 (FP16) models with batch sizes of 1-256 and sequence lengths of 128-204. Stated performance uplift is expressed as the average TPS over the (3) LLMs tested. Results may vary.