Dell has expanded its collaboration with AMD, now offering Team Red’s top-of-the-line Instinct MI300X AI accelerators in its cutting-edge servers along with new generative AI solutions. With the rapid evolution of AI, the fusion of open-source technologies and state-of-the-art hardware acceleration is driving industry innovation. Dell Technologies and AMD are at the forefront, providing on-premises infrastructure solutions that are up to 75% more cost-effective than public cloud IaaS and are designed to empower enterprise generative AI applications.
The PowerEdge XE9680 with AMD Instinct MI300X accelerators offers high-performance capabilities designed for companies leveraging generative AI, featuring eight MI300X accelerators, 1.5 TB of HBM3 memory, and 42 petaFLOPS of peak theoretical FP8 performance with sparsity precision.
This powerful configuration enables faster training and inference of large language models, helping organizations deliver AI-driven insights and innovative applications more efficiently. Recent testing on a single-node configuration has demonstrated industry-leading total cost of ownership:
- Deploying the 70B Llama 2 parameter model on a single AMD Instinct MI300X accelerator on a Dell PowerEdge XE9680 server.
- Deploying eight concurrent instances of the model using all eight AMD Instinct MI300X accelerators available on the Dell PowerEdge XE9680 server.
- Finetuning the 70B Llama 2 parameter model with FP16 precision on a Dell PowerEdge XE9680 server with eight AMD Instinct MI300X accelerators.
With simplified deployment through Dell OpenManage Enterprise, intelligent automation via APEX AIOps software, and enhanced security through integrated cyber recovery and a Zero Trust approach, the XE9680 server enables organizations to rapidly deploy and scale their GenAI solutions while maintaining a robust security posture. Announced in May and available today, the Dell Validated Design for Generative AI with AMD is the next step in Dell’s Generative AI solutions that facilitate reliable GenAI deployment by enterprises. This design guide provides organizations and developers with comprehensive guidance to implement LLM inference and model customization, as well as advanced techniques like fine-tuning and retrieval augmented generation (RAG). Leveraging open standards and reducing the need for proprietary AI software suites, developers can simplify development and freely customize workflows with open-source LLM models from partners like Hugging Face and Meta.
Accelerating modern workloads. Innovation and scalability enabling efficient and agile enterprises.
- With AMD Instinct MI300X accelerators, the Dell Validated Design enables near-linear scalability and low-latency distributed GenAI training and inference.
- PowerScale F710 provides faster time-to-air with a massive boost in streaming performance that accelerates all stages of the AI pipeline.
- Dell PowerSwitch Z9664F-ON, with 64 400GbE ports, offers low-latency and high-throughput Ethernet fabrics for modern AI clusters.
- The AI-optimized Broadcom Thor2 NIC card offers 400G, interconnecting the MI300X accelerators with the industry’s lowest power requirements.
