Recent announcements indicate that the Intel Lunar Lake CPU platform is demonstrating impressive AI inference performance with the latest Meta LLaMA 3.2 models. Intel has published an article outlining the performance gains LLaMA 3.2 achieves with Intel AI hardware, such as Intel Gaudi AI accelerators, Intel EO processors, Intel Core Ultra “Lunar Lake” CPUs, and Intel Arc graphics. In a LinkedIn post, Robert Hallock, Vice President and General Manager of Client AI and Technical Marketing at Intel, stated that Intel Lunar Lake Core Ultra processors are delivering outstanding performance with LLaMA 3.2.
According to the claims, the flagship Intel Core Ultra 9 288V “Lunar Lake” CPU can achieve an incredibly low latency of just 28.5 ms for 32 input tokens and 31.4 ms for 1024 input tokens for the 3B model. This translates to about 32-35 tokens per second, which is an impressive performance for AI inference.
Intel is working on its AI inference framework called OpenVINO, which optimizes performance on Intel hardware to enhance both performance and efficiency on PCs and edge devices.
