MediaTek’s next high-end SoC can run Meta Llama 2 model locally

MediaTek has announced that its next high-end SoC will have the ability to run Facebook parent company Meta’s major language model Llama 2 locally. The software has been adapted for this purpose.

The SoC has ‘Transformer backbone acceleration’, so says MediaTek. Chips for AI functions will also have access to more dram bandwidth to run Llama 2 and other generative AI applications. So far, a lot of generative AI works on smartphones with cloud applications.

MediaTek says nothing about the performance of the language model on the SoC or the space required for the language model on storage. The processor designer expects applications based on Llama 2 to appear before the end of the year. This also applies to the SoC, probably the Dimensity 9300. Rumors have been going around about this for months. MediaTek follows competitor Qualcomm, which previously said that flagship smartphones and PCs with Qualcomm Socs will also be able to run Llama 2 locally from 2024.