Partnership between companies wants to standardize benchmarks for AI

A partnership of companies led by Meta and IBM wants to standardize benchmarks for AI applications. In addition, the coalition, including Hugging Face and AMD, wants to stimulate the open development of AI tools.

In addition to companies, universities are also part of the partnership that has been named AI Alliance. as stated in the announcement. Participants include AMD, Anyscale, CERN, Cerebras, Cleveland Clinic, Cornell University, Dartmouth, Dell Technologies, Hugging Face, Imperial College London, Intel, INSAIT, Linux Foundation, NASA, NSF, Oracle, Red Hat, Roadzen, ServiceNow, Sony Group, Stability AI, University of California Berkeley, University of Illinois, University of Notre Dame, The University of Tokyo and Yale University.

The group wants to develop, release and standardize AI benchmarks in order to properly measure the performance of AI systems. The group also wants to encourage the open development of artificial intelligence, so that major steps are not taken behind closed doors and other researchers can watch. This should keep development responsible and limit risks.

It is still unknown what this collaboration will look like in practice. There are many such partnerships and some lead to success, while others fade into oblivion. Well-known examples of partnerships include the Wireless Power Consortium that develops the Qi standard for wireless charging and the Connectivity Standards Alliance that creates the smart home standard Matter.