Microsoft’s most recent “small language model,” Phi-2, is less than 13 billion parameters and performs as well as or better than certain larger open-source Llama 2 models.

The “Phi” family of small language models (SLMs) from Microsoft Research’s Machine Learning Foundations team has been available for a few months. These models perform remarkably well across a range of benchmarks.

Among the SLMs that are currently in use, the first model, the 1.3 billion parameter Phi-1, achieved state-of-the-art performance on Python coding (particularly on the HumanEval and MBPP benchmarks).

“We are now releasing Phi-2, a 2.7 billion-parameter language model that demonstrates outstanding reasoning and language understanding capabilities, showcasing state-of-the-art performance among base language models with less than 13 billion parameters,” the company said in an update.

For researchers looking to explore mechanistic interpretability, enhance safety, or refine experimentation on a range of tasks, Phi-2 is a perfect playground.

“We have made Phi-2 available in the Azure AI Studio model catalogue to foster research and development on language models,” said Microsoft.

The field of natural language processing has undergone a radical transformation as a result of the enormous expansion of language models to hundreds of billions of parameters.

However, it is still unclear if strategic training decisions, such as data selection, might lead to the development of such emergent talents on a smaller scale.

“Our line of work with the Phi models aims to answer this question by training SLMs that achieve performance on par with models of much higher scale (yet still far from the frontier models),” said Microsoft.

Additionally, the business has thoroughly tested several frequently utilized research community queries.

“We observed a behaviour by the expectation we had given the benchmark results,” said the tech giant.

Be a part of Elets Collaborative Initiatives. Join Us for Upcoming Events and explore business opportunities. Like us on Facebook , connect with us on LinkedIn and follow us on Twitter.

"Exciting news! Elets technomedia is now on WhatsApp Channels Subscribe today by clicking the link and stay updated with the latest insights!" Click here!