The proposing company EJIE and researchers from the University of the Basque Country, with the support of the Basque Government, have developed and released for the use of citizens and developers a high-quality open-weight LLM for Basque (latxa.eu), currently the best open model for Basque and close to GPT4o and Claude 3.5 in performance. In order to be competitive with the best closed models further developments are needed.
The goal of this project is to improve the linguistic proficiency of the public LLM while continuing to offer skills that are similar to newer closed models, and thus compete with a new service in the LLM market for Basque. The goal requires that we scale-up a novel ground-breaking approach (which has been already shown to be effective for small LLMs) to larger LLMs. More specifically, the method has been shown to work for 8B LLMs and in order to scale them up to 70B significant compute is needed, which is not available to the proposing company.
The aim of this project is that the company will deliver a new product and service, a 70B-scale open conversational model for Basque which will be made available to the general public and innovative industry, under the auspices of the Basque Government. The model will surpass or equal the linguistic proficiency of current closed models while maintaining competitive skills. The model will be uploaded to public repositories with an open license and the company will use its dedicated computational infrastructure to serve the new product in inference.
Principal Investigator, Company and Country
Oscar Guadilla, Basque Government Informatic Society- EJIE, Spain