Hemit HF, St Olavs and NorwAI plan to test and validate LLMs in the health sector

Hemit HF, the technology enterprise of Helse Midt-Norge, and the St Olavs Hospital are planning to try out whether NorwAI's newest language model will support the hypothesis that smaller, Norwegian language models, trained and fine-tuned for the health domain, will perform better than large, international generic models.

All partners to-be are excited about the plans:

- In the specialist health service, many are eager for digital tools that improve their working life. We are anxious to see whether language models adapted to the Norwegian health service can contribute to this, says CEO Trond Utne at Hemit HF.

- It is an exciting research challenge to find good benchmarks for language models within a sector where the tasks vary significantly between different fields, from chronic mental health to surgery in all its forms, says Professor Eric Monteiro at NorwAI.

Interesting sector

In the language model work, the research center has shifted its activities in the past year from building generic Norwegian models to a greater extent developing and training models according to domain-specific orders. Here, health is an interesting sector. The parties are now investigating the conditions for joint work.

NorwAI is currently training a new language model of 24 billion parameters based on the Magistral architecture of Mistral (NorwAI-Magistral-24B-reasoning) which is due to be completed in November 2025. A beta model was demonstrated at the NorwAI Innovate conference in September. The model is an important starting point for the plans NorwAI and Hemit are now pursuing.

Improved quality and efficiency

Hemit HF provide IT services for specialized healthcare to the Central Norway Regional Health Authority (Helse Midt-Norge) in the counties of Trøndelag and Møre og Romsdal. Their mission is to enhance patient care through innovative ICT solutions while ensuring the continuous availability of critical hospital systems. A goal to the planned project is to improved quality and efficiency in healthcare through exploring, testing and validating Norwegian-based trustworthy LLMs.

St Olav Hospital also take part in project planning. The University Hospital is organizing a clinical advisory board to add their expertise in support of the work and will be part of a steering committee for “HelseLLM”.

Healthcare organizations possess data that is not openly available. Using these in controlled forms can be a prerequisite for creating specialized models for selected, designated tasks that solve relevant clinical challenges with better results than internationally available general models.

A motivation 

In Norway, hundreds of doctors have already participated in the exploration of whether and how language models can contribute to everyday work on a test basis. However, the scattered experiments have not been scaled up further on a large scale so far. This motivates Hemit HF.

- We want to explore whether language models adapted to the Norwegian health service can provide better digital support for the health service than the general language models available in the market. If the hypothesis is correct, we want to create a plan for how this can be used nationally, says Trond Utne, who adds that the four health regions in Norway are collaborating closely on better solutions.

By Rolf D. Svendsen

2025-10-30