Italian artificial intelligence startup iGenius Inc. announced today the launch of Colosseum 355B, its new state-of-the-art foundation large language model designed for highly regulated industries to provide businesses the confidence that their data won’t be compromised.

IGenius specializes in AI for enterprises working in highly regulated sectors, such as finance and public administration. The company develops LLMs to power its business intelligence agent, Crystal, an AI solution designed to work within these industries while providing high performance, customization and maintaining data privacy.

To build Colosseum 355B, the company collaborated with Nvidia Corp. to accelerate the development of the model. It was built using the Nvidia AI Enterprise software platform and used the DGX Cloud AI platform to orchestrate more than 3,000 Nvidia H100 GPUs.

According to Nvidia, the work was completed in two months and the result was a 355 billion-parameter model that supports more than 50 languages, excels at coding and is optimized to fit on a single H100 GPU node.

IGenius said the model was pre-trained using a method known as FP8 precision, meaning it uses eight-bit floating point data, offering a significant reduction in memory usage. This size reduction allows the model to cut inference costs by 50% without having to convert the model, which could reduce accuracy or quality.

Being small enough to run on a single H100 GPU also means that a regulated enterprise can run the model within its firewall on-premises without the need to host in the cloud. This provides the capability for an organization to host their own proprietary LLM and maintain complete control.

“Colosseum is a powerful AI model poised to unlock new opportunities for sovereign nations across the most highly regulated industries,” said Alexis Bjorlin, vice president of DGX Cloud at Nvidia.

The model is specially designed for both continued pre-training and fine-tuning. CPT is a cost-effective alternative to pretraining, a way to do further training of a base pretrained LLM using a large domain of text documents to augment the model’s general knowledge with more specific information. That allows organizations to build their own specialized AI models that can adapt to long-term needs without losing general knowledge.

Furthering the company’s collaboration with Nvidia, Colosseum 355B is packaged as an Nvidia NIM microservice and available as an application programming interface via the company’s Nvidia API catalog. Nvidia’s NIM microservices provide secure containerized AI models that can be deployed across clouds, data centers and workstations.

Image: Pixabay