The Cerebras AI Model Studio

The Cerebras AI Model Studio is a simple pay by the model computing service powered by dedicated clusters of Cerebras CS-2’s and hosted by Cirrascale Cloud Services. It is a purpose-built platform, optimized for training large language models on dedicated clusters of millions of cores. It provides deterministic performance, requires no distributed computing headaches, and is push-button simple to start.

The Problem

Training large Transformer models such as GPT and T5 on traditional cloud platforms can be painful, expensive, and time consuming. Gaining access to large instances typically offered in the cloud can often takes weeks just to get access. Networking, storage, and compute can cost extra, and setting up the environment is no joke. Models with tens of billions of parameters end up taking weeks to get going and months to train.

If you want to train in less time, you can attempt to reserve additional instances – but unpredictable inter-instance latency, makes distributing AI work difficult, and achieving high performance across multiple instances challenging .

Our Solution

The Cerebras AI Model Studio makes training large Transformer models fast, easy, and affordable. With Cerebras, you have millions of cores, predictable performance, no parallel distribution headaches – all of this enables you to quickly and easily run existing models on your data or to build new models from scratch optimized for your business.

A dedicated cloud-based cluster powered by Cerebras CS-2 systems with millions of AI cores for large language models and generative AI:

• Train 1-175 billion parameter models quickly and easily
• No parallel distribution pain: single-keystroke scaling over millions of cores
• Zero DevOps or firewall pain: simply SSH in and go
• Push-button performance: models in standard PyTorch or TensorFlow
• Flexibility: pre-train or fine-tune models with your data
• Train in a known amount of time, for a fixed fee

Discover the Benefits of the Cerebras AI Model Studio

1. Dataset size is dependent upon the model architecture used for training and the task. If you are unsure, our world-class engineers will be happy to help

2. Domain similarity assess if the data used to pre-train a generic model and your data are similar enough such that your fine-tuned model will perform well on downstream tasks

White-Glove Support with Cerebras Experts

With White-Glove Support, Cerebras thought leaders will fine-tune a model on the Cerebras Wafer-Scale Cluster on your behalf and will deliver you trained weights. Contact us directly for pricing.

Fine-Tuning - Standard Offering Pricing

Introductory pricing below. These prices represent blocks of dedicated cluster time for the chosen model. Additional system time is available at an hourly rate as needed.

* T5 tokens to train from the original T5 paper. Chinchilla scaling laws not applicable.

** Note that GPT-J was pre-trained on ~400B tokens. Fine-tuning jobs can employ a wide range of dataset sizes, but often use order 1-10% of the pre-training tokens. As such, one might fine-tune a model like GPT-J with ~4-40B tokens. We provide estimated wall clock time to fine-tune train the model checkpoints above with 10B tokens on Cerebras AI Model Studio and an AWS p4d instance in the table above to give you a sense of how much time jobs of this scale could take.

Additional Services Available

Cirrascale and Cerebras provides additional services as needed, such as:

• Bigger dedicated clusters to are available to reduce time to accuracy and work on larger models
• Additional cluster time for hyperparameter tuning, pre-production training runs, post-production continuous pre-training or fine-tuning is available by the hour
• CPU hours from Cirrascale for dataset preparation
• CPU or GPU support from Cirrascale for production model inference

Fixed-Price Production Model Training

Introductory pricing below. These prices represent blocks of dedicated cluster time for the chosen model. Additional system time is available at an hourly rate as needed.

* T5 tokens to train from the original T5 paper. Chinchilla scaling laws not applicable.

** Expected number of days, based on training experience to date, using a 4-node Cerebras Wafer-Scale Cluster. Actual training of model may take more or less time.

Interested? Sign Up for a Free Trial

Sometimes it makes sense to try before you buy. We offer free trials to qualified companies looking seriously at the Cerebras Cloud. Our free trial enables you to familiarize yourself with Cerebras’ simple programming model based on Python and standard ML frameworks PyTorch and TensorFlow. You'll be able to experience push-button model scaling from 1-20B parameters.

The 2-Day Free Trial includes the ability to run models with several model implementations to review and choose from. You'll also have a curated Pile dataset to train upon while in a secure, dedicated access to the programming environment. We provide technical documentation and training guides to help you get started. It's easy, so sign up to try the Cerebras AI Model Studio today.