“Foundation models for science” are AI models that can solve a range of scientific problems within one or more science domains. They typically refer large models that take scientific data, not human language, as input and produce some output.
I’ll put examples of foundation models for science here as I find them.
“Frontier models for science” is a term made up by FASST. I’m not sure if it is meant to be different from foundation models for science.
Characteristics
Vision transformers can be used to process scientific imaging data, but they typically require very long sequences to handle high-resolution imagery. This is different from LLMs for chatbots which downsample or crop images before analyzing them.1
Example models
| Model | Creator | Year | Domain | Parameters |
|---|---|---|---|---|
| ORBIT | ORNL | 2024 | Climate | 113 B |
| Aurora | Microsoft | 2024 | Climate | 1.3 B |
| FourCastNet | NVIDIA | 2022 | Climate | 100 M2 |
| TEDDY | Merck | 2025 | Single Cell Bio | 70-400 M |
There’s also a paper where the authors “advocate for developing [foundation models] for power grids”3 but it doesn’t actually present a trained model.
Footnotes
-
This has to be inferred since the paper only says the model requires 10 GB of GPU memory and took 1024 GPU-hours of A100 time. The model used 8 AFNO blocks (888K per block) and a 12-layer vision transformer (~7.08M per layer). ↩
-
Foundation models for the electric power grid - ScienceDirect ↩