Glenn's Digital Garden

OpenAI

This page serves as a locus for everything related to OpenAI.

Software stack

OpenAI has disclosed the following about their training software stack:

  • They have used Ray for training GPT 3.5 and GPT 4.0.1 It is unclear if they have used it for training since then, or if they use it for inferencing at all.
  • They have used Kubernetes on their large training clusters.2 See Kubernetes.
  • They have used Apache Spark for data preprocessing. This was mentioned in the GPT-3 paper.

For inferencing, their stack appears to include:

  • Cosmos DB for conversation state (see ChatGPT)
  • Codex’s web UI uses Temporal to store workflow state3

In addition, they have disclosed:

  • They use a private monorepo for their code. This was stated in some video they posted about testing with data that they know wasn’t in the training dataset.
  • Their observability platform is built on ClickHouse,4 Fluent Bit, and Azure Blob5

Training techniques

Infrastructure

See Microsoft supercomputers and Stargate.

Business

See OpenAI x Microsoft.

Footnotes

  1. https://www.anyscale.com/glossary/what-is-ray ↩

  2. https://openai.com/index/scaling-kubernetes-to-7500-nodes/ ↩

  3. Of course you can build dynamic AI agents with Temporal | Temporal ↩

  4. What is observability in 2026? Why it’s an analytics problem and why your database matters. | Engineering | ClickHouse Resource Hub ↩

  5. Why OpenAI chose ClickHouse for petabyte-scale observability ↩

Graph View

Backlinks