Anthropic

This page serves as a locus for all things Anthropic.

Software stack

Anthropic has disclosed the following about their inferencing stack:

• They inference using NVIDIA GPUs, Google TPUs, and AWS’s Trainium.¹
• They use Kubernetes in their serving environment²
• They appear to use JAX.¹
• They pin sessions to specific servers to improve KV cache hit rate.¹
• They provision pools of GPUs to specifically handle long contexts.¹ This makes sense, given they likely use context parallelism to support long contexts.

Their observability stack was described in 2025 at ClickHouse Open House 2025:²

• DataDog Vector for batched ingest
• ClickHouse
• Prometheus
• OpenTelemetry (Collector?)

Anthropic also uses hex for something.³ Maybe observability?²

Footnotes

1. A postmortem of three recent issues \ Anthropic ↩ ↩² ↩³ ↩⁴

2. How Anthropic is using ClickHouse to scale observability for the AI era ↩ ↩² ↩³

3. The AI Analytics Platform for your whole team | Hex - Under the “trusted by” logo wall. ↩