Infinia is a scale-out storage system being developed by DDN as its competitor to modern data platforms such as VAST and WEKA. It is fundamentally a key-value store built on b-epsilon trees upon which S3 and block “services” are exposed; NFS and a proprietary file protocol are on the roadmap. It advertises multitenancy as a first-class feature, positioning it well for hyperscale AI cloud environments, but curiously, its implementation of multitenancy relies on access controls rather than cryptographically secure tenant isolation.
It feels like what would happen if an old-school parallel file system designer created a scale-out hyperscale storage platform without ever having worked in a hyperscale environment. Many of its design choices and user/tenant interfaces reflect the mentality that a storage administrator is a full-time job held by someone who enjoys knowing the internals of their storage system.
History
Formerly known as DDN RED, it Infinia been reinvented several times over the last decade before ever seeing the light of day as a block+object storage system. Its core architecture has been led by Eric Barton, an influential figure in both the development of Lustre and DAOS. When Intel divested itself of DAOS, many of the key developers went on to work on Infinia.
Architecture
Infinia bears a lot of similarity to the original incarnation of DAOS in terms of complexity. It relies on b-epsilon trees to index bulk data chunks. Its control plane relies on etcd.
Current capabilities
As of Infinia v2.x,
- Infinia supports only one cluster in one realm. Cross-site/federated clustering is clearly on the roadmap, but it is not implemented.
- It doesn’t handle its own logs. You must forward them to an external solution, much like you would forward the logs from a parallel file system to an external stack (e.g., based on Victoria, Grafana, Prometheus)
- Only supports multitenancy at the access layer. Doesn’t ensure cryptographic isolation, so a compromise in the Infinia software itself or the underlying infrastructure exposes all tenants.
- S3 API limitations (not sure how common these are in third-party S3 implementations):
- no object checksums
- no IPv6
- no bucket tagging