Glenn's Digital Garden

DAOS

Distributed Asynchronous Object Storage

DAOS is a high-performance, all-NVMe parallel storage system developed over the course of over a decade by the HDF Group, Whamcloud, and Intel. It shares a common lineage with many of the original creators of Lustre, but it was designed from the ground up to not adhere to POSIX semantics and instead offer a native object interface.

It is the storage system for Aurora.

History

Off the top of my head:

  • I first became aware of DAOS in 2015, when it was a research effort funded by the DOE as part of the FastForward/DesignForward programs. The grant was given to the HDF group and Whamcloud a few years earlier. Eric Barton was leading the effort, and DAOS was really complicated.
  • Intel acquired Whamcloud, and DAOS along with it, in 2012.1
  • At some point between 2014 and 2018, Eric Barton retired from Intel. Johann Lombardi takes over as DAOS chief architect. Eric goes on to join DDN to develop what is now Infinia.
  • Intel sells Whamcloud to DDN in 20181 but keeps the DAOS contract and development. DAOS becomes a part of Aurora. A lot of the complexity introduced by Eric Barton is removed from DAOS.
  • Johann Lombardi leaves Intel in 2023.
  • The DAOS Foundation is formed in late 2023. Its founding members are Intel, Argonne, HPE, Google, and Enkata.
    • Intel, Argonne, and HPE are tied to the hip because of Aurora
    • Enkata spun out of Croit, a company developing DAOS features
    • Google bet on DAOS as the foundation for its ParallelStore product
  • Intel divests its DAOS development team to HPE in late 2024.
  • Johann Lombardi joins the HPE DAOS team in February 2025.2

There really are two different realizations of DAOS which adds to the confusion around its history: the Eric Barton version, which went on to be realized in Infinia, and the Johann Lombardi version, which lives on in the DAOS Foundation. The Eric Barton DAOS was documented in an SC16 paper3 but the version of DAOS that lives on in Aurora and in the open-source community is very different from that.

Footnotes

  1. Intel gobbles Lustre file system expert Whamcloud • The Register ↩ ↩2

  2. Post | LinkedIn ↩

  3. Lofstead et al. DAOS and friends. 2016. ↩

Graph View

Backlinks