Shift Left: Headless Data Architecture, Part 2

The headless data architecture is a formalization of a data access layer at the center of an organization, providing consistent data access for operational and analytical use cases. It requires identifying work to shift left from downstream, creating a single standardized set of data through streams and tables. This approach reduces downstream costs by shifting work to the left, simplifying data creation, access, and use. The traditional multi-hop architecture is slower, expensive, brittle, and can lead to similar-yet-different data sets, which can cause conflicts and loss of trust. A shift-left approach provides a more cost-effective way to create, access, and use data, with the logical top level being the data product, composed of a stream and its related table. Data products are trustworthy, standardized, and reusable, simplifying getting the data needed by teams and services. Shifting left is modular and incremental, allowing selective choice of loads to shift left, validation, and replacement of existing jobs. This approach unlocks unparalleled data access across the organization, providing freedom from coordinating multiple copies of data and using suitable processing or query engines.