article thumbnail

Data Mesh Principles and Logical Architecture

Martin Fowler

Last year, my colleague Zhamak Dehghani introduced the notion of the Data Mesh , shifting from the notion of a centralized data lake to a distributed vision of data.

article thumbnail

What is data architecture? A framework to manage data

CIO

Data architecture definition Data architecture describes the structure of an organizations logical and physical data assets, and data management resources, according to The Open Group Architecture Framework (TOGAF). An organizations data architecture is the purview of data architects.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

The key to operational AI: Modern data architecture

CIO

From customer service chatbots to marketing teams analyzing call center data, the majority of enterprises—about 90% according to recent data —have begun exploring AI. For companies investing in data science, realizing the return on these investments requires embedding AI deeply into business processes.

article thumbnail

Governing data products using fitness functions

Martin Fowler

Decentralized data management requires automation to scale governance effectively. Fitness functions are a powerful automated governance technique my colleagues have applied to data products within the context of a Data Mesh.

article thumbnail

An Architect’s Guide for Selecting Scalable, Data-Layer Technologies

There’s no getting around it: selecting the right foundational data-layer components is crucial for long-term application success.

article thumbnail

Agentic AI design: An architectural case study

CIO

In our real-world case study, we needed a system that would create test data. This data would be utilized for different types of application testing. The requirements for the system stated that we need to create a test data set that introduces different types of analytic and numerical errors.

article thumbnail

Why CIOs must lead the charge on ESG – and why enterprise architecture is the key

CIO

In an era marked by heightened environmental, social and governance (ESG) scrutiny and rapid artificial intelligence (AI) adoption, the integration of actionable sustainable principles in enterprise architecture (EA) is indispensable. Training a single AI model emits as much as five average cars over their lifetimes.

article thumbnail

Use Cases for Apache Cassandra®

From understanding its distributed architecture to unlocking its incredible power for industries like healthcare, finance, retail and more, experience how Cassandra® can transform your entire data operations.

article thumbnail

7 Pitfalls for Apache Cassandra in Production

Apache Cassandra is an open-source distributed database that boasts an architecture that delivers high scalability, near 100% availability, and powerful read-and-write performance required for many data-heavy use cases.

article thumbnail

The Next-Generation Cloud Data Lake: An Open, No-Copy Data Architecture

In an effort to be data-driven, many organizations are looking to democratize data. However, they often struggle with increasingly larger data volumes, reverting back to bottlenecking data access to manage large numbers of data engineering requests and rising data warehousing costs.

article thumbnail

Partner Webinar: A Framework for Building Data Mesh Architecture

Speaker: Jeremiah Morrow, Nicolò Bidotti, and Achille Barbieri

Data teams in large enterprise organizations are facing greater demand for data to satisfy a wide range of analytic use cases. Yet they are continually challenged with providing access to all of their data across business units, regions, and cloud environments.

article thumbnail

Embedding BI: Architectural Considerations and Technical Requirements

While data platforms, artificial intelligence (AI), machine learning (ML), and programming platforms have evolved to leverage big data and streaming data, the front-end user experience has not kept up. Traditional Business Intelligence (BI) aren’t built for modern data platforms and don’t work on modern architectures.

article thumbnail

Build Your Open Data Lakehouse on Apache Iceberg

Speaker: Veena Vasudevan and Jason Hughes

With data stored in vendor-agnostic files and table formats like Apache Iceberg, the open lakehouse is the best architecture to enable data democratization. By moving analytic workloads to the data lakehouse you can save money, make more of your data accessible to consumers faster, and provide users a better experience.

article thumbnail

Top Considerations for Building an Open Cloud Data Lake

Data fuels the modern enterprise — today more than ever, businesses compete on their ability to turn big data into essential business insights. Increasingly, enterprises are leveraging cloud data lakes as the platform used to store data for analytics, combined with various compute engines for processing that data.

article thumbnail

The Unexpected Cost of Data Copies

An organization’s data is copied for many reasons, namely ingesting datasets into data warehouses, creating performance-optimized copies, and building BI extracts for analysis. Read this whitepaper to learn: Why organizations frequently end up with unnecessary data copies.