article thumbnail

Fundamentals of Data Engineering

Xebia

The following is a review of the book Fundamentals of Data Engineering by Joe Reis and Matt Housley, published by O’Reilly in June of 2022, and some takeaway lessons. This book is as good for a project manager or any other non-technical role as it is for a computer science student or a data engineer.

article thumbnail

AI data readiness: C-suite fantasy, big IT problem

CIO

If youre spending so much time to keep the lights on for operational side of data and cleansing, then youre not utilizing your domain experts for larger strategic tasks, he says. Data hygiene, data quality, and data security are all topics that weve been talking about for 20 years, Peterson says.

Data 201
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

What is data architecture? A framework to manage data

CIO

To do this, organizations should identify the data they need to collect, analyze, and store based on strategic objectives. Ensure data governance and compliance. Choose the right tools and technologies.

article thumbnail

Deletion Vectors in Delta Live Tables: Identifying and Remediating Compliance Risks

Perficient

Our Databricks Practice holds FinOps as a core architectural tenet, but sometimes compliance overrules cost savings. There is a catch once we consider data deletion within the context of regulatory compliance. However; in regulated industries, their default implementation may introduce compliance risks that must be addressed.

article thumbnail

Why thinking like a tech company is essential for your business’s survival

CIO

We developed clear governance policies that outlined: How we define AI and generative AI in our business Principles for responsible AI use A structured governance process Compliance standards across different regions (because AI regulations vary significantly between Europe and U.S.

Company 186
article thumbnail

Ducklake: A journey to integrate DuckDB with Unity Catalog

Xebia

Unity Catalog gives you centralized governance, meaning you get great features like access controls and data lineage to keep your tables secure, findable and traceable. Unity Catalog can thus bridge the gap in DuckDB setups, where governance and security are more limited, by adding a robust layer of management and compliance.

article thumbnail

Cloudera Data Engineering 2021 Year End Review

Cloudera

Since the release of Cloudera Data Engineering (CDE) more than a year ago , our number one goal was operationalizing Spark pipelines at scale with first class tooling designed to streamline automation and observability. The post Cloudera Data Engineering 2021 Year End Review appeared first on Cloudera Blog.