Remove Data Engineering Remove Software Engineering Remove Testing
article thumbnail

Giving more tools to software engineers: the reorganization of the factory

Erik Bernhardsson

I had my first job as a software engineer in 1999, and in the last two decades I've seen software engineering changing in ways that have made us orders of magnitude more productive. Because someone made the economic decision that the cost of building that software was too high. Supply-demand of software engineers.

article thumbnail

Fishtown Analytics raises $29.5M Series B for its data engineering platform

TechCrunch

Fishtown Analytics , the Philadelphia-based company behind the dbt open-source data engineering tool, today announced that it has raised a $29.5 The company is building a platform that allows data analysts to more easily create and disseminate organizational knowledge. million Series A round in April.

article thumbnail

What is a data engineer? An analytics role in high demand

CIO

What is a data engineer? Data engineers design, build, and optimize systems for data collection, storage, access, and analytics at scale. They create data pipelines that convert raw data into formats usable by data scientists, data-centric applications, and other data consumers.

article thumbnail

The future of data: A 5-pillar approach to modern data management

CIO

This article proposes a methodology for organizations to implement a modern data management function that can be tailored to meet their unique needs. By modern, I refer to an engineering-driven methodology that fully capitalizes on automation and software engineering best practices.

Data 167
article thumbnail

Fundamentals of Data Engineering

Xebia

The following is a review of the book Fundamentals of Data Engineering by Joe Reis and Matt Housley, published by O’Reilly in June of 2022, and some takeaway lessons. This book is as good for a project manager or any other non-technical role as it is for a computer science student or a data engineer.

article thumbnail

Our First Netflix Data Engineering Summit

Netflix Tech

Engineers from across the company came together to share best practices on everything from Data Processing Patterns to Building Reliable Data Pipelines. The result was a series of talks which we are now sharing with the rest of the Data Engineering community! In this video, Sr.

article thumbnail

Maintaining conventions in dbt projects with dbt-bouncer

Xebia

In traditional software engineering projects, challenges like these are overcome with automated tooling; directory structures encourage a standardised file layout, pre-commit offers config-based formatting and tools like flake8 offer linting capabilities. repos: - repo: [link] rev: v2.0.6 Loaded config from dbt-bouncer-example.yml.