Remove Data Engineering Remove Off-The-Shelf Remove Storage
article thumbnail

Giving more tools to software engineers: the reorganization of the factory

Erik Bernhardsson

I had my first job as a software engineer in 1999, and in the last two decades I've seen software engineering changing in ways that have made us orders of magnitude more productive. Mediocre software exists because someone wasn't able to hire better engineers, or they didn't have time, or whatever.

article thumbnail

Supercharge your Airflow Pipelines with the Cloudera Provider Package

Cloudera

Many customers looking at modernizing their pipeline orchestration have turned to Apache Airflow, a flexible and scalable workflow manager for data engineers. A provider could be used to make HTTP requests, connect to a RDBMS, check file systems (such as S3 object storage), invoke cloud provider services, and much more.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

7 data trends on our radar

O'Reilly Media - Ideas

Whether you’re a business leader or a practitioner, here are key data trends to watch and explore in the months ahead. Increasing focus on building data culture, organization, and training. The demand for data skills (“the sexiest job of the 21st century”) hasn’t dissipated. Tools for secure and privacy-preserving analytics.

Trends 85
article thumbnail

Why Reinvent the Wheel? The Challenges of DIY Open Source Analytics Platforms

Cloudera

In their effort to reduce their technology spend, some organizations that leverage open source projects for advanced analytics often consider either building and maintaining their own runtime with the required data processing engines or retaining older, now obsolete, versions of legacy Cloudera runtimes (CDH or HDP).

article thumbnail

Supply Chain Control Tower: Enhancing Visibility and Resilience

Altexsoft

Leading executives focus on building resilient and intelligent supply chains that can withstand the turmoil due to data-based proactive decisions. “Control towers are the artificial intelligence (AI) of supply chain. Everyone wants to have it, but nobody quite knows how it works.” Christian Titze, vice president analyst at Gartner.

article thumbnail

The Good and the Bad of Apache Kafka Streaming Platform

Altexsoft

It offers high throughput, low latency, and scalability that meets the requirements of Big Data. The technology was written in Java and Scala in LinkedIn to solve the internal problem of managing continuous data flows. What does the high-performance data project have to do with the real Franz Kafka’s heritage?

article thumbnail

Fleet Maintenance Software: Technology Behind Preventive and Predictive Vehicle Servicing

Altexsoft

Taking good care of your fleet assets pays off by prolonging their lifecycle, increasing efficiency, and reducing the probability of failures. Prevention is better than cure. If you think vehicle breakdowns are inevitable, we got news for you. These risks and losses can – and have to! – be avoided with proactive maintenance.