InfoQ Culture Methods

FEBRUARY 24, 2023

In this article, Tariq King describes the metaverse concept, discusses its key engineering challenges and quality concerns, and then walks through recent technological advances in AI and software testing that are helping to mitigate these challenges.

Testing 120

Testing Engineering Software Technology 120

Dzone - DevOps

SEPTEMBER 6, 2024

Considering data engineering and data science, Astro and Apache Airflow rise to the top as important tools used in the management of these data workflows. This article compares Astro and Apache Airflow, explaining their architecture, features, scalability, usability, community support, and integration capabilities.

Comparison 91

Comparison Microservices Data Engineering Software Development 91

Erik Bernhardsson

DECEMBER 15, 2020

I had my first job as a software engineer in 1999, and in the last two decades I've seen software engineering changing in ways that have made us orders of magnitude more productive. Mediocre software exists because someone wasn't able to hire better engineers, or they didn't have time, or whatever.

Software Engineering 304

Software Engineering Engineering Tools Software 304

InfoQ Culture Methods

JULY 28, 2020

To become a machine learning engineer, you have to interview. You have to gain relevant skills from books, courses, conferences, and projects. Include technologies, frameworks, and projects on your CV. In an interview, expect that you will be asked technical questions, insight questions, and programming questions.

Artificial Inteligence 122

Artificial Inteligence Machine Learning Engineering How To 122

TechCrunch

DECEMBER 10, 2020

As remote work continues to solidify its place as a critical aspect of how businesses exist these days, a startup that has built a platform to help companies source and bring on one specific category of remote employees — engineers — is taking on some more funding to meet demand. Turing is essentially tapping into both concepts.

Engineering 325

Engineering Artificial Inteligence Lambda Machine Learning 325

Altexsoft

JUNE 29, 2021

This article will focus on the role of a machine learning engineer, their skills and responsibilities, and how they contribute to an AI project’s success. The role of a machine learning engineer in the data science team. The focus here is on engineering, not on building ML algorithms.

Artificial Inteligence 105

Artificial Inteligence Machine Learning Engineering Data Engineering 105

InfoQ Culture Methods

APRIL 7, 2020

QCon returned to London this past March for its fourteenth year in the city, attracting over 1,600 senior developers, architects, data engineers, team leads, and CTOs. This article provides a summary of the key takeaways. By Abel Avram.

Data Engineering 66

Data Engineering Engineering Data Development 66

CIO

OCTOBER 6, 2020

Earlier this year, Microsoft announced it would retire 15 certifications and 44 exams, including MCSA: BI Reporting, which validated data analysts' expertise using Power BI and Excel to visualize data via dashboards and reports. To read this article in full, please click here

Data 240

Data Data Engineering Report Engineering 240

The Crazy Programmer

NOVEMBER 6, 2021

Big Data Engineer. Another highest-paying job skill in the IT sector is big data engineering. And as a big data engineer, you need to work around the big data sets of the applications. Not only this, but you also need to use coding skills, data warehousing, and visualizing skills. Conclusion.

Artificial Inteligence 290

Artificial Inteligence Blockchain Artificial Intelligence Software Review 290

CIO

SEPTEMBER 29, 2023

Most relevant roles for making use of NLP include data scientist , machine learning engineer, software engineer, data analyst , and software developer. Relevant job roles include image processing engineer or scientist, robotics engineer, AI research scientist, quality control analyst, AR or VR developer, among others.

Generative AI 357

Generative AI Machine Learning Artificial Inteligence ChatGPT 357

InfoQ Culture Methods

JUNE 19, 2024

Gen AI Assistants play to the strengths of professionals with a breadth of experience, particularly software developers who can describe what they want the LLM to complete and critically evaluate the result. These tools enable us to swiftly cross divides of domain language and scale large repetitive tasks down to interesting ones on a human scale.

Generative AI 115

Generative AI Project Management Artificial Inteligence Software 115

InfoQ Culture Methods

JANUARY 13, 2023

This article is a summary of the 2022 software trends podcast. 2022 was another year of significant technological innovations and trends in the software industry and communities. The InfoQ podcast co-hosts met last month to discuss the major trends from 2022, and what to watch in 2023.

Trends 104

Trends Software Report Innovation 104

Dzone - DevOps

OCTOBER 30, 2023

In this article, we explore the synergy between software-based architecture and the development of interoperability solutions for IoT to provide insights relevant to software developers and data engineers. To tackle this challenge head-on, software-based architectures are emerging as powerful solutions.

IoT 95

IoT Architecture Software Data Engineering 95

InfoWorld

MARCH 8, 2024

LinkedIn has decided to open source its data management tool, OpenHouse, which it says can help data engineers and related data infrastructure teams in an enterprise to reduce their product engineering effort and decrease the time required to deploy products or applications.

Open Source 72

Open Source Tools Data Engineering Storage 72

InfoQ Culture Methods

FEBRUARY 22, 2022

The article explores optimizing test execution, saving machine resources, and reducing feedback time to developers. Test suites may be computationally expensive, compete with each other for available hardware, or simply be so large as to cause considerable delay until their results are available. By Gregor Endler, Marco Achtziger.

Artificial Inteligence 109

Artificial Inteligence Machine Learning Testing Development 109

InfoQ Culture Methods

SEPTEMBER 1, 2023

There’s an increasing concern about the energy use and corresponding carbon emissions of generative AI models. And while the concerns may be overhyped, they still require attention, especially as generative AI becomes integrated into our modern life.

Generative AI 105

Generative AI Energy Architecture System 105

InfoQ Culture Methods

MAY 24, 2023

GDPR compliance should be a default feature in every application that handles PII (Personally Identifiable Information). Most organizations have an impression that GDPR is a luxury feature that needs special tools to implement.

Programming 115

Programming Compliance Applications Tools 115

Harvard Business Review

MAY 10, 2024

They determined that, although data engineers and data scientists typically take on most responsibility from conception to production of AI development lifecycles, non-technical leaders can play a key role in ensuring the integration of responsible AI.

How To 128

How To Data Engineering Research Engineering 128

InfoQ Culture Methods

JANUARY 23, 2024

Generative AI will profoundly transform communication and information sharing over the next decade, but the change will be uneven across industries and roles. Organizations should empower workers to use AI augmentation thoughtfully, while building literacy on capabilities and limits.

Generative AI 59

Generative AI Industry Organization Data Engineering 59

InfoQ Culture Methods

JANUARY 7, 2022

This article presents how to use data science to detect wastes and impediments, and concepts and related information that help teams to figure out the root cause of impediments they struggle to get rid of.

Software Development 94

Software Development Data Software Trends 94

InfoWorld

DECEMBER 4, 2023

But there was plenty other news to spark the interest of database managers, data scientists, data engineers, and developers, including new extract, transform, load (ETL) services, a new Cost Optimization Hub, and revamped enterprise pricing tier for AWS' cloud-based development tool, dubbed Amazon CodeCatalyst.

AWS 65

AWS Artificial Intelligence Artificial Inteligence Generative AI 65

InfoQ Culture Methods

JULY 17, 2020

To mix the power of the data and the importance of people to offer business intelligence is a key point nowadays. Innovation is not only about the most advanced technology, management and processes are the new era of startups' innovation. The result is not only the most imporant thing, the way you do it more important.

Agile 123

Agile Innovation Culture Business Intelligence 123

Dzone - DevOps

FEBRUARY 7, 2023

As digital transformation reaches more industries, the number of data points generated is growing exponentially. As such, data integration strategies to collect such large volumes of data from different sources in varying formats and structures are now a primary concern for data engineering teams.

Strategy 98

Strategy Data Data Engineering Analysis 98

InfoQ Culture Methods

DECEMBER 28, 2020

As 2020 is coming to an end, we created this article listing some of the best posts published this year. This collection was hand-picked by nine InfoQ Editors recommending the greatest posts in their domain. It's a great piece to make sure you don't miss out on some of the InfoQ's best content.

Continuous Delivery 89

Continuous Delivery Google Cloud Data Engineering Microservices 89

Perficient

MARCH 23, 2023

Databricks Streaming and Apache Flink are two popular stream processing frameworks that enable developers to build real-time data pipelines, applications and services at scale. Comparison Databricks is an integrated platform for data engineering, machine learning, data science and analytics built on top of Apache Spark.

Data 83

Data Artificial Inteligence Machine Learning Data Engineering 83

Crunchbase News

JUNE 26, 2024

Editor’s note: This article is part of an ongoing series in which Crunchbase News interviews active investors in artificial intelligence. Hawkeye analyzes issues with the software stack based on prompts from engineers, chats with the team, and provides solutions. Each one of us will have a genie. billion in 2021.

Artificial Intelligence 101

Artificial Intelligence Artificial Inteligence Generative AI Software Review 101

CIO

FEBRUARY 14, 2023

From architectures and databases to feature stores and feature engineering, a myriad of variables must work in sync for this to be accomplished. Flexible and easy-to-work-with data models are the oil that makes the engine for building models run smoothly. Data and ML model development fundamentally depend on one another.

Artificial Inteligence 284

Artificial Inteligence Strategy Data Machine Learning 284

InfoQ Culture Methods

FEBRUARY 25, 2021

Developing ML with agile has a few challenges that new teams coming up in the space need to be prepared for - from new roles like Data Scientists to concerns in reproducibility and dependency management. Machine learning is a powerful new tool, but how does it fit in your agile development? By Jay Palat.

Artificial Inteligence 118

Artificial Inteligence Machine Learning Agile Development 118

Altexsoft

FEBRUARY 11, 2023

This suggests that today, there are many companies that face the need to make their data easily accessible, cleaned up, and regularly updated. Hiring a well-skilled data architect can be very helpful for that purpose. What is a data architect? What is the main difference between a data architect and a data engineer?

Data 87

Data Data Engineering Big Data Architecture 87

Data Virtualization

DECEMBER 23, 2021

Reading Time: 5 minutes As I embark on my new journey with Denodo and data virtualization, I am now frequently asked, “What is data virtualization?” Unless I am talking to a data engineer or a data architect (this would help explain it, if you.

Virtualization 52

Virtualization Data Data Engineering Analysis 52

Data Virtualization

DECEMBER 23, 2021

Reading Time: 5 minutes As I embark on my new journey with Denodo and data virtualization, I am now frequently asked, “What is data virtualization?” Unless I am talking to a data engineer or a data architect (this would help explain it, if you.

Virtualization 52

Virtualization Data Data Engineering Analysis 52

O'Reilly Media - Ideas

AUGUST 18, 2020

That is, products that are laser-focused on one aspect of the data science and machine learning workflows, in contrast to all-in-one platforms that attempt to solve the entire space of data workflows. The Two Cultures of Data Tooling. Lessons Learned from Data Warehouse and Data Engineering Platforms.

Machine Learning 143

Machine Learning Artificial Inteligence Data Data Engineering 143

InfoQ Culture Methods

DECEMBER 6, 2019

As part of the 2019 end-of-year-summary content, this article collects together a list of recommended presentation recordings from the InfoQ editorial team. By Charles Humble, Ben Linders, Arthur Casals, Manuel Pais, Erik Costlow, Shane Hastie, Shaaron A Alvares, Srini Penchikala, Michael Redlich, Steef-Jan Wiggers, Daniel Bryant.

Data Engineering 52

Data Engineering Architecture Programming Engineering 52

InfoQ Culture Methods

JANUARY 19, 2022

In this podcast summary Thomas Betts, Wes Reisz, Shane Hastie, Charles Humble, Srini Penchikala, and Daniel Bryant discuss what they have seen in 2021 and speculate a little on what they hope to see in 2022. Topics explored included: hybrid working and the importance of ethics and sustainability within technology.

Technical Review 94

Technical Review Sustainability Technology Artificial Inteligence 94

Exadel

OCTOBER 18, 2022

No matter the industry, a practical data science approach brings data scientists, stakeholders, and data engineers together to drive business value. The role of a data science team is to use one of three approaches, depending on the maturity of an organization. Data capability and infrastructure assessment.

Data 52

Data Data Engineering Infrastructure Industry 52

InfoQ Culture Methods

SEPTEMBER 6, 2019

Is it all about customer-centric business, or is there any data left? Can we integrate data analytics and customer empathy? This article explores how we can move towards a more customer-centric business and what information we require in order to understand the most valuable thing we have: our customer.

Analytics 63

Analytics Agile Data Big Data 63

Dataiku

AUGUST 30, 2022

This is a guest article by Sooter Saalu. Sooter is a data professional with over four years of experience working across data-centric roles. He has a working knowledge of Python, SQL, and several BI and data engineering tools.

Data Engineering 52

Data Engineering Education Engineering Data 52