article thumbnail

Getting infrastructure right for generative AI

CIO

For generative AI, a stubborn fact is that it consumes very large quantities of compute cycles, data storage, network bandwidth, electrical power, and air conditioning. Infrastructure-intensive or not, generative AI is on the march. of the overall AI server market in 2022 to 36% in 2027.

article thumbnail

Supercharge your auto scaling for generative AI inference – Introducing Container Caching in SageMaker Inference

AWS Machine Learning - AI

Today at AWS re:Invent 2024, we are excited to announce the new Container Caching capability in Amazon SageMaker, which significantly reduces the time required to scale generative AI models for inference. In our tests, we’ve seen substantial improvements in scaling times for generative AI model endpoints across various frameworks.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

Together raises $20M to build open source generative AI models

TechCrunch

Generative AIAI that can write essays, create artwork and music, and more — continues to attract outsize investor attention. According to one source, generative AI startups raised $1.7 billion in Q1 2023, with an additional $10.68 billion worth of deals announced in the quarter but not yet completed.

article thumbnail

The ElliQ eldercare robot gets a hardware upgrade, generative AI for improved conversations

TechCrunch

In some parts of the world (read: Japan, primarily), eldercare has been an important robotics focus for decades. In recent years, other markets have begun exploring the space. Labrador Robotics’ home assistive system is a good example here in the States.

Hardware 253
article thumbnail

Should finance organizations bank on Generative AI?

CIO

As I work with financial services and banking organizations around the world, one thing is clear: AI and generative AI are hot topics of conversation. Financial organizations want to capture generative AI’s tremendous potential while mitigating its risks. In short, yes. But it’s an evolution. billion by 2032.

article thumbnail

Gartner projects major IT spending increases for 2025

CIO

growth this year, with data center spending increasing by nearly 35% in 2024 in anticipation of generative AI infrastructure needs. This spending on AI infrastructure may be confusing to investors, who won’t see a direct line to increased sales because much of the hyperscaler AI investment will focus on internal uses, he says.

article thumbnail

The mainframe’s future in the age of AI

CIO

If there’s any doubt that mainframes will have a place in the AI future, many organizations running the hardware are already planning for it. Many Kyndryl customers seem to be thinking about how to merge the mission-critical data on their mainframes with AI tools, she says.

Survey 200