Red Hat is Acquiring AI Optimization Startup Neural Magic (techcrunch.com) 4
Red Hat, the IBM-owned open source software firm, is acquiring Neural Magic, a startup that optimizes AI models to run faster on commodity processors and GPUs. From a report: The terms of the deal weren't disclosed. MIT research scientist Alex Matveev and professor Nir Shavit founded Somerville, Massachusetts-based Neural Magic in 2018, inspired by their work in high-performance execution engines for AI. Neural Magic's software aims to process AI workloads on processors and GPUs at speeds equivalent to specialized AI chips (e.g. TPUs). By running models on off-the-shelf processors, which usually have more available memory, the company's software can realize these performance gains.
Big tech companies like AMD and a host of other startups, including NeuReality, Deci, CoCoPie, OctoML and DeepCube, offer some sort of AI optimization software. But Neural Magic is one of the few with a free platform and a collection of open source tools to complement it. Neural Magic had so far managed to raise $50 million in venture capital from backers like Andreessen Horowitz, New Enterprise Associations, Amdocs, Comcast Ventures, Pillar VC and Ridgeline Ventures.
Big tech companies like AMD and a host of other startups, including NeuReality, Deci, CoCoPie, OctoML and DeepCube, offer some sort of AI optimization software. But Neural Magic is one of the few with a free platform and a collection of open source tools to complement it. Neural Magic had so far managed to raise $50 million in venture capital from backers like Andreessen Horowitz, New Enterprise Associations, Amdocs, Comcast Ventures, Pillar VC and Ridgeline Ventures.
systemd-ai (Score:3)
Re: (Score:2)
Fortunately, Red Hat has throttled the mad dash to replace the operating system with systemd that Lennart Pottering pursued so aggressively until his switch to Microsoft. I suspect that was a primary reason he left Red Hat: they were reining in his enthusiasm after some notable disasters.
Yet another API? (Score:2)
That's the only way I can imagine this functioning.
Even OpenACC (a CUDA alternative) could run stuff on the CPU instead of a GPU. Making something run optimally on a wide range of devices is not easy though. Add to that it was already slower/worse than CUDA in general... with little motivation for NVIDIA to spend a lot of effort changing that.
Re: (Score:2)
Or sell vLLM support and services.
Meta contracted them to improve LLama support in vLLM for instance. Redhat might want to start offering consultancy for companies who want to finetune and run open source models with internal data too.