DataPelago has created a new engine called Nucleus that dramatically speeds up data processing for AI and analytics.
It outperforms Nvidia’s cuDF library by large margins while working across different types of hardware.
Today’s GPUs are powerful, but older software often wastes their potential, making faster tools like Nucleus especially valuable.
This shift could have dramatic implications for Nvidia.
For years, enterprises have leaned on GPUs (Graphics Processing Units) to handle ever-growing mountains of data, leveraging their ability to run thousands of calculations in parallel for AI and analytics workloads. Every generative AI model, recommendation engine, and analytics dashboard depends on data libraries to prepare, join, and transform massive datasets.
Yet the industry faces a quiet challenge: Despite advances in hardware, performance often stalls at scaling limits because the software stack struggles to fully exploit the hardware’s capabilities. Many legacy data libraries were optimized for CPUs, not GPUs. As a result, memory bandwidth and compute throughput often go underutilized, and every time data moves between CPU and GPU, much of the performance advantage evaporates.
To address this, Nvidia launched cuDF in 2018 as part of its open-source RAPIDS suite—a GPU-accelerated DataFrame library that quickly became the gold standard for data operations. It delivered speedups over CPU-based libraries and better utilization of GPU hardware.
But cuDF also has limits. It requires an Nvidia GPU with ample memory and CUDA support, ruling out environments without compatible hardware. In many ways, cuDF became the industry’s ceiling: powerful enough to accelerate AI and analytics pipelines, yet constrained by the quirks of GPU architecture itself.
Now, California-based data startup DataPelago says it has surpassed those limits with its universal data processing engine, Nucleus. Built atop Nvidia hardware, Nucleus reportedly delivers performance gains so steep they could reset the economics of GPU acceleration. In a benchmark test, Nucleus outpaced cuDF by 38.6 times on hash joins, eight times on sorts, and 10 times on filters and projections.
“To fully realize the benefits of GPUs, data processing engines need to fully leverage the hardware’s strengths while compensating for its limitations,” says DataPelago CEO Rajan Goyal—something he argues demands fresh algorithms built for data workloads.
The implications go far beyond engineering bragging rights. Cloud GPUs are expensive, and enterprises face pressure to maximize every compute cycle. Faster data processing means lower cloud bills and quicker time-to-insight. Goyal says Nucleus is designed to run on any hardware and handle any type of data, while integrating with existing frameworks without requiring changes to customer applications.
“We slot into the existing environments that developers are already working in,” he adds.
Empowering Enterprise AI with a Hardware-Neutral Approach
DataPelago’s benchmark test ran on standard public-cloud servers with both entry-level Tesla T4 and high-end H100 GPUs. The test mimicked real-world tasks: moving data from CPU to GPU, processing it, and returning results to the host. Using the same dataset and harness, Nucleus was compared head-to-head with cuDF on core AI and analytics operations.
“We wanted to improve the performance ceiling for GPUs, and the only way to do that credibly was to compare ourselves directly to cuDF,” says Goyal. He notes that accelerating data prep by an order of magnitude gives businesses the capacity to process exponentially more information for AI training and retrieval tasks, keeping systems up-to-date.
The engine achieved these results by redesigning its execution layer to handle complex workloads, including kernel fusion, native multi-column support, and optimized handling of variable-length data such as strings. Interestingly, while Nucleus runs on Nvidia’s CUDA framework and GPUs, it delivers higher performance—essentially out-engineering Nvidia on its own tech stack.
DataPelago president JG Chirapurath says Nucleus delivers “far greater performance from existing hardware investments” and stresses that enterprises prefer solutions that build on what they already have rather than forcing a rip-and-replace.
Goyal argues that cuDF is tightly coupled to Nvidia’s GPU ecosystem, creating vendor lock-in and limiting hardware flexibility. This dependence restricts open innovation and ties enterprises to Nvidia’s roadmap.
“Nucleus is designed to work across any hardware (not only GPUs), while also handling any type of data and supporting any query engine. It is designed to lift the performance ceiling of any hardware,” claims Goyal. The engine also includes built-in intelligence that automatically maps data operations to the most suitable hardware and dynamically reconfigures tasks to maximize performance.
Software Over Silicon: The Emerging Battle for Enterprise AI Efficiency
If DataPelago’s approach takes hold, enterprises may begin prioritizing universality and efficiency over single-vendor ecosystems when building AI infrastructure. Still, analysts caution that benchmark results often look stronger in controlled tests than in real-world production, and risks remain if the hardware landscape evolves quickly.
“The offering will appeal to those looking to avoid vendor lock-in,” says Alvin Nguyen, senior analyst at Forrester. “But with tools like AMD’s CUDA translation for its data center GPUs, the real advantage is if you’re also targeting CPUs and Field Programmable Gate Arrays (FPGAs). There is a large population of developers experienced with NVIDIA’s ecosystem, so moving away from NVIDIA now means a bigger short-term investment in other options.”
Nguyen also notes that progress on transformer-based workloads, such as training large foundational models, is slowing compared with prior years. As inferencing begins to outpace training, raw GPU horsepower is no longer the main driver. “A more balanced view, including the software layer, is a smart way to look at things.”
Still, investors are buying in. DataPelago has raised $47 million in seed and Series A funding from Eclipse, Qualcomm Ventures, and Taiwania Capital, and recently hired industry veteran JG Chirapurath as president. CEO Goyal himself worked at Cisco and Oracle before founding the company.
For years, the AI industry has fixated on chip shortages and the race for ever-more powerful GPUs. Nucleus points instead to a different kind of competition. If the biggest performance gains now come from software rather than hardware, the battleground could shift from chip foundries to algorithmic innovation. The future of AI infrastructure may depend less on building bigger chips and more on rethinking how we harness the ones we already have.
“Hardware neutrality is strategic differentiation, not just technical capability. Enterprises want infrastructure investments that remain valuable as technology evolves,” says Chirapurath. “My long-term vision is positioning DataPelago as the universal data processing foundation that accelerates the next decade of AI and analytics innovation. We’re making previously impossible applications economically feasible.”
