Evolving needs catalyze innovation.
This is applicable to the creation of the Apache NiFi product at the heart of Datavolo, but it’s also relevant to why the time is right for Datavolo.
Joe Witt is the original leader of the NiFi project - a technology created at the United States’ NSA which came to the OSS world through a Federal Technology Transfer Program. The needs of the NSA had evolved - it was essential to build a system that could route data with different modalities (highly structured information, very unstructured documents, photos, audio, etc.), and do it based on a systematic and dynamic understanding of priorities and a complex network infrastructure. This is not something the streams processing or queuing world could do then, nor can it do it now. The needs changed and Joe was at the heart of an important innovation - one that helped save lives. The technology is widely used across the Government sector today.
As the work was made open source, Apache NiFi has gained traction around the world. NiFi has 14M+ Docker pulls, 450+ code contributors and is in production use at thousands of enterprises worldwide in all industries. In particular, it has heavy use in higher regulation industries such as Government, Healthcare, Finance, and Telecommunications because of its strong focus on security and governance. The customers that have come to understand they need it, are effectively only able to achieve the goals for operating parts of their business at the time and pace of the business, not in batch. And that insight is why we think now is the time to get behind co-founders, Joe and Luke and the rest of the Datavolo team in order to meet the burgeoning needs in data management.
We think more will be written about this, but it's become clear that the relational data model, which we have effectively built the entire modern economy on, is now going to be a peer to a new data model that is rising to meet the emerging needs of AI. We are at an inflection point.
We believe the opportunity that AI presents is going to drive momentous changes in data management. Building AI systems that are part of running the business, moment to moment, is creating a need to rethink the data architecture. Applications built with AI inference require the right data at the right time - extrapolating data patterns and extracting what is needed is not how databases have been built, which were historically oriented to transactional events and batch processing. We have the opportunity with AI to build systems that are working with the business - assisting a sales agent, supply chain manager, field technician, or any number of the countless jobs people are doing every day.
The detailed technical dive into what Joe and Luke wanted to build underscored their innovative approach to data handling, management, and utilization. This wasn't about creating just another data platform looking to arbitrage a gap in the market offerings; it was about laying the groundwork for a future where data is not just managed but is also intelligently leveraged to meet these new needs coming from AI.
I had the pleasure of meeting Joe and then Luke via a mutual friend, as they were thinking about what to do next. Joe’s focus was on how to make a significant impact, and it’s just fantastic that answering that question led him back to the NiFi work in partnership with our Creation strategy. This is a technology ahead of its time, having rapidly matured over the past year, and we believe is poised to transform data management.
