Bindplane this week revealed that later this year, it will leverage artificial intelligence (AI) to add an ability to automatically identify log types, apply parsers, and optimize configurations to streamline the management of OpenTelemetry data.
Company CEO Mike Kelly said a Pipeline Intelligence platform will make it possible for organizations to automate 70-80% of the tasks required to manage a data pipeline. Initially, Bindplane will provide local processing capabilities, followed by an ability to store telemetry data in a way that will enable DevOps teams that opt in to that option to apply artificial intelligence (AI) to telemetry data.
That capability is now critical more than ever as the number of OpenTelemetry pipelines and the amount of telemetry data being collected continue to increase, he added. Organizations are not only using OpenTelemetry to instrument applications, they are also increasingly using it to collect telemetry data from security and networking platforms as well, noted Kelly.
Additionally, generative artificial intelligence workloads are now also starting to add massive amounts of telemetry data to pipelines as well, he added. As a result, many organizations will soon reach a breaking point in terms of how telemetry data is currently managed, noted Kelly.
Manually building custom parsers to collect that data today can require multiple days per log type. As the number of log types continues to increase, the need to automatically apply parsers to log types becomes more acute, said Kelly. Pipeline Intelligence now makes it possible to parse and configure a log type in a few minutes, he added.
Bindplane claims that the level of automation represents $50,000-$60,000 in redirected value per year for a data engineer making $200,000 annually. While the cost of collecting telemetry data may vary widely from one organization to another, there is no doubt that organizations are starting to be overwhelmed by it. There is, of course, plenty of opportunity to optimize existing pipelines, but it’s now also only a matter of time before more of them will need to be created to accommodate the amount of telemetry data being collected in parallel.
While existing large language models (LLMs) can be used to help explain how to address some telemetry issues, there will be a need for AI technologies that are specifically built for telemetry data, said Kelly. A general-purpose LLM isn’t as useful for telemetry as many had initially hoped, he noted.
In the meantime, most organizations will discover in the coming year that they will need to simplify the way they collect telemetry data and then automate as many of those workflows as possible, added Kelly.
The value of telemetry data is, of course, often overlooked, but there can be no observability of DevOps workflows without it. Many members of DevOps teams are spending a lot more time collecting and managing telemetry data at the expense of other tasks. An opportunity to automate the management of telemetry data creates an opportunity for DevOps engineers to spend more time fixing the issues surfaced by their observability and monitoring tools. After all, no matter how critical an application is, there are still only so many hours in a day to resolve an issue.
