News

The state of data matters. Microsoft knows a thing or two about network health and backend monitoring, so this month came forward with the launch of Drasi. As an open-source project from the Microsoft Azure Incubations team, Drasi is designed to manage change detection and reaction in complex software systems from a data status perspective.

Letโ€™s first ask why something like Drasi might be needed, before we look at how it works.

This technology is aligned for use in real-time event-driven architectures that serve compute and processing jobs in complex networks supported by vector or time-series databases, or both. Suppose a database needs to serve a rapid transaction retail processing system, a sensor recording system on a wind turbine, or a financial trading application. In these scenarios, the database is put under a high degree of pressure from core applications that make regular calls to the database to read existing data and to ingest new data records.

AWS

Software application developers that need to track the state of a changing database in these environments will need to make multiple โ€œpollingโ€ connections to it to assess its state at any given time, which means network throughput and expenditure on system energy.ย 

But itโ€™s easier to read a smaller computing entity such as a log (or even a log database) than an entire database in its own right.ย 

Path Of Least Resistance

As a result of this core truth, Drasi monitors the logs created by the database, rather than the source motherlode of data itself. Because every database interaction (write, read, insert, save, update, delete) creates a database log, Drasi is built to analyze logs and so understand the changing state of data within the core database.

Alleviating database load in this way enables Drasi to help software engineers detect critical events within the data layer and take a more immediate approach to action that is tuned to business objectives.ย ย 

Event-driven systems are now being deployed in the locations noted above and increasingly across the edge compute space that serves the Internet of Things (IoT). This is efficient decoupling of services, but it comes with challenges as โ€œeventsโ€ grow in frequency and complexity. Additional complexity arises when data is stored in various different formats and silos, as would not be unexpected in a complex distributed system. Achieving real-time responses inside these systems is crucial, but processing delays can occur due to network latency, congestion or slow event processing.

Inefficient โ€˜Pollingโ€™ Mechanismsย 

โ€œCurrently, developers struggle to build event-handling mechanisms because available libraries and services rarely offer an end-to-end, unified framework for change detection and reaction. They must often piece together multiple tools, resulting in complex, fragile architectures that are hard to maintain and scale. For example, existing solutions may rely on inefficient polling mechanisms [to query the database] or require constant querying of data sources, leading to performance bottlenecks and increased resource consumption. Also, many change detection tools lack true real-time capabilities, utilizing batch processing, data collation or delayed event analysis. For businesses that need immediate reactions, even these slight delays can lead to missed opportunities or risks,โ€ said Mark Russinovich, chief technology officer, deputy chief information security officer and technical fellow, Microsoft Azure.

Automated Appropriate Actions

Drasi simplifies the automation of appropriate intelligent reactions in dynamic systems of the kind described here. It is capable of delivering what the technology industry loves to call โ€œreal-time actionable insightsโ€ without the overhead of traditional data processing methods. This can be described as a lightweight approach to tracking system changes by watching for events in logs and change feeds, without copying data to a central data lake or repeatedly querying data sources.

โ€œApplication developers use database queries to define which changes to track and express logical conditions to evaluate change data,โ€ explained Russinovich. โ€œDrasi then determines if any changes trigger updates to the result sets of those queries. If they do, it executes context-aware reactions based on business needs. This streamlined process reduces complexity, ensures timely action while the data is most relevant and prevents important changes from slipping through the cracks.โ€ย 

Drasi has three components: Sources, Continuous Queries, Reactions:

  • Sources connect to data sources to continuously monitor for critical changes. A Source tracks application logs, database updates, or system metrics and gathers relevant information in real-time.
  • Continuous Queries exist instead of manual, point-in-time queries. Comparatively self-explanatory then, this continuous process constantly evaluates incoming changes based on predefined criteria.ย 
  • Drasi executes registered automated reactions that are capable of sending alerts, updating other systems, or performing remediation steps.

A practical example that showcases Drasiโ€™s real-world applicability is its use in smart building management.ย 

โ€œFacilities managers typically use dashboards to monitor the comfort levels of their spaces and need to be alerted when there are deviations in these levels. With Drasi, creating an always-accurate dashboard was simple. The building spaces are represented in a Microsoft Azure Cosmos DB database, which records room condition updates. A Drasi Source reads the change logs of the Azure Cosmos DB database and passes this change data to Continuous Queries that calculate the comfort levels for individual rooms and provide aggregate values for entire floors and the building itself. A Reaction for SignalR receives the output of the Continuous Queries and directly drives updates to a browser-based dashboard,โ€ said Russinovich.

Drasi has been submitted to the Cloud Native Computing Foundation (CNCF) as a Sandbox project. To learn more and get started with Drasi licensed under the Apache 2.0 license, visit drasi.io.

Techstrong TV

Click full-screen to enable volume control
Watch latest episodes and shows

Networking Field Day

SHARE THIS STORY

RELATED STORIES