SAGE Data Streaming

Overview

The SAGE cyberinfrastructure (www.sageplatform.org) is an advanced software-defined sensor network designed to leverage AI at the edge for real-time data collection and analysis in IoT environments. SAGE supports transformative research by enabling edge-based processing of high-resolution data from diverse sensors, including cameras, air quality monitors, and LIDAR systems. This capability is vital for applications such as environmental monitoring, wildlife tracking, and urban analytics.

As part of the National Data Platform (NDP) initiative, SAGE integrates seamlessly into a robust workflow for data processing and dissemination.

The architecture, illustrated in the figure above, highlights the collaboration of several key components:

SAGE: Collects and processes data at the edge, exposing an API for data retrieval
Data Stream Adapter: Connects SAGE with the R-Pulsar Framework, converting data into formats suitable for real-time analysis
R-Pulsar Framework: Extends cloud capabilities to IoT devices, enabling local data analysis, decision-making, and transformation in real time
Data Sink Adapter: Routes processed data to its destination, such as OSDF Origin, for further use or storage
Metadata Catalog: Logs metadata throughout the workflow to ensure data traceability and management
User & OSDF Origin: End-users access or subscribe to data streams, while OSDF Origin serves as a repository or platform for processed data.

Explore the Jupyter Notebook demo for a detailed walkthrough

Objective

To create a cutting-edge cyberinfrastructure that harnesses artificial intelligence and edge computing to enable real-time data analysis and decision-making. By deploying a network of smart sensors and integrating advanced machine learning, SAGE aims to provide scientists with powerful tools to monitor, understand, and respond to environmental and societal phenomena with unprecedented speed and precision.

Key Features

SciDx data streaming
Real time data analysis
Data set registration

Collaborators

This use case highlights collaboration between the University of Utah, Sage Continuum (https://sagecontinuum.org/), and the NSF.