Real-time sensor data streams (original) (raw)

An extensible infrastructure for processing distributed geospatial data streams

2006

Although the processing of data streams has been the focus of many research efforts in several areas, the case of remotely sensed streams in scientific contexts has received little attention. We present an extensible architecture to compose streaming image processing pipelines spanning multiple nodes on a network using a scientific workflow approach. This architecture includes (i) a mechanism for stream query dispatching so new streams can be dynamically generated from within individual processing nodes as a result of local or remote requests, and (ii) a mechanism for making the resulting streams externally available. As complete processing image pipelines can be cascaded across multiple interconnected nodes in a dynamic, scientist-driven way, the approach facilitates the reuse of data and the scalability of computations. We demonstrate the advantages of our infrastructure with a toolset of stream operators acting on remotely sensed data streams for realtime change detection.

A stream database server for sensor applications

2002

We present a framework for stream data processing that incorporates a stream database se11Jer as a fundamental component. The server operates as the stream control iflterjace between arrays of distributed data stream sources and end-user clients thaJ access and analyze the streams. The underlying framework provides novel stream managemem and query processing mechanisms to support the online acquisition, management, storage, non-blocking query. and imegration of data streams for distributed muLti-sensor networks. In this paper, we define OUT stream model and stream representation for the stream database, and we describe the functionality alld implementation of key components of the stream processing framework, including the query processing interface for source streams, the stream manager, the stream buffer manager, non-blocking query execution, and a new class ofjoin aLgorithms for joining multipLe data streams constrained by a sliding time window. We conduct experiments using real data streams to evaluate the performance of the new aLgoritluns against traditional stream join aLgorithms. The experiments show significant performance improvements and aLso demonstrate the flexibility of our system ;n handling data streams. A muLti-sensor network appLicatioll for the intelligent detection of lwzardous materials ;s presented to illustrate the capabilities ofourframework.

An Environmental Monitoring System for Managing Spatiotemporal Sensor Data over Sensor Networks

Sensors (Basel, Switzerland), 2012

In a wireless sensor network, sensors collect data about natural phenomena and transmit them to a server in real-time. Many studies have been conducted focusing on the processing of continuous queries in an approximate form. However, this approach is difficult to apply to environmental applications which require the correct data to be stored. In this paper, we propose a weather monitoring system for handling and storing the sensor data stream in real-time in order to support continuous spatial and/or temporal queries. In our system, we exploit two time-based insertion methods to store the sensor data stream and reduce the number of managed tuples, without losing any of the raw data which are useful for queries, by using the sensors' temporal attributes. In addition, we offer a method for reducing the cost of the join operations used in processing spatiotemporal queries by filtering out a list of irrelevant sensors from query range before making a join operation. In the results o...

Real-Time Stream Data Management

SpringerBriefs in Computer Science, 2019

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

MaD-WiSe: a distributed stream management system for wireless sensor networks

2010

Wireless sensor networks (WSN) are composed of several sensors having limited memory, processing power, communication bandwidth, and energy, which cooperate in performing a given task. The use of the database paradigm has emerged in the last few years as a viable solution to manage data in such a context. In this paper we present the MaD-WiSe system, a distributed query processing framework that moves the processing of the query into the network. MaD-WiSe reconsiders various aspects related to database system design and it reinterprets them according to the WSN constraints and requirements. In particular it considers the aspects related to the definition of a query language to formalize the queries, a stream model to manage data acquired by the sensors, a query algebra to define the operators that actually perform the query, and energy efficiency and query optimization strategies for saving energy.

The Case for a Signal-Oriented Data Stream Management System

2007

Sensors capable of sensing phenomena at high data rates-on the order of tens to hundreds of thousands of samples per second-are useful in many industrial, civil engineering, scientific, networking, and medical applications. In these applications, high-rate streams of data produced by sensors must be processed and analyzed using a combination of both event-stream and signal-processing operations. This paper motivates the need for a data management and continuous query processing architecture that integrates these two different desired classes of functions into a single, unified software system. The key goals of such a system include: the ability to treat a sequence of samples that constitute a "signal segment" as a basic data type; ease of writing arbitrary event-stream and signal-processing functions; the ability to process several million samples per second on conventional PC hardware; and the ability to distribute application code across both PCs and sensor nodes.

SensorStream: A Semantic Real-Time Stream Management System

2012

As data proliferates at increasing rates, the need for real-time stream processing applications increases as well. In the same way that data stream management systems have emerged from the database community, there is now a similar concern in managing dynamic knowledge among the Semantic Web community. Unfortunately, early relevant approaches are to a large extent theoretical and do not present convincing evidence of their efficiency in real dynamic environments. In this paper, we present a framework for the effective, real-time processing of streaming data and we define and analyze in depth its key components. Our framework serves as a basis for the implementation of the SensorStream prototype, on which we run numerous performance and scalability measurements that outline its behaviour and demonstrate its suitability and scalability for solutions that require real-time information processing from distributed and heterogeneous data sources.

CAREER: Data Management for Ad-Hoc Geosensor Networks

2011

is working on the topic of spatio-temporal data streams, especially the data stream model support of sensor data streams capturing continuous phenomena. He will defend his thesis proposal in June 2011. Name: Nural, Arda Worked for more than 160 Hours: Yes Contribution to Project: Arda Nural has started as a Ph.D. student at the University of Maine in Fall 2003. His research topic is in the area of mobile geosensor networks and data streams management systems. In particular, he focuses on the detection of emerging flock patterns in crowds of moving objects and the tracking of topological behavior such as splitting and merging of flocks, and the identification of leader patterns. He defended successfully defended his thesis proposal in April 2009. Name: Jin, Guang Worked for more than 160 Hours: Yes Contribution to Project: Guang Jin was a Ph.D. student and graduated successfully in April 2009. His research area was in the realm of quantitative and qualitative detection of events in geosensor networks. Name: Xiao, Danqing Worked for more than 160 Hours: Yes Contribution to Project: The female student is currently funded by the correlated NSF project 'Monitoring dynamic spatial fields using responsive geosensor networks', PI M. Worboys, Co-PI S. Nittel. She successfully defended her MS thesis in April 2010, and her topic was detection of non-topological changes in geosensor networks.

Integration of Reliable Sensor Data Stream Management into Digital Libraries

Lecture Notes in Computer Science, 2007

Data Stream Management (DSM) addresses the continuous processing of sensor data. DSM requires the combination of stream operators, which may run on different distributed devices, into stream processes. Due to the recent advantages in sensor technologies and wireless communication, the amount of information generated by DSM will increase significantly. In order to efficiently deal with this streaming information, Digital Library (DL) systems have to merge with DSM systems. Especially in healthcare, the continuous monitoring of patients at home (telemonitoring) will generate a significant amount of information stored in an e-health digital library (electronic patient record). In order to stream-enable DL systems, we present an integrated data stream management and Digital Library infrastructure in this work. A vital requirement for healthcare applications is however that this infrastructure provides a high degree of reliability. In this paper, we present novel approaches to reliable DSM within a DL infrastructure. In particular, we propose information filtering operators, a declarative query engine called MXQuery, and efficient operator checkpointing to maintain high result quality of DSM. Furthermore, we present a demonstrator implementation of the integrated DSM and DL infrastructure, called OSIRIS-SE. OSIRIS-SE supports flexible and efficient failure handling to ensures complete and consistent continuous data stream processing and execution of DL processes even in the case of multiple failures.

Issues in data stream management

2003

Abstract Traditional databases store sets of relatively static records with no pre-defined notion of time, unless timestamp attributes are explicitly added. While this model adequately represents commercial catalogues or repositories of personal information, many current and emerging applications require support for on-line analysis of rapidly changing data streams.