← Back to Sessions

Development of Near Real Time Data Streams from NOTA Borehole Strainmeters

Session: Advances in Real-Time Geophysical Network Operations and Data Analytics

Type: Oral

Date: 4/22/2021

Presentation Time: 05:30 PM Pacific

Description: 

The National Science Foundation (NSF) funded Network of the Americas (NOTA) borehole strainmeter (BSM) network is made of 85 Gladwin Tensor Strainmeters (GTSMs), which have been operated for the last 15 years returning data as hourly downloads. Recent interest in potentially using BSMs in earthquake early warning (EEW) has driven a development effort to upgrade the network to near real time.

A team at UNAVCO is currently developing modern tools for data processing and distribution based on a publish-subscribe messaging system using the Apache Kafka platform. The goal is to create a robust and scalable framework that can handle sensor agnostic timeseries data input. We plan to publish data at multiple processing levels to individual topics, allowing users to subscribe to a topic and receive the data product that best suits their needs. To date we have begun development for GNSS, Strain, Tilt, and Met data on a local 10-node Kubernetes cluster.

We detail here the prototype system being built to stream data in near real time from the NOTA BSMs. Due to their age, the GTSM data loggers are unable support streaming directly. Instead, we have new firmware that supports writing a local data file every few seconds. These 20 sps files will be pulled from the logger, parsed, and produced into Kafka messages. Users interested in the lowest latency data can subscribe to these raw messages. We also plan to build modules that consume the raw streams and produce higher level products including dynamic, areal, and shear strains. These raw and processed streams can also be directed to a database such as TimescaleDB, which users can query to recover past data. We are also experimenting with a frontend such as Grafana for viewing the data streams.

A prototype system currently on test achieves latencies of roughly one to four seconds for 20 sps data over the cell network. This effort is very much a work in progress, and we look forward to discussing these issues with others working on similar problems.

Presenting Author: Mike H. Gottlieb

Student Presenter: No

Authors