Application of Big Data Technologies on Real-Time Seismic Data Stream Processing
Session: Advances in Real-Time Geophysical Network Operations and Data Analytics
Type: Oral
Date: 4/22/2021
Presentation Time: 05:00 PM Pacific
Description:
The ever-growing seismic data connected to a real-time seismic data stream processing system requires a robust processing capacity, a stable message exchanging center, and a scalable data cache. One practical solution is to divide and conquer, in which parallel processing and distributed storage and caching are essential. The newly developed seismic monitoring technologies also face challenges when integrating with traditional tight-coupled earthquake monitoring systems. In this study, we developed a real-time seismic data stream processing system named SeisFLINK based on big data technologies with a completely different architecture from previously. The architecture of the system has three layers; the bottom layer provides hardware and operating system support; the middle layer is the big data technology supporting system that provides solutions for message distribution, distributed real-time stream processing, caching, and archiving, which are powered by Apache Kafka, Apache Flink, Redis, and TDengine, respectively; and seismic applications and additional tools are developed on the top layer and could exchange message and seismic data with message exchanging center and data caching center by universal standard interfaces. Big data stream processing systems guarantee a powerful seismic data flow system and their features naturally fulfill the requirements of parallel processing and distributed storage and caching. Loose-coupled architecture makes the system more modular, easier to integrate newly developed applications and upgrade or replace modules and subsystems separately. The independence of the operating system and hardware increases its flexibility that can be configured on a cluster of physical computers or virtual machines in the Cloud.
Presenting Author: Xinghui Huang
Student Presenter: No
Authors
Xinghui Huang Presenting Author hsxyhq@126.com China Earthquake Networks Center |
Yanlu Ma Corresponding Author mayl@seis.ac.cn China Earthquake Networks Center |
Tong Chen chentong@seis.ac.cn China Earthquake Networks Center |
Xuejun Han hxj@seis.ac.cn China Earthquake Networks Center |
Hongfeng Chen chf@seis.ac.cn China Earthquake Networks Center |
|
|
|
|
Application of Big Data Technologies on Real-Time Seismic Data Stream Processing
Category
Advances in Real-Time Geophysical Network Operations and Data Analytics