← Back to Sessions

WITHDRAWN Using Multi-Resolution Grids and Mcc-F1 curve to Improve Aftershock Forecast Testability

Description: 

W/D Testing a forecast model is key to further improving it. Aftershock forecasts are usually provided for each cell of a gridded region, and the spatial forecasts are often evaluated by using the Receiver Operating Characteristic (ROC) curve. It draws a binary comparison of earthquake occurrences or non-occurrence for each grid cell. Previous tests suggested that Coulomb stress is not a good predictor of aftershocks despite its large use in literature. However, the results of the ROC tests were questioned because data imbalance can largely bias the results.

We analyze the feasibility of ROC for evaluating aftershock forecast models. We conduct a synthetic experiment by creating a perfect Coulomb model, in which all earthquakes are made to occur in the positive stress change regions, and compare it with simpler distance-based all-positive forecasts. The experiment shows that ROC favors the models that only tend to forecast the active regions, while the models trying to forecast the quiet regions are at a disadvantage, thereby highlighting flaws in using ROC for aftershock forecast ranking.

We suggest a two-fold improvement in the testing strategy. We propose Matthews Correlation Coefficient (MCC) and F1 curve as a replacement for the ROC curve. Furthermore, we suggest using a multi-resolution test grid adapted to the earthquake density. We conduct the same synthetic experiment to evaluate forecast models using MCC-F1 and a radial grid (a simple example of a multi-resolution grid). With this strategy, we can better differentiate between the Coulomb forecast model and the non-informative model, particularly in the case of existing outliers. We also use this strategy to test forecast models for Chi-Chi and Landers aftershocks. For evaluating those real scenarios, we use the recently introduced Quadtree approach to generate multi-resolution grids. Despite the improved tests, we find that the simple distance-based model outperforms the Coulomb model in both cases, indicating that MCC-F1 and multi-resolution grids do not alter the ranking of the models as long as the Coulomb model is not adjusted to local conditions.

Session: New Methods and Models for More Informative Earthquake Forecasting

Type: Oral

Date: 4/19/2023

Presentation Time: 11:30 AM (local time)

Presenting Author: Muhammad Asim Khawaja

Student Presenter: Yes

Invited Presentation: 

Authors