Expanding the Value of Confidence Estimates from Neural Network Classifiers

Session: Leveraging Advanced Detection, Association and Source Characterization in Network Seismology [Poster]
Type: Poster
Date: 4/30/2020
Time: 08:00 AM
Room: Ballroom
Description: 

Deep neural networks have limited context for learning beyond the expectations encoded in their objective functions, which often reward models for unreasonably high decision confidence. Despite the quickly evolving frontier of solutions for uncertainty quantification, we have a limited understanding of what simple and accessible solutions offer us. Here we explore how well we can improve the fidelity of confidence reporting on event classification in Utah using several different regularization approaches, including enforcing domain expectations for network seismology during learning. The purpose of this work is to test if these methods are able to provide analysts with decision support beyond opaque deterministic responses. We test this by assessing which methods increase the fidelity of confidence values and if subsequently credible confidence thresholds can be established. A second objective of this work is to establish baseline performance for these scalable and accessible approaches before moving on to probabilistic methods where outcomes produced under the simplifying assumptions required for computational tractability will need to outperform the methods explored here in order to be beneficial for seismic monitoring.

Presenting Author: Lisa Linville

Authors