Description: Seismic event detection and phase picking are the base of many seismological workflows. In recent years, several publications demonstrated that deep learning approaches significantly outperform classical approaches and even achieve human-like performance under certain circumstances. However, as most studies differ in the datasets and exact evaluation tasks studied, it is yet unclear how the different approaches compare to each other. Furthermore, there are no systematic studies how the models perform in a cross-domain scenario, i.e., when applied to data with different characteristics. Here, we address these questions by conducting a large-scale benchmark study. We compare six previously published deep learning models on eight datasets covering local to teleseismic distances and on three tasks: event detection, phase identification and onset time picking. Furthermore, we compare the results to a classical Baer-Kradolfer picker. Overall, we observe the best performance for EQTransformer, GPD and PhaseNet, with EQTransformer having a small advantage for teleseismic data. Furthermore, we conduct a cross-domain study, in which we analyze model performance on datasets they were not trained on. We show that trained models can be transferred between regions with only mild performance degradation, but not from regional to teleseismic data or vice versa. As deep learning for detection and picking is a rapidly evolving field, we ensured extensibility of our benchmark by building our code on standardized frameworks and making it openly accessible. This allows model developers to easily compare new models or evaluate performance on new datasets, beyond those presented here. Furthermore, we make all trained models available through the SeisBench framework, giving end-users an easy way to apply these models in seismological analysis.

Description: Machine Learning (ML) methods have seen widespread adoption in seismology in recent years. The abilityof these techniques to efficiently infer the statistical properties of large datasets often provides significantimprovements over traditional techniques when the number of data are large(»millions of examples). Withthe entire spectrum of seismological tasks, e.g., seismic picking and detection, magnitude and source propertyestimation, ground motion prediction, hypocentre determination; among others, now incorporating ML ap-proaches, numerous models are emerging as these techniques are further adopted within seismology. To evaluatethese algorithms, quality controlled benchmark datasets that contain representative class distributions are vital.In addition to this, models require implementation through a common framework to facilitate comparison.Accessing these various benchmark datasets for training and implementing the standardization of models iscurrently a time-consuming process, hindering further advancement of ML techniques within seismology. Thesedevelopment bottlenecks also affect ’practitioners’ seeking to deploy the latest models on seismic data, withouthaving to necessarily learn entirely new ML frameworks to perform this task. We present SeisBench as a soft-ware package to tackle these issues. SeisBench is an open-source framework for deploying ML in seismology.SeisBench standardises access to both models and datasets, whilst also providing a range of common processingand data augmentation operations through the API. Through SeisBench, users can access several seismologicalML models and benchmark datasets available in the literature via a single interface. SeisBench is built to beextensible, with community involvement encouraged to expand the package. Having such frameworks availablefor accessing leading ML models forms an essential tool for seismologists seeking to iterate and apply the nextgeneration of ML techniques to seismic data.