In:
Genome Biology, Springer Science and Business Media LLC, Vol. 24, No. 1 ( 2023-05-17)
Abstract:
Computational methods represent the lifeblood of modern molecular biology. Benchmarking is important for all methods, but with a focus here on computational methods, benchmarking is critical to dissect important steps of analysis pipelines, formally assess performance across common situations as well as edge cases, and ultimately guide users on what tools to use. Benchmarking can also be important for community building and advancing methods in a principled way. We conducted a meta-analysis of recent single-cell benchmarks to summarize the scope, extensibility, and neutrality, as well as technical features and whether best practices in open data and reproducible research were followed. The results highlight that while benchmarks often make code available and are in principle reproducible, they remain difficult to extend, for example, as new methods and new ways to assess methods emerge. In addition, embracing containerization and workflow systems would enhance reusability of intermediate benchmarking results, thus also driving wider adoption.
Type of Medium:
Online Resource
ISSN:
1474-760X
DOI:
10.1186/s13059-023-02962-5
Language:
English
Publisher:
Springer Science and Business Media LLC
Publication Date:
2023
detail.hit.zdb_id:
2040529-7
Permalink