Using Machine Learning to Enable Big Data Analysis within Human Review Time Budgets

Abstract:

The quantity of astronomical observations collected by today’s instruments far exceeds the capability of manual inspection by domain experts. Scientists often have a fixed time budget of a few hours spend to perform the monotonous task of scanning through a live stream or data dump of candidates that must be prioritized for follow-up analysis. Today’s and next generation astronomical instruments produce millions of candidate detection per day, and necessitate the use of automated classifiers that serve as “data triage” in order to filter out spurious signals. Automated data triage enables increased science return by prioritizing interesting or anomalous observations for follow-up inspection, while also expediting analysis by filtering out noisy or redundant observations. We describe three specific astronomical investigations that are currently benefiting from data triage techniques in their respective processing pipelines.