Best Practices and Processes for Choosing Research Priorities

Abstract:

Individuals, teams, departments, organizations, funding agencies, committees, and others all need to select desirable research priorities from many possible alternatives. One cannot do everything, one cannot afford everything, so what to select?

Essays and reports since Weinberg (1963) have suggested criteria for choosing science topics. Popper et al (2000) reviewed and summarized all that had gone before in the subject of setting priorities; their main conclusions were that the underlying principles were the promotion of excellence and relevance. Sea Change (2015) from the NRC/OSB focused on four criteria. From most important to least important, they were transformative science, societal impacts, readiness, and partnership potential; these four criteria embodied the essence of the suggestions from Weinberg on, framed with the pragmatism of ORPISS (2007).

Getting to the final set of priorities from many candidates involves a sequence of formal or informal processes, only the last of which is the application of the selected, weighted criteria. As developed by professional prioritization experts, the best-practice steps and processes are:

• Collection of input candidates from the community.
• Clustering and parsing/rephrasing of the input to eliminate redundancy and repetition and develop statements at a useful level of specificity. (NOTE:there is no counting of input to see how many times a particular topic was mentioned. The goal is diversity in the input, not a popularity contest.)
• Development of the selection criteria, and weighting the chosen criteria.
• Application of the selection criteria to the clustered/adjusted candidates.

Finally, two more best practices:

• Do continuing sanity checks, to avoid losing sight of the goals of the effort.
• Resist the temptation to just sit around a table and talk about it to arrive at the priorities, which depends too much on who the specific members of the prioritization team are, and provides no transparency or explanation of why those specific priorities were selected.