Replication Markets
Definition: A replication market is an environment where users bet on the replicability of certain effects. Forecasters are incentivized to make accurate predictions and the top successful forecasters receive monetary compensation or contributorship for their bets. The rationale behind a replication market is that it leverages the collective wisdom of the scientific community to predict which effect will most likely replicate, thus encouraging researchers to channel their limited resources to replicating these effects.
Related terms: Citizen science, Crowdsourcing, Replicability, Reproducibility
References:
- Liu, Y., Gordon, M., Wang, J., Bishop, M., Chen, Y., Pfeiffer, T., Twardy, C., & Viganola, D. (2020). Replication Markets: Results, Lessons, Challenges and Opportunities in AI Replication. ArXiv:2005.04543 . http://arxiv.org/abs/2005.04543
- Tierney, W., Hardy III, J. H., Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., & others. (2020). Creative destruction in science. Organizational Behavior and Human Decision Processes, 161, 291â309. https://doi.org/10.1016/j.obhdp.2020.07.002
- Tierney, W., Hardy III, J., Ebersole, C. R., Viganola, D., Clemente, E. G., Gordon, M., & others. (2021). A creative destruction approach to replication: Implicit work and sex morality across cultures. Journal of Experimental Social Psychology, 93, 104060. https://doi.org/10.1016/j.jesp.2020.104060
- Replication Markets. (n.d.). Replication Markets â Reliable research replicatesâŠyou can bet on it. Replication Markets. https://www.replicationmarkets.com/
Originally drafted by: Ali H. Al-Hoorie
Reviewed by: Mahmoud Elsherif, Leticia Micheli, Sam Parsons