St Andrews Research Repository

St Andrews University Home
View Item 
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  •   St Andrews Research Repository
  • University of St Andrews Research
  • University of St Andrews Research
  • University of St Andrews Research
  • View Item
  • Login
JavaScript is disabled for your browser. Some features of this site may not work without it.

Examining the generalizability of research findings from archival data

Thumbnail
View/Open
Delios_2022_PNAS_Examining_generalizability_CC.pdf (843.2Kb)
Date
26/07/2022
Author
Delios, Andrew
Clemente, Elena Guilia
Wu, Tao
Tan, Hongbin
Wang, Yong
Gordon, Michael
Viganola, Domenico
Chen, Zhaowei
Dreber, Anna
Johannesson, Magnus
Pfeiffer, Thomas
Generalizability Tests Forecasting Collaboration
Uhlmann, Eric Luis
Keywords
Research reliability
Generalizability
Archival data
Reproducibility
Context sensitivity
ZA4050 Electronic information resources
DAS
Metadata
Show full item record
Altmetrics Handle Statistics
Altmetrics DOI Statistics
Abstract
This initiative examined systematically the extent to which a large set of archival research findings generalizes across contexts. We repeated the key analyses for 29 original strategic management effects in the same context (direct reproduction) as well as in 52 novel time periods and geographies; 45% of the reproductions returned results matching the original reports together with 55% of tests in different spans of years and 40% of tests in novel geographies. Some original findings were associated with multiple new tests. Reproducibility was the best predictor of generalizability—for the findings that proved directly reproducible, 84% emerged in other available time periods and 57% emerged in other geographies. Overall, only limited empirical evidence emerged for context sensitivity. In a forecasting survey, independent scientists were able to anticipate which effects would find support in tests in new samples.
Citation
Delios , A , Clemente , E G , Wu , T , Tan , H , Wang , Y , Gordon , M , Viganola , D , Chen , Z , Dreber , A , Johannesson , M , Pfeiffer , T , Generalizability Tests Forecasting Collaboration & Uhlmann , E L 2022 , ' Examining the generalizability of research findings from archival data ' , Proceedings of the National Academy of Sciences of the United States of America , vol. 119 , no. 30 , e2120377119 . https://doi.org/10.1073/pnas.2120377119
Publication
Proceedings of the National Academy of Sciences of the United States of America
Status
Peer reviewed
DOI
https://doi.org/10.1073/pnas.2120377119
ISSN
0027-8424
Type
Journal article
Rights
Copyright © 2022 the Author(s). Published by PNAS. This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND).
Description
This research project benefitted from Ministry of Education (Singapore) Tier 1 Grant R-313-000-131-115 (to A. Delios), National Science Foundation of China Grants 72002158 (to H.T.) and 71810107002 (to H.T.), grants from the Knut and Alice Wallenberg Foundation (to A. Dreber) and the Marianne and Marcus Wallenberg Foundation (through a Wallenberg Scholar grant; to A. Dreber), Austrian Science Fund (FWF) Grant SFB F63 (to A. Dreber), grants from the Jan Wallander and Tom Hedelius Foundation (Svenska Handelsbankens Forskningsstiftelser; to A. Dreber), and an Research & Development (R&D) research grant from Institut Européen d'Administration des Affaires (INSEAD) (to E.L.U.). Dmitrii Dubrov, of the G.T.F.C., was supported by the National Research University Higher School of Economics (HSE University) Basic Research Program.
Collections
  • University of St Andrews Research
URI
http://hdl.handle.net/10023/26021

Items in the St Andrews Research Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

Advanced Search

Browse

All of RepositoryCommunities & CollectionsBy Issue DateNamesTitlesSubjectsClassificationTypeFunderThis CollectionBy Issue DateNamesTitlesSubjectsClassificationTypeFunder

My Account

Login

Open Access

To find out how you can benefit from open access to research, see our library web pages and Open Access blog. For open access help contact: openaccess@st-andrews.ac.uk.

Accessibility

Read our Accessibility statement.

How to submit research papers

The full text of research papers can be submitted to the repository via Pure, the University's research information system. For help see our guide: How to deposit in Pure.

Electronic thesis deposit

Help with deposit.

Repository help

For repository help contact: Digital-Repository@st-andrews.ac.uk.

Give Feedback

Cookie policy

This site may use cookies. Please see Terms and Conditions.

Usage statistics

COUNTER-compliant statistics on downloads from the repository are available from the IRUS-UK Service. Contact us for information.

© University of St Andrews Library

University of St Andrews is a charity registered in Scotland, No SC013532.

  • Facebook
  • Twitter