Registered Reports and Replications
Registered Reports and Replications are a publication format that emphasize the importance of the research question and the quality of methodology by conducting the peer review prior to data collection and analysis. High quality protocols are then provisionally accepted for publication before data collection commences. This format is designed to minimize publication bias and research bias in hypothesis-driven research, while also allowing the flexibility to conduct exploratory (unregistered) analyses and report serendipitous findings.
The submission and review process for Registered Reports and Replications is divided into two distinct stages.
- Stage 1 submission: Authors submit manuscripts including only an Introduction, Methods (including proposed analyses), and Pilot Data (where applicable).
- Stage 1 review: Reviewers evaluate study proposals before data are collected, assessing the importance of the research question, feasibility of the methodology, and analysis pipeline.
- In principle acceptance (IPA): Manuscripts that pass peer review will be issued an IPA, indicating that the article will be published pending successful completion of the study.
- Stage 2 submission: Following study completion, authors submit their finalized manuscript for re-review, now including Results and Discussion sections.
- Stage 2 review: Reviewers appraise whether the authors’ adhered to the preregistered experimental procedures and that any conclusions are justified given the data.
Information for Authors
The cornerstone of the Registered Reports format is that a significant part of the manuscript is assessed prior to data collection. The submission and review process is consequently divided into two stages.
Stage 1: Initial manuscript submission and review
Initial submissions should include the stage 1 manuscript (requirements detailed below) and a brief cover letter. Authors are welcome to first submit presubmission enquiries for advice on the likely suitability of a study as a Registered Report.
Please include a brief scientific case for consideration and an anticipated timeline for completing the study if the initial submission is accepted in principle.
This section should include a review of the relevant literature that motivates the research question and a full description of the experimental aims and hypotheses. Please note that following IPA, the Introduction section cannot be altered.
The methods section should include:
- A full description of proposed sample characteristics, including criteria for data inclusion and exclusion (e.g. outlier extraction). Procedures for objectively defining exclusion criteria due to technical errors or for any other reasons must be specified, including details of how and under what conditions data would be replaced.
- A description of experimental procedures in sufficient detail to allow another researcher to repeat the methodology exactly, without requiring further information.
- Proposed analysis pipeline, including all preprocessing steps, and a precise description of all planned analyses.
- Optional pilot data can be included to establish proof of concept, effect size estimations, or feasibility of proposed methods. Any pilot experiments will be published with the final version of the manuscript and will be clearly distinguished from data obtained for the pre-registered experiment(s).
Following stage 1 peer review manuscripts will either be rejected, given the opportunity to revise or offered an in-principle acceptance (IPA). An IPA indicates that the article will be published pending completion of the approved methods and analytic procedures, in addition to a defensible interpretation of the results.
Authors are reminded that any deviation from the stated experimental procedures, regardless of how minor it may seem to the authors, could lead to rejection of the manuscript at Stage 2. In cases where the pre-registered protocol is altered after IPA due to unforeseen circumstances (e.g. change of equipment or unanticipated technical error), the authors must consult the Editors immediately for advice, and prior to the completion of data collection. Minor changes to the protocol may be permitted according to editorial discretion. In such cases, IPA would be preserved and the deviation reported in the Stage 2 submission. If the authors wish to alter the experimental procedures more substantially following IPA but still wish to publish their article as a Registered Report then the manuscript must be withdrawn and resubmitted as a new Stage 1 submission.
Note that registered analyses must be undertaken, but additional unregistered analyses can also be included in a final manuscript (see below).
Stage 2: Full manuscript submission and review
Authors will be asked to complete their study and resubmit their manuscripts within 12 months of receiving an in-principle acceptance. Failure to meet the agreed timetable may result in a stage 1 manuscript being considered withdrawn.
A stage 2 submission should include the following additions:
- Confirmation in the resubmission Cover Letter that all non-pilot data was collected after the data of IPA, and analysed in the manner previously described. Any unforeseen changes in methods and analyses must be clearly noted.
- Please note that the Introduction cannot be altered from the approved stage 1 submission, and the stated hypothesis cannot be amended of appended.
Results & Discussion
- The outcome of all registered analyses must be reported in the manuscript, except in rare instances where a registered and approved analysis is subsequently shown to be logically flawed or unfounded. In such cases, the authors, reviewers, and editor must agree that a collective error of judgment was made and that the analysis is inappropriate. In such cases the analysis would still be mentioned in the Methods but omitted with justification from the Results.
- It is reasonable that authors may wish to include additional analyses that were not included in the registered submission. For instance, a new analytic approach might become available between IPA and stage 2 review, or a particularly interesting and unexpected finding may emerge. Such analyses are admissible but must be clearly justified in the text, appropriately caveated, and reported in a separate section of the Results titled “Exploratory analyses”. Authors should be careful not to base their conclusions entirely on the outcome of statistically significant post hoc analyses.
- Authors reporting null hypothesis significance tests are required to report exact p-values and effect sizes for all inferential analyses.
- Raw and processed date must be made freely available in accordance with our data sharing policies.
Please see our Registered Report and Replications submission guidelines for details of the additional Declarations that must be included, as with standard article types.
Manuscripts can be withdrawn at the authors’ discretion following in-principle acceptance. However, the journal will publish the manuscript’s proposed title, author names, the abstract from the approved stage 1 submission and a brief reason for the failure to complete the study.
Information for Reviewers
This guide for reviewers contains information on the peer-review process for Registered Reports. If you have been invited to review for the journal and have any queries, please do not hesitate to contact our Editorial Team.
The review process for Registered Reports is divided into two stages. At stage 1, reviewers are asked to assess study proposals before data are collected. At stage 2, reviewers consider the full study, including results and interpretation.
Stage 1: Initial manuscript submission and review
Stage 1 manuscripts will include only an Introduction, Methods (including proposed analyses), and Pilot Data (where applicable). In considering papers at stage 1, we ask reviewers to consider assess:
- The importance of the research question(s).
- The logic, rationale, and plausibility of the proposed hypotheses.
- The soundness and feasibility of the methodology and analysis pipeline (including statistical power analysis where appropriate).
- Whether the clarity and degree of methodological detail is sufficient to exactly replicate the proposed experimental procedures and analysis pipeline.
- Whether the authors have pre-specified sufficient outcome-neutral tests for ensuring that the results obtained are able to test the stated hypotheses, including positive controls and quality check.
Following stage 1 peer review, manuscripts will be accepted, offered the opportunity to revise, or rejected outright. Manuscripts that pass peer review will be issued an in principle acceptance (IPA), indicating that the article will be published pending successful completion of the study according to the pre-registered methods and analytic procedures, as well as a defensible and evidence-based interpretation of the results. Authors will be asked to complete their study within 12 months of IPA.
Stage 2: Full manuscript submission and review
Following completion of the study, authors will complete the manuscript, including Results and discussion sections. These stage 2 manuscripts will more closely resemble a regular article format. The manuscript will then be returned to the reviewers, who will be asked to appraise:
- Whether the data are able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls).
- Whether the Introduction, rationale and stated hypotheses are the same as the approved stage 1 submission (required).
- Whether the authors adhered precisely to the registered experimental procedures.
- Whether any unregistered post hoc analyses added by the authors are justified, methodologically sound, and informative.
- Whether the authors’ conclusions are justified given the data.
Reviewers at stage 2 may suggest that authors report additional post hoc tests on their data; however authors are not obliged to do so unless such tests are necessary to satisfy one or more of the stage 2 review criteria. Please note that editorial decisions will not be based on the perceived importance, novelty, or conclusiveness of the results.
Cognitive Research: Principles and Implications is affiliated with The Psychonomic Society
- ISSN: 2365-7464
2016 Journal Metrics
37 days from submission to first decision
48 days from acceptance to publication
226.0 Usage Factor