From its homepage:

ReScience lives on GitHub where each new implementation of a computational study is made available together with comments, explanations and tests. Each submission takes the form of a pull request that is publicly reviewed and tested in order to guarantee that any researcher can re-use it.

So, let's say you are a reviewer: What does this all mean? We have a page with the editorial process but for a first review experience with us it might lack detail on how to proceed. I present here a step by step guide.

This blog post is not meant as a replacement for the content of the website http://rescience.github.io/ but rather as a complement to it.

The setup for GitHub first

As ReScience lives on GitHub, it is necessary to have an account with them first. With that, you can log in at https://github.com/.

How submissions are prepared

New papers enter the editorial process via a special repository: https://github.com/ReScience/ReScience-submission To prepare the paper, the authors have proceeded to the following steps:

1. Visit https://github.com/ReScience/ReScience-submission and click the fork button

2. Clone you own repository (I assume the username pdebuyl everywhere. Change it to fit your needs indeed).

pierre@pc$ git clone https://github.com/pdebuyl/ReScience-submission.git
Cloning into 'ReScience-submission'...
remote: Counting objects: 55, done.
remote: Total 55 (delta 0), reused 0 (delta 0), pack-reused 55
Receiving objects: 100% (55/55), 117.27 KiB | 0 bytes/s, done.
Resolving deltas: 100% (12/12), done.

Look into it:

pierre@pc$ cd ReScience-submission
pierre@pc$ ls
article  code  data  notebook  README.md

The repository is made of placeholders for an article, for code, notebooks and data. The submission to ReScience contains all of that.

After some writing of the manuscript and of code, the authors commit their changes and push them to their fork of the repository (for this process, it is necessary to generate SSH keys, see the GitHub help page).

git push origin master

Then, they make a "Pull Request"

Their submission will now appear in the pull requests of Rescience-submission

How to get your copy of a submission

The submission of the authors now "lives" at two locations: the pull request page of Rescience-submission and in the repository of the authors. The submission template for ReScience requests to git the explicit link to the repository, here https://github.com/pdebuyl/ReScience-submission, so that it is easy to find. Follow this link (the link named Repository in the section Replication) and there the green button clone or download. Copy-paste the clone URL and execute

pierre@pc$ git clone git@github.com:pdebuyl/ReScience-submission.git
pierre@pc$ cd ReScience-submission

Now, the repository is populated with code from the authors and also the source and PDF for the paper.

For the code, we expect the subdirectory code/ to have instructions in the README.md file. Those should be sufficient for you to check that the figures and/or other results in the paper are correct. From this point of view, the code/ directory should be a self-contained software package in the sense that it contain all the source code, scripts and instructions for use.

Reviewer guidelines

The editorial page lists what is expected from the review. The discussion itself takes place on the pull request page for the submission.

When the paper is eventually accepted, it is archived at https://github.com/ReScience-Archives and on Zenodo, where it is accessible via a DOI link. The submission pull request is then closed (without merge).