Longtime readers of the Galaxy Zoo blog will be familiar with the peer review process from the many posts here describing it. The time elapsed between a paper’s submission and its acceptance (if it is accepted) can be long or short, and papers from the Zoo have sampled the whole spectrum.
The process with our paper on supermassive black holes growing in bulgeless galaxies took about 4 months: we submitted the paper in July, received comments and suggestions from the anonymous referee in August, then modified the paper based on the referee’s report and re-submitted it in October. This week, the paper was accepted by MNRAS.
The initial report from the referee was extremely thorough and constructive, and incorporating his/her comments helped to significantly improve the paper. The referee pointed out, for example, that although the paper emphasized the lack of significant mergers in the evolutionary histories of the sample, the bulgeless nature of the sample excludes not just mergers but any violent evolutionary process that can disrupt a disk to the point where it transfers a significant fraction of its stars from a disk into a bulge or pseudobulge. That was certainly a fair point, so we changed our discussion to include further consideration of the implications of those evolutionary processes being excluded.
And we made some other changes, too, including expanded discussion of why our results differ from some other studies and additional description of how we might be affected by dust in these galaxies (and why we think we aren’t). There were also some very interesting questions that we couldn’t really answer within the scope of this paper, but that we had asked ourselves too and that have already formed the basis for additional projects now underway. Overall, this was a classic example of what the peer review process was meant to be.
The accepted version of the paper will soon be available on the arXiv for anyone to download. In the spirit of openness, I had hoped to include the referee’s report and our response in the additional materials on the arXiv, but the referee did not give permission to do so. That’s fine — it’s anonymous and it’s perfectly acceptable if the referee prefers the exact contents of the report to be private as well. Hopefully he/she approves of my summary!
Note: as soon as it’s published, the paper will also be added to the Zooniverse Publications page, which coincidentally happens to have been released today as the first day of the Zooniverse Advent calendar. Have a look — Galaxy Zoo’s contributions are impressive and we’re joined by many, many others.
With the first Galaxy Zoo paper submitted (kudos to Kate and Anze!), we’d like to describe to you what happens next. What’s scientific publishing all about? How does it work? If you’ve followed the blog and the forum, you have a pretty good idea of the first part of the scientific process: discovery!
We set out on the Galaxy Zoo project in part to test whether spiral galaxies in different parts of the sky somehow have spins that align, as has been claimed by earlier work. Kate and Anze have commented on the motivation for this work and blogged about how we did find an effect, were startled by it and so started the bias test to understand it. Kate and Anze used the bias test data to show conclusively that in the case of Galaxy Zoo it was an effect with the observers and that the universe isn’t mad.
This is one of the amazing and unique things about science. Good scientists spend most of their time arguing against the effects they see in their own data, to avoid falling into traps of seeing only what they expect to see. To see how unique and amazing this is, try to imagine a politician arguing against a piece of legislation s/he is sponsoring! This process of double, triple, and quadruple-checking one’s own work is a very important part of science.
Once we were convinced that we really understood what is going on, we could then write up our conclusions in the form of a scientific paper. Steven wrote here about the process of writing a paper; Kate went through the same process Steven described. Over the past few weeks, she passed her paper around to the rest of the Galaxy Zoo team for comments. Kate’s paper has thus passed through the first check — her own examination of her results — and the second — amongst the team itself.
The next step in scientific research is to submit the paper to a journal. This has now happened, and the paper Land et al. (2008) (where “et al.” means “and the rest,” including YOU!!) has been submitted to the top UK journal Monthly Notices of the Royal Astronomical Society (MNRAS).
The editor of this journal will now select an anonymous referee who can comment on the scientific and technical merits of the paper. The referee is another astronomer or cosmologist whom the editor can ask for an expert assessment of the work. He or she will have a few weeks to read it, think about it, and then make a number of recommendations to the editor of the journal. There are three options. The referee can reject the paper outright. This generally happens very rarely, except in highly competitive top journals like Nature and Science. They can support publication of the paper, asking for only a few minor modifications. This also happens quite rarely, though! The most common outcome is for her to write a “referee report,” suggesting a number of modifications and ask for clarifications. The referee might have questions about some part of the analysis, suggest some alternative thoughts and ideas, or criticise the methodology. Sometimes referees can be hostile to a paper; but often, they are genuinely helpful and constructive.
After receiving the report, we get a few weeks to digest it and modify the paper according to the referee’s comments, and argue against the points raised that we disagree with. This process may repeat itself a number of times if the referee isn’t happy with our modifications, and so it can often take weeks and months for a paper to get to a decision by the editor (acceptance or rejection). If a referee is being particularly unreasonable, we can write to the editor requesting a new referee. In extreme circumstances, we could even choose to submit the paper to a different journal and hope for a more reasonable referee.
The whole process is generally known as peer review since the referee is a peer — a fellow scientist and expert in the field. If the paper is accepted, it will appear both in the online and print version of the journal after another few weeks or months. A paper accepted in such a journal is then considered peer-reviewed.
So, if Kate’s paper hasn’t yet been peer-reviewed why is the paper already “public”? It’s general practice in astrophysics to post papers as preprints on a web server called astro-ph. Astro-ph is updated daily to make all papers publicly accessible for anyone. Most people post their papers there when they submit them to journals so they are available immediately. Some wait till the paper is accepted. Thus, not everything on astro-ph is peer-reviewed! In fact, in cosmology, some like to submit preprints to astro-ph before submitting so to allow the community to comment before the draft is submitted to a journal.
It’s important to note that something said in a “peer-reviewed” paper isn’t necessarily true. The point of peer-review is to weed out obviously flawed paper whose logic has holes or whose data don’t support the conclusion. Knowing that a paper has been peer-reviewed should give you extra confidence that its results are believable – that means that an expert in the field has read through the paper and thinks its conclusions are believable.It’s really just the first step of proper “peer-review,” because the process continues. As the community of astrophysicists digests the paper, they too pass judgement on whetherthey consider the paper important and whether they believe the conclusion. Thus, in the years after publication, other astrophysicists might deem Land et al. (2008) a key paper and cite it in the future, commenting on it positively. Or they might disagree with it, but that would still be a sign that it was important enough to comment on. Or it might just fade into obscurity if astronomers don’t consider it important. That’s the historical legacy of a paper – and that’s the ultimate peer-review.