Pnas Response To Letter

The PNAS has refused to publish the Letter to PNAS. The PNAS has provided the following non-responsive explanation dated September 11, 2008.

Note how the PNAS hides behind anonymity to justify its failure to address the five errors identified in the Letter to PNAS:

A member of the Editorial Board has evaluated the letter and concluded that PNAS cannot publish it for the following reasons:

From what I take to be the underlying issue from the numbered points, Mr. Schlafly's main concern has to do with the fact that one experiment failed to yield a statistically significant result, and this happened to be the experiment with the largest sample size. Every experiment has limited power to detect a difference of any given magnitude, and so in a series of experiments some may yield non-significant results even when the null hypothesis is false. The non-significant experiment may even be the one with the largest sample size. There is nothing exceptional in this--it is a matter of chance. Nevertheless, from a statistical point of view, it is proper to combine the results of independent experiments, as Blount et al. did correctly in their original paper. If the overall result is significant, as it is in this case, then the whole series of tests is regarded as significant. Mr. Schlafly seems to suggest that experiments differing in sample size cannot be combined in an overall analysis, and if this is what he is suggesting, he is wrong.

I think Letters published in PNAS should raise points that in themselves, or in conjunction with the authors' response, should be of wide interest to the readership of PNAS or should illuminate some obscure or subtle point. The issues raised by Mr. Schlafly are neither obscure nor subtle, but are part of everyday statistical analysis at a level too elementary to need rehearsal in the pages of PNAS.

Mr. Schlafly's final comment about release of data is uncalled for. My understanding is that the authors have made the relevant materials available on their web site. This seems to me to meet the requirement that "data collected with public funds belong in the public domain." If Mr. Schlafly believes that the disclosure is incomplete, that is an issue that needs to be argued with the original funding agency, not with the readers of PNAS.

This response can be discussed at Talk:PNAS Response to Letter

See also[edit]

• Letter to PNAS