Original Research Policy still needs to be worked out[edit]

I came here from page [1], which states:

Not original research. Articles should be aimed to be excellent encyclopedia articles, and thus are summations of what is known about a topic. Hence, while articles may sum up their topics in novel ways, they should not do so in ways that imply new theories or analyses that in academic contexts would require peer review for publishing. In other words, they should not contain original research or observations. See the original research policy.

(Note: "that in academic contexts would require peer review for publishing" ?? That doesn't seem applicable to anything!)

However, I found a blank page.

Before I do elaborate editing, and certainly before starting new articles, I like the rules to be a little sharper defined. As the term is identical to that of Wikipedia, Larry certainly had in mind something similar to the policy of the same name in Wikipedia (Larry please correct me if I'm wrong). However, I don't recall to have seen in Wikipedia a summary phrasing as here above; and also CZ's neutrality policy is slightly different. Thus the adapted policy will itself probably be a bit original!

In particular, it must fit in with the following requirements that were set out on the same page:

The standards of a good Citizendium article are complex:

Accurate. Articles should have a high standard of accuracy. Editors should review every substantive claim made, and be of the opinion that the claim is well justified, before approving the article. This does not imply that every fact needs a reference; indeed over-referencing articles, can make them unwieldy. References should be selected with care; ideally they should be from authoritative sources and should be verifiable online. References should be given to direct the reader to particularly notable sources of fact or opinion, or to facts the truth of which might reasonably be questioned. (Note: that last sentence appears to state the inverse of what is meant!)

Neutral. Articles must not take a stand on controversial issues. They should report on controversies rather than engaging in them, reporting every side as sympathetically as possible consistent with the sympathetic representation of competing sides, and doling out limited space, where necessary, according to (in the case of mainly academic controversies) the proportion of opinion among experts or, in some broader controversies, the general public whose native language is the language of the compendium. See the neutrality policy.

(Side note: emphasis mine. I see a conflict between "Accurate" according to expert knowledge - surely the first concern of CZ, and "Neutral" in proportion of knowledge among an English-only (or even American-only?!) speaking general public).

The purpose of a CZ article is to allow the reader to make up his or her own mind on any controversial topic, not to 'lead' the reader to a particular conclusion. The reader might be reasonably led by the weight or quality of evidence, but should not be led by rhetorical devices or by selective presentation of evidence. An article that is tranparently seeking to be balanced and fair is more likely to be given credibility than article which appears to be designed to promote a particular position.

At first impression, the particular situation of CZ with expert editors is so different that it may conflict with the current Wikipedia formulation of No Original Research. As asserted on the "We're not Wikipedia page" (I'll emphasize in bold for my concerns):

Our article policies differ. Our aim is to craft compelling introductory narratives, not mere collections of data. We are encouraging our contributors to create coherent, readable, extended narratives that actually do the job of introducing a topic to people who need an introduction to the topic. We are actively discouraging articles that take the form of mere disconnected summaries of subtopics, or other "modular" collections of data that could easily be reshuffled and reorganized. Such "articles" are dull and not likely to be read all the way through.

I suspect that a modular approach is often the only possible solution to satisfy both of Wikipedia's policies concerning neutrality and no original research - some kind of possibly "original narrative" may be the only way to organize all the relevant expert-known information into a smooth, non-modular article. (Larry, could you comment?)

We use an older version of the neutrality policy. Wikipedia has added all sorts of bells and whistles to its original neutrality policy. We've gone back to one of the original versions. And we don't use the neologisms "NPOV" and "POV"; we use the old-fashioned English words "neutral" and "biased." And we actually take the neutrality policy seriously; for many Wikipedia articles, the policy seems to be on hold.

Probably equally old versions of Wikipedia's policies will best match each other to start with.

We take a more sensible approach to citing sources. The editors we have on board actually create the sort of sources that Wikipedia cites. We do cite sources, of course, but we have a sensible approach to doing so. We cite sources because doing so helps the reader. We do not cite sources in order to settle internal disputes, or to "prove" a point to contributors. As seasoned researchers, we know that people can find sources for all sorts of ridiculous claims.

This appears to prescribe the writing of articles that would be called "original" in Wikipedia.

Harald van Lintel 16:17, 27 November 2007 (CST)

Yes, this is all contradictory. I particularly object to the phrase about "the general public whose native language is" English, which has nothing to do with neutrality. I asked Larry to remove this phrase and he has not. The Editorial Council needs to take a strong hold on this issue, because it is not in good shape at this time. --Martin Baldwin-Edwards 16:25, 27 November 2007 (CST)

I agree that wikipedia would consider some of what we do as "original research" and this is one thing that sets us apart. In other words, the original research policy at wikipedia should be different than it is here precisely because we have editors with real names that place their reputations as the source. --D. Matt Innis 18:01, 27 November 2007 (CST)

My own interpretation of this passage differs from that offered above:

"The editors we have on board actually create the sort of sources that Wikipedia cites."

Granted, it is open to the interpretation that original research is OK on the citizendium, or that editors, not authors, may use the citizendium as a forum for the publication of original research. My interpretation of this passage does not contradict other documents, because I interpret it as "our editors create the sources [encyclopedias like] the wikipedia cites, when they are at work on their day jobs.". I didn't interpret it as stating that our editors would publish original research here.

Given that it has proven open to other interpretations this passage should probably be re-written. But I think it should be viewed as merely ambiguous, not contradictory.

Cheers! George Swan 09:16, 28 November 2007 (CST)

Thanks for the clarification! Yes, probably that is what was meant. However, in that case the suggestion is not entirely correct, as experts on Wikipdia do exactly the same. Thus it's a bit unfair. Even without ambiguity it would need rephrasing, more accurately pointing out the real difference with Wikipedia.

Regards, Harald van Lintel 13:03, 28 November 2007 (CST)

First version, mostly based on Wikipedia[edit]

I'll now import a version of Wikipedia, of about the same time period as the NPOV policy (see argument here above); next I'll immediately make a few modifications as I see fit (such as eliminating some unnecessary complexity and perhaps modernize the header). Next it's for others to comment and improve. Harald van Lintel 16:22, 27 November 2007 (CST)

However, I now discovered that the corresponding policy on Wikipedia is a much more recent development, it only started in 2003 and even the 2004 version is a very poor mix between neutrality and no original research... IMHO the "least bad" versions are the most recent ones. Thus I now selected a version of yesterday and start from there. Harald van Lintel 16:56, 27 November 2007 (CST)

OK, done! :-)

I made many important changes, but it still strongly resembles the Wikipedia article from which it was derived (eventhough I think that it's already better). Thus I guess that it's not (yet) the time to put the "Live" tag on. Harald van Lintel 18:24, 27 November 2007 (CST)

Basically, I'm not comfortable with a brand new author writing a very long, consequential policy out of whole cloth. I've deleted the policy from the page, but I'll put Harald's proposal on CZ:Original Research Policy/HVL proposal. The next step is for any such proposal to get attention from editors and for it to be made into an Editorial Council resolution. --Larry Sanger 18:52, 27 November 2007 (CST)

Larry, as expressed above I did not write that policy, the changes consisted mostly of deletions of Wikipedia bells and fiddles. An Editorial council procedure sounds fine to me, I did not realise that that is the standard route (the most important neutrality policy does not give the impression that it is the result of such a resolution). Harald van Lintel 01:49, 28 November 2007 (CST)

Yes, you're right, Larry. I should have said so, too. This is a major policy issue, and needs experienced editors to hash out a policy for the Editorial Council to examine. --Martin Baldwin-Edwards 19:03, 27 November 2007 (CST)

I'm a very experiencd Wikipedia editor, and I have been involved in improving the NPOV and NOR policies of Wikipedia. Anyway, I seem to have initiated progress on this issue of lacking policy, which is certainly good! Harald van Lintel 01:49, 28 November 2007 (CST)

reconciling goals[edit]

The Wikipedia "no original research" policy has as an explicit goal keeping out cranks. Citizendium's editor policy has as an explicit goal attracting people who do useful original research. Presumably, CZ wants to keep out cranks as well, but that creates a potential conflict. The conflict is, however, not that terribly difficult to resolve, I think. Much of what is ready to appear in Citizendium will be based on existing published research - the latest observations and interpretations may not really be ready to appear in an encyclopedia. However, sometimes something new really is significant enough to appear, and hopefully, sometimes it will be a CZ author or editor who does the research which should be included in the encyclopedia now.

My suggestion: Peer review. CZ's Original Research policy should allow authors and editors to include the results of their own research, provided it has been subjected to independent peer review. This will usually mean publication in a journal with which the author is not affiliated, but we could conceive of allowing CZ editors to provide a preliminatry review of a work which is otherwise "in press". Perhaps work which has only been reviewed by CZ editors would be kept on a subpage, until outside peer review is complete. Anthony Argyriou 20:00, 27 November 2007 (CST)

Work that is peer reviewed by publication in a journal is also allowed in Wikipedia. Thus it's fine but nothing new. Published knowledge from peer reviewed journals is never "Original research" of the encyclopdia that mentions it!

However, internal peer review of new material in CZ would be something different. Harald van Lintel 01:59, 28 November 2007 (CST)

Sorry if this is answered above, but this comment caught my eye. CZ policy implies that internal review is acceptable? Chris Day (talk) 02:30, 28 November 2007 (CST)

Alright, now I get the point, I just read Anthony's comments above in full. I'm not sure we need to be so up to date that we are preceeding the publication of work. Is that really the role of an encyclopedia? As Larry mentions below, there is an avenue for signed articles that are not part of the article itself. Chris Day (talk) 02:35, 28 November 2007 (CST)

I thought about this recently during one of my musings, and it occurred to me that a lot of research is done and published in scientific journals. Now until the results of the research are accepted as scientific fact and are evidenced as such, it seems like they would fall under the category of "original research". Why don't we include a subpage on articles that is designed for theories and areas of research on subjects and explicitly label it as such? At least that way we could indicate that there is a reasonable uncertainty as to the confirmation of this content, but it still pertains to the particular subject matter. I think this would do well. --Robert W King 20:22, 27 November 2007 (CST)

Ha! Apologies Anthony, we had about the same idea. --Robert W King 20:27, 27 November 2007 (CST)

See CZ:Signed Articles --Larry Sanger 20:36, 27 November 2007 (CST)

Policy in a nutshell[edit]

Joe Quick put http://citizendium.org/fundamentals.html to my attention.

According to the Fudamentals, content must be: "based on common experience, published, credible research, and expert opinion".

That seems to allow for (or even demand!) opinions that are a little more "Original Research" than is allowed in Wikipdia.

Harald van Lintel 18:05, 7 January 2008 (CST)

I'm pretty sure it means "pre-existing common experience, published, credible research, and expert opinion."; I'm not sure that developing theories and original research on CZ (with the exception of eduzendium of course) is good practice. --Robert W King 18:08, 7 January 2008 (CST)

The difference is the subtle difference between the two definitions of original research. One defined as: a researcher developing something "original" that he/she is the only one that is working on it and the other as: an expert synthesizing multiple sources into a rational discussion of the issues. The first is obviously original research. The latter is also considered original research on wikipedia. Because at WP the expert is not afforded any more weight than any other editor, any synthesis of information cannot be allowed unless that right is given to everyone - so everyone must support their writing with citations. Therefore, on wikipedia, if it isn't written by a reliable and verifiable source then it has to be left out. However, here at Citizendium, we have the experts that create those sources. They use their real names and place their reputations on the article. We want them to digest the information for us into something that only someone who has the full spectrum of knowledge can give us. This does not mean that they can "make up anything they want". Most experts can readily cite sources for anything they write, but we don't expect them to (once we have vetted them). That 'digestion' of information is what experts do, and indeed what society asks them to do. That, I think, is the essence that will allow Citizendium to create the reliable source we are looking for. The challenge is in wording the policy to allow this. --D. Matt Innis 20:04, 7 January 2008 (CST)

Thanks Matt, that's exactly what I had in mind! Usually articles can be written without being original in any way, but in cases where the published literature is full of confusion and/or errors the choice is between a poor article that is a good reflection of the poor state of affairs, or a good article that does contain an "original" synthesis, even one that takes a clear position.

Note that I don't think it's a challenge at all to phrase the policy like that, quite to the contrary! In Wikipedia the phrasing to allow a synthesis but not one that corresponds to original thought was problematic, and I saw that sometimes experts have ignored that rule for the sake of clarity (and I won't ring the bell on them!) :-)

Harald van Lintel 11:04, 12 January 2008 (CST)

Matt seems to state this very well. I'd add that there is room even for authors under experts to do what Matt is talking about in the second type. Butler was largely written that way, and its one reason why WP's version of that article will never be near half of what CZ's is (unless they import ours or just break policy, of course). Stephen Ewen 20:33, 7 January 2008 (CST)

Good point about authors as well. I think authors should be able to synthesize, with the understanding that it has passed the review of the editors of the workgroups involved. --D. Matt Innis 20:59, 7 January 2008 (CST)

Matt's statement is excellent and points to the central reason CZ is intellectual superior to Wikipedia. The kids over there have a great deal of trouble evaluating the validity of sources--every dog on the web looks alike to them. They rarely look at scholarly articles, for example.Richard Jensen 14:05, 8 January 2008 (CST)

The intellectually superior don't stereotype. I, for one, use only the finest resources in writing Wikipedia articles. James Hare 21:48, 13 January 2008 (CST)

I don't see all sources as the same, nor do many users I know. It is probably best to stay on topic rather than bash Wikipedia. I think it is well established that many people here hate Wikipedia ;) Aaron Schulz 21:52, 13 January 2008 (CST)

First adapted version[edit]

I now did a first try to adapt the policy description to Citizendium - just a first suggestion -CZ:Original_Research_Policy/HVL_proposal Harald van Lintel 12:06, 12 January 2008 (CST)

I'm glad Harald took the intitiative! But I have one small disagreement regarding procedure. The propsal overlooks the key point that CZ has expert editors who know the literature who can make a determination. They must bve part of the processs, or CZ will be just like Wikipedia with its indeterminant nasty fights. The danger is that poorly informed CZ contributors will make repeated challenges and tie an article up in knots-- and this has recently happened at CZ in several cases I know about. So I suggest this change:

1. old: Any material that is likely to be challenged must be supported by a reliable source
2. new: Any material that is challenged by a CZ editor must be supported by a reliable source

The older phrasing seems to demand preemptive footnoting of every single sentence, which violates our CZ style and puts an unnecessary burden on our authors. The revised form means that a critic who spots supposed original research needs to have the support of one of our expert editors.

Secondly the "Citing Oneself" section is unnecessary and contradictory. My proposal solves the problem more simply.Richard Jensen 13:22, 12 January 2008 (CST)

It may be unnecessary, but I miss the contradiction. Where?

Also, the phrasing likely to be challenged certainly doesn't demand preemptive footnoting of every single sentence! :-)) On the other hand, there is nothing wrong with a wealth of footnotes.

Harald van Lintel 18:07, 30 January 2008 (CST)

What if an author challenges some text? Also, as far as I know, you are not allowed to cite yourself. --Robert W King 13:24, 12 January 2008 (CST)

Authors challenge texts all the time and that does not change. But if an author says it's forbidden original research then the author has to first convince an editor. People can cite published sources (including their own); it's citations to unpublished work that is disallowed as being original research.Richard Jensen 13:29, 12 January 2008 (CST)

I'm pretty sure you can't cite your own work as evidence of anything. That would be like me publishing a patently wrong paper about how oxygen is poison, and then citing it as sufficient evidence as to why breathing is dangerous for humanity, and using that to affirm my authority on a particular topic related to the material. --Robert W King 13:32, 12 January 2008 (CST)

articles published in scholarly journals go through an elaborate vetting process. Once they appear they can be the base for CZ articles. And yes, appearance of your papers in scholarly journals makes you an expert but does not make you a CZ editor. CZ has its own elaborate process for choosing editors -- it's what makes us so different from Wikipedia. The editors are all about making quality control decisions and they should be integrated into the Original Research policy. Consider this scenario: author X writes statement A in an article. reader Y disagrees with A and denounces it as original research, which makes it forbidden. The Constable therefore removes the forbidden text A, on the basis of one reader Y who has no qualifications to say what is or is not original. I suggest it is much better to have editor Z look into the matter and make the decision whether A is forbidden or not. Richard Jensen 13:48, 12 January 2008 (CST)

That's all well and good (as is policy) but that doesn't have anything to do with citing yourself. --Robert W King 13:54, 12 January 2008 (CST)

I'd say that only authors can have any "OR" and that it be limited to synthesizing data in peer-review mainstream journals. However, I'd object to someone citing their own synthesis of their own data/articles. Aaron Schulz 14:12, 12 January 2008 (CST)

CZ policy is to attrack published scholars. They will of course be writing CZ articles that synthesize their own scholarly publications. It's perverse for an encyclopedia to trust uncredentialed authors and distrust credentialed scholars. You see a lot of that at Wikipedia, and that is what drove me and a lot of others to Citizendium, which honors and respects scholarship and the scholars who produce it. Richard Jensen 14:15, 12 January 2008 (CST)

I don't recall saying authors could do that either. Aaron Schulz 14:41, 12 January 2008 (CST)

I'd suggest that if a scholar has put together some synthesis or some published work that perhaps another editor or author should review the suggested work to make that a part of the article space. It's the exact same reason why Larry Sanger is a topic informant on himself for the article about him here. --Robert W King 14:23, 12 January 2008 (CST)

It's the people who do the hard orginal research that has been accepted by journals and publishers who best understand the topic and who are the best people to write an encyclopedia. If we attract some leading scholars on China, we want them to write about China, not astrophysics. People who distrust experts don't read encyclopedias--they get their ideas from passing conversations at the grocery store or snippets from a TV show.Richard Jensen 14:34, 12 January 2008 (CST)

You forget so soon that's what workgroups are for, Richard. Besides it is unlikely that here in CZ, you will find someone who is in the field of Astrophysics editing Geography articles. I think it is safe to assume that people here tend to gravitate to "what they know". --Robert W King 14:36, 12 January 2008 (CST)

That sounds like a good idea. Aaron Schulz 14:41, 12 January 2008 (CST)

Yes, I assume that people will write about what they know! But here's the rub. If Y sees statement A in an article, and Y thinks that statement is wrong, then A can adduce some evidence and debate it with X on the talk page. Constables have no role. But if Y says that a true statement A is Original Research, then the burden of proof suddenly shifts to X who has to prove it's not original research or it gets erased by constables. I want an editor Z in the process who will decide if Y's allegation has any merit. The reason is that we cannot allow ignorance to have a privileged position; only experts are able to say whether or not A is original research. In real life it's not easy to tell whether or not a true statement A is "original research" (not allowed) or a synthesis (allowed). You need to be very familiar with the published literature to make that call. (and indeed this issue did come up already, regarding the Oriental article.Richard Jensen 15:00, 12 January 2008 (CST)

I am opposed to this text in principle, as it purports to tell editors of CZ how to do their job. Any rules on how editors should proceed within CZ should emanate from the Editorial Council, which is the only body with the expertise necessary to draw up rules of guidance on editorial policy. Martin Baldwin-Edwards 15:08, 12 January 2008 (CST)

I must be misreading Martin somehow. ?? What does he mean by "telling editors how to do their job"? My amendment does not make editors do anything one way or the other. It requires authors who complain to take their complaints to the editorial officials in charge. I agree with Martin that the Eduitorial Council is the proper place to set policy on Original Research.Richard Jensen 15:44, 12 January 2008 (CST)

Same problem here: we all agree on that! The draft text is just a first attempt to put in clear text what the Editorial Council seems to be saying but hasn't been saying clearly enough to be sure... Let's hope that it helps to get clarity about an existing policy that definitely needs to be more than a referral to an empty page! Harald van Lintel 17:45, 30 January 2008 (CST)

That is not what this draft proposal says, Richard. For example, I have just concluded some original research in which I have created my own statistics and analytical diagrams from a restricted access microdataset. I have some problem with the people who commissioned the research, who ask for sources for my data creation [because they don't know about original research in Greece, as nobody ever does it]. The same would happen on CZ, under this proposal, that my unique data could not be reproduced on the grounds that (a) It would be self-citation; and (b) the publisher will not be reputable. Or another example: I have published many things as working papers because I am sick of the incompetent refereeing processes and idiots who are running the major journals on migration. These working papers are cited in all refereed journals, in World Bank and IMF publications etc, as if they were refereed journal arrticles. These would all be considered unacceptable sources, under this proposal.

In other words, this draft proposal reproduces exactly what is wrong with current academia at its worst, and is not useful for established experts. We already know what is serious scholarship and a set of rules can only cause problems. Martin Baldwin-Edwards 16:19, 12 January 2008 (CST)

That can't be right: information that is found in refereed journals and other respected sources is never rejected as "Original Research", not even in Wikipedia (the adapted text is more tolerant). Which sentence should be changed in order to avoid such a misunderstanding? Harald van Lintel 17:51, 30 January 2008 (CST)

Am i correct in reading Martin as saying CZ should not ban any original research just because it's original? Richard Jensen 17:15, 12 January 2008 (CST)

That is not exactly what I am saying. I am questioning the concept of original versus published. These are matters which Editors should decide on a case-by-case basis. I do not want a set of rules telling me what is acceptable when I make an editorial decision. I am sure you agree with this, Richard. Martin Baldwin-Edwards 17:18, 12 January 2008 (CST)

Isn't this covered by CZ:Signed articles? --Robert W King 17:20, 12 January 2008 (CST)

Not really, Robert. I mean that editors must be the people who decide what constitutes citeable material. Publication of genuinely original research should go into signed articles, as you say. Martin Baldwin-Edwards 17:46, 12 January 2008 (CST)

I think Martin has it right. Our policy should say something to the effect the regular (unsigned) articles are not the place for first publication of significant amounts of original research. (Perhaps a rule: if an article contains more than 500 words of original research then it should spinoff a signed article.) Richard Jensen 19:06, 12 January 2008 (CST)

I think that that doesn't work as the level of original research is not in direct relation to the number of words - and it's certainly against CZ's current No Original Research policy! Harald van Lintel 05:41, 2 February 2008 (CST)

what we want is original research to be spun off in signed articles; an article under 500 words doesn't make much sense. A paragraph of original research of say 250 words will not weaken CZ and should be encouraged. The original Wikipedia prohibition is not very useful at CZ because we have editors who appreciate and can deal with scholarship in contrast to the Wikikids who can't easily recognize or handle original research. Richard Jensen 09:29, 2 February 2008 (CST)

I can write a crackpot theory that is 125 words, but that doesn't mean it should belong in CZ. --Robert W King 09:44, 2 February 2008 (CST)

all crackpot additions get deleted whether or not they are original. The issue is whether someone can try to erase correct information because it is not sourced--that happens all the time on Wikipedia. Richard Jensen 10:56, 2 February 2008 (CST)

To be honest, I'm truly baffled at the hostility to the proposal of allowing an expert to cite his own work. Its like saying to the expert; 'Your life's work and research is not acceptable, please find someone else's citation' Can no-one else see the lack of common sense in this scenario? Denis Cavanagh 12:28, 13 January 2008 (CST)

I agree. This was part of my point. We have other editors who can pick up on any problems: it is not as if we are WP. Martin Baldwin-Edwards 16:40, 13 January 2008 (CST)

Sorry, but where is this "hostility to the proposal of allowing an expert to cite his own work"? I don't see it! Any author is of course allowed to cite his/her own work, and that is explicitly mentioned in my proposed version. Harald van Lintel 17:56, 30 January 2008 (CST)

It's in the discussion above, rather than in your text, Harald. Martin Baldwin-Edwards 18:59, 30 January 2008 (CST)

OK Harald van Lintel 05:41, 2 February 2008 (CST)

readily available information[edit]

I have a question regarding "original research" and easily-verified facts. How do we handle this. Example: For an article I want the release date of a game, or the publishing date of a book. I own the game or book in question, so I look at the cover/impressum and take it from there. Does that fall under "original research"? Do I still have to google some Amazon or whatever link to provide a reference? --Tom Vogt 18:29, 30 January 2008 (CST)

Original research I think is usually meant as taking a bunch of information, and synthesizing it to ascertain some point or prove some theory. --Robert W King 18:31, 30 January 2008 (CST)

Absolutely not, Tom! The idea is that journals and other venues are for first expressions of novel ideas, theories, and findings, not CZ. Stephen Ewen 20:46, 30 January 2008 (CST)

On the notion of original research[edit]

An earlier policy statement said: "Articles should be aimed to be excellent encyclopedia articles, and thus are summations of what is known about a topic. Hence, while articles may sum up their topics in novel ways, they should not do so in ways that imply new theories or analyses that in academic contexts would require peer review for publishing. In other words, they should not contain original research or observations."

I thought I would make a first stab at expanding the policy. Actually, I am going to as it were lay the philosophical foundations for the policy, which I hope will explain, to those who follow this very abstract prose, why this sort of policy is needed and what is essential nature is. I am deliberately ignoring what Wikipedia has written about it. I originally devised the concept of the policy (and named it) for Nupedia, and when I look back at Wikipedia policy pages, I often find them to be badly "clarified."

The central intuition behind the original research policy is that there is an identifiable class of claim (and whole theories, arguments, narratives, etc.) that constitutes an original contribution to knowledge. It is this notion of "originality" that causes the familiar difficulties with the original research policy: what should count as an "original contribution to knowledge," such that we should omit such things from CZ?

I have found repeatedly in attempting to answer ti esti questions (Greek: "what is," the sort of question that Socrates asked about the various virtues) it is essential to understand what function the concept being defined is supposed to serve. In the present case, we can ask: what is the function of the notion of "original contribution to knowledge"? And this question has a specific practical answer, because this is a practical project; we are quite literally putting the concept to use. So, what is the use of the concept? We advert to the notion of an "original contribution to knowledge" in explaining why certain claims (etc.) require vetting before either publication or else, in a collaborative project that involves a "publish, then filter" approach, approval. That is, we want to omit claims (arguments, results, and in general, "research") that have not been but ought to be professionally vetted before being made. And that is why we speak of "original research" and "original contribution to knowledge." The questions then become: what constitutes "professional vetting," anyway? And what claims, in general, should be professionally vetted before being made?

But before answering those questions, first we should explain why we want to omit this class of claim or "research," however it is to be more carefully described. There is a clear and, to my mind, incontrovertible explanation. It is that professional vetting justifies the claim, and claims that are not adequately justified should not be published (or, as I said, approved). Absent professional vetting, we are thrown back upon our own resources to gauge the justifiability of the claim. If someone says, "I have done a study of X, and my results are R"--and the study is not of published research about some subject, but of the subject matter itself--and wants it included in a CZ article, then what can we say? Who is in a position to evaluate the study? You might say, "Our editors, of course; they're experts." But we are asking our editors to do what is, I think, a fundamentally different (and much more difficult) sort of editorial task than any they now have. We'd be asking them to gauge the trustworthiness of a researcher, research methods, analysis, and whether the the results are significant contribution (worth mentioning in CZ). In short, we'd be asking them to do all the things that the staff (editor and reviewers) of an academic or professional journal do, or (depending on the claims made) that the editorial staff of a serious, conscientious newspaper do.

You might say that there could be some way to set up a wiki (or other bottom-up collaborative software) that would enable distributed, self-selecting groups of professionals to come together and do this editorial work. I would agree (though I don't think this suggestion has been by any means demonstrated or proven). But there are at least two really good reasons that we don't attempt this.

First, we are an open project, and allow non-experts to work alongside experts. If we were to allow the publishing of original research within CZ articles, nonexpert authors would necessarily be involved in the vetting process, because they are full-fledged members of the project. It is not as if there are particular sentences and sections of articles that only editors may work on. The problem, then, is that non-experts are (perhaps by definition) not good judges of original research. They lack the tools to determine what is a valid and important contribution to knowledge; if they had such tools, they could for that reason count as experts.

A second, related reason is that reference publishing, or summing up existing published knowledge, is simply a radically different sort of thing from peer reviewed publishing of original research. The required editorial processes are different and can be kept separate. For the sake of simplicity, it is best to stick to one or the other, and not try to mix them together. Perhaps in the future we might want to experiment in such a radical way, but not yet.

There is a third reason, actually. It is that, however possible it might be to set up a wiki to accomplish "collaborative peer review," it will very probably not take any such form as the way CZ uses wiki software. In particular, it will not take the form of self-selecting people working at will on single, unsigned descriptions of topics. We would have to establish that someone's original research is actually worth describing in an article, and that is something that cannot be done in the lines of the article itself. At least it would have to be done on a talk page; and then, of course, you can immediately see that a talk page is hardly the place to first post, and certify, some research. No, you'd want to use some other system, better suited to the purpose. And then, of course, you're proposing a new publishing system, the results of which Citizens can then choose to cite in their articles, or not.

I know this doesn't answer those two important questions I listed above (what constitutes professional vetting and when must it be done), but it does set us up better, I hope, to discuss them intelligently. --Larry Sanger 15:13, 2 February 2008 (CST)

Larry makes some useful points. In the humanities, the journals and book publishers do not have a system for vetting 500 word statements. The simplest solution is for authors to not claim that a particular statement is based on original research. (they therefore get no credit for publishing original research.) Richard Jensen 18:33, 2 February 2008 (CST)

"The simplest solution is for authors to not claim that a particular statement is based on original research." That's a loophole. I could simply substantiate that some point I invented and am claiming is valided by published, peer-reviewed research X, Y, and Z when in fact the conclusion I have reached is invalid and bogus, and according to that idea as long as I state that it is not based on original research, that would make it "ok", assuming no one took the time to invalidate my claim. --Robert W King 18:54, 2 February 2008 (CST)

Robert is assuming we have malevolent people trying to sneak falsehood and deception into CZ. That is indeed a problem at Wikipedia (mostly from high school kids changing a date or a number), but I have never seen it here at CZ--I have never seen a statement in a CZ article which in my opinion the author thought was false. I think we start with the assumption that our authors are honest. As for OR, no author on CZ to my knowledge has ever stated that this argument or that is totally original. (In history and politics, it is very rare indeed to come across a totally original idea of any importance. very rare.)Richard Jensen 19:16, 2 February 2008 (CST)

I'm assuming worst-case on purpose, but I must agree that I haven't seen it here either. I guess it's a poor attempt at forward-thinking/prevention? Sometimes I worry too much. --Robert W King 19:48, 2 February 2008 (CST)

The function of the concept of original research[edit]

(I'm simply continuing the essay I started above now. I might not finish it in this installment, either!)

I said that the central intuition behind the original research policy as follows. First I said that we can define a useful concept of "an original contribution to knowledge," and we can get at that concept by determining how it functions, i.e., how we use it. Then I said that we advert to the notion of an original contribution to explain why claims require "vetting" before publishing. I said that certain claims require professional vetting before being made, because otherwise they are not justified. This is my wedge into the concept, then. What claims are not justified unless they are first "vetted" by experts (whatever we take "vetting" to mean--mind you, that is probably very important here)?

Just to clarify, let me stipulate that there are many sorts of claims that do not require expert vetting in order for us to consider them justified: common observations, such as that the sun is yellowish and that most babies are born with ten fingers and ten toes; any claims that are reported simultaneously and widely in many different sources, such as I assume Kennedy's "Ich bin ein Berliner" ("I am a jelly donut") line was; the mere reportage of an informant's words themselves (i.e., we can quote autobiographical remarks directly without the interposition of an editor, saying that the informant did in fact say that, whether what he said was true or not); and no doubt others.

And let me stipulate that there are also many sorts of claims that do require expert (or at least careful editorial) vetting in order for us to consider them justified; for example, the claim that a drug was shown by a certain longitudinal study to have certain effects, and the study has never been published anywhere; a new philosophical analysis of justice, never before published, even if made by some famous philosopher; or the claim that a certain person said a certain thing at a certain time and place, when only a few witnessed the remark and it was not immediately reported publicly, and where issues of context, history, etc., might be very germane; and no doubt many others.

Now, armed with these examples, I would like to try to explain why, lacking expert vetting, the latter claims should not be considered justified, while the former claims perhaps should be. First of all, it will be very clarify that I am not using "justified" in any very stringent sense. It does not mean "incontrovertibly established" or "proven," but simply "more or less blameless or in the clear, from the point of view of sound editorial practice." Anyone who writes that JFK said, "Ich bin ein Berliner" without citing a source is in the clear (at least from the point of view of justifying the claim in an encyclopedia--for a school paper, for example, maybe matters would be different), because the line was reported widely (I assume) and is now common knowledge.

I would actually add to the category of "justified" claims those claims that people make that are common knowledge in a particular place, or among a certain ethnic group, or in a particular discipline. In other words, while some knowledge might be very far from common, if you ask the "experts" for a "source" for a certain piece of "common knowledge," P, about X, they will say, "But anyone who knows anything about X knows that P is correct." For instance, I happen to know that Flattop Mountain (Alaska) indeed appears flat on top, from Anchorage, and it is pretty flat for a person standing on top of the mountain, too. This is something that anyone physically able can ascertain for himself, and many a hiker has done so. So, even while there might be some authoritative sources that state this as a fact (such as the book 55 Ways to the Wilderness), no one needs to cite any such source in order to be justified in making the claim. Of course, when I say that, I mean anyone in CZ (and most publications). Indeed, if a magazine author submitted an article to National Geographic, making these claims about Flattop Mountain, her editor would not fact-check that particular claim, because it is the sort of claim that any competent person could be expected to get right.

The latter observation gives us a clue here. There are certain claims that, we think, any competent observer can make justifiably; there are others, however, that not just any competent observer can make without any confirmation or vetting. More complicated claims about science and history are examples of the latter. And let us return to the examples of claims that need expert vetting: reports of longitudinal studies, new philosophical analyses, and quotations obscured by memory and context. I claim that what these claims all have in common is that their validity, importance, or reliability is not simply obvious on its face to any competent observer, and that expert observers are notably better judges of such matters. We can explain further in each case why the experts are better judges, as well: scientists familiar with drugs and drug research can tell far better than anyone else whether a longitudinal study was well designed, whether the conclusions follow, and so forth; philosophers can tell far better than anyone whether a new theory of justice is an original, important contribution to the literature, because they are familiar with the various theories of justice and also with the quality of the newer ones on offer; the relevant specialist editors and historians have far finer-tuned notions of how memory can be deceptive in certain circumstances, and whether the words are likely to be exactly right, and whether the informant might be lying for self-serving purposes.

So here's my hypothesis. The claims that we consider to be justified only when vetted by experts are those claims experts are better at judging than the average competent person.

Well, I'm going to leave it at that for now. I'll have more to say later, hopefully not in a few more months! --Larry Sanger 15:29, 13 April 2008 (CDT)

Imported conversation[edit]

I imported this conversation from the Alternative medicine (theories) article concerning citation to facilitate decision making on the CZ:Original Research Policy. D. Matt Innis 20:39, 23 December 2008 (UTC)

Citations[edit]

I have no intention of getting into Other Place-style "citation" demands for the source of 16 ounces being in a pound, but I do think it is a reasonable request, when a book, not readily available, refers to "figures" or "studies" from sources such as:

• World Health Organization
• New England Journal of Medicine
• University of Toronto
• Centers for Disease Control

that it is unreasonable to request that a reasonable amount of information is given to let the reader find the original source. There are a number of places in military history where I will cite a secondary source (e.g., the online Center for the Study of Intelligence) who quotes a document, I will give as much information as possible to identify the original document. For example, I cite an online document by Harold Ford, one of the CIA historians, who, in his document, cites "Saigon telepouch SAIG 5624 (IN 69402), 19 December 1967. CIA files, Job No. 80B01721R, O/D/NFAC, Box 2, "Substantive Policy Files, DDI Vietnam Files, Folder 5." The Ford reference is <ref name=FordEpi3>{{citation | first = Harold R. | last = Ford | year = 1997 | publisher = Center for the Study of Intelligence, [[Central Intelligence Agency]] |title=CIA and the Vietnam Policymakers: Three Episodes 1962 - 1968 | id = Ford Episode 3 |contribution = Episode 3 1967-1968: CIA, the Order-of-Battle Controversy, and the Tet Offensive | url = https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/cia-and-the-vietnam-policymakers-three-episodes-1962-1968/epis3.html}}</ref>

Now, I recognize one would likely have to go to the U.S. National Archives to obtain the original of this document, but there is enough information to find it there. I might be able to prevail on a local colleague to get it, or, with luck, get an archivist to copy it.

Most contemporary World Health Organization documents are online. The NEJM will vary about free availability, but there will almost certainly be an abstract available in PUBMED or at the NEJM website. Some of their articles are made available, free, in full text, but all of their content, going back quite a number of years, is available to subscribers, including reference libraries.

I would really appreciate, especially when the source of a document may be critical of its position, such as an alternative medicine advocate citing a medical journal, or vice versa, that there is at least some reasonable chance to verify the original material. Howard C. Berkowitz 23:16, 22 December 2008 (UTC)