Talk:Bayes' theorem/Archive 1

This page neither gives a good statement of Bayes' theorem nor even hints at the content of the special case stated by Bayes in the 18th century, in which the prior and posterior distributions were continuous rather than discrete.

If you can do better than this, please edit the article to make it better. This is how Wikipedia grows and improves.

---

I have now largely rewritten this article.

Thank you! The Anome

---

No professional who uses Bayes' theorem daily, nor anyone who writes on theoretical aspects of Bayesian inference ever refers to "a priori" probabilities nor to "a posteriori" probabilities, but rather to "prior" and "posterior" probabilities. Nor do those terms make sense, in view of the usual meanings of "a priori" and "a posteriori" in epistemology, which is the field to which they properly belong. Those phrases are sometimes found used by mathematicians who are not familiar with related theoretical issues, but not by anyone else.

---

Would a simple derivation of Bayes' theorem from first principles be useful here?

Sure. 141.140.6.185 21:31 Nov 16, 2002 (UTC)

---

I thought that the definition of conditional probability, in a strict mathematical sense, is "Bayes' Theorem". That is, this is really a definition, and the proof is a justification of why this definition applies to real-world situations.

That is, the definitition of P(A | B) = P(A intersect B) / P(B). The conclusions are just algebraic maniupulation of this definition.

Well, you can easily get Bayes' Theorem by manipulating the definition of conditional probability in a simple way: but the deep question, when you've done the derivation, is what does this mean? This leads to the two schools of Bayesianism and Frequentism, and leads most people to a complete re-examination of the idea of probability. The axiomatic approach to probability is one get-out. Declaring the problem to be a non-problem is another. Becoming a Bayesian is the most common option.

---

Bayes' theorem is equally valid from the frequentist and Bayesian perspectives. What is not valid according to frequentism is the assignment to hypotheses, of probabilities that admit no frequency interpretation, such as the proposition that there was life on Mars a billion years ago, or that the mass of the planet Neptune is between two specified quantities. But before anyone says Bayes' theorem is at most trivially different from the definition of conditional probability, it must be remembered that there is a continuous version of Bayes' theorem, which say how to get the posterior probability density when given the prior density and the likelihood function. -- Mike Hardy

---

I've only been contributing to wikipedia actively again for a couple of days, and would like your comment on my proposal, before going ahead and doing it: I think that the readability of this article would be greatly improved if the equations involving division would be rendered with Tex, rather than just written out the way they are now. WDYT? --snoyes 22:02 Feb 16, 2003 (UTC)

For sure. TeX has been available on Wikipedia only since the beginning of 2003, and this article is older than that. As are lots of others, which are slowly being re-set in TeX whenever indiviidual Wikipedians with TeX skills get to them. Michael Hardy 22:07 Feb 16, 2003 (UTC)

---

i am missin a "decision tree"! see the German Verision.

---

Removed the text:

The correct punctuation is Bayes's theorem although loose use of the apostrophe appears to be common in the field, including many textbooks and the net. There was only one Reverend Bayes, and singular nouns always take apostrope-s to denote posession. See apostrophe. The formal rule is unambiguous.

1) It is not relevant to the article itself and spoils the flow.

2) It is incorrect. The use of the apostrophe in Bayes' Theorem is covered in Apostrophe_(punctuation) under the fourth point of Things to Note.

Spellbinder 14:46, 10 Dec 2003 (UTC)

I guess it does spoil the flow of the text...still, check this out (from The Phrase Finder, http://phrases.shu.ac.uk/ (and, mea culpa, I didn't read the wiki entry on apostrophe properly... I just assumed it'd agree with standard usage (cringe ;-)

From "A Manual of Style" (12th ed., University of Chicago Press, 1969): The possessive case of singular nouns is formed by the addition of an apostrophe and an "s," and the possessive of plural nouns (except for a few irregular plurals) by the addition of an apostrophe only. This general rule is well understood and for common nouns needs no examples to illustrate its application. There is only one notable exception to the rule for common nouns, a case wherein tradition and euphony dictate the use of the apostrophe only: for appearance' (conscience', righteousness', etc.) sake

PROPER NOUNS. The general rule covers proper nouns as well as common, including most names of any length ending in sibilants: Burns's poems, Marx's theories, Czar Nicholas's assassination, . . . , Dickens's novels, PLURAL: the Rosses' and the Williamses' lands

Exceptions are the names "Jesus" and "Moses" and Greek (or hellenized) names of more than one syllable ending in "es": Jesus' nativity, Moses' leadership, Euripides' plays, Demosthenes' orations, Rameses' tomb, Xerxes' army

YMMV

---

Hi there, 152.78.0.29 - at least I presume it's you! ;-)

My Fowler's Modern English Usage gives the clue as to why it's Bayes' Theorem and not Bayes's Theorem:

"It was formerly customary, when a word ended in -s, to write its possessive with an apostrophe but no additional s."

That's why those forms like Jesus' nativity hang around. Because Bayes' Theorem has been around a long time, it's still got its original possessive, which is now set in stone. I have a maths degree and it was always Bayes' Theorem and never Bayes's Theorem though I do agree that the latter form would be the correct one if the theorem were named today.

However, it's rather difficult to work that info into the article without going into a long digression about the history of the apostrophe, so I took the easy option and reverted it.

BTW, you can easily sign your contributions in talk by using four ~ (I can't write that in full here else it'll just replace it with my signature!) See Wikipedia:Talk_page for info on it.

Spellbinder 15:41, 10 Dec 2003 (UTC)

When did it cease to be customary? Fowler wrote in the 1920s. I am not aware that that practice is no longer standard. I suspect that is because I am an American: We still write "Mr. Smith" rather than "Mr Smith" and we still call that particular punctuation mark a "period" rather than by the more recent name "full stop". Michael Hardy 23:08, 27 Dec 2003 (UTC)

Hi Spellbinder. Thanks for the "~" tip (and indeed the Talk_page tip). I'm very new to wiki and am feeling my way around here; I figure the best way to learn is to try things out!

[the other thing I haven't yet worked out is how to get email notification of when a page is changed. I presume it's possible (unless people hit the "reload" button every five minutes). How does this work?]

best wishes

152.78.0.29 15:48, 10 Dec 2003 (UTC)

---

152.78.0.29 Since the chat's getting a bit off-topic, go to my talk-page at User_talk:Spellbinder. But give me a chance to update that! :-) Spellbinder 16:04, 10 Dec 2003 (UTC)

---

I've added some historical discussion, including a statement of Bayes's main result. Only problem is -- why does the formula (Prop 9) appear as LaTeX source instead of being rendered? I put it inside "langle math rangle" brackets but it didn't work. Can anyone fix it or tell me how? Wile E. Heresiarch 20:10, 27 Dec 2003 (UTC) -- OK, LaTeX is fixed now, great. Wile E. Heresiarch 16:51, 1 Jan 2004 (UTC)

---

This article is a bit disorganized.

• "Justification for Bayes' theorem" treads the same ground as "Bayes' theorem in probability theory".
• There's an example, some additional technical discussion, and then another example.
• "A worked example" has something about cookies and then a discussion of likelihood functions.
• "Bayesianism" is all the way at the end of the article.

Here's how I would suggest editing it.

• Strike "Justification for Bayes' theorem" section.
• Put both examples in one section, and put that section just before the end of the article.
• Move discussion of likelihood functions (now under "A worked example") into the technical discussion.
• Move "Bayesianism" to just after the technical discussion and before the examples, since Bayesianism is an interpretation of the just-described rule of algebra which justifies the examples.

I'm interested in your comments on this scheme. Wile E. Heresiarch 16:51, 1 Jan 2004 (UTC)

I've made the changes outlined above. Wile E. Heresiarch 03:31, 7 Jan 2004 (UTC)

---

I've made some minor changes.

1. Struck "Its discrete version may appear to go little beyond an identity...". This doesn't mean anything; every algebraic identity is a tautology, and so it "doesn't go beyond" the axioms and assumptions it's derived from. I think the original author meant to say it's a simple theorem, and I that's how I stated it elsewhere in the article.
2. Rephrased "A frequent error is to think..." -- whether the identification of Bayesianism with Bayes' rule is an error, is beside the point; the point is to very briefly state what Bayesianism is about. I've done so.
3. Rephrased "... while it says nothing 'new' ...". Same issue as first point above.
4. Explicitly stated a derivation of the theorem. Yes, it's short, but important enough to try to make sure everybody gets it.
5. Reworked the stuff around "often embellished using the [[law of total probability]]:". As noted in law of total probability, the term is ambiguous; it seems unwarranted to appeal to a "law of total probability" in a derivation.

Hope this makes it clearer -- Wile E. Heresiarch 17:43, 1 Jan 2004 (UTC)

---

I erased some comments that referred to old versions of the article. The page history still has them of course. Somebody please do likewise with the stuff about use of the apostrophe, i.e., somebody who was participating in that discussion. Wile E. Heresiarch 16:16, 2 Jan 2004 (UTC)

---

(i) I see that the equation near the top (under "Proposition 9 in the essay") has had the binomial coefficients struck out. I put them in originally because I was following Covarrubias's gloss (link at bottom of article) and he shows those coefficients. I would like to make the equation in the article accurately represent what Bayes wrote. However, on looking at the essay again, I can't tell for sure whether they're needed or not -- Bayes phrased everything in terms of geometric figures, and it's hard for me to map it into algebra. I'll study the essay some more and see if I can't figure it out. (ii) FWIW I don't think the use of the forward slash is an improvement over writing it numerator above denominator. Wile E. Heresiarch 03:31, 7 Jan 2004 (UTC)

• I've reverted the equation under "Proposition 9" to the form which contains the binomial coefficients. This is consistent with Stigler's exposition. (See Stigler 1982 in the refs.) I also looked at the original essay, and, although it is quite difficult to read, I see that Rule 1 (for computation) contains a term E, and "E being the coefficient of a^p b^q when (a + b)^n is expanded". I speculate that Bayes didn't see the binomial coefficients could be canceled, since he didn't actually write out the formula in the manner shown in the article. The whole topic of "what did Bayes prove", it turns out, is quite subtle; in fact, it's not clear that Bayes was a Bayesian. But that's a debate for another article. Wile E. Heresiarch 19:57, 7 Jan 2004 (UTC)

---

TeX rendering appears to be broken again, or there is a subtle bug in the formula I can't see. -- The Anome 20:32, 7 Jan 2004 (UTC)

---

Hello. I've reverted the change, dated 23:04, 16 Feb 2004, made to the paragraph "What is Bayesian about Proposition 9..." to the previous revision. The changed text stated, "That is, one assigns probabilities not to events that are random, to propositions that are uncertain, and the same algebra is used as in frequentist statistical inference." (1) Bayes did in fact assign probabilities to events, so the distinction made here doesn't apply to Bayes. (2) Bringing frequentism into the picture is a digression; the important point is that for Bayes, there was a single reasoning method which covered both the event and the parameter that governs it. Happy editing, Wile E. Heresiarch 23:20, 17 Feb 2004 (UTC)

---

BTW I like the stuff that User:Miguel put in about how Bayes' side-stepped some difficulties about the interpretation of the result. I am considering adding a heading to the Thomas Bayes article -- "Was Bayes a Bayesian?" Maybe someone else would like to do that! In this context the remarks by Stigler (JRSS Series A, 145:250-258, 1982) are especially relevant. Happy editing, Wile E. Heresiarch 23:58, 17 Feb 2004 (UTC)

I did not read the original article by Bayes, but rather the corresponding entry in

I. Todhunter, A History of the Mathematical Theory of Probability from the time of Pascal to that of Laplace (1865).

Looking back at my notes (translated into Spanish by myself from Todhunter) it seems that Bayes stated his question in Bayesian terms but then set out to solve it by assuming it is equivalent to the statement involving billiard balls. A frequentist could then argue that Bayes' original question is meaningless, and therefore not equivalent to the question about billiard balls. I'd like to get a copy of Todhunter from the library and track down the actual statement. Even better would be to get Bayes' article, which was reprinted as

J. Bayes, An Essay towards solving a problem in the Doctrine of Chances, Biometrica, 45 (1958).

— Miguel Wed Feb 18 01:05 GMT 2004

Whoops! the Biometrica reference is already in the main article. — Miguel Wed Feb 18 01:07 GMT 2004

I've gone ahead and put a "Was Bayes a Bayesian?" section in the Thomas Bayes article. The answer is quite subtle, I think! By the way, there is a link to an electronic version of the essay (in original notation) in the Bayes' theorem article. Happy editing, Wile E. Heresiarch 15:41, 20 Feb 2004 (UTC)

---

Hello. I wonder if the examples section can be tightened up by removing the cookies example. The medical test example is no more difficult, and a lot more interesting. I welcome your comments. Wile E. Heresiarch 02:29, 8 Apr 2004 (UTC)

I don't think that removing the cookie example is necessary. It is a `toy' example, yes, but it is easy to understand. The medical test example is more realistic and interesting, but contains concepts (false positives) with which the lay reader may not be instantly familiar. I don't think simple-but-dull examples detract from articles such as this one; the more the merrier. Ben Cairns 13:37, 12 Apr 2004 (UTC)