An Introduction to Bayes' Theorem
(and it's connection to the Rx-Bayes App)
Transcript of lecture video above, with screenshots
0:00 - Hi there, this is Dr Hakeem, presenting an introduction to Bayes theorem and reasoning, specifically with some focus on medicine, but really intended as a gentle introduction to the background of Bayes' Theorem, and how the mathematics works a little bit behind the scenes without diving into any deep or complicated Algebra equations. This is actually a lecture I gave in Austin, Texas, in February 2018. It was entitled "Labs, Damn Lies and Statistics: An exploration of medicine, math, probability, error and belief." I'm Dr. Zain Hakeem. I'm a concierge physician at RiverRock Medical Clinic, in Austin, Texas, and, the creator of the Rx-Bayes Medical App, which is intended to do Bayesian reasoning in an intuitive way so that clinicians, residents, and premed students can understand and learn how to reason in a more bayesian way. So through this lecture we're going to go through the math and understand how Bayes theorem works, and then we're going to look at the App and how that simplifies things.
1:11 - We're going to look at how labs, and x-rays, can have errors and how they have to have errors; why error is built into the entire concept of labs and x-rays and pathology in fact. We're going to look at where pretest estimates come from, and then we're going to start to do some philosophical reflections, talk about some of the implications of bayesian reasoning.
I want to start by putting in your mind the idea of a 30 year old young female coming in complaining of Breast Cancer. This is a very prototypical example, but it makes the point quite well. She never had any risk factors, never found any lumps or bumps, but just got worried and ended up convincing a physician to give her a mammogram. That was suspicious for cancer. She then had a biopsy done and the biopsy actually confirmed the cancer, and she now comes in wanting to get treated.
2:05 - And the question is, should she get treated? It seems obvious, but if it seems obvious, I think you're in for a pretty wild ride to see how Bayes' theorem clarifies that the answer is not nearly as obvious as it seems.
So, to get there we have to go through the groundwork and that'll be a bit painful, but bear with me ... getting through the mathematics can sometimes be a little bit of a hassle, but, hopefully it'll be worth it in the end and you'll understand where the mathematics came from. The main thing for me is that I don't want anybody to feel that they have to take the the mathematics on faith. I want everyone to feel that they understand where the mathematics came from and why bayes' theorem is giving the implications that it's giving. We're gonna start with a real example of comparing lab-made diamonds to natural diamonds and trying to tell the difference.
3:02 - Lab-made diamond diamonds have gotten very sophisticated and now the only way to tell the difference between a lab-made diamond at a natural, or mined diamond, is to look at some of the stress patterns in the stone under fluoroscopy, using a device like this.
So imagine that you're a jewel merchant. How do you tell the difference between a supply of lab-made diamonds and natural real diamonds, if they're visually indistinguishable? By using a device.
So, let's suppose a new device is coming on the market and at the factory, the company who makes the device is trying to test that device for accuracy. They mark 100 real diamonds and 9,900 lab-made diamonds and they put them in a bag together and they run all 10,000 diamonds through the machine, and then they look at the results after the fact.
Of the real diamonds, the machine got it right and called those real diamonds as real, 70% of the time. Out of the lab-made diamonds, the machine was right 90% of the time, and said that they were in fact lab-made.
So far so good. But now suppose the company improves the quality of the device and releases it as device version two. And it's 99% accurate for both scenarios. It's 99% accurate calling lab-made diamonds as lab-made, and 99% accurate calling real diamonds as real, but it's wrong 1% of the time in both cases.
4:25 - So let's go through some of the math. If you chose a diamond at random, with no device to help you, the probability that that diamond would be real is 1%, because 100 of 10,000 were real. We call that the pretest probability. That's the probability before doing any kind of testing.
So, remember that first device that was 70% accurate real diamonds and 90% accurate? Let's see how that device does. If we pick a diamond at random and we feed it into the machine, and the machine pops out and says, "Hey! I think this diamond is real!" ... What's the probability that the machine is right?
5:16 - Well we can do that math. So we take all the times where the machine says that it's real, because that's what it told us. And out of all the times the machine says it's real, it's right 70 times, and it's wrong 990 times. And we can divide those to get a percent correct. So, the percentage of the time where the machine 1) says it's real, and 2) the machine is correct.
And it turns out that if the machine says it's real, it's only correct 6.6% of the time. But what's happened is that using the machine has increased the probability, or certainty, that that diamond was real from 1% up to 6.6%.
5:52 - So choosing at random, we could only be 1% certain that this diamond was real, but now that the machine told us that it thinks it's real, now the probability is 6.6%.
So let's see the more accurate device. How does the more accurate device perform? Does that help us out? It's 99% accurate. Surely that must help improve the quality.
Same process. We take a diamond at random, we feed it into the machine and the machine pops out and says that it thinks it's real. Now, what's the probability? Well, we can do the math the exact same way.
We take all the times where the machine says it's real. We divide the times that the machine is right by the total number of times tested, and we find out the percent-correct - and the machine is right 50% of the time.
6:54So this has gone from a 1% certainty of choosing at random, and now that the machine has told us that it thinks it's real, now we can be 50% certain it is real.
But wait a minute! Wasn't that device 99% accurate? Why is it only 50%?!?!?!?
Well, that's the counterintuitiveness of the math of Bayes Theorem. And when I learned it I was very surprised, and it didn't quite make sense ... how can the 99% accurate machine only be right 50% of the time?
But, thhere was a specific example that I think clarified it for me and hopefully it'll clarify it for you as well: the case where there are no real diamonds.