Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

A basic course in statistics will inform you of why a 99.99% accurate test should be looked at with skepticism when diagnosing a rare disease. Yet we see the fancy 9s and think somehow this many 9s is enough.


Sure, but that’s not what we are talking about


Diagnosis of rare diseases is a subset of "true" statements, no?


he's referring to frequency. a 2.9T parameter processing a 1M token data will inevitably produce wrong "false" tokens, guaranteed.

So even if you have a 99.99% pure date set of the "true" variety, the errors will aggregate to some very less than 99.99% characteristics.

Realm of big numbers problem.


Really? Laws of probability works against your argument.

Such a basic course in statistics needs to be scrutinized.


If 1 in 10,000 people have a disease, then a "test" which always reports the patient doesn't have the disease will be correct 99.99% of the time. "99.99% accuracy" should be "looked at with skepticism" in that it doesn't tell you what you need to know to understand the quality of a a test for a rare disease (a classifier under conditions of severe class imbalance); at a minimum, you would want to understand it's false positive and false negative rate, not (just) it's overall error rate.

See example "A": https://en.wikipedia.org/wiki/Base_rate_fallacy


If only stating the obvious was a fallacy :)

You appear not to have understood probability theory my friend. You will never get 100% in this universe for anything. What if "its a simulation" or "a dream" arguments ensures you never acheive 100%.

Bayes probability theory will be a good start for you.

https://en.wikipedia.org/wiki/Bayes%27_theorem


Well, I do agree that all measurements contain error, but the point wasn't that the error rate would be greater than 0% but that a single headline summary of error can't always distinguish between good and bad tests.


Every breath you take, there is a 99.99% chance that everything is normal, and a 0.01% chance that you breathe mild acid which horribly burns and causes a massive coughing fit.

It probably don’t cause long term damage unless that breath happened to be more important than normal, like while driving right as a child runs into the road.

Would you act differently knowing that you had one of these occasional acid breaths? Even if it only happened once per day on average (0.001%)


You are correct, but this is a made up example, not a practical matter. You’re falling into the same trap as the author.


You must have come to rescue him :) from the "incorrect" trap.


And your point is? Laws of probability doesn't discriminate.

There is always a probability a planet killer GRB hits us and we all die. Should we take precautions in our everyday life. Or should we wear helmet while walking on the road pray a coconut not falling on our head is just 99.999%.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: