Now that we’ve learned the basic probability concepts (in LO10), the different schools of probability thoughts (in LO11 Day 1), we have enough of languages to talk about probability in the real world. Or more specifically…

How and why probability goes wrong in the real world.

Or to make it less negatively-sounding…

How and why human instincts, intuitions, and judgments on probability contradicts what the laws of probability suggest.

As a start, watch this 3blue1brown video on Bayes’ Theorem (15 mins). If you don’t like watching videos and prefer readings, you may alternatively read this interactive webpage that contains the same material (but with less animation/visualization). As a side practice, try to identify the school of probability thought this video belongs to, and any hidden probability assumption(s) it discusses. The answers can be found below.

---

Spoiler prevention space

 

 

 

 

 

 

 

 

---

Answer: the video falls under the Bayesian school of thought. Towards the end when it talks about whether you should assume Steve is a randomly sampled American, it gets at the assumption of equal probability outcomes.

Conjunction Fallacy

Aside from an introduction to Bayes’ theorem, the video also introduces the conjunction fallacy along the way without using the term explicitly. The conjunction fallacy refers to the Linda experiment where almost all participants chose the less likely event. The name comes from the fact that the second event (Linda is a bank teller AND is active in the feminist movement) is a conjunction (“AND”) of two logical statements.

Base Rate Fallacy (a.k.a. Medical Test Paradox)

Now, watch (at least the first part of) this 3blue1brown video on this more advanced concept in very similar context. (21 mins in total). You should watch at least the first 6 minutes that focus on introducing the fallacy/paradox; the later parts are on “fixing” the fallacy/paradox and are optional, since we more focus on identifying the fallacy/paradox in this class. (There is also a longer article by The Decision Lab on the same topic; consider this one optional)

Gambler’s Fallacy and Hot Hand Fallacy

Read this article by The Decision Lab on these two fallacies. They are two sides of the same coin: in repeated experiments (compound experiments that involve when the same experiment is repeated again and again) where the assumption of independence among all experiments is appropriate, the Gambler’s Fallacy happens when human incorrectly bias their belief of future trials against the recent outcomes, and the Hot Hand Fallacy happens when they bias towards the recent outcomes.

Simpson’s Paradox

Lastly, watch this short video by minutephysics on Simpson’s paradox (6 mins), which occurs when categorized data exhibits opposite trends when analyzed in aggregate vs. broken down by category. Aside from introducing the paradox, the video rightfully caveats against taking statistics out of context (the part where it says “more money makes you a cat”).

There are more fallacies and paradoxes out there (Shao-Heng’s favorite is the Bertrand’s paradox, if you crave for more), but the ones above are what this class can reasonably cover.

Schools of Probability Thoughts

Strictly speaking, LO10 was all math definitions and operations. We did contextualize the concepts using real-world examples, but we did not really touch the following fundamental question:

What do people actually mean when talking about probabilities? 

Turns out there are many different schools of thoughts when it comes to probability, just like there are different schools of thoughts when it comes to what AI is, or how to categorize kinds of AI systems.

Read the following articles:

• Interpretations of Probability by 1000wordphilosophy
• Comparing Frequentist and Bayesian Approaches by Statology

The first article is philosophical, whereas the second is from a data scientist’s perspective, making them complement each other.

After reading the two articles, you may realize that they talk about related but seemingly different concepts, and it is hard to synthesize between the philosophical and data scientific perspectives. Below is a table of how we think of the three schools of probability thoughts in this class:

"School of thought" (we will use these terms)	Closely related (or equivalent) to	Theoretical or Empirical?	Objective or Subjective?	What is probability/uncertainty in this school of thought?	Notes
Pure	Abstract; Classical; Logical	Theoretical	Objective	Probability is just a mathematical concept. It may or may not model the real world, and we might not care.	May or may not assume equally likely outcomes (also called the Principle of Indifference in the 1000wordphilosophy article)
Frequentist	Propensity	Empirical	Objective	Probability is just the long-run frequency that some event we care about occurs. There is a true underlying frequency. The more data/evidence we get, the closer we are to that truth.	Everything that EXCLUSIVELY draws from data falls under this school of thought.
Bayesian		Both	Subjective	Probability is just someone's belief about things. Data and evidence can lead to changes/updates in beliefs, but there is no such thing of a truth.	Can be purely theoretical, but since it is subjective, most of the time it is used in an empirical context.

In other words, when asked to identify the school of probability thought something falls under, you should answer one of Pure, Frequentist, or Bayesian, justified by evidence provided in the scenario.

Hidden/Implicit Assumptions in Probability Thoughts

1. Assumption of equally likely outcomes

As already mentioned in the 1000wordphilosophy article, this assumption  is frequently used in pure/classical probability. You should already be quite familiar with this, as our entire LO10 operates under this assumption.

As a ridiculous example of misusing the assumption, suppose we are interested in whether it will rain tomorrow in Durham, NC. We can define the two outcomes as {rain, no rain}. With the assumption of equally likely outcomes, the probability that it rains tomorrow is clearly one half, or 50%. Now, we can be more granular and define three outcomes, {no rain, light rain, heavy rain}. Now the assumption of equally likely outcomes tells us the probability that it rains tomorrow is two thirds, or about 66%. Clearly, at least one of these models is “wrong”. (As you’ll see below, both models are very far off empirically.)

2. Assumption of independent events

Another common hidden/implicit assumption in probabilistic reasoning is the assumption of independence between two or more events. This is especially prevalent when people reason about compound experiments involving many parallel or sequential experiments of the same kind. From LO10, we have learned that this assumption leads to very nice simplifications when calculating the probability of an event. Which is usually fine and inconsequential if we stay in pure/classical probability land, but if we also happen to care about modeling the real world reasonably well, we need to be vigilant when holding these assumptions, especially if without explicitly stating it.

As an example, our lovely city of Durham, NC has about 108 days of precipitation annually (note: this is a frequentist claim). We can simplify this a bit and assume it is 1 out of 3 days (although 108/365 is closer to 30%). What is the probability that it rains two days in a row in Durham then? If your gut feeling is to say about 1 out of 9, you just successfully applied the assumption of independence. But the precipitation events cannot really be assumed independent (think about multiple-day storms or hurricanes).

3. Assumption of stationarity

A third common one is the assumption of stationarity, which means the probabilistic model does not change with time (or any other unaccounted factors that may change with time).

Continuing our discussion on rainy days in Durham. Is it accurate to say the probability that it rains on a typical summer day in Durham and that for a typical winter day in Durham are both 30%? With data and some common knowledge about how seasons work*, we know the probability should be slightly higher for summer than for winter.

*Do not take this for granted. Many parts of the world do not have four distinguishable seasons. Some parts of the world do not even have two seasons.

Note that these assumptions are not always wrong/inappropriate. They can be reasonable if applied in appropriate contexts. We should use our best judgment in determining whether an assumption is appropriate in a context. Some of this is subjective.

There are more hidden/implicit assumptions out there that involve more complicated probability theory concepts beyond the scope of this not-a-math class. But you are encouraged to explore them yourself (potentially with AI, if you choose so).

For the probability LO, there is some built-in autonomy in the reference material to (hopefully) boost your motivation.

The concepts to learn in this LO are the following:

• Definitions of basic probability concepts (outcome, experiment, event, sample space, compound experiment)
• Basic probability axioms
• Disjoint vs. joint events
• Independent vs. not independent events
• Conditional probability

There are at least four different ways to learn these concepts, as we outline below. You should choose at least one route based on the following considerations:

1. your preferred style of the material – see the media form and “mathyness” information below
2. your future plans of study – if you have aspirations of taking data science, discrete math, majoring in CompSci/Statistics/Math, etc., it might make sense to do the version that is closest to your future study
3. whether you decided to use AI in your learning for this LO. The AI option is obviously only available to those of you who decided to use AI.

• CS216 (Everything Data) reference videos:
  • Content: two short videos (26 mins in total) designed for a data science class with running examples of application in medicine/vaccine.
  • Media form: video with text-based slides (no visualization)
  • “Mathyness”: medium. Prof. Fain talks in natural language, while the slides do use corresponding math notations.
  • Links: Video 1 (Outcomes, Events, Probabilities); Video 2 (Joint and Conditional Probability)
  • Note: no need to worry about the reference to a dataframe around 13:15 of Video 1, as that is a data science-specific reference.

• CS230 (Discrete Math) reference readings:
  • Content: two subsections of a textbook chapter (16 pages in total) designed for a CompSci discrete math class.
  • Media form: pure text with a few diagrams, with Shao-Heng’s PDF annotations marking what paragraphs to skip
  • “Mathyness”: extremely rigorous math notations
  • Link: In class Box folder

• Static Web Resource (MathIsFun.com):
  • Content: several interactive webpages
  • Media form: text-based but with illustrated examples and hyperlinks to related concepts
  • “Mathyness”: extremely lay language, does not use set notations, etc.
  • Links: Probability, Independent Events, Conditional Probability
  • Caveat: These few pages are somewhat repetitive and rely heavily on the assumption of equally-likely outcomes, which is not always true (we will discuss this in LO11). They also use terms such as dependent events that are not mathematically rigorous. What they are good at is avoiding math notations and providing links to related concepts.

• Generative AI with Internet Search capability (so for example, use Duke-accessible/paid ChatGPT instead of DukeGPT):
  • Content: you decide. For example, generative AI models can find all the materials above with accurate prompting.
  • Media form: you decide. You can explicitly instruct the AI to find material in your favorite media form.
  • “Mathyness”: you decide. You can explicitly instruct the AI to find material at the level of “mathyness” you prefer.
  • Note for prompting AI: be sure to give AI the following:
    • Context: this is for our class, CS/Edu171, which is a first-year class on learning theories, AI literacy, etc. You can even feed it this very website.
    • Task: the task here is to find appropriate resources for you to learn the concepts outlined above. So, give the AI the list of concepts.
    • Directions: what exactly do you want from the AI? Likely at least the links to whatever websites they are referring to.
    • Other important directions, like the media form and mathyness you prefer.
  • Notes: it is your own responsibility to check carefully that the material you got collectively covers all of the concepts. All the caveats we have discussed about using AI to learn in this class still apply. Finally, remember the task here for you is to learn these concepts. It is not just to find the appropriate resources–that’s the task for the AI.