🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 143 (from laksa157)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa143.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta/\')), getAhrefsUnparsedNoserviceFromURL(\'https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta/\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/beta\/","crawl_time":1774674071,"first_indexed_time":1610333737,"http_code":200,"src_unparsed":"io,github!chrispiech,\/probabilityForComputerScientists\/en\/part4\/beta\/ s443","src_root_hash":"2566890010099092343","history_drop_reason":null,"meta_title":"Beta Distribution","meta_descriptions":[],"attrs_boilerpipe_text":"The Beta distribution is the distribution most often used as the distribution of probabilities. In this section we are going to have a very meta discussion about how we represent probabilities. Until now probabilities have just been numbers in the range 0 to 1. However, if we have uncertainty about our probability, it would make sense to represent our probabilities as random variables (and thus articulate the relative likelihood of our belief).\nBeta Random Variable\nNotation:\nX\n∼\nBeta\n(\na\n,\nb\n)\nDescription:\nA belief distribution over the value of a probability\np\nfrom a Binomial distribution after observing\na\n−\n1\nsuccesses and\nb\n−\n1\nfails.\nParameters:\na\n>\n0\n, the number successes + 1\nb\n>\n0\n, the number of fails + 1\nSupport:\nx\n∈\n[\n0\n,\n1\n]\nPDF equation:\nf\n(\nx\n)\n=\nB\n(\na\n,\nb\n)\n⋅\nx\na\n−\n1\n⋅\n(\n1\n−\nx\n)\nb\n−\n1\nwhere\nB\n(\na\n,\nb\n)\n=\nΓ\n(\na\n)\nΓ\n(\nb\n)\nΓ\n(\na\n,\nb\n)\nCDF equation:\nNo closed form\nExpectation:\nE\n[\nX\n]\n=\na\na\n+\nb\nVariance:\nVar\n(\nX\n)\n=\na\nb\n(\na\n+\nb\n)\n2\n(\na\n+\nb\n+\n1\n)\nPDF graph:\nParameter\na\n:\nParameter\nb\n:\nWhat is your Belief in\np\nAfter 9 Heads in 10 Flips?\nImagine we have a coin and we would like to know its true probability of coming up heads,\np\n. We flip the coin 10 times and observe 9 heads and 1 tail. What is your belief in\np\nbased off this evidence? Using the definition of probability we could guess that\np\n≈\n9\n10\n. That number is a very rough estimate, especially since it is only based off 10 coin flips. Moreover the \"point-value\"\n9\n10\ndoes not have the ability to articulate how uncertain it is.\nCould we instead have a random variable for the true probability? Formally, let\nX\nrepresent the true probability of the coin coming up heads. We don't use the symbol\nP\nfor random variables, so\nX\nwill have to do. If\nX\n=\n0.7\nthen the probability of heads is 0.7.\nX\nmust be a continuous random variable with support\n[\n0\n,\n1\n]\nsince probabilities are continuous values which must be between 0 and 1.\nBefore flipping the coin, we could say that our belief about the coin's heads probability is uniform:\nX\n∼\nUni\n(\n0\n,\n1\n)\n. Let\nH\nbe a random variable for the number of heads and let\nT\nbe a random variable for the number of tails observed. What is\nP\n(\nX\n=\nx\n|\nH\n=\n9\n,\nT\n=\n1\n)\n?\nThat probability is hard to think about! However it is much easier to reason about the probability with the condition reveresed:\nP\n(\nH\n=\n9\n,\nT\n=\n1\n|\nX\n=\nx\n)\n. This term asks the question: what is the probability of seeing 9 heads and 1 tail in 10 coin flips, given that the true probability of a heads is\nx\n. Convince yourself that this probability is just a binomial probability mass function with\nn\n=\n10\nexperiements, and\np\n=\nx\nevaluated at\nk\n=\n9\nheads:\nP\n(\nH\n=\n9\n,\nT\n=\n1\n|\nX\n=\nx\n)\n=\n(\n10\n9\n)\nx\n9\n(\n1\n−\nx\n)\n1\nWe are presented with a perfect context for\nBayes' theorem with random variables\n. We know a conditional probability in one direction and we would like to know it in the other:\nf\n(\nX\n=\nx\n|\nH\n=\n9\n,\nT\n=\n1\n)\n=\nP\n(\nH\n=\n9\n,\nT\n=\n1\n|\nX\n=\nx\n)\n⋅\nf\n(\nX\n=\nx\n)\nP\n(\nH\n=\n9\n,\nT\n=\n1\n)\nBayes Theorem\n=\n(\n10\n9\n)\nx\n9\n(\n1\n−\nx\n)\n1\n⋅\nf\n(\nX\n=\nx\n)\nP\n(\nH\n=\n9\n,\nT\n=\n1\n)\nBinomial PMF\n=\n(\n10\n9\n)\nx\n9\n(\n1\n−\nx\n)\n1\n⋅\n1\nP\n(\nH\n=\n9\n,\nT\n=\n1\n)\nUniform PDF\n=\n(\n10\n9\n)\nP\n(\nH\n=\n9\n,\nT\n=\n1\n)\nx\n9\n(\n1\n−\nx\n)\n1\nConstants to front\n=\nK\n⋅\nx\n9\n(\n1\n−\nx\n)\n1\nRename constant\nLets take a look at that function. For now we can let\nK\n=\n110\n. Regardless of\nK\nwe will get the same shape, just scaled:\nWhat a beautiful image. It tells us relatively likelihood over the probability that is governing our coinflips. Here are a few observations from this chart:\nEven after only 10 coin flips we are very confident that the true probability is > 0.5\nIt is almost 10 times more likely that\nX\n=\n0.9\nas it is that\nX\n=\n0.6\n.\nf\n(\nX\n=\n1\n)\n=\n0\n, which makes sense. How could we have flipped that one tail if the probability of heads was 1?\nWait but why?\nIn the derivation above for\nf\n(\nX\n=\nx\n|\nH\n=\n9\n,\nT\n=\n1\n)\nwe made the claim that\nP\n(\nH\n=\n9\n,\nT\n=\n1\n)\nis a constant. A lot of folks find that hard to believe. Why is that the case?\nIt may be helpful to juxtapose\nP\n(\nH\n=\n9\n,\nT\n=\n1\n)\nwith\nP\n(\nH\n=\n9\n,\nT\n=\n1\n|\nX\n=\nx\n)\n. The later says \"what is the probability of 9 heads, given the true probability is\nx\n\". The former says \"what is the probability of 9 heads, under all possible assignments of\nx\n\". If you wanted to calculate\nP\n(\nH\n=\n9\n,\nT\n=\n1\n)\nyou could use the law of total probability:\nP\n(\nH\n=\n9\n,\nT\n=\n1\n)\n=\n∫\ny\n=\n0\n1\nP\n(\nH\n=\n9\n,\nT\n=\n1\n|\nX\n=\ny\n)\nf\n(\nX\n=\ny\n)\nThat is a hard number to calculate, but it is in fact a constant with respect to\nx\n.\nBeta Derivation\nLet's generalize the derivation from the previous section, using\nh\nfor the number of observed heads and\nt\nthe number of observed tails.\nIf we let\nH\n=\nh\nbe the event that we saw\nh\nheads, and let\nT\n=\nt\nbe the event that we saw\nt\ntails in\nh\n+\nt\ncoinflips. We want to calculate the probability density function\nf\n(\nX\n=\nx\n|\nH\n=\nh\n,\nT\n=\nt\n)\n. We can use the exact same series of steps, starting with Bayes Theorem:\nf\n(\nX\n=\nx\n|\nH\n=\nh\n,\nT\n=\nt\n)\n=\nP\n(\nH\n=\nh\n,\nT\n=\nt\n|\nX\n=\nx\n)\nf\n(\nX\n=\nx\n)\nP\n(\nH\n=\nh\n,\nT\n=\nt\n)\nBayes Theorem\n=\n(\nh\n+\nt\nh\n)\nx\nh\n(\n1\n−\nx\n)\nt\nP\n(\nH\n=\nh\n,\nT\n=\nt\n)\nBinomial PMF, Uniform PDF\n=\n(\nh\n+\nt\nh\n)\nP\n(\nH\n=\nh\n,\nT\n=\nt\n)\nx\nh\n(\n1\n−\nx\n)\nt\nMoving terms around\n=\n1\nc\n⋅\nx\nh\n(\n1\n−\nx\n)\nt\nwhere \nc\n=\n∫\n0\n1\nx\nh\n(\n1\n−\nx\n)\nt\nd\nx\nThe equation that we arrived at when using a Bayesian approach to estimating our probability defines a probability density function and thus a random variable. The random variable is called a Beta distribution, and it is defined as follows:\nThe Probability Density Function (PDF) for\nX\n∼\nBeta\n(\na\n,\nb\n)\nis:\nf\n(\nX\n=\nx\n)\n=\n{\n1\nB\n(\na\n,\nb\n)\nx\na\n−\n1\n(\n1\n−\nx\n)\nb\n−\n1\nif \n0\n<\nx\n<\n1\n0\notherwise\nwhere \nB\n(\na\n,\nb\n)\n=\n∫\n0\n1\nx\na\n−\n1\n(\n1\n−\nx\n)\nb\n−\n1\nd\nx\nA Beta distribution has\nE\n[\nX\n]\n=\na\na\n+\nb\nand\nV\na\nr\n(\nX\n)\n=\na\nb\n(\na\n+\nb\n)\n2\n(\na\n+\nb\n+\n1\n)\n. All modern programming languages have a package for calculating Beta CDFs. You will not be expected to compute the CDF by hand in CS109.\nTo model our estimate of the probability of a coin coming up heads: set\na\n=\nh\n+\n1\nand\nb\n=\nt\n+\n1\n. Beta is used as a random variable to represent a belief distribution of probabilities in contexts beyond estimating coin flips. For example perhaps a drug has been given to 6 patients, 4 of whom have been cured. We could express our belief in the probability that the drug can cure patients as\nX\n∼\nBeta\n(\na\n=\n5\n,\nb\n=\n3\n)\n:\nNotice how the most likely belief for the probability of curing a patient, is\n4\n\/\n6\n, the fraction of patients cured. This distribution shows that we hold a non-zero belief that the probability could be something other than\n4\n\/\n6\n. It is unlikely that the probability is 0.01 or 0.09, but reasonably likely that it could be 0.5.\nBeta as a Prior\nYou can set\nX\n∼\nBeta\n(\na\n,\nb\n)\nas a prior to reflect how biased you think the coin is apriori to flipping it. This is a subjective judgment that represent\na\n+\nb\n−\n2\n\"imaginary\" trials with\na\n−\n1\nheads and\nb\n−\n1\ntails. If you then observe\nh\n+\nt\nreal trials with\nh\nheads you can update your belief. Your new belief would be,\nX\n∼\nBeta\n(\na\n+\nh\n,\nb\n+\nt\n)\n. Using the prior\nBeta\n(\n1\n,\n1\n)\n=\nUni\n(\n0\n,\n1\n)\nis the same as saying we haven't seen any \"imaginary\" trials, so apriori we know nothing about the coin. Here is the proof for the distribution of\nX\nwhen the prior was a Beta too:\nIf our prior belief is\nX\n∼\nBeta\n(\na\n,\nb\n)\n, then our posterior is\nBeta\n(\na\n+\nh\n,\nb\n+\nt\n)\n:\nf\n(\nX\n=\nx\n|\nH\n=\nh\n,\nT\n=\nt\n)\n=\nP\n(\nH\n=\nh\n,\nT\n=\nt\n|\nX\n=\nx\n)\nf\n(\nX\n=\nx\n)\nP\n(\nH\n=\nh\n,\nT\n=\nt\n)\nBayes Theorem\n=\n(\nh\n+\nt\nh\n)\nx\nh\n(\n1\n−\nx\n)\nt\n⋅\n1\nc\n⋅\nx\na\n−\n1\n(\n1\n−\nx\n)\nb\n−\n1\nP\n(\nH\n=\nh\n,\nT\n=\nt\n)\nBeta PMF, Uniform PDF\n=\nK\n⋅\nx\nh\n(\n1\n−\nx\n)\nt\n⋅\nx\na\n−\n1\n(\n1\n−\nx\n)\nb\n−\n1\nCombine Constants\n=\nK\n⋅\nx\na\n+\nh\n−\n1\n(\n1\n−\nx\n)\nb\n+\nt\n−\n1\nCombine Like Bases\nWhich is the PDF of\nBeta\n(\na\n+\nh\n,\nb\n+\nt\n)\nIt is pretty convenient that if we have a Beta prior belief, then our posterior belief is also Beta. This makes Betas especially convenient to work with, in code and in proof, if there are many updates that you will make to your belief over time. This property where the type of distribution is the same before and after an observation is called a conjugate prior.\nQuick question: Are you allowed to just make up priors and imaginary trials? Some folks think that is fine (they are called Bayesians) and some folks think that you shouldn't make up prior beliefs (they are called frequentists). In general, for small data it can make you much better at making predictions if you are able to come up with a good prior belief.\nObservation: There is a deep connection between the beta-prior and the uniform-prior (which we used initially). It turns out that\nBeta\n(\n1\n,\n1\n)\n=\nUni\n(\n0\n,\n1\n)\n. Recall that\nBeta\n(\n1\n,\n1\n)\nmeans 0 imaginary heads and 0 imaginary tails.","attrs_markdown":"[Probability For Computer Science](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/index.html)\n\n- [Notation Reference](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/intro\/notation) [Core Probability Reference](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/intro\/core_probability_ref) [Random Variable Reference](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/intro\/all_distributions) [Python Reference](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/intro\/python) [Calculators](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/intro\/calculators)\n\n- Part 1: Core Probability\n- [Counting](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/counting) [Combinatorics](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/combinatorics) [Definition of Probability](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/probability) [Equally Likely Outcomes](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/equally_likely) [Probability of **or**](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/prob_or) [Conditional Probability](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/cond_prob) [Independence](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/independence) [Probability of **and**](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/prob_and) [Law of Total Probability](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/law_total) [Bayes' Theorem](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/bayes_theorem) [Log Probabilities](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/log_probabilities) [Many Coin Flips](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part1\/many_flips)\n- [Applications](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/beta\/#part1_examples)\n  - [Enigma Machine](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/enigma)\n  - [Serendipity](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/serendipity)\n  - [Random Shuffles](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/random_shuffles)\n  - [Random Graphs](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/counting_graphs)\n  - [Set Diversity](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/diversity_shapes)\n  - [Bacteria Evolution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/bacteria_evolution)\n\n- Part 2: Random Variables\n- [Random Variables](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/rvs) [Probability Mass Functions](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/pmf) [Expectation](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/expectation) [Variance](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/variance) [Bernoulli Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/bernoulli) [Binomial Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/binomial) [Poisson Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/poisson) [More Discrete Distributions](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/more_discrete) [Categorical Distributions](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/categorical) [Continuous Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/continuous) [Uniform Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/uniform) [Exponential Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/exponential) [Normal Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/normal) [Binomial Approximation](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part2\/binomial_approx)\n- [Applications](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/beta\/#part2_examples)\n  - [100 Binomial Problems](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/100_binomial_problems)\n  - [Winning Series](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/winning_series)\n  - [Approximate Counting](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/approximate_counting)\n  - [Jury Selection](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/jury)\n  - [Grading Eye Inflamation](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/grading_eye_inflammation)\n  - [Grades are Not Normal](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/grades_not_normal)\n  - [Curse of Dimensionality](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/curse_of_dimensionality)\n  - [Algorithmic Art](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/algorithmic_art)\n\n- Part 3: Probabilistic Models\n- [Joint Probability](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/joint) [Marginalization](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/marginalization) [Multinomial](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/multinomial) [Continuous Joint](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/continuous_joint) [Inference](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/inference) [Bayesian Networks](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/bayesian_networks) [Independence in Variables](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/independent_vars) [Correlation](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/correlation) [General Inference](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/computational_inference)\n- [Applications](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/beta\/#part3_examples)\n  - [Fairness in AI](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/fairness)\n  - [Federalist Paper Authorship](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/federalist)\n  - [Name to Age](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/name2age)\n  - [Probability of Baby Delivery](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/prob_baby_delivery)\n  - [Bayesian Carbon Dating](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/bayesian_carbon_dating)\n  - [Digital Vision Test](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/digital_vision_test)\n  - [Bridge Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/bridge_distribution)\n  - [Expectation of Sum Proof](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/expectation_of_sums)\n  - [Bayesian Viral Load Test](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/bayesian_viral_load_test)\n  - [CS109 Logo](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/dart_logo)\n  - [Tracking in 2D](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/tracking_in_2D)\n\n- Part 4: Uncertainty Theory\n- [Beta Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/beta) [Adding Random Variables](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/summation_vars) [Central Limit Theorem](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/clt) [Sampling](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/samples) [Bootstrapping](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/bootstrapping) [Algorithmic Analysis](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/algorithmic_analysis) [Information Theory](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/information_theory) [Distance Between Distributions](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/divergence)\n- [Applications](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/beta\/#part4_examples)\n  - [Thompson Sampling](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/thompson)\n  - [Night Sight](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/night_sight)\n  - [P-Hacking](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/p_hacking)\n  - [Differential Privacy](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/differential_privacy)\n\n- Part 5: Machine Learning\n- [Parameter Estimation](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part5\/parameter_estimation) [Maximum Likelihood Estimation](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part5\/mle) [Maximum A Posteriori](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part5\/map) [Machine Learning](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part5\/machine_learning) [Naïve Bayes](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part5\/naive_bayes) [Logistic Regression](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part5\/log_regression) [Diffusion](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part5\/diffusion)\n- [Applications](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part4\/beta\/#part5_examples)\n  - [MLE Normal Demo](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/mle_demo)\n  - [MLE Pareto Distribution](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/mle_pareto)\n  - [MLE Mixture Model](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/examples\/mixture_models)\n\n- Ⓒ Chris Piech, Stanford University\n\n# Beta Distribution\n***\nThe Beta distribution is the distribution most often used as the distribution of probabilities. In this section we are going to have a very meta discussion about how we represent probabilities. Until now probabilities have just been numbers in the range 0 to 1. However, if we have uncertainty about our probability, it would make sense to represent our probabilities as random variables (and thus articulate the relative likelihood of our belief).\n\n**Beta Random Variable**\n\n|   |   |\n|---|---|\n| Notation: | X ∼ Beta ( a , b ) |\n| Description: | A belief distribution over the value of a probability p from a Binomial distribution after observing a − 1 successes and b − 1 fails. |\n| Parameters: | a \\> 0, the number successes + 1   b \\> 0, the number of fails + 1 |\n| Support: | x ∈ \\[ 0 , 1 \\] |\n| PDF equation: | f ( x ) \\= B ( a , b ) ⋅ x a − 1 ⋅ ( 1 − x ) b − 1 where B ( a , b ) \\= Γ ( a ) Γ ( b ) Γ ( a , b ) |\n| CDF equation: | No closed form |\n| Expectation: | E \\[ X \\] \\= a a \\+ b |\n| Variance: | Var ( X ) \\= a b ( a \\+ b ) 2 ( a \\+ b \\+ 1 ) |\n| PDF graph: |   |\n\nParameter\n\na\n\n:\n\nParameter\n\nb\n\n:\n## What is your Belief in p After 9 Heads in 10 Flips?\nImagine we have a coin and we would like to know its true probability of coming up heads, p. We flip the coin 10 times and observe 9 heads and 1 tail. What is your belief in p based off this evidence? Using the definition of probability we could guess that p ≈ 9 10. That number is a very rough estimate, especially since it is only based off 10 coin flips. Moreover the \"point-value\" 9 10 does not have the ability to articulate how uncertain it is.\n\nCould we instead have a random variable for the true probability? Formally, let X represent the true probability of the coin coming up heads. We don't use the symbol P for random variables, so X will have to do. If X \\= 0\\.7 then the probability of heads is 0.7. X must be a continuous random variable with support \\[ 0 , 1 \\] since probabilities are continuous values which must be between 0 and 1.\n\nBefore flipping the coin, we could say that our belief about the coin's heads probability is uniform: X ∼ Uni ( 0 , 1 ). Let H be a random variable for the number of heads and let T be a random variable for the number of tails observed. What is P ( X \\= x \\| H \\= 9 , T \\= 1 )?\n\nThat probability is hard to think about! However it is much easier to reason about the probability with the condition reveresed: P ( H \\= 9 , T \\= 1 \\| X \\= x ). This term asks the question: what is the probability of seeing 9 heads and 1 tail in 10 coin flips, given that the true probability of a heads is x. Convince yourself that this probability is just a binomial probability mass function with n \\= 10 experiements, and p \\= x evaluated at k \\= 9 heads: P ( H \\= 9 , T \\= 1 \\| X \\= x ) \\= ( 10 9 ) x 9 ( 1 − x ) 1\n\nWe are presented with a perfect context for [Bayes' theorem with random variables](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/en\/part3\/inference\/). We know a conditional probability in one direction and we would like to know it in the other: f ( X \\= x \\| H \\= 9 , T \\= 1 ) \\= P ( H \\= 9 , T \\= 1 \\| X \\= x ) ⋅ f ( X \\= x ) P ( H \\= 9 , T \\= 1 ) Bayes Theorem \\= ( 10 9 ) x 9 ( 1 − x ) 1 ⋅ f ( X \\= x ) P ( H \\= 9 , T \\= 1 ) Binomial PMF \\= ( 10 9 ) x 9 ( 1 − x ) 1 ⋅ 1 P ( H \\= 9 , T \\= 1 ) Uniform PDF \\= ( 10 9 ) P ( H \\= 9 , T \\= 1 ) x 9 ( 1 − x ) 1 Constants to front \\= K ⋅ x 9 ( 1 − x ) 1 Rename constant\n\nLets take a look at that function. For now we can let K \\= 110. Regardless of K we will get the same shape, just scaled:\n\nWhat a beautiful image. It tells us relatively likelihood over the probability that is governing our coinflips. Here are a few observations from this chart:\n\n1. Even after only 10 coin flips we are very confident that the true probability is \\> 0.5\n2. It is almost 10 times more likely that\n   X\n   \n   \\=\n   \n   0\\.9\n   as it is that\n   X\n   \n   \\=\n   \n   0\\.6\n   .\n3. f\n   \n   (\n   \n   X\n   \n   \\=\n   \n   1\n   \n   )\n   \n   \\=\n   \n   0\n   , which makes sense. How could we have flipped that one tail if the probability of heads was 1?\n\n***Wait but why?***\n\nIn the derivation above for f ( X \\= x \\| H \\= 9 , T \\= 1 ) we made the claim that P ( H \\= 9 , T \\= 1 ) is a constant. A lot of folks find that hard to believe. Why is that the case?\n\nIt may be helpful to juxtapose P ( H \\= 9 , T \\= 1 ) with P ( H \\= 9 , T \\= 1 \\| X \\= x ). The later says \"what is the probability of 9 heads, given the true probability is x\". The former says \"what is the probability of 9 heads, under all possible assignments of x\". If you wanted to calculate P ( H \\= 9 , T \\= 1 ) you could use the law of total probability: P ( H \\= 9 , T \\= 1 ) \\= ∫ y \\= 0 1 P ( H \\= 9 , T \\= 1 \\| X \\= y ) f ( X \\= y ) That is a hard number to calculate, but it is in fact a constant with respect to x.\n## Beta Derivation\nLet's generalize the derivation from the previous section, using h for the number of observed heads and t the number of observed tails.\n\nIf we let H \\= h be the event that we saw h heads, and let T \\= t be the event that we saw t tails in h \\+ t coinflips. We want to calculate the probability density function f ( X \\= x \\| H \\= h , T \\= t ). We can use the exact same series of steps, starting with Bayes Theorem: f ( X \\= x \\| H \\= h , T \\= t ) \\= P ( H \\= h , T \\= t \\| X \\= x ) f ( X \\= x ) P ( H \\= h , T \\= t ) Bayes Theorem \\= ( h \\+ t h ) x h ( 1 − x ) t P ( H \\= h , T \\= t ) Binomial PMF, Uniform PDF \\= ( h \\+ t h ) P ( H \\= h , T \\= t ) x h ( 1 − x ) t Moving terms around \\= 1 c ⋅ x h ( 1 − x ) t where c \\= ∫ 0 1 x h ( 1 − x ) t d x\n\nThe equation that we arrived at when using a Bayesian approach to estimating our probability defines a probability density function and thus a random variable. The random variable is called a Beta distribution, and it is defined as follows:\n\nThe Probability Density Function (PDF) for X ∼ Beta ( a , b ) is: f ( X \\= x ) \\= { 1 B ( a , b ) x a − 1 ( 1 − x ) b − 1 if 0 \\< x \\< 1 0 otherwise where B ( a , b ) \\= ∫ 0 1 x a − 1 ( 1 − x ) b − 1 d x A Beta distribution has E \\[ X \\] \\= a a \\+ b and V a r ( X ) \\= a b ( a \\+ b ) 2 ( a \\+ b \\+ 1 ). All modern programming languages have a package for calculating Beta CDFs. You will not be expected to compute the CDF by hand in CS109.\n\nTo model our estimate of the probability of a coin coming up heads: set a \\= h \\+ 1 and b \\= t \\+ 1. Beta is used as a random variable to represent a belief distribution of probabilities in contexts beyond estimating coin flips. For example perhaps a drug has been given to 6 patients, 4 of whom have been cured. We could express our belief in the probability that the drug can cure patients as X ∼ Beta ( a \\= 5 , b \\= 3 ):\n\n![](https:\/\/chrispiech.github.io\/probabilityForComputerScientists\/img\/chapters\/beta.png)\n\nNotice how the most likely belief for the probability of curing a patient, is 4 \/ 6, the fraction of patients cured. This distribution shows that we hold a non-zero belief that the probability could be something other than 4 \/ 6. It is unlikely that the probability is 0.01 or 0.09, but reasonably likely that it could be 0.5.\n\n## Beta as a Prior\nYou can set X ∼ Beta ( a , b ) as a prior to reflect how biased you think the coin is apriori to flipping it. This is a subjective judgment that represent a \\+ b − 2 \"imaginary\" trials with a − 1 heads and b − 1 tails. If you then observe h \\+ t real trials with h heads you can update your belief. Your new belief would be, X ∼ Beta ( a \\+ h , b \\+ t ). Using the prior Beta ( 1 , 1 ) \\= Uni ( 0 , 1 ) is the same as saying we haven't seen any \"imaginary\" trials, so apriori we know nothing about the coin. Here is the proof for the distribution of X when the prior was a Beta too:\n\nIf our prior belief is\n\nX\n\n∼\n\nBeta\n\n(\n\na\n\n,\n\nb\n\n)\n\n, then our posterior is\n\nBeta\n\n(\n\na\n\n\\+\n\nh\n\n,\n\nb\n\n\\+\n\nt\n\n)\n\n:\n\nf\n\n(\n\nX\n\n\\=\n\nx\n\n\\|\n\nH\n\n\\=\n\nh\n\n,\n\nT\n\n\\=\n\nt\n\n)\n\n\\=\n\nP\n\n(\n\nH\n\n\\=\n\nh\n\n,\n\nT\n\n\\=\n\nt\n\n\\|\n\nX\n\n\\=\n\nx\n\n)\n\nf\n\n(\n\nX\n\n\\=\n\nx\n\n)\n\nP\n\n(\n\nH\n\n\\=\n\nh\n\n,\n\nT\n\n\\=\n\nt\n\n)\n\nBayes Theorem\n\n\\=\n\n(\n\nh\n\n\\+\n\nt\n\nh\n\n)\n\nx\n\nh\n\n(\n\n1\n\n−\n\nx\n\n)\n\nt\n\n⋅\n\n1\n\nc\n\n⋅\n\nx\n\na\n\n−\n\n1\n\n(\n\n1\n\n−\n\nx\n\n)\n\nb\n\n−\n\n1\n\nP\n\n(\n\nH\n\n\\=\n\nh\n\n,\n\nT\n\n\\=\n\nt\n\n)\n\nBeta PMF, Uniform PDF\n\n\\=\n\nK\n\n⋅\n\nx\n\nh\n\n(\n\n1\n\n−\n\nx\n\n)\n\nt\n\n⋅\n\nx\n\na\n\n−\n\n1\n\n(\n\n1\n\n−\n\nx\n\n)\n\nb\n\n−\n\n1\n\nCombine Constants\n\n\\=\n\nK\n\n⋅\n\nx\n\na\n\n\\+\n\nh\n\n−\n\n1\n\n(\n\n1\n\n−\n\nx\n\n)\n\nb\n\n\\+\n\nt\n\n−\n\n1\n\nCombine Like Bases\n\nWhich is the PDF of\n\nBeta\n\n(\n\na\n\n\\+\n\nh\n\n,\n\nb\n\n\\+\n\nt\n\n)\n\nIt is pretty convenient that if we have a Beta prior belief, then our posterior belief is also Beta. This makes Betas especially convenient to work with, in code and in proof, if there are many updates that you will make to your belief over time. This property where the type of distribution is the same before and after an observation is called a conjugate prior.\n\nQuick question: Are you allowed to just make up priors and imaginary trials? Some folks think that is fine (they are called Bayesians) and some folks think that you shouldn't make up prior beliefs (they are called frequentists). In general, for small data it can make you much better at making predictions if you are able to come up with a good prior belief.\n\nObservation: There is a deep connection between the beta-prior and the uniform-prior (which we used initially). It turns out that Beta ( 1 , 1 ) \\= Uni ( 0 , 1 ). Recall that Beta ( 1 , 1 ) means 0 imaginary heads and 0 imaginary tails.","attrs_readable_markdown":null,"meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

20 days ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0.7 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta/
Last Crawled	2026-03-28 05:01:11 (20 days ago)
First Indexed	2021-01-11 02:55:37 (5 years ago)
HTTP Status Code	200
Meta Title	Beta Distribution
Meta Description	null
Meta Canonical	null
Boilerpipe Text	The Beta distribution is the distribution most often used as the distribution of probabilities. In this section we are going to have a very meta discussion about how we represent probabilities. Until now probabilities have just been numbers in the range 0 to 1. However, if we have uncertainty about our probability, it would make sense to represent our probabilities as random variables (and thus articulate the relative likelihood of our belief). Beta Random Variable Notation: X ∼ Beta ( a , b ) Description: A belief distribution over the value of a probability p from a Binomial distribution after observing a − 1 successes and b − 1 fails. Parameters: a > 0 , the number successes + 1 b > 0 , the number of fails + 1 Support: x ∈ [ 0 , 1 ] PDF equation: f ( x ) = B ( a , b ) ⋅ x a − 1 ⋅ ( 1 − x ) b − 1 where B ( a , b ) = Γ ( a ) Γ ( b ) Γ ( a , b ) CDF equation: No closed form Expectation: E [ X ] = a a + b Variance: Var ( X ) = a b ( a + b ) 2 ( a + b + 1 ) PDF graph: Parameter a : Parameter b : What is your Belief in p After 9 Heads in 10 Flips? Imagine we have a coin and we would like to know its true probability of coming up heads, p . We flip the coin 10 times and observe 9 heads and 1 tail. What is your belief in p based off this evidence? Using the definition of probability we could guess that p ≈ 9 10 . That number is a very rough estimate, especially since it is only based off 10 coin flips. Moreover the "point-value" 9 10 does not have the ability to articulate how uncertain it is. Could we instead have a random variable for the true probability? Formally, let X represent the true probability of the coin coming up heads. We don't use the symbol P for random variables, so X will have to do. If X = 0.7 then the probability of heads is 0.7. X must be a continuous random variable with support [ 0 , 1 ] since probabilities are continuous values which must be between 0 and 1. Before flipping the coin, we could say that our belief about the coin's heads probability is uniform: X ∼ Uni ( 0 , 1 ) . Let H be a random variable for the number of heads and let T be a random variable for the number of tails observed. What is P ( X = x \| H = 9 , T = 1 ) ? That probability is hard to think about! However it is much easier to reason about the probability with the condition reveresed: P ( H = 9 , T = 1 \| X = x ) . This term asks the question: what is the probability of seeing 9 heads and 1 tail in 10 coin flips, given that the true probability of a heads is x . Convince yourself that this probability is just a binomial probability mass function with n = 10 experiements, and p = x evaluated at k = 9 heads: P ( H = 9 , T = 1 \| X = x ) = ( 10 9 ) x 9 ( 1 − x ) 1 We are presented with a perfect context for Bayes' theorem with random variables . We know a conditional probability in one direction and we would like to know it in the other: f ( X = x \| H = 9 , T = 1 ) = P ( H = 9 , T = 1 \| X = x ) ⋅ f ( X = x ) P ( H = 9 , T = 1 ) Bayes Theorem = ( 10 9 ) x 9 ( 1 − x ) 1 ⋅ f ( X = x ) P ( H = 9 , T = 1 ) Binomial PMF = ( 10 9 ) x 9 ( 1 − x ) 1 ⋅ 1 P ( H = 9 , T = 1 ) Uniform PDF = ( 10 9 ) P ( H = 9 , T = 1 ) x 9 ( 1 − x ) 1 Constants to front = K ⋅ x 9 ( 1 − x ) 1 Rename constant Lets take a look at that function. For now we can let K = 110 . Regardless of K we will get the same shape, just scaled: What a beautiful image. It tells us relatively likelihood over the probability that is governing our coinflips. Here are a few observations from this chart: Even after only 10 coin flips we are very confident that the true probability is > 0.5 It is almost 10 times more likely that X = 0.9 as it is that X = 0.6 . f ( X = 1 ) = 0 , which makes sense. How could we have flipped that one tail if the probability of heads was 1? Wait but why? In the derivation above for f ( X = x \| H = 9 , T = 1 ) we made the claim that P ( H = 9 , T = 1 ) is a constant. A lot of folks find that hard to believe. Why is that the case? It may be helpful to juxtapose P ( H = 9 , T = 1 ) with P ( H = 9 , T = 1 \| X = x ) . The later says "what is the probability of 9 heads, given the true probability is x ". The former says "what is the probability of 9 heads, under all possible assignments of x ". If you wanted to calculate P ( H = 9 , T = 1 ) you could use the law of total probability: P ( H = 9 , T = 1 ) = ∫ y = 0 1 P ( H = 9 , T = 1 \| X = y ) f ( X = y ) That is a hard number to calculate, but it is in fact a constant with respect to x . Beta Derivation Let's generalize the derivation from the previous section, using h for the number of observed heads and t the number of observed tails. If we let H = h be the event that we saw h heads, and let T = t be the event that we saw t tails in h + t coinflips. We want to calculate the probability density function f ( X = x \| H = h , T = t ) . We can use the exact same series of steps, starting with Bayes Theorem: f ( X = x \| H = h , T = t ) = P ( H = h , T = t \| X = x ) f ( X = x ) P ( H = h , T = t ) Bayes Theorem = ( h + t h ) x h ( 1 − x ) t P ( H = h , T = t ) Binomial PMF, Uniform PDF = ( h + t h ) P ( H = h , T = t ) x h ( 1 − x ) t Moving terms around = 1 c ⋅ x h ( 1 − x ) t where c = ∫ 0 1 x h ( 1 − x ) t d x The equation that we arrived at when using a Bayesian approach to estimating our probability defines a probability density function and thus a random variable. The random variable is called a Beta distribution, and it is defined as follows: The Probability Density Function (PDF) for X ∼ Beta ( a , b ) is: f ( X = x ) = { 1 B ( a , b ) x a − 1 ( 1 − x ) b − 1 if 0 < x < 1 0 otherwise where B ( a , b ) = ∫ 0 1 x a − 1 ( 1 − x ) b − 1 d x A Beta distribution has E [ X ] = a a + b and V a r ( X ) = a b ( a + b ) 2 ( a + b + 1 ) . All modern programming languages have a package for calculating Beta CDFs. You will not be expected to compute the CDF by hand in CS109. To model our estimate of the probability of a coin coming up heads: set a = h + 1 and b = t + 1 . Beta is used as a random variable to represent a belief distribution of probabilities in contexts beyond estimating coin flips. For example perhaps a drug has been given to 6 patients, 4 of whom have been cured. We could express our belief in the probability that the drug can cure patients as X ∼ Beta ( a = 5 , b = 3 ) : Notice how the most likely belief for the probability of curing a patient, is 4 / 6 , the fraction of patients cured. This distribution shows that we hold a non-zero belief that the probability could be something other than 4 / 6 . It is unlikely that the probability is 0.01 or 0.09, but reasonably likely that it could be 0.5. Beta as a Prior You can set X ∼ Beta ( a , b ) as a prior to reflect how biased you think the coin is apriori to flipping it. This is a subjective judgment that represent a + b − 2 "imaginary" trials with a − 1 heads and b − 1 tails. If you then observe h + t real trials with h heads you can update your belief. Your new belief would be, X ∼ Beta ( a + h , b + t ) . Using the prior Beta ( 1 , 1 ) = Uni ( 0 , 1 ) is the same as saying we haven't seen any "imaginary" trials, so apriori we know nothing about the coin. Here is the proof for the distribution of X when the prior was a Beta too: If our prior belief is X ∼ Beta ( a , b ) , then our posterior is Beta ( a + h , b + t ) : f ( X = x \| H = h , T = t ) = P ( H = h , T = t \| X = x ) f ( X = x ) P ( H = h , T = t ) Bayes Theorem = ( h + t h ) x h ( 1 − x ) t ⋅ 1 c ⋅ x a − 1 ( 1 − x ) b − 1 P ( H = h , T = t ) Beta PMF, Uniform PDF = K ⋅ x h ( 1 − x ) t ⋅ x a − 1 ( 1 − x ) b − 1 Combine Constants = K ⋅ x a + h − 1 ( 1 − x ) b + t − 1 Combine Like Bases Which is the PDF of Beta ( a + h , b + t ) It is pretty convenient that if we have a Beta prior belief, then our posterior belief is also Beta. This makes Betas especially convenient to work with, in code and in proof, if there are many updates that you will make to your belief over time. This property where the type of distribution is the same before and after an observation is called a conjugate prior. Quick question: Are you allowed to just make up priors and imaginary trials? Some folks think that is fine (they are called Bayesians) and some folks think that you shouldn't make up prior beliefs (they are called frequentists). In general, for small data it can make you much better at making predictions if you are able to come up with a good prior belief. Observation: There is a deep connection between the beta-prior and the uniform-prior (which we used initially). It turns out that Beta ( 1 , 1 ) = Uni ( 0 , 1 ) . Recall that Beta ( 1 , 1 ) means 0 imaginary heads and 0 imaginary tails.
Markdown	[Probability For Computer Science](https://chrispiech.github.io/probabilityForComputerScientists/en/index.html) - [Notation Reference](https://chrispiech.github.io/probabilityForComputerScientists/en/intro/notation) [Core Probability Reference](https://chrispiech.github.io/probabilityForComputerScientists/en/intro/core_probability_ref) [Random Variable Reference](https://chrispiech.github.io/probabilityForComputerScientists/en/intro/all_distributions) [Python Reference](https://chrispiech.github.io/probabilityForComputerScientists/en/intro/python) [Calculators](https://chrispiech.github.io/probabilityForComputerScientists/en/intro/calculators) - Part 1: Core Probability - [Counting](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/counting) [Combinatorics](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/combinatorics) [Definition of Probability](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/probability) [Equally Likely Outcomes](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/equally_likely) [Probability of or](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/prob_or) [Conditional Probability](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/cond_prob) [Independence](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/independence) [Probability of and](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/prob_and) [Law of Total Probability](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/law_total) [Bayes' Theorem](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/bayes_theorem) [Log Probabilities](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/log_probabilities) [Many Coin Flips](https://chrispiech.github.io/probabilityForComputerScientists/en/part1/many_flips) - [Applications](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta/#part1_examples) - [Enigma Machine](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/enigma) - [Serendipity](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/serendipity) - [Random Shuffles](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/random_shuffles) - [Random Graphs](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/counting_graphs) - [Set Diversity](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/diversity_shapes) - [Bacteria Evolution](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/bacteria_evolution) - Part 2: Random Variables - [Random Variables](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/rvs) [Probability Mass Functions](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/pmf) [Expectation](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/expectation) [Variance](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/variance) [Bernoulli Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/bernoulli) [Binomial Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/binomial) [Poisson Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/poisson) [More Discrete Distributions](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/more_discrete) [Categorical Distributions](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/categorical) [Continuous Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/continuous) [Uniform Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/uniform) [Exponential Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/exponential) [Normal Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/normal) [Binomial Approximation](https://chrispiech.github.io/probabilityForComputerScientists/en/part2/binomial_approx) - [Applications](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta/#part2_examples) - [100 Binomial Problems](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/100_binomial_problems) - [Winning Series](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/winning_series) - [Approximate Counting](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/approximate_counting) - [Jury Selection](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/jury) - [Grading Eye Inflamation](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/grading_eye_inflammation) - [Grades are Not Normal](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/grades_not_normal) - [Curse of Dimensionality](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/curse_of_dimensionality) - [Algorithmic Art](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/algorithmic_art) - Part 3: Probabilistic Models - [Joint Probability](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/joint) [Marginalization](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/marginalization) [Multinomial](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/multinomial) [Continuous Joint](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/continuous_joint) [Inference](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/inference) [Bayesian Networks](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/bayesian_networks) [Independence in Variables](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/independent_vars) [Correlation](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/correlation) [General Inference](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/computational_inference) - [Applications](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta/#part3_examples) - [Fairness in AI](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/fairness) - [Federalist Paper Authorship](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/federalist) - [Name to Age](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/name2age) - [Probability of Baby Delivery](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/prob_baby_delivery) - [Bayesian Carbon Dating](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/bayesian_carbon_dating) - [Digital Vision Test](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/digital_vision_test) - [Bridge Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/bridge_distribution) - [Expectation of Sum Proof](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/expectation_of_sums) - [Bayesian Viral Load Test](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/bayesian_viral_load_test) - [CS109 Logo](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/dart_logo) - [Tracking in 2D](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/tracking_in_2D) - Part 4: Uncertainty Theory - [Beta Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta) [Adding Random Variables](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/summation_vars) [Central Limit Theorem](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/clt) [Sampling](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/samples) [Bootstrapping](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/bootstrapping) [Algorithmic Analysis](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/algorithmic_analysis) [Information Theory](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/information_theory) [Distance Between Distributions](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/divergence) - [Applications](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta/#part4_examples) - [Thompson Sampling](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/thompson) - [Night Sight](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/night_sight) - [P-Hacking](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/p_hacking) - [Differential Privacy](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/differential_privacy) - Part 5: Machine Learning - [Parameter Estimation](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/parameter_estimation) [Maximum Likelihood Estimation](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/mle) [Maximum A Posteriori](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/map) [Machine Learning](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/machine_learning) [Naïve Bayes](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/naive_bayes) [Logistic Regression](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/log_regression) [Diffusion](https://chrispiech.github.io/probabilityForComputerScientists/en/part5/diffusion) - [Applications](https://chrispiech.github.io/probabilityForComputerScientists/en/part4/beta/#part5_examples) - [MLE Normal Demo](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/mle_demo) - [MLE Pareto Distribution](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/mle_pareto) - [MLE Mixture Model](https://chrispiech.github.io/probabilityForComputerScientists/en/examples/mixture_models) - Ⓒ Chris Piech, Stanford University # Beta Distribution * The Beta distribution is the distribution most often used as the distribution of probabilities. In this section we are going to have a very meta discussion about how we represent probabilities. Until now probabilities have just been numbers in the range 0 to 1. However, if we have uncertainty about our probability, it would make sense to represent our probabilities as random variables (and thus articulate the relative likelihood of our belief). Beta Random Variable \| \| \| \|---\|---\| \| Notation: \| X ∼ Beta ( a , b ) \| \| Description: \| A belief distribution over the value of a probability p from a Binomial distribution after observing a − 1 successes and b − 1 fails. \| \| Parameters: \| a \> 0, the number successes + 1 b \> 0, the number of fails + 1 \| \| Support: \| x ∈ \[ 0 , 1 \] \| \| PDF equation: \| f ( x ) \= B ( a , b ) ⋅ x a − 1 ⋅ ( 1 − x ) b − 1 where B ( a , b ) \= Γ ( a ) Γ ( b ) Γ ( a , b ) \| \| CDF equation: \| No closed form \| \| Expectation: \| E \[ X \] \= a a \+ b \| \| Variance: \| Var ( X ) \= a b ( a \+ b ) 2 ( a \+ b \+ 1 ) \| \| PDF graph: \| \| Parameter a : Parameter b : ## What is your Belief in p After 9 Heads in 10 Flips? Imagine we have a coin and we would like to know its true probability of coming up heads, p. We flip the coin 10 times and observe 9 heads and 1 tail. What is your belief in p based off this evidence? Using the definition of probability we could guess that p ≈ 9 10. That number is a very rough estimate, especially since it is only based off 10 coin flips. Moreover the "point-value" 9 10 does not have the ability to articulate how uncertain it is. Could we instead have a random variable for the true probability? Formally, let X represent the true probability of the coin coming up heads. We don't use the symbol P for random variables, so X will have to do. If X \= 0\.7 then the probability of heads is 0.7. X must be a continuous random variable with support \[ 0 , 1 \] since probabilities are continuous values which must be between 0 and 1. Before flipping the coin, we could say that our belief about the coin's heads probability is uniform: X ∼ Uni ( 0 , 1 ). Let H be a random variable for the number of heads and let T be a random variable for the number of tails observed. What is P ( X \= x \\| H \= 9 , T \= 1 )? That probability is hard to think about! However it is much easier to reason about the probability with the condition reveresed: P ( H \= 9 , T \= 1 \\| X \= x ). This term asks the question: what is the probability of seeing 9 heads and 1 tail in 10 coin flips, given that the true probability of a heads is x. Convince yourself that this probability is just a binomial probability mass function with n \= 10 experiements, and p \= x evaluated at k \= 9 heads: P ( H \= 9 , T \= 1 \\| X \= x ) \= ( 10 9 ) x 9 ( 1 − x ) 1 We are presented with a perfect context for [Bayes' theorem with random variables](https://chrispiech.github.io/probabilityForComputerScientists/en/part3/inference/). We know a conditional probability in one direction and we would like to know it in the other: f ( X \= x \\| H \= 9 , T \= 1 ) \= P ( H \= 9 , T \= 1 \\| X \= x ) ⋅ f ( X \= x ) P ( H \= 9 , T \= 1 ) Bayes Theorem \= ( 10 9 ) x 9 ( 1 − x ) 1 ⋅ f ( X \= x ) P ( H \= 9 , T \= 1 ) Binomial PMF \= ( 10 9 ) x 9 ( 1 − x ) 1 ⋅ 1 P ( H \= 9 , T \= 1 ) Uniform PDF \= ( 10 9 ) P ( H \= 9 , T \= 1 ) x 9 ( 1 − x ) 1 Constants to front \= K ⋅ x 9 ( 1 − x ) 1 Rename constant Lets take a look at that function. For now we can let K \= 110. Regardless of K we will get the same shape, just scaled: What a beautiful image. It tells us relatively likelihood over the probability that is governing our coinflips. Here are a few observations from this chart: 1. Even after only 10 coin flips we are very confident that the true probability is \> 0.5 2. It is almost 10 times more likely that X \= 0\.9 as it is that X \= 0\.6 . 3. f ( X \= 1 ) \= 0 , which makes sense. How could we have flipped that one tail if the probability of heads was 1? Wait but why?** In the derivation above for f ( X \= x \\| H \= 9 , T \= 1 ) we made the claim that P ( H \= 9 , T \= 1 ) is a constant. A lot of folks find that hard to believe. Why is that the case? It may be helpful to juxtapose P ( H \= 9 , T \= 1 ) with P ( H \= 9 , T \= 1 \\| X \= x ). The later says "what is the probability of 9 heads, given the true probability is x". The former says "what is the probability of 9 heads, under all possible assignments of x". If you wanted to calculate P ( H \= 9 , T \= 1 ) you could use the law of total probability: P ( H \= 9 , T \= 1 ) \= ∫ y \= 0 1 P ( H \= 9 , T \= 1 \\| X \= y ) f ( X \= y ) That is a hard number to calculate, but it is in fact a constant with respect to x. ## Beta Derivation Let's generalize the derivation from the previous section, using h for the number of observed heads and t the number of observed tails. If we let H \= h be the event that we saw h heads, and let T \= t be the event that we saw t tails in h \+ t coinflips. We want to calculate the probability density function f ( X \= x \\| H \= h , T \= t ). We can use the exact same series of steps, starting with Bayes Theorem: f ( X \= x \\| H \= h , T \= t ) \= P ( H \= h , T \= t \\| X \= x ) f ( X \= x ) P ( H \= h , T \= t ) Bayes Theorem \= ( h \+ t h ) x h ( 1 − x ) t P ( H \= h , T \= t ) Binomial PMF, Uniform PDF \= ( h \+ t h ) P ( H \= h , T \= t ) x h ( 1 − x ) t Moving terms around \= 1 c ⋅ x h ( 1 − x ) t where c \= ∫ 0 1 x h ( 1 − x ) t d x The equation that we arrived at when using a Bayesian approach to estimating our probability defines a probability density function and thus a random variable. The random variable is called a Beta distribution, and it is defined as follows: The Probability Density Function (PDF) for X ∼ Beta ( a , b ) is: f ( X \= x ) \= { 1 B ( a , b ) x a − 1 ( 1 − x ) b − 1 if 0 \< x \< 1 0 otherwise where B ( a , b ) \= ∫ 0 1 x a − 1 ( 1 − x ) b − 1 d x A Beta distribution has E \[ X \] \= a a \+ b and V a r ( X ) \= a b ( a \+ b ) 2 ( a \+ b \+ 1 ). All modern programming languages have a package for calculating Beta CDFs. You will not be expected to compute the CDF by hand in CS109. To model our estimate of the probability of a coin coming up heads: set a \= h \+ 1 and b \= t \+ 1. Beta is used as a random variable to represent a belief distribution of probabilities in contexts beyond estimating coin flips. For example perhaps a drug has been given to 6 patients, 4 of whom have been cured. We could express our belief in the probability that the drug can cure patients as X ∼ Beta ( a \= 5 , b \= 3 ): ![](https://chrispiech.github.io/probabilityForComputerScientists/img/chapters/beta.png) Notice how the most likely belief for the probability of curing a patient, is 4 / 6, the fraction of patients cured. This distribution shows that we hold a non-zero belief that the probability could be something other than 4 / 6. It is unlikely that the probability is 0.01 or 0.09, but reasonably likely that it could be 0.5. ## Beta as a Prior You can set X ∼ Beta ( a , b ) as a prior to reflect how biased you think the coin is apriori to flipping it. This is a subjective judgment that represent a \+ b − 2 "imaginary" trials with a − 1 heads and b − 1 tails. If you then observe h \+ t real trials with h heads you can update your belief. Your new belief would be, X ∼ Beta ( a \+ h , b \+ t ). Using the prior Beta ( 1 , 1 ) \= Uni ( 0 , 1 ) is the same as saying we haven't seen any "imaginary" trials, so apriori we know nothing about the coin. Here is the proof for the distribution of X when the prior was a Beta too: If our prior belief is X ∼ Beta ( a , b ) , then our posterior is Beta ( a \+ h , b \+ t ) : f ( X \= x \\| H \= h , T \= t ) \= P ( H \= h , T \= t \\| X \= x ) f ( X \= x ) P ( H \= h , T \= t ) Bayes Theorem \= ( h \+ t h ) x h ( 1 − x ) t ⋅ 1 c ⋅ x a − 1 ( 1 − x ) b − 1 P ( H \= h , T \= t ) Beta PMF, Uniform PDF \= K ⋅ x h ( 1 − x ) t ⋅ x a − 1 ( 1 − x ) b − 1 Combine Constants \= K ⋅ x a \+ h − 1 ( 1 − x ) b \+ t − 1 Combine Like Bases Which is the PDF of Beta ( a \+ h , b \+ t ) It is pretty convenient that if we have a Beta prior belief, then our posterior belief is also Beta. This makes Betas especially convenient to work with, in code and in proof, if there are many updates that you will make to your belief over time. This property where the type of distribution is the same before and after an observation is called a conjugate prior. Quick question: Are you allowed to just make up priors and imaginary trials? Some folks think that is fine (they are called Bayesians) and some folks think that you shouldn't make up prior beliefs (they are called frequentists). In general, for small data it can make you much better at making predictions if you are able to come up with a good prior belief. Observation: There is a deep connection between the beta-prior and the uniform-prior (which we used initially). It turns out that Beta ( 1 , 1 ) \= Uni ( 0 , 1 ). Recall that Beta ( 1 , 1 ) means 0 imaginary heads and 0 imaginary tails.
Readable Markdown	null
Shard	143 (laksa)
Root Hash	2566890010099092343
Unparsed URL	io,github!chrispiech,/probabilityForComputerScientists/en/part4/beta/ s443