Beta Gamma Distributions

[!NOTE] Context: Gaussian works for unbounded data (-∞ to ∞). But probabilities are bounded [0, 1], and waiting times are positive [0, ∞). This is where Beta and Gamma shine.

1. The Beta Distribution: Probability of Probabilities

The Beta distribution is defined on the interval [0, 1]. This makes it the perfect model for probabilities themselves.

Think of it as modeling your belief about a coin’s bias p.

• Prior: Before flipping, you think the coin is fair (Beta(1, 1) = Uniform).
• Data: You see 10 Heads and 0 Tails.
• Posterior: Your belief shifts heavily towards p=1.

Conjugate Priors (The Magic)

Beta is the Conjugate Prior for the Binomial distribution.

• Rule: Posterior = Prior + Data
• If Prior ~ Beta(α, β)
• And Data ~ Binomial(Successes=S, Failures=F)
• Then Posterior ~ Beta(α + S, β + F)

Intuition:

• α (Alpha): “Virtual Successes” + Observed Successes.
• β (Beta): “Virtual Failures” + Observed Failures.

2. Real World Case Study: A/B Testing

Imagine you are running an A/B test for a new “Buy Now” button color.

• Variant A (Blue): 100 views, 5 clicks.
• Variant B (Red): 100 views, 8 clicks.

Is Red better? Or is it just luck?

We can model the Conversion Rate (CTR) of each variant as a Beta Distribution.

1. Prior: Assume we know nothing. Beta(1, 1).
2. Posterior A: Beta(1+5, 1+95) = Beta(6, 96).
3. Posterior B: Beta(1+8, 1+92) = Beta(9, 93).

We then calculate P(CTR_B > CTR_A) by comparing the two distributions.

---

3. Interactive: Bayesian A/B Tester

Simulate an A/B test. Click “Success” (Conversion) or “Failure” (No Conversion) to update the Beta distributions for Variant A and Variant B. Watch how they narrow (become more confident) as you collect data.

Variant A (Blue)

α: 1, β: 1

Variant B (Purple)

α: 1, β: 1

4. The Gamma Distribution: Waiting for Events

The Gamma distribution is defined on [0, ∞).

• Exponential: Waiting time for the first event.
• Gamma: Waiting time for the k-th event.

Parameters:

• k (or α): Number of events.
• θ (or scale): Average time between events.

---

5. Code Implementation

Java Example (Bayesian Update)

A simple class to track the state of a coin/conversion rate.

import org.apache.commons.math3.distribution.BetaDistribution;

public class BayesianCounter {
    private double alpha; // Successes + 1
    private double beta;  // Failures + 1

    public BayesianCounter() {
        // Uniform Prior
        this.alpha = 1.0;
        this.beta = 1.0;
    }

    public void addSuccess() { this.alpha++; }
    public void addFailure() { this.beta++; }

    // Get the Mean (Expected CTR)
    public double getMean() {
        return alpha / (alpha + beta);
    }

    // Get 95% Credible Interval
    public double[] getCredibleInterval() {
        BetaDistribution dist = new BetaDistribution(alpha, beta);
        return new double[] {
            dist.inverseCumulativeProbability(0.025),
            dist.inverseCumulativeProbability(0.975)
        };
    }

    public static void main(String[] args) {
        BayesianCounter variantA = new BayesianCounter();

        // Simulate 5 clicks, 95 non-clicks
        for(int i=0; i<5; i++) variantA.addSuccess();
        for(int i=0; i<95; i++) variantA.addFailure();

        System.out.printf("Mean CTR: %.4f%n", variantA.getMean());
        double[] ci = variantA.getCredibleInterval();
        System.out.printf("95%% CI: [%.4f, %.4f]%n", ci[0], ci[1]);
    }
}

Go Example (Bayesian Update)

package main

import (
	"fmt"
	"gonum.org/v1/gonum/stat/distuv"
)

type BayesianCounter struct {
	Alpha float64
	Beta  float64
}

func NewCounter() *BayesianCounter {
	return &BayesianCounter{Alpha: 1.0, Beta: 1.0}
}

func (b *BayesianCounter) AddSuccess() { b.Alpha++ }
func (b *BayesianCounter) AddFailure() { b.Beta++ }

func (b *BayesianCounter) GetCredibleInterval() (float64, float64) {
	dist := distuv.Beta{Alpha: b.Alpha, Beta: b.Beta}
	return dist.Quantile(0.025), dist.Quantile(0.975)
}

func main() {
	variantA := NewCounter()

	// Simulate 5 clicks, 95 non-clicks
	for i := 0; i < 5; i++ { variantA.AddSuccess() }
	for i := 0; i < 95; i++ { variantA.AddFailure() }

	// Mean = Alpha / (Alpha + Beta)
	mean := variantA.Alpha / (variantA.Alpha + variantA.Beta)
	lower, upper := variantA.GetCredibleInterval()

	fmt.Printf("Mean CTR: %.4f\n", mean)
	fmt.Printf("95%% CI: [%.4f, %.4f]\n", lower, upper)
}

Summary

• Beta is the probability of a probability.
• Conjugate Prior: Posterior = Prior + Data.
• A/B Testing is naturally Bayesian: Start with a prior, update with user data.