Théo Charron

Bernoulli distribution

The Bernoulli distribution describes the outcome of a single trial with two outcomes: success (coded 1) or failure (coded 0). It is the simplest indicator variable and the building block of binomial and counting models.

Characteristics

CharacteristicFormulaDescription
Notation

Sometimes written or

Parameter

Success probability; often

SupportTwo mutually exclusive integer values
Probability mass functionPMF on the binary support
Cumulative distribution function

if ,
if ,


if

Jump of height at

Moment-generating functionDerivatives at 0 yield moments
Characteristic functionFourier transform of the law
MeanAverage success probability on one trial
Variance

Maximized at ()

Median

if , if

Not unique if

Mode

if , if

Bimodal if

Skewness

Positive if , negative if

Excess kurtosis

Lighter tails than the normal at

Shannon entropy

Convention ; maximized at

Fisher information

For one observation; increases as moves away from

Kullback-Leibler divergence

Between two Bernoulli laws with parameters and

Properties and theorems

Property / theoremFormulaDescription
Indicator variable

Every Bernoulli is the indicator of an event with probability

ComplementSuccess and failure are symmetric under inversion
i.i.d. sum

independent trials → binomial law

Integer moments

for

Since on

Maximum variance

Reached at ; bounds single-trial uncertainty

Zero product

and are never simultaneously nonzero

MGF of sums

For i.i.d. variables; identifies the binomial

Degenerate cases

,

Point masses at the parameter boundaries
Bernoulli process

i.i.d.

Sequence of independent trials; partial sums are binomial

Applications

Default modeling

A loan defaulting (yes/no) over a horizon is often modeled as a Bernoulli with parameter PD (probability of default).

Binary classification

Classifier output (positive/negative), marketing conversion, binary signal: the target or prediction error is naturally Bernoulli.

Independent trials

Coin flips, noisy bit transmission, winning/losing trades under i.i.d. assumptions: each repetition is a Bernoulli trial.

Building block for counting laws

Sums of i.i.d. Bernoulli variables yield the binomial; in the rare regime (large , small ), the Poisson law appears as a limit.

Statistical inference

ConceptFormulaDescription
Sample

observed trials; inference targets proportion

MLE estimatorEmpirical success rate; unbiased, consistent, efficient
Estimator variance

Decays as ; largest at

Wald interval

Normal approximation; acceptable when and are large enough

Clopper-Pearson interval

Exact proportion interval with

Proportion Z-test

Under and large

Conjugate prior (Bayesian)The beta law is conjugate to Bernoulli

Important remarks

  • A single trial () cannot estimate with a margin of error; inference practically concerns an i.i.d. Bernoulli sample.

  • The Wald interval can yield bounds outside for near 0 or 1 and moderate .

  • For small samples, prefer Clopper-Pearson or the Wilson interval.

  • The rule of thumb and guides use of the normal approximation.

Computational methods

MethodDescriptionApplications
Pseudo-random generation

, return 1 if

Default simulation, Monte Carlo, bootstrap
NumPy

np.random.binomial(1, p) or np.random.default_rng().binomial(1, p)

Vectorized draws on a portfolio
SciPy

scipy.stats.bernoulli(p) — PMF, CDF, rvs, moments

Exact calculations and simulation
Estimation and CIs

proportion_confint (statsmodels) — Wilson, Clopper-Pearson

Robust confidence intervals for proportions
Chargement…

References

• Wikipedia, Bernoulli distribution: https://en.wikipedia.org/wiki/Bernoulli_distribution

• Special case of the exponential family; see also Fundamentals and Discrete distributions.