What is a Sigma-Algebra? I find the name awfully off-putting. I prefer to think of it as simply a smaller version of the entire sample space that only includes events that can be measured or that have any sort of meaningful size.

To be able to have a sigma-algebra you need to have a set. You then proceed to construct a subset by first including the empty set and the entire set; so the subset now has 2 elements.

Formally speaking, a sigma-algebra is a collection of subsets of some larger set that satisfies a bunch of properties – namely, that they can be measured (and in our case, measured using the probability measure).

DEFINITION

A collection of subsets of the sample space \Omega is called a sigma-algebra on \Omega if

  • both the entire sample space and the empty set are also in the collection,
  • if a subset E is in the collection then so is its complement \overline{E} ,
  • for all countable collections of subsets E_i , where i \in \mathbb{N}, the union of any or all of them is also in the collection: \bigcup_{i=1}^{\infty}E_i \in \mathcal{F}

Sigma-algebras form one of the three fundamental parts of what is called a probability space. The other two parts are the entire sample space, \Omega , and the probability measure \mathbb{P} . Essentially we say that a probability space is a triple (\Omega,\mathcal{F},\mathbb{P}) .

Why are probability spaces set up like this? Why can’t we just have our sample space and perform experiments, cataloging events as we go, assigning probabilities as we see fit? Well, you can. You can just have the sample space and a probability measure \mathbb{P} , define it on all subsets of the sample space, and you have yourself a genuine finite probability space: (\Omega,\mathbb{P}) . This is a very natural way for anyone to begin their study of probability spaces.

So a \sigma-algebra is just a definition of which sets may be considered as events. Elements not in sigma-algebra simply have no defined probability measure. Basically  sigma-algebras are the “patch” that lets us avoid some pathological behaviors of mathematics, namely non-measurable sets.

The three requirements of a sigma-algebra can be considered as consequences of what we would like to do with probability:

  1. Closure under countable unions.
  2. Closure under countable intersections.
  3. Closure under complements.

The countable unions and countable intersections components are direct consequences of the non-measurable set issue. Closure under complements is a consequence of the Kolmogorov axioms: if \mathbb{P}(A) = 2/3 then the complement of that event \mathbb{P}(\overline{A}) had better be 1/3. So without the closure under complements property you could have undefined complement events (which should always be allowed to happen), or complement events could have negative or greater than 100% probabilities. Also, closure under complements and the Kolmogorov axioms allow us to say things like:

\mathbb{P}(A\cup\overline{A}) = \mathbb{P}(A) + 1 - \mathbb{P}(A) = 1