For the longest time I wanted to cover measure based probability in my notes.
I recently took a couple of courses by Jen Corcoran on coursera and when I came across her online course on measure based probability, I decided to focus on it.
118.1 Lesson 1: Measurable Sets
This lesson is a rather informal introduction to the measurable sets. We will define measures in lesson 2, but we get to build up to it by trying to understand what it means to measure a set.
This video introduces measure-theoretic probability, an advanced course that unifies discrete and continuous probability. The instructor, Jem, explains that measure theory is a language that allows for a more comprehensive discussion of probability concepts.
The core of measure theory revolves around sets. Jem poses a thought experiment, asking the viewer to “measure” an irregular 2D set, highlighting that “measure” needs a clear definition, which can include area, length, volume, or other interpretations.
The video then delves into key definitions:
\Omega : A non-empty set representing the sample space of an experiment, encompassing all possible outcomes.
Event (A) : A subset of \Omega.
Probability (P) : A function that assigns numbers between zero and one to subsets of \Omega.
Jem elaborates on fields (or algebras), which are collections of subsets of \Omega with three crucial properties:
- Contains Omega: The collection must include the entire sample space.
- Closed under complements: If a set is in the collection, its complement must also be.
- Closed under finite unions: The union of any finite number of sets in the collection must also be in the collection.
This property implies closure under finite intersections due to De Morgan’s laws.
The video distinguishes \sigma-fields (or \sigma-algebras) from fields
A \sigma-field is a field that is also closed under countable unions. This means that a \sigma-field is automatically a field, but a field is not necessarily a \sigma-field.
Finally, Jem provides examples of fields and \sigma-fields:
- Trivial \sigma-field
- Consists only of \Omega and the empty set.
- Power set of \Omega
- The collection of all possible subsets of \Omega, which forms a \sigma-field.
- \sigma-field generated by A
- A small \sigma-field formed by a set A, its complement, the empty set, and \Omega.
118.1.1 Measure theory is focused on sets!
118.1.2 Lebesgue Measure
Let \Omega be a non-empty set.
Think of \Omega = \{1, 2, 3, 4, 5\} corresponding to a sample space of an experiment e.g. outcomes of tossing a coin.
Definition 118.1 (Sample Space) Is the set of all possible outcomes of an experiment
Then we can define a measure \mu on \Omega by assigning a non-negative number to each outcome.
For example, we could define \mu as follows:
The set \Omega is any set, but we can think about it as an abstraction of our coin toss sample space.
Jen point out that it is an abstract set and I take it to mean that we might not be able to come up with an experiment that has the its sample space, i.e. outcomes as the subsets in \Omega but we can still define a measure on it.
Think of \mathbb{P} as a function that assigns a number between 0 and 1 to each subset of \Omega.
\mathbb{P}(2^{\Omega}) \to [0,1 ]
For more general \Omega, assigning values of P for all subsets of \Omega can be difficult. We need to restrict the subsets of \Omega that we can assign probabilities to.
PErhaps we can assign probabilities to some subsets of \Omega but not all.
118.1.3 Alternative formulations of a \sigma-field
In this slide we consider a two variant of the definition of a field that are equivalent to the original definition.
In the first condition we can replace \Omega with \emptyset as we can always get one from the other using the complements operation in the second property.
Likewise in the third condition we can replace finite unions with finite intersections as we can get one from the other using the complements operation and de Morgan’s laws.
In this slide Jem demonstrates how we can use de Morgan’s law to show that if a collection of sets is closed under complements and finite unions, then it is also closed under finite intersections.
A, B \in \mathcal{F}
In this slide Jem extends the idea of a field to a \sigma-algebra.
The difference being that a \sigma-algebra is closed under finite unions of its members \sigma-algebra is a field that is also closed under countable unions.
We will often want to extend the a result from finite unions to countable unions, and this is where \sigma-algebras come in handy.
118.2 Three Examples of \sigma-fields
- \mathcal{F}=\{\Omega, \emptyset\} is a \sigma-field. It is called the trivial \sigma-field.
- \mathcal{F}=\mathcal{P}(\Omega), the power set of \Omega, is a \sigma-field. It is called the discrete \sigma-field.
- \mathcal{F}=\{\emptyset, A, A^c, \Omega\} is a \sigma-field generated by a set A.
we now show that the field of sets that we have defined closed under countable unions of 2 sets.
118.2.1 Counter example
A field that is not a \sigma-field
Jem also demonstrates an example of a field that is not a \sigma-field using intervals on the real number line, illustrating that while it might be closed under finite unions, it may not be closed under countable (infinite) unions.
We can take the countable union of the sets in \mathcal{F}, which is the set of all rational numbers in the interval [0,1]. This set is not in \mathcal{F}, so \mathcal{F} is not closed under countable unions and therefore not a \sigma-field.
Let’s show that \mathcal{F} is not a \sigma-field.
First we define
A_n = \left\{\frac{1}{n}\right\} \forall n \in \mathbb{N}
Then their union is
\bigcup_{n=1}^{\infty} A_n = \left\{\frac{1}{n} : n \in \mathbb{N}\right\}
but this set is not in \mathcal{F}, so \mathcal{F} is not closed under countable unions and therefore not a \sigma-field.
In the next video will discuss the consider the idea of generating \sigma-fields from a set and a special type of \sigma-field called the Borel \sigma-field.
This \sigma-field, is generated by the open sets in a topological space. The Borel \sigma-field is important in measure theory and probability because it allows us to define measures on a wide variety of sets, including those that are not necessarily open or closed.