Introduction to Probability

Algebras of sets

◼
  • Let X be a set, and consider the power set (X) of X. Then a subset Σ⊆(X) is called an algebra if it satisfies the following:
    • ◼
  • X∈Σ
    • ◼
  • Σ is closed under complements in X. That is if A∈Σ, then X\A∈Σ
    • ◼
  • Σ is closed under finite unions. That is, if A,B∈Σ, then A⋃B∈Σ
    • ◼
  • Examples:
    • ◼
  • For every set X, (X) is an algebra of sets.
    • ◼
  • Note that X∈(X)
    • ◼
  • Let A∈(X) then X/A is also a subset of X, thus X/A∈(X)
    • ◼
  • Take A,B∈(X)then A⋃B is a subset of X so A⋃B∈(X)
    • ◼
  • V={{},{a},{b,c},{a,b,c}}⊂({a,b,c}) is an algebra of sets of {a,b,c}. Notice that {b}, {c}, {a,b},{a,c} are not elements of the algebra mentioned here.
    • ◼
  • {a,b,c}∈V
    • ◼
  • If A∈V we need to check that X\A ∈V. We check this for individual sets.
    • ◼
  • If A,B∈V then we need to check that A⋃B ∈V. We check this by checking each pair
    • ◼
  • Notes:

    • σ-Algebras

    ◼
  • Let X be a set, and consider the power set (X) of X. Then a subset Σ⊆(X) is called a σ-algebra over X if it satisfies the following:
    • ◼
  • X∈Σ
    • ◼
  • Σ is closed under complements in X. That is if A∈Σ, then X\A∈Σ
    • ◼
  • Σ is closed under countable unions. That is, if a countable collection
    A
    0
    ,
    A
    1
    ,...
    of sets are in Σ, then so is A
    =
    ⋃
    n∈
    A
    i
    • ◼
  • Notes:
    • ◼
  • Notice that {} is an element of every σ-algebra since X is in Σ and X\X={} is in Σ.
    • ◼
  • Note that every σ-algebra is an algebra.

    • Measures and Probability measures.

    Measures

    ◼
  • Let X be a set and Σ a σ-algebra over X. A function μ:Σ is called a measure if it satisfies the following properties:
    • ◼
  • For all A∈Σ, μ(A)≥0. This is called non-negativity.
    • ◼
  • μ({}) = μ(∅) = 0
    • ◼
  • If
    {
    A
    i
    }
    i∈
    is a collection of pairwise disjoint sets in Σ, then
    μ(
    ⋃
    i∈
    A
    i
    )=
    ∑
    i∈
    μ(
    A
    i
    )
    • ◼
  • A common measure that we will be using in this section is the counting measure. Given any set X, the counting measure on any σ-algebra is the measure corresponding to the cardinality of the subsets in the σ-algebra
    • ◼
  • Example: Let X = {a,b,c} and Σ={{},{a},{b,c},{a,b,c}} then the counting measure assigns:
    • ◼
  • μ({}) = 0
    • ◼
  • μ({a}) = 1
    • ◼
  • μ({b,c}) = 2
    • ◼
  • μ({a,b,c})=3

    • Probability measures

    ◼
  • A probability measure is a measure p on X such that p(X) = 1.
    • ◼
  • Examples:
    • ◼
  • Consider a finite set X. Consider the counting measure μ on (X). Then a probability measure can be obtained by taking for each A∈(X)
    p(A):=
    μ(A)
    μ(X)
    =
    |A|
    |X|
    • ◼
  • Let X = {a,b,c} and Σ={{},{a},{b,c},{a,b,c}} then the probability counting measure assigns:
    • ◼
  • p({}) = 0
    • ◼
  • p({a}) =
    1
    3
    • ◼
  • p({b,c}) =
    2
    3
    • ◼
  • p({a,b,c})=
    3
    3
    =1

    • Notes on measures:

    ◼
  • In this subsection we consider a set X, a σ-algebra Σ over X, a measure μ on Σ, and A,B ∈Σ, and
    {
    A
    i
    }
    i∈
    ⊆Σ
    .
    • ◼
  • If A⊆B then μ(A)≤μ(B)
    • ◼
  • μ(
    ⋃
    i∈
    A
    i
    )<=
    ∑
    i∈
    μ(
    A
    i
    )
    with the
    A
    i
    not necessarily pairwise disjoint.

    • Probability space

    ◼
  • A probability space is a triple (Ω, ℱ, P) where:
    • ◼
  • Ω is a set called the sample space which contains all possible outcomes.
    • ◼
  • ℱ is a σ-algebra of Ω which is considered as a set of events
    • ◼
  • P is a probability measure on ℱ.
    • ◼
  • Examples:
    • ◼
  • Consider the experiment of rolling two dice in order. We can then have Ω
    =
    "(1,1)"
    "(1,2)"
    "(1,3)"
    "(1,4)"
    "(1,5)"
    "(1,6)"
    "(2,1)"
    "(2,2)"
    "(2,3)"
    "(2,4)"
    "(2,5)"
    "(2,6)"
    "(3,1)"
    "(3,2)"
    "(3,3)"
    "(3,4)"
    "(3,5)"
    "(3,6)"
    "(4,1)"
    "(4,2)"
    "(4,3)"
    "(4,4)"
    "(4,5)"
    "(4,6)"
    "(5,1)"
    "(5,2)"
    "(5,3)"
    "(5,4)"
    "(5,5)"
    "(5,6)"
    "(6,1)"
    "(6,2)"
    "(6,3)"
    "(6,4)"
    "(6,5)"
    "(6,6)"
    .
    • ◼
  • We can consider ℱ=(Ω). So we can have sets like “Rolling a 1 on the first die” = {(1,1),(1,2),(1,3),(1,4),(1,5),(1,6)} or “rolling a total of 7” = {(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)}” or “Rolling a total of 7 or a 1 on the first die” = {(1,1),(1,2),(1,3),(1,4),(1,5),(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)} as events.
    • ◼
  • We can consider the measure p to be the counting probability measure. So for example the probability of rolling a 1 on the first die would be
    |{(1,1),(1,2),(1,3),(1,4),(1,5),(1,6)|
    |Ω|
    =
    6
    36
    =
    1
    6
    .
    • ◼
  • Example 2
    • ◼
  • Ω
    2
    ={1,2,3,4,5,6,7,8,9,10,11,12}​
    ℱ
    2
    =(Ω)P({1})=0P({12}) =
    1
    36
    ​P({2}) =
    1
    36
    ​P({3}) =
    1
    18
    ​​
    P({7})=
    1
    6
    ​​
    P({4})=
    3
    36
    ​P({5})
    =
    1
    9
    ​P({6}) =
    5
    36
    ​P({8}) =
    5
    36
    ​P({9}) =
    1
    9
    ​P({10}) =
    1
    12
    ​P({11}) =
    1
    18

    • Conditional probability

    ◼
  • Many times we are interested in determining how different events interact with each other. More so in how the probabilities change or not given that a certain event has happened.
    • ◼
  • Let us consider the three events mentioned above; A = “Rolling a 1 on the first die” and B = “rolling a total of 7” and C=A⋃B = “Rolling a total of 7 or a 1 on the first die”. What is the probability Rolling a 1 on the first die given that we rolled a total of 7? To calculate this we consider the sample space as being reduced to the events inside of B. Thus we are looking at the probability of
    A⋂B
    over the probability of B. In symbols we would represent this as
    P(A|B)=
    P(A⋂B)
    P(B)
    and read it as “The probability of A given B”.
    • ◼
  • So A= {(1,1),(1,2),(1,3),(1,4),(1,5),(1,6)}, B = {(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)} and C = {(1,1),(1,2),(1,3),(1,4),(1,5),(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)}
    • ◼
  • The above only makes sense then when the probability of B is non-zero.
    • ◼
  • The actual calculation in this case is
    P(A|B)=
    P(A⋂B)
    P(B)
    =
    P({(1,6)})
    P({(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)})
    =
    1/36
    6/36
    =
    1
    6
    Let us calculate all other conditional probabilities.
    • ◼
  • P(B|A)=
    1
    6
    • ◼
  • P(A|C)=
    6
    11
    • ◼
  • P(C|A)=1
    • ◼
  • P(B|C)=
    6
    11
    • ◼
  • P(C|B)=1
    • ◼
  • Let us now calculate the probabilities of the individual events happening
    • ◼
  • P(A) =
    1
    6
    • ◼
  • P(B) =
    1
    6
    • ◼
  • P(C) =
    11
    36
    • ◼
  • There are a few interesting observations.
    • ◼
  • Both P(C|A) and P(C|B) are 1. This is because in general X⊆Y implies that P(Y|X) = 1.
    • ◼
  • P(B|A) = P(B) and P(A|B)=P(A). When this happens we say that A and B are independent.