Bayesian Networks
Causality and Metaphysics
The fundamental issue of causal inference is that causal effects cannot be measured directly.
Causal Diagram:
- graphical representation of data generating process (DGP):
- node: variables in the DGP
- arrows: show direction of causation
- large number of test subjects can “skew” proportions
Components
Recall Bayes Theorem:
Recall The Law of Total Probability:
: posterior probability : prior probability : evidence
In general, it is not effective to use The Law of Total Probability in this scenario.
Probability Model: a joint distribution over a set of random variables.
Marginal Distributions are sub-tables which eliminate variables. Marginalization (summing out) is combing collapsed rows by adding.
Marginalization
Given
Recall Independence:
Implications of Independence in Bayes Theorem
In other words, given independence,
Assuming conditional independence, we can say things like:
Causal vs. Bayesian Inference
Bayesian Statistical Framework:
- process of interest
- data collection (evidence)
- build model
- update model
Example
Causal Diagram:
Bayesian Network:
- with a Bayes Net, the arrow does not necessarily indicate causality
- instead, it is indicating the child node has a conditionally dependent relationship with the parent node and is conditionally independent of non-parent nodes (naive assumption).
The Bayesian Network
The point of a Bayes Net is to represent full joint probability distributions, and to encode an interrelated set of conditional independence/probability statements.
- nodes (events)
- conditional probability tables (CPTs), relating those events
- describe how variables interact locally
- chain together local interactions to estimate global, indirect interactions
To put another way, Bayes Nets implicitly encode joint distributions as a product of the local conditional probabilities.
Keeping in mind that each node is conditionally independent of its other predecessors, given its parents, we can order in such a way that:
Using Causal Diagrams to Construct Bayes Nets
Although a Bayes Net is not necessarily a Causal Diagram, we should create the network in such a way that it flows from cause to effect
- Nodes: What is the set of variables we need to model?
- order them:
- best if ordered such that causes precede effects
- order them:
- Links: For each node
, do:- choose a minimal set of parents
, such that - for each parent, insert arcs (links) from parent to
- write down conditional probability table (CPT)
- choose a minimal set of parents
Bayesian Network Example
We can use this network to answer questions such as:
Which if we were to write this out in it’s entirety using formulas/algebra, it would be a cumbersome process. However, using a Bayesian Network to answer this question, we can follow the flow of the diagram to simplify things.
Causal vs. Diagnostic Modeling
Diagnostic: observing an effect leads to competition between possible causes.
: Rock in Shoe : Deformed Foot : Foot Hurts
We need to diagnose which is most likely: