Quiz 1
BST 570-1: Probabilistic Machine Learning, Fall 2026
Name: ____________________________
Time: 60 minutes. Total: 10 points, ten questions, one point each. Closed book; no notes, no devices. A calculator is allowed. Questions 1 to 3 and 9 are true/false; the rest are multiple choice with exactly one correct answer. Circle your answer at the question and copy it into the answer box below. Scratch work is not graded, but several questions need it. The material is weeks 1 to 5 and Assignment 1.
Notation follows the lectures: lowercase \(p\) for mass functions and densities, capital \(P\) for probabilities of events, \(X \perp Y \mid Z\) for conditional independence.
Answers. Write your answers here as well; this box is what is graded. True/false questions take T or F, multiple choice a letter.
| Q1. | __________ | Q6. | __________ |
| Q2. | __________ | Q7. | __________ |
| Q3. | __________ | Q8. | __________ |
| Q4. | __________ | Q9. | __________ |
| Q5. | __________ | Q10. | __________ |
Part A: Reading independence off a graph
Questions 1 to 5 refer to the graph below. An observational study asks whether prescribing a statin (\(A\)) lowers the five-year risk of a cardiovascular event (\(Y\)). The directed graph is the analysts’ model of how the data arose.
- \(Z\): the prescribing physician’s general preference for statins. It has a direct effect on treatment and no direct effect on any other variable.
- \(L\): the patient’s baseline risk (age, blood pressure, family history). Sicker patients are more likely to be prescribed the drug and more likely to have an event.
- \(A\): whether the patient was prescribed a statin.
- \(M\): the patient’s LDL cholesterol one year later. The statin affects the outcome only through its effect on LDL cholesterol.
- \(Y\): whether the patient had a cardiovascular event within five years.
- \(S\): whether the patient was enrolled in a follow-up registry. Treated patients and patients who had an event are both more likely to be enrolled.
- \(R\): whether the patient returned the registry’s follow-up survey, which is sent only to registry members.
For Questions 1 to 3, mark True exactly when the displayed independence is implied by d-separation in the graph.
Question 1 (1 point)
\(Z \perp L\): the prescriber’s preference is independent of the patient’s baseline risk.
True / False
Question 2 (1 point)
\(Z \perp Y \mid A\): once we know whether the patient was treated, the prescriber’s preference tells us nothing further about the outcome.
True / False
Question 3 (1 point)
\(A \perp Y \mid L, M, R\): given baseline risk, LDL cholesterol, and whether the survey was returned, treatment and outcome are independent.
True / False
Questions 4 and 5 use numbers. Suppose every variable is binary, with \(1\) meaning yes: \(L = 1\) is high baseline risk, \(A = 1\) is treated, \(M = 1\) is LDL brought under control, and \(Y = 1\) is an event. After summing out \(Z\), the joint distribution factorizes as \(p(l)p(a \mid l)p(m \mid a)p(y \mid l,m)\). Here \(P(L = 1) = 0.3\); the remaining probabilities are given in three separate tables below. Variables \(S\) and \(R\) are not needed.
Treatment model
| \(L\) | \(P(A = 1 \mid L)\) |
|---|---|
| 0 | 0.2 |
| 1 | 0.6 |
LDL model
| \(A\) | \(P(M = 1 \mid A)\) |
|---|---|
| 0 | 0.2 |
| 1 | 0.8 |
Outcome model
| \(L\) | \(M\) | \(P(Y = 1 \mid L,M)\) |
|---|---|---|
| 0 | 0 | 0.10 |
| 0 | 1 | 0.05 |
| 1 | 0 | 0.40 |
| 1 | 1 | 0.20 |
Question 4 (1 point)
The five-year risk among treated patients, \(P(Y = 1 \mid A = 1)\), is
\(0.060\)
\(0.114\)
\(0.145\)
\(0.161\)
Question 5 (1 point)
Adjusting for \(L\) blocks the backdoor path \(A \leftarrow L \to Y\): compare treated and untreated patients within each level of \(L\), then average over the population distribution of \(L\). The adjusted risk under treatment \(a\) is \[ P_{\text{adj}}(a) = \sum_{l} P(L = l)\, P(Y = 1 \mid A = a, L = l), \qquad a \in \{0, 1\}. \] The adjusted risk difference \(P_{\text{adj}}(1) - P_{\text{adj}}(0)\) is
\(+0.024\)
\(-0.057\)
\(-0.075\)
\(-0.120\)
Part B: EM
Question 6 (1 point)
Counts \(y_1, \ldots, y_N\) come from a two-component mixture: \(z_i \sim \text{Categorical}(\pi_1, \pi_2)\) and \(y_i \mid z_i = k \sim \text{Poisson}(\lambda_k)\), independently over \(i\). With responsibilities \(r_{ik} = P(z_i = k \mid y_i, \theta^{(t)})\) and \(N_k = \sum_i r_{ik}\), the M-step maximizes \[ Q(\theta \mid \theta^{(t)}) = \sum_{i=1}^N \sum_{k=1}^2 r_{ik} \big[\log \pi_k + y_i \log \lambda_k - \lambda_k - \log y_i!\big]. \] Four counts and their current responsibilities for component 1 are shown below. Treat the displayed responsibility values as exact.
| \(i\) | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| \(y_i\) | 0 | 2 | 6 | 8 |
| \(r_{i1}\) | 1.00 | 0.75 | 0.25 | 0.00 |
The M-step gives \((\pi_1, \lambda_1, \lambda_2) =\)
\((0.50,\; 1.5,\; 6.5)\)
\((0.50,\; 2.0,\; 4.0)\)
\((0.25,\; 1.5,\; 6.5)\)
\((0.50,\; 1.5,\; 4.0)\)
Question 7 (1 point)
The lecture showed that for any distributions \(q_i(z_i)\) and any \(\theta\), \[ \ell(\theta) = \mathcal{L}(q, \theta) + \sum_{i=1}^N \text{KL}\big(q_i \,\|\, p(z_i \mid y_i, \theta)\big), \] and that EM’s monotonicity follows from the chain \[ \ell(\theta^{(t+1)}) \;\ge\; \mathcal{L}(q^{(t+1)}, \theta^{(t+1)}) \;\ge\; \mathcal{L}(q^{(t+1)}, \theta^{(t)}) \;=\; \ell(\theta^{(t)}). \] Exactly one of the following statements is correct. Which?
If the E-step is replaced by hard assignments, with \(q_i\) a point mass at \(\arg\max_k r_{ik}\), the log likelihood \(\ell\) still never decreases, because the M-step still maximizes \(\mathcal{L}\) over \(\theta\).
Initializing a Gaussian mixture with \(r_{ik} = 1/K\) for every \(i\) and \(k\) and starting with the M-step reaches the same fit as a random start, only after more iterations.
Two runs from different random starts that converge to parameters differing only in the order of the components have exactly the same log likelihood.
The equality \(\mathcal{L}(q^{(t+1)}, \theta^{(t)}) = \ell(\theta^{(t)})\) holds because the M-step maximizes \(Q(\theta \mid \theta^{(t)})\) over \(\theta\).