Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
ISBN: 978-0-387-31073-2
日程
- 7/6 開始.
- (夏学期)原則毎週の金曜日、17：00−.
- (7月-9月)火曜日もこの輪読に充てる.
- (10月以降)火曜日17:00-
進め方
- 数式と図表を理解することができればOK. 本文は、斜体語句の意味と数式と図表を理解するために読む.
- ひとつの章を複数人で担当してよい.
- 興味ある部分だけの、スポット参加歓迎.
- 原則, 章の順番によらず, 担当者が好きな章を選んでよい.
- ただし, 章間につながりがある場合があるので選ぶ際にはじゅうぶん注意してください.
- 章間の依存関係に気がついた人は, このページ末尾に書き足してください.
参加予定者
yuusaku-t, kazuo-h, masahiko-h, manab-ki, hideharu-o, shimbo, matsu, yotaro-w, masayu-a, junta-m
分担
(早く終われば繰り上がる可能性があります.)
- 7/6, 10 kazuo-h (Chapter 1)
- 7/13, 16 manab-ki, masahiko-h (Chapter 3)
- 7/20, 24 shimbo (Chapter 9)
- 7/27, 31, 8/3, 21 kazuo-h (Chapter 2)
- 8/21, 24, 31 matsu (Chapter 4)
- 8/28, 9/4, 7 shimbo (Chapter 11)
- 9/11, 18 shimpei-m (Chapter 6)
- 10/9, 12, 16, 23 junta-m (Chapter 7)
- 10/23, 30, 11/6, 16 yotaro-w (Chapter 8 前半)
- 11/27, 12/4 inui (Chapter 8 後半)
- 12/11, 18, 25, 1/8 manab-ki 他 (Chapter 10)
- 12/11, 18, 25 manab-ki (10.1, 10.2)
- 12/25 ikumi-s (10.3), yotaro-w(10.4)
- 1/8 manab-ki (10.5), junta-m (10.6)
- 1/15 yotaro-w (10.7)
- 1/22, 29, 2/5 hideharu-o (Chapter 13)
- 2/12 shimbo (Chapter 12)
- 2/26, 3/4 kazuo-h (Chapter 12)
目次
| 章節 | 章題 | ページ | 担当（2〜4人OK） | 日程（案） |
| Contents | xiii | - | - | |
| 1 Introduction | 1 | kazuo-h | 7/6, 10 | |
| 1.1 | Example: Polynomial Curve Fitting | 4 | ||
| 1.2 | Probability Theory | 12 | ||
| 1.3 | Model Selection | 32 | ||
| 1.4 | The Curse of Dimensionality | 33 | ||
| 1.5 | Decision Theory | 38 | ||
| 1.6 | Information Theory | 48 | ||
| 2 Probability Distributions | 67 | kazuo-h, matsu | 7/27, 31, 8/3, 21 | |
| 2.1 | Binary Variables | 68 | ||
| 2.2 | Multinomial Variables | 74 | ||
| 2.3 | The Gaussian Distribution | 78 | ||
| 2.4 | The Exponential Family | 113 | ||
| 2.5 | Nonparametric Methods | 120 | ||
| 3 Linear Models for Regression | 137 | manab-ki, masahiko-h | 7/13, 16 | |
| 3.1 | Linear Basis Function Models | 138 | ||
| 3.2 | The Bias-Variance Decomposition | 147 | ||
| 3.3 | Bayesian Linear Regression | 152 | ||
| 3.4 | Bayesian Model Comparison | 161 | ||
| 3.5 | The Evidence Approximation | 165 | ||
| 3.6 | Limitations of Fixed Basis Functions | 172 | ||
| 4 Linear Models for Classication | 179 | matsu | 8/21, 24 | |
| 4.1 | Discriminant Functions | 181 | ||
| 4.2 | Probabilistic Generative Models | 196 | ||
| 4.3 | Probabilistic Discriminative Models | 203 | ||
| 4.4 | The Laplace Approximation | 213 | ||
| 4.5 | Bayesian Logistic Regression | 217 | ||
| 5 Neural Networks | 225 | kazuo-h | 後回し | |
| 5.1 | Feed-forward Network Functions | 227 | ||
| 5.2 | Network Training | 232 | ||
| 5.3 | Error Backpropagation | 241 | ||
| 5.4 | The Hessian Matrix | 249 | ||
| 5.5 | Regularization in Neural Networks | 256 | ||
| 5.6 | Mixture Density Networks | 272 | ||
| 5.7 | Bayesian Neural Networks | 277 | ||
| 6 Kernel Methods | 291 | shimpei-m | 9/11, 18 | |
| 6.1 | Dual Representations | 293 | ||
| 6.2 | Constructing Kernels | 294 | ||
| 6.3 | Radial Basis Function Networks | 299 | ||
| 6.4 | Gaussian Processes | 303 | ||
| 7 Sparse Kernel Machines | 325 | junta-m | 10/5, 12 | |
| 7.1 | Maximum Margin Classiers | 326 | ||
| 7.2 | Relevance Vector Machines | 345 | ||
| 8 Graphical Models | 359 | yotaro-w,inui | 10/23, 30 | |
| 8.1 | Bayesian Networks | 360 | ||
| 8.2 | Conditional Independence | 372 | ||
| 8.3 | Markov Random Fields | 383 | ||
| 8.4 | Inference in Graphical Models | 393 | ||
| 9 Mixture Models and EM | 423 | shimbo | 7/20, 24 | |
| 9.1 | K-means Clustering | 424 | ||
| 9.2 | Mixtures of Gaussians | 430 | ||
| 9.3 | An Alternative View of EM | 439 | ||
| 9.4 | The EM Algorithm in General | 450 | ||
| 10 Approximate Inference | 461 | manabu-ki | ||
| 10.1 | Variational Inference | 462 | ||
| 10.2 | Illustration: Variational Mixture of Gaussians | 474 | ||
| 10.3 | Variational Linear Regression | 486 | ||
| 10.4 | Exponential Family Distributions | 490 | ||
| 10.5 | Local Variational Methods | 493 | ||
| 10.6 | Variational Logistic Regression | 498 | ||
| 10.7 | Expectation Propagation | 505 | ||
| 11 Sampling Methods | 523 | shimbo | 8/28, 9/4, 7 | |
| 11.1 | Basic Sampling Algorithms | 526 | ||
| 11.2 | Markov Chain Monte Carlo | 537 | ||
| 11.3 | Gibbs Sampling | 542 | ||
| 11.4 | Slice Sampling | 546 | ||
| 11.5 | The Hybrid Monte Carlo Algorithm | 548 | ||
| 11.6 | Estimating the Partition Function | 554 | ||
| 12 Continuous Latent Variables | 559 | harendra-b | ||
| 12.1 | Principal Component Analysis | 561 | ||
| 12.2 | Probabilistic PCA | 570 | ||
| 12.3 | Kernel PCA | 586 | ||
| 12.4 | Nonlinear Latent Variable Models | 591 | ||
| 13 Sequential Data | 605 | hideharu-o | ||
| 13.1 | Markov Models | 607 | ||
| 13.2 | Hidden Markov Models | 610 | ||
| 13.3 | Linear Dynamical Systems | 635 | ||
| 14 Combining Models | 653 | |||
| 14.1 | Bayesian Model Averaging | 654 | ||
| 14.2 | Committees | 655 | ||
| 14.3 | Boosting | 657 | ||
| 14.4 | Tree-based Models | 663 | ||
| 14.5 | Conditional Mixture Models | 666 | ||
章間の依存関係
(間違いあれば修正してください.)
- 3 章
- (少なくとも)一箇所, 2 章の数式を引用している.
- 7 章
- 4章の結果を参照している箇所が多い
- 6 章に依存.
- 8 章
- 3-7 章とは独立に読める?
- 9 章
- 2.3.9 Mixtures of Gaussians
- ancestral sampling ... 8.1.2 節
- 8 章 (グラフィカルモデルの図) に依存 (でもほとんど問題ないレベル).
- 9.3.3 Bernoulli 分布については 2.1 節 (p.69).
- 9.3.4 節は 3.5 節に依存. 後半, Relevance Vector Machine は 7.2 節.
- 10 章
- Mixtures of Gaussians は 9 章 (2 章?) が初出.
- 11 章
- ancestral sampling (8章）などの用語が出てくるが, ほぼ他の章と独立して読める.
- 12 章
- Kernel PCA は 6 章 Kernel に依存.
- EM アルゴリズムが出てくるため, 9 章に依存.