Table of Contents


We meet on Thursdays 11:30am - 1:00pm in 32-D407.

Feel free to add papers to the paper stack.

To join the reading group, feel free to:

Next meeting:

November 17

Nonuniform fast Fourier transforms using min-max interpolation.
Fessler, Jeffrey A., and Bradley P. Sutton.
IEEE transactions on signal processing 51.2 (2003): 560-574. fessler_nufft.pdf

November 3

GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models

October 20, 2022

Understanding Diffusion Models: A Unified Perspective
Calvin Luo

October 13, 2022

Gaussian Processes for Machine Learning (Book - chapters 1 and 2)
Edward Rasmussen, Christopher Williams

October 6, 2022

Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot, Franck Gabriel, Clément Hongler
NeurIPS 2018

September 29, 2022

Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot, Franck Gabriel, Clément Hongler
NeurIPS 2018

September 22, 2022

A Structured Dictionary Perspective on Implicit Neural Representations
Gizem Yüce, Guillermo Ortiz-Jiménez, Beril Besbinar, Pascal Frossard
CVPR 2022

September 15, 2022

Vox2Cortex: Fast Explicit Reconstruction of Cortical Surfaces from 3D MRI Scans with Geometric Deep Neural Networks
Fabian Bongratz, Anne-Marie Rickmann, Sebastian Pölsterl, Christian Wachinger
CVPR 2022

September 08, 2022

We briefly discussed a stack of papers on self-supervision for image time-series:

Slow and steady feature analysis: higher order temporal coherence in video
Dinesh Jayaraman and Kristen Grauman
CVPR 2016

Learning and using the arrow of time
Donglai Wei, Joseph J. Lim, Andrew Zisserman, and William T. Freeman
CVPR 2018

Joint-task self-supervised learning for temporal correspondence
Xueting Li, Sifei Liu, Shalini De Mello, Xiaolong Wang, Jan Kautz, and Ming-Hsuan Yang
NeurIPS 2019

Learning correspondence from the cycle-consistency of time
Xiaolong Wang, Allan Jabri, and Alexei A. Efros
CVPR 2019

Learning pixel trajectories with multiscale contrastive random walks
Zhangxing Bian, Allan Jabri, Alexei A. Efros, and Andrew Owens
CVPR 2022

August 26, 2022

Space-Time Correspondence as a Contrastive Random Walk
Allan Jabri, Andrew Owens, Alexei Efros
NeurIPS 2020

August 19, 2022

Noether Networks: meta-learning useful conserved quantities
Alet, Ferran, Dylan Doblar, Allan Zhou, Josh Tenenbaum, Kenji Kawaguchi, Chelsea Finn
NeurIPS 2021

August 12, 2022

BIGPrior: Towards Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration
Majed El Helou, Sabine Süsstrunk

August 5, 2022

Prototypical Networks for Few-shot Learning
Jake Snell, Kevin Swersky, Richard S. Zemel
NeurIPS 2017

July 29, 2022

Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, Ren Ng
NeurIPS 2020

July 22, 2022

Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot, Franck Gabriel, Clément Hongler
NeurIPS 2018

July 15, 2022

Reconstructing continuous distributions of 3D protein structure from cryo-EM images
Ellen D. Zhong, Tristan Bepler, Joseph H. Davis, Bonnie Berger
ICLR 2020

July 8, 2022

Chapter 6.1 of Numerical Optimization by Nocedal and Wright
The BFGS Method and its Convergence Analysis (pg. 135-143, 153-160)

July 1, 2022

Chapter 5.2 of Numerical Optimization by Nocedal and Wright
Non-Linear Conjugate Gradient (pg. 121-134)

June 24, 2022

Chapter 5.1 of Numerical Optimization by Nocedal and Wright
Convergence Rate and Preconditioning (pg. 112-120)

June 17, 2022

Chapter 5.1 of Numerical Optimization by Nocedal and Wright
Linear Conjugate Gradient Method (pg. 101-111)

June 10, 2022

DiffusionNet: Discretization Agnostic Learning on Surfaces
Nicholas Sharp, Souhaib Attaiki, Keenan Crane, Maks Ovsjanikov

June 3, 2022

Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics
M Niemeyer, L Mescheder, M Oechsle, A Geiger fessler_nufft.pdf

April 15, 2022

An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby

Paper stack:

Vision Transformer papers

April 22, 2022

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, Baining Guo


Emerging Properties in Self-Supervised Vision Transformers
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, Armand Joulin


Masked Autoencoders Are Scalable Vision Learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick

Contrastive Learning papers

Debiased Contrastive Learning (NeurIPS 2021); Chuang et al.

MICCAI 2019 papers

Learning Interpretable Features via Adversarially Robust Optimization; Khakzar et al.

Graph Neural Networks for Interpreting Task-fMRI Biomarkers; Li et al.

A Surface-theoretic Approach for Statistical Shape Modeling; Ambellan et al.

Journal papers

Geometric deep learning: going beyond Euclidean data; Bronstein et al. (may also be an associated review article?)

MICCAI 2018 papers

Construction of a Spatiotemporal Statistical Shape Model of Pediatric Liver from Cross-Sectional Data; Atsushi Saito et al.

Fast CapsNet for Lung Cancer Screening; Aryan Mobiny, Hien Van Nguyen

CompNet: Complementary Segmentation Network for Brain MRI Extraction, Raunak Dey, Yi Hong

Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation; Tanya Nair et al.

Generative discriminative models for multivariate inference and statistical mapping in medical imaging; Erdem Varol et al.

Roto-translation covariant convolutional networks for medical image analysis; Erik Bekkers et al.

Uncertainty in multitask learning: Joint representations for probabilistic MR-only radiotherapy planning; Felix Bragman et al.

Factorized spatial representation learning: Application in semi-supervised myocardial segmentation; Agisilaos Chartsias et al.

Hierarchical Spherical Deformation for Shape Correspondence; Lyu et al.

Using the Anisotropic Laplace Equation to Compute Cortical Thickness; Joshi et al.

3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes; Wong et al.

Deep Multi-Structural Shape Analysis: Application to Neuroanatomy; Gutierrez-Becker, B., and Wachinger, C.

rfDemons: Resting fMRI-Based Cortical Surface Registration Using the BrainSync Transform; Joshi et al.

ICCV 2017 papers

Deformable Convolution Networks; Dai et al.

IPMI 2017 papers

Unsupervised domain adaptation in brain lesion segmentation with adversarial networks; Kamnitsas et al.

Spectral kernels for probabilistic analysis and clustering of shapes; Folgoc et al.

MICCAI 2017 papers

Intraoperative Organ Motion Models with an Ensemble of conditional Generative Adversarial Networks; Hu et al.

A multi-armed bandit to smartly select a training set from big medical data; Becker et al.

Skin Disease Recognition Using Deep saliency features and multimodal learning of Dermoscopy and clinical images; Ge et al.

X-Ray in-depth decomposition: revealing the latent structures; Albarqouni et al.

Deep adversarial networks for biomedical image segmentation utilizing unannotated networks; Zhang et al.

Semi-supervised Deep Learning for Fully Convolutional Networks; Baur et al.

TandemNet: Distilling Knowledge from Medical Images Using Diagnostic Reports as Optional Semantic References; Zhang et al.

Towards automatic semantic segmentation in volumetric ultrasound; Yang et al.

The active atlas: combining 3D anatomical models with texture detectors; Chen et al.

Nonrigid image registration using Multi-scale 3D Convolutional Neural Networks; Sokooti et al.

Robust nonrigid registration through agent-based action learning; Krebs et al.

Online statistical inference for Large-Scale Binary Images; Chung et al.

Efficient deformable motion correction for 3-D abdominal MRI using manifold regression; Chen et al.

Learning and incorporating shape models for semantic segmentation; Ravishankar et al.

End to end unsupervised deformable image registration with a convolutional neural network; de Vos et al.

Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations; Sudre et al.

Adversarial training and dilated convolutions for Brain MRI segmentation; Moeskops et al.

Older papers

Sparse Kernel Machines for Discontinuous Registration and Nonstationary Regularization, Christoph Jud (University of Basel) Nadia MÃri, Philippe C. Cattin

fast implementation of registration: Fast Deformable Image Registration with Non-Smooth Dual Optimization, Martin Rajchl (Imperial College London, Robarts, Ontario); John S.H Baxter, Wu Qiu, Ali R. Khan, Aaron Fenster, Terry M. Peters, Daniel Rueckert, Jing Yuan

Image Registration for Placenta Reconstruction, Floris Gaisser, Toshio Chiba, Pieter Jonker

Tissue-Volume Preserving Deformable Image Registration for 4DCT Pulmonary Images, Bowen Zhao, Joohyun Song, Geoffrey Hugo, Yue Pan, Sarah Gerard, Kaifang Du, Taylor Patton, Joseph Reinhardt, John Bayouth, Gary Christensen

Past meetings:

Spring 2022

April 8

Deep Diffeomorphic Normalizing Flows

Hadi Salman, Payman Yadollahpour, Tom Fletcher, Kayhan Batmanghelich

April 1, 2022

DeepGEM: Generalized Expectation-Maximization for Blind Inversion
Angela Gao, Jorge Castellanos, Yisong Yue, Zachary Ross, Katherine Bouman

March 18, 2022

The Optimal Noise in Noise-Contrastive Learning Is Not What You Think
Chehab, Gramfort, Hyvarinen

February 25, 2022

Flood-Filling Networks
Michał Januszewski, Jeremy Maitin-Shepard, Peter Li, Jörgen Kornfeld, Winfried Denk, Viren Jain

Fall 2021

Dec 1

C. Chen et al., Realistic Adversarial Data Augmentation for MR Image Segmentation, MICCAI 2020chen-etal-realistic-aug.pdf

Nov 17

A Semantic Loss Function for Deep Learning with Symbolic Knowledge, ICML 2018 (

Nov 10

R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. PNAS, 2005. geometric-diffusion1.pdf

A survey of applications for background reading: R. Talmon, I. Cohen, S. Gannot, and R. R. Coifman. Diffusion maps for signal processing. IEEE Signal Processing magazine, 2013. talmon_diffusion_sp.pdf

November 3

Deformable Slice-to-Volume Registration for Motion Correction of Fetal Body and Placenta MRI by Uus et al., 2020 slice-to-volume-reg-placenta.pdf

Background reading:

  1. Reconstruction of fetal brain MRI with intensity matching and complete outlier removal by Kuklisova Murgasova Deprez et al., 2012 reconstruction_fetal_brain_mri_deprez2012.pdf
  2. A survey on slice to volume reconstruction: Slice-to-volume medical image registration: a survey by Ferrante et al., 2017 slice-to-vol-survey.pdf
October 20

Representation Learning with Contrastive Predictive Coding


October 6

Estimation of Non-normalized Statistical Models by Score Matching
Aapo Hyvarinen

September 29

Robust Compressed Sensing MRI with Deep Generative Priors
Ajil Jalal, et al.

September 22

Bayesian Learning via Stochastic Gradient Langevin Dynamics
Max Welling and Yee Whye Teh

For proof of convergence of Langevin dynamics (both continuous and discrete) to the posterior:
Langevin Diffusions and Metropolis-Hastings Algorithms
G.O. Roberts and O. Stramer

September 1

HodgeNet: Learning Spectral Geometry on Triangle Meshes
Dmitriy Smirnov and Justin Solomon

Summer 2021

August 25

MeshCNN: A Network with an Edge
Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, Daniel Cohen-Or

August 11

CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A. Efros, Trevor Darrell

August 4

Beyond pixel-wise supervision for segmentation: A few global shape descriptors might be surprisingly good!
Kervadec et al. 2021 (MIDL, Best Paper Award)

Spring 2021

May 10

Deep Parametric Continuous Convolutional Neural Networks
Shenlong Wang,Simon Suo,Wei-Chiu Ma, Andrei Pokrovsky, and Raquel Urtasun

May 3

Meta-Learning with Latent Embedding Optimization
Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, Raia Hadsell

April 26

Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, Ren Ng

April 12

A Simple Framework for Contrastive Learning of Visual Representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton

April 5

AutoInt: Automatic Integration for Fast Neural Volume Rendering
David B. Lindell, Julien N. P. Martel, Gordon Wetzstein

March 29

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng

March 22

Image Registration via Stochastic Gradient Markov Chain Monte Carlo (

March 15

Evidential Deep Learning to Quantify Classification Uncertainty (

March 8

Spherical Deformable U-Net: Application to Cortical Surface Parcellation and Development Prediction
Zhao et al.spherical_unet.pdf

March 1

RARE: Image Reconstruction using Deep Priors Learned without Ground Truth
Jiaming Liu, Yu Sun, Cihat Eldeniz, Weijie Gan, Hongyu An, Ulugbek S. Kamilov

Feb 22

The little engine that could: Regularization by denoising (RED)
Yaniv Romano, Michael Elad, and Peyman Milanfar red.pdf

Feb 8

B-spline Parameterized Joint Optimization of Reconstruction and K-space Trajectories (BJORK) for Accelerated 2D MRI
Guanhua Wang, Tianrui Luo, Jon-Fredrik Nielsen, Douglas C. Noll, Jeffrey A. Fessler

Feb 1

CBAM: Convolutional Block Attention Module
Sanghyun Woo, Jongchan Park, Joon-Young Lee, In So Kweon
ECCV 2018

Attention Augmented Convolutional Networks
Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, Quoc V. Le
ICCV 2019

Jan 25

Linear Predictability in Magnetic Resonance Imaging Reconstruction: Leveraging Shift-Invariant Fourier Structure for Faster and Better Imaging
Justin P. Haldar, Kawin Setsompop

Jan 11
Jan 4

Information-Theoretic Segmentation by Inpainting Error Maximization

Fall 2020

Dec 14

Dubois, Yann, Douwe Kiela, David J. Schwab, and Ramakrishna Vedantam. “Learning Optimal Representations with the Decodable Information Bottleneck.” Advances in Neural Information Processing Systems 33 (2020).

November 23

Geometric deep learning: going beyond Euclidean data

November 2

Dimensionality Reduction by Learning an Invariant Mapping

October 26

Deep Compressed Sensing

Wu et. al, in ICML 2019

October 19

Domain Generalization for Medical Imaging Classification with Linear-Dependency Regularization

September 21

Invariant Risk Minimization

September 14

Neural Tangent Kernel: Convergence and Generalization in Neural Networks

Summer 2020

August 31

Semi-Supervised Learning with Ladder Networks

August 24

Structural Autoencoders Improve Representations for Generation and Transfer

August 17

Training Generative Adversarial Networks with Limited Data, Karras et al.

August 10

A Fourier Perspective on Model Robustness in Computer Vision
Yin et al, 2020

Summer 2020 Book

Elements of Causal Inference Peters, Janzing, and Scholkopf [pdf]

July 27

Chapters 6.1, 6.2, 6.3, 6.4, 6.5 of Elements of Causal Inference [pdf]

July 20

Chapter 4.2 and 5 of Elements of Causal Inference [pdf]

July 13
June 29
June 22

Chapters 1 and 2 of Elements of Causal Inference [pdf]

June 15

Tutorial on Variational Autoencoders, Carl Doesch, 2016:

Auxiliary material, Kingma and Welling tutorial on VAEs, 2019:

June 8

Yen-Chun Chen and Linjie Li and Licheng Yu and Ahmed El Kholy and Faisal Ahmed and Zhe Gan and Yu Cheng and Jingjing Liu. “UNITER: UNiversal Image-TExt Representation Learning.” arXiv:1909.11740 (2019).

June 1

Oord, Aaron van den, Yazhe Li, and Oriol Vinyals. “Representation learning with contrastive predictive coding.” arXiv preprint arXiv:1807.03748 (2018).

May 25


May 18

Noise2Noise: Learning Image Restoration without Clean Data

Extending Stein’s unbiased risk estimator to train deep denoisers with correlated pairs of noisy images

Spring 2020

May 11

k-Space Deep Learning for Accelerated MRI. Yoseob Han, Leonard Sunwoo, Jong Chul Ye, IEEE TMI.

May 4

DeepSphere: A graph-based spherical CNN Michael Defferrard, Martino Milani, and Frederick Gussett. ICLR 2020 deepsphere_cnn.pdf

April 27

Louizos, Christos, Uri Shalit, Joris M. Mooij, David Sontag, Richard Zemel, and Max Welling. “Causal effect inference with deep latent-variable models.” In Advances in Neural Information Processing Systems, pp. 6446-6456. 2017.

April 13

UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction Leland McInnes, John Healy, James Melville

April 6, 2020

Neural Discrete Representation Learning Aaron van den Oord, Oriol Vinyals, Koray Kavukcuoglu

March 30, 2020

Unsupervised Learning with Stein’s Unbiased Risk Estimator

March 16, 2020

How Good is the Bayes Posterior in Deep Neural Networks Really?

March 9, 2020
March 2, 2020

Deep Complex Networks

February 24, 2020

Learning to Explain: An Information-Theoretic Perspective on Model Interpretation

February 10, 2020

Towards Automatic Concept-based Explanations

February 2, 2020

Explanation by Progressive Exaggeration Sumedha Singla, Brian Pollack, Junxiang Chen, Kayhan Batmanghelich Updated version:

January 27, 2020

Noise-contrastive estimation: A new estimation principle for unnormalized statistical models

January 13, 2020

Putting An End to End-to-End:Gradient-Isolated Learning of Representations

January 6, 2020

Burda et al, Importance weighted autoencoders (ICLR 2016):

Fall 2019

16 December

The Thermodynamic Variational Objective (NeurIPS 2019):

9 December

Domain-Adversarial Training of Neural Networks, JMLR 2016:

25 November

Zhou et al, Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation, ICCV 2019

2 December

Show, attend and tell: Neural image caption generation with visual attention

18 November

Compositional Attention Networks for Machine Reasoning:

4 November

Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis

28 October

Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology

A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology

21 October

A Topology Layer for Machine Learning

15 October

Predicting Slice-to-Volume Transformation in Presence of Arbitrary Subject Motion, Hou et al., MICCAI 2018

Fast Volume Reconstruction From Motion Corrupted Stacks of 2D Slices

7 October

High Accuracy Optical Flow Estimation Based on a Theory for Warping

Determining Optical Flow

30 September

Group Equivariant Convolutional Networks, Taco S. Cohen, Max Welling

September 23

Andrearczyk et al, Exploring local rotation invariance in 3D CNNs with steerable filters, MIDL 2019.

September 16

Freeman and Adelson, The design and use of steerable filters, IEEE PAMI 1991 design_and_use_of_steerable_filters.pdf

September 9

Weiler et al, 3D Steerable CNNs: Learning RotationallyEquivariant Features in Volumetric Data, NeurIPS 2018.

Summer 2019

August 21

Dihn et al, Density estimation using Real NVP, ICLR 2017

Gomez et al, The reversible residual network: Backpropagation without storing activations, NeurIPS 2017

August 14

Diederik P. Kingma, Prafulla Dhariwal: Glow: Generative Flow with Invertible 1×1 Convolutions, NeurIPS 2018

August 7

Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, Olivier Bachem; Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations; Proceedings of the 36th International Conference on Machine Learning, PMLR 97:4114-4124, 2019.

July 17

Computational Optimal Transport by Gabriel Peyre and Marco Cuturi: Finish 2.4-2.5; 3.1-3.4

July 3

Computational Optimal Transport by Gabriel Peyre and Marco Cuturi: Read up to the end of 2.4

June 24

Justin Solomon’s slides from his tutorial talk on optimal transport

June 27

Automated Treatment Planning in Radiation Therapy using Generative Adversarial Networks; Mahmood et al. mahmood18a.pdf

Spring 2019

May 13

The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision; Jiayuan Mao et al.

April 29

Fully Automatic 3D Reconstruction of the Placenta and its Peripheral Vasculature in Intrauterine Fetal MRI; Torrents-Barrena et al. automatic_placenta_seg.pdf

April 22

Semi-supervised learning for segmentation under semantic constraint; Pierre-Antoine Ganaye et al.

Mar 18

Improved Training of Wasserstein GANs

Mar 11
Mar 4

Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer

Feb 25

Spectral Representations for Convolutional Neural Networks:

Feb 11

Greff et al., “Highway and Residual Networks learn Unrolled Iterative Estimation”.

Jan 14 and Jan 22

Note: we’re meeting on Tue, Jan 22, 1pm for a one-time off-schedule meeting (because Mon Jan 21 is a holiday), focusing on Sec 4 and 5 of:

Neural Ordinary Differential Equations
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, David Duvenaud
NeurIPS 2018. pdf

Related papers for further reading:
- Rousseau et al, Residual Networks as Geodesic Flows of Diffeomorphisms, ArXiv 2018. pdf
Seems very related concurrent development by different authors.
- Grathwohl et al, FFJORD: Free-form continuous dynamics for scalable reversible generative models, ICLR 2019. pdf
Continuation of the continuous flows from the Neural-ODEs

Jan 7

A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. Jo Schlemper, Jose Caballero, Joseph V. Hajnal, Anthony Price and Daniel Rueckert. IEEE TMI 2017. //

Fall 2018

Dec. 10

K. Hammernik, T. Klatzer, E. Kobler, M.P. Recht, D.K. Sodickson, T. Pock, F. Knoll
arning a Variational Network for Reconstructionof Accelerated MRI Data.
Magnetic Resonance in Medicine, 2018

Dec. 3

Bo Zhu, Jeremiah Z. Liu, Stephen F. Cauley, Bruce R. Rosen, Matthew S. Rosen
Image reconstruction by domain-transform manifold learning.
Nature Letters

Nov. 19

Simon A. A. Kohl, Bernardino Romera-Paredes, Clemens Meyer, Jeffrey De Fauw, Joseph R. Ledsam, Klaus H. Maier-Hein, S. M. Ali Eslami, Danilo Jimenez Rezende, Olaf Ronneberger
A Probabilistic U-Net for Segmentation of Ambiguous Images.
NIPS 2018

Nov. 5

Finn, C., Abbeel, P., Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In: ICML. (2017)

Oct. 29

Gabriel Maicas, Andrew P. Bradley, Jacinto C. Nascimento, Ian Reid, Gustavo Carneiro

“Training Medical Image Analysis Systems like Radiologists” MICCAI 2018

Oct. 22

Martin Szummer and Tommi Jaakkola. “Information Regularization with Partially Labeled Data.” Advances in neural information processing systems. 2003.

Follow up paper:

Oct. 1 & Oct. 15

Grandvalet, Yves, and Yoshua Bengio. “Semi-supervised learning by entropy minimization.” Advances in neural information processing systems. 2005.

Summer 2018

Aug 28

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra, “Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization”, ICCV 2017,

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba, “Learning Deep Features for Discriminative Localization”, CVPR 2016,

Aug 21

David Bau*, Bolei Zhou*, Aditya Khosla, Aude Oliva, Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. CVPR 2017.

Other papers of interest from Bolei Zhou’s webpage:

Aug 7 and Aug 14

S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, W. Samek: On Pixel-wise Explanations for Non-Linear Classifier Decisions by Layer-wise Relevance Propagation. PLOS ONE, 10(7): e0130140 (2015)

July 31

Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Marius Staring, and Ivana Isgum, “End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network .” DLMIA-MICCAI, 2017.

July 24

Geoffrey E Hinton, Sara Sabour, Nicholas Frosst. Matrix capsules with EM routing.

July 17

Sabour, Sara, Nicholas Frosst, and Geoffrey E. Hinton. “Dynamic routing between capsules.” Advances in Neural Information Processing Systems. 2017.

Additional papers for more reading:




July 10

Spring 2018

May 1 and May 16

Kingma, Diederik P., et al. “Semi-supervised learning with deep generative models.” Advances in Neural Information Processing Systems. 2014.

April 24

Kainz et al.: “Fast Volume Reconstruction From Motion Corrupted Stacks of 2D Slices”


April 10

Mengye Ren and Richard S. Zemel. “End-to-End Instance Segmentation with Recurrent Attention.” arXiv preprint arXiv:1605.09410 (2015).

April 3

Lipton, Zachary C., et al. “Learning to diagnose with LSTM recurrent neural networks.” arXiv preprint arXiv:1511.03677 (2015).

March 27

Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, and Vladlen Koltun “A Probabilistic Model for Component-Based Shape Synthesis” SIGGRAPH / ACM Transactions on Graphics 31(4), 2012

Mar 13

Conjugate gradient algorithm and gradient preconditioning.

Chapters 11.2 and 11.3 in Numerical Algorithms by Justin Solomon

We will briefly discuss conjugate gradients then focus on preconditioning.

Mar 6

Conjugate gradient algorithm.

A (shorter) section from the numerical recipes book: c10-6.pdf

More background: Jonathan Richard Shewchuk. An Introduction to the Conjugate Gradient Method Without the Agonizing Pain. This is a nice (although long) tutorial paper: painless-conjugate-gradient.pdf

Feb 13

From Label Maps to Generative Shape Models: A Variational Bayesian Learning Approach; IPMI 2017 elhabian2017.pdf

Feb 06

Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery; IPMI 2017 schlegl2017.pdf

Jan 30

Error correction boosting for learning fully Convolutional networks with limited data; MICCAI 2017, Roy et al. roy.pdf

Jan 23

Christoph Baur, Shadi Albarqouni, Nassir Navab. Semi-supervised Deep Learning for Fully Convolutional Networks. baur.pdf

Fall 2017

Dec 12

Zhang Y., Yang L., Chen J., Fredericksen M., Hughes D.P., Chen D.Z. (2017) Deep Adversarial Networks for Biomedical Image Segmentation Utilizing Unannotated Images.

Dec 5

Kingma, Diederik P., and Max Welling. “Auto-encoding variational bayes.” arXiv preprint arXiv:1312.6114 (2013). vae_2013.pdf

Useful lecture notes for reviewing variational approximation: 437_approximations.pdf

Nov 28

Juan Eugenio Iglesias - Globally optimal coupled surfaces for semi-automatic segmentation of medical images iglesias2017.pdf

Nov 14

Topology-controlled Reconstruction of Multi-labelled Domains from Cross-sections; Ed Chien (Justin’s postdoc) will present. multitopo_tog.pdf

Nov 07

We will focus on:

  • Ravishankar et al - Learning and Incorporating Shape Models for Semantic Segmentation ravishankar2017.pdf
  • Milletari et al - Integrating Statistical Prior Knowledge into Convolutional Neural Networks milletari2017.pdf

This reading will be related to that of Oct 24th, when we discussed:

  • Oktay et al - Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation oktay2017a.pdf

Oct 31

Phillips Tech Talk: Computational Neurology and Computational Pathology. The talk is at 4PM in 32-G882. The link to register is here.

Oct 24 - Shape priors with CNNs

  • Oktay et al - Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation oktay2017a.pdf
  • Milletari et al - Integrating Statistical Prior Knowledge into Convolutional Neural Networks milletari2017.pdf
  • Ravishankar et al - Learning and Incorporating Shape Models for Semantic Segmentation ravishankar2017.pdf

Oct 17

Vertex Clustering model for disease progression: Application to Cortical Thickness Images; Marinescu et al. marinescu2017.pdf

Sep 26

We will read Sections 5.3 to 5.4 of bv_cvxbook.pdf.

Sep 19

We will read Sections 5.1 to 5.2.

Sep 12

We will read from section 4.2.3 onwards.

Sep 5

We reviewed the first third of Ch. 4 (read through section 4.2.3)

Summer 2017

We will be reading Convex Optimization by Stephen Boyd this summer. bv_cvxbook.pdf

Aug 1

Read section 2.6 on Dual Cones

July 27

We will continue reading Ch. 2 (read through section 2.5). See also a helpful set of slides for Chapter 2.

July 18

We will start Ch. 2 on Convex Sets (read through section 2.3)

Spring 2017

May 9

We will go back to the paper we started with:

Chen et al., Sparse Projections of Medical Images onto Manifolds []

May 2

Nonlinear Component Analysis as a Kernel Eigenvalue Problem Bernhard Scholkopf, Alexander Smola, Klaus-Robert Muller. AKA Kernel PCA. kpca.pdf

For those who want to take a look at a more formal treatment of the Sobolev kernel and such, here is another tutorial: AN INTRODUCTION TO THE THEORY OF REPRODUCING KERNEL HILBERT SPACES VERN I. PAULSENrkhs.pdf

April 25

We will go over the second part of the notes.

April 11

March 21

Fowlkes, Charless, Serge Belongie, Fan Chung, and Jitendra Malik. “Spectral grouping using the Nystrom method.” IEEE transactions on pattern analysis and machine intelligence 26, no. 2 (2004): 214-225. fowlkes_spectralgrouping_nystrom.pdf

Mar 7

Chen et al., Sparse Projections of Medical Images onto Manifolds []

Feb 28

Daniel Moyer, Boris A. Gutman, Joshua Faskowitz, Neda Jahanshad, and Paul M. Thompson. A Continuous Model of Cortical Connectivity. MICCAI 2016 []

Feb 21

Yang Xiao, Roland Kwitt, and Marc Niethammer. “Fast Predictive Image Registration.” International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis. Springer International Publishing, 2016. []

Feb 7

Isola et al., Image-to-Image Translation with Conditional Adversarial Networks, []

IAP 2017

Jan 31

Ronneberger et al. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI 2015. unet.pdf

Jan 24

Oktay et al. Multi-Input Cardiac Image Super-Resolution using Convolutional Neural Networks. MICCAI 2016. []

Jan 17

Jan 10

Barycentric Subspace Analysis: a new Symmetric Group-wise Paradigm for Cardiac Motion Tracking. []

Fall 2016

Nov 22

Unsupervised Freeview Groupwise Cardiac Segmentation Using Synchronized Spectral Network unsupervised.pdf

Nov 15

Bilateral Weighted Adaptive Local Similarity Measure for Registration in Neurosurgery kochan16.pdf

Nov 8

Fast Fully Automatic Segmentation of the Human Placenta from Motion Corrupted MRI. kainz_miccai106a.pdf

Nov 1

SpineNet: Automatically Pinpointing Classification Evidence in Spinal MRIs. jamaludin16.pdf

Oct 25

MICCAI 2016 debrief.

Please come with 1-3 papers that you liked at MICCAI 2016 and want to discuss about!

Oct 11

Aditya Khosla will talk about his work on CNN for Medical Tasks.

Potentially relevant paper:

Oct 4

Antonio Torralba will discuss his experience with NNs.

Sep 27

We’ll continue with Goodfellow et al’s book., chapter 9 (CNNs)

Sep 13

We’ll continue with Goodfellow et al’s book., up to and including 6.5 (Backprop)

Summer 2016

Aug 25

We’ll continue with Goodfellow et al’s book., up to and including 6.4

Aug 11

We’re starting a series of discussions on Deep learning.

We’ll start with Goodfellow et al’s book. (Please scan through Part I and review whatever is necessary)

On Aug 11th, we’ll read Part II, Chapter 6.

July 28

Adrien Depeursinge will tell us about his work on texture classification.

June 17, June 23, July 14

Vercauteren et al, Diffeomorphic Demons: Efficient Non-parametric Image Registration: diffeodemons-neuroimage08-vercauteren.pdf

For those interested in further development: symlogdemons-miccai08-vercauteren.pdf

For those need basic background of demons algorithm: thirion98.pdf

June 9

Avants, Brian B., Charles L. Epstein, Murray Grossman, and James C. Gee. “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain.” Medical image analysis 12, no. 1 (2008): 26-41. MedIA version: avants-media.pdf symmetric_diffeomorphic_image_registration.pdf

June 2

C. Studholme, D.L.G. Hill, D.J. Hawkes, “An overlap invariant entropy measure of 3D medical image alignment”, Pattern Recognition 32 (1999) 71—86 normalizedmi.pdf (Discussion leader: Danielle)

July 21

Horn et al, Determining Optical Flow: opticalflow.pdf

Spring 2016

May 17

Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images. D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes. IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 18, NO. 8, AUGUST 1999. rueckert-ffd.pdf

May 10

Floris F. Berendsen, Uulke A. van der Heide, Thomas R. Langerak, Alexis N.T.J. Kotte, Josien P.W. Pluim. Free-form image registration regularized by a statistical shape model: application to organ segmentation in cervical MR. CVIU 2013. plum-2013.pdf

May 3

Papież, B. W., Heinrich, M. P., Fehrenbach, J., Risser, L., & Schnabel, J. A. (2014). An implicit sliding-motion preserving regularisation via bilateral filtering for deformable image registration. Medical image analysis, 18(8), 1299-1311. 1-s2.0-s1361841514000784-main.pdf

April 26

Tanya Schmah, Laurent Risser, and Franncois-Xavier Vialard, 2013 - Left-Invariant Metrics for Diffeomorphic Image Registration with Spatially-Varying Regularisation. leftinvariantmetrics.pdf diffeomorphic_image_matching_with_left-invariant_metrics.pdf(Discussion Leader - Miaomiao)

April 19

Tong et al, 2013 - Segmentation of MR images via discriminative dictionary learning and sparse coding: Application to hippocampus labeling. dictlearning.pdf (Discussion Leader - Danielle)

April 12

Bhatia et al, 2014 - Hierarchical Manifold Learning for Regional Image Analysis. bhatia2014.pdf - Discussion Leader: Adrian

April 5

Kim, Jaechul, Ce Liu, Fei Sha, and Kristen Grauman. “Deformable spatial pyramid matching for fast dense correspondences.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013. 2013cvpr_dsp.pdf - Discussion Leader: Ray

One of its applications in image-guided radiotherapy:

Mazur, Thomas R., et al. “SIFT-based dense pixel tracking on 0.35 T cine-MR images acquired during image-guided radiation therapy with application to gating optimization.” Medical physics 43.1 (2016): 279-293. sift-based_dense_pixel_tracking_on_0.35_t_cine-mr_images_acquired_during_imageguided_radiation_therapy_with_application_to_gating_optimization.pdf

March 29

Matthieu Lê, Jan Unkelbach, Nicholas Ayache, Hervé Delingette. GPSSI: Gaussian Process for Sampling Segmentations of Images. MICCAI 2015. le-miccai-2015.pdf

Feb 23

Mapping Stacked Decision Forests to Deep and Sparse Convolutional Neural Networks for Semantic Segmentation Disc. leader: Greg, 20160209_ciccarelli_rfdnn.pdf

Feb 2

Conditional Regression Forests for Human Pose Estimation, Sun et al, CVPR 2012 skt_cvpr2012.pdf Discussion Leader: Danielle

If you’d like some background on Regression Forests, you can look at these chapters from “Decision Forests for Computer Vision and Medical Image Analysis”, Criminisi and Shotton, eds.: decisionforestschap3.pdf, decisionforestschap5.pdf

IAP 2016

Jan 25

We will have Mert Sabuncu present his work on longitudinal analysis:


related on his MICCAI 2015 paper sabuncu-miccai-2015.pdf.

Jan 4

We will march off to Sarang Joshi’s tutorial on diffeomorphic registration. Here’s some holiday reading: François-Xavier Vialard , Laurent Risser, Daniel Rueckert, Colin J. Cotter. Diffeomorphic 3D Image Registration via Geodesic Shooting Using an Efficient Adjoint Calculation. IJCV 2011. vialard-ijcv.pdf

Note from the tutorial: liegroups.pdf

Matlab code used in the tutorial: gaussiansplines.txt and flow.txt

Fall 2015

Dec 10

Simultaneous Longitudinal Registration with Group-Wise Similarity Prior. Greg M. Fleishman , Boris A. Gutman, P. Thomas Fletcher, Paul M. Thompson. IPMI 2015 fleishman-ipmi-2015.pdf

Nov 19

[MICCAI2015] Uncertainty-driven Forest Predictors for Vertebra Localization and Segmentation. David Richmond, Dagmar Kainmueller, Ben Glocker, Carsten Rother, Gene Myers.

Nov 12

q-Space Deep Learning for Twelve-Fold Shorter and Model-Free Diffusion MRI Scans golkov15.pdf.

Nov 5

Neher et. al. A Machine Learning Based Approach to Fiber Tractography Using Classifier Voting. MICCAI 2015. neher15.pdf

Oct 29

Soheil Hor, Mehdi Moradi. Scandent Tree: A Random Forest Learning Method for Incomplete Multimodal Datasets. MICCAI 2015. scandent-trees-miccai-2015.pdf

Oct 19 (Monday)

We sorted out the list to read.

Sep 24

We will be reading Miaomiao Zhang’s latest paper: zhang_ipmi2015.pdf

For more context we have, Ashburner and Friston’s paper: ashburner_ni11.pdf

And also the paper we read last week!

Sep 17

We will read Miller et al’s paper on the application of geodesic shooting in diffeomorphic image registration: miller_jmiv06.pdf

A more recent paper by Ashburner and Friston is also available for further reading: ashburner_ni11.pdf

Summer 2015

Aug 27, Sep 3

We will read Vercauteren et al, Diffeomorphic Demons: Efficient Non-parametric Image Registration: diffeodemons-neuroimage08-vercauteren.pdf

For those interested in further development of this line of research: symlogdemons-miccai08-vercauteren.pdf

Aug 20

No meeting

Aug 13

We will continue reading Beg et al.

Additional resources:

Review of calculus of variations: calculusofvariations.pdf

Danielle’s notes on Beg et al: dfpace_beg_etal.pdf

Aug 6

We will read Beg et al, Computing large deformation metric mappings via geodesic flows of diffeomorphisms, IJCV 2005. beg_lddmm.pdf

July 30

We will continue going over Danial Lashkari’s notes.

July 23

We will continue going over Danial Lashkari’s notes. Additionally, this is another resource: diffgeo.pdf

July 16

We will go over Danial Lashkari’s notes from the last time we all learned about lie algebra and manifolds.

Spring 2015

May 8, 22

Durrleman, Stanley, et al. “Toward a comprehensive framework for the spatiotemporal statistical analysis of longitudinal shape data.” International journal of computer vision 103.1 (2013): 22-59. durrleman_et_al_2013.pdf

April 24, May 1

Reshef DN, Reshef YA, Finucane HK, et al. Detecting Novel Associations in Large Datasets. Science (New York, N.y). 2011;334(6062):1518-1524. doi:10.1126/science.1205438. mic.pdf

The supplementary material that we will discuss: som.pdf Section 3 discussed the approximation algorithm, though sections 1 and 2 are likely to also be useful for the discussion!

April 17

MICCAI: Zhang, Miaomiao, and P. Thomas Fletcher. “Bayesian Principal Geodesic Analysis in Diffeomorphic Image Registration.” Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014. Springer International Publishing, 2014. 121-128. bayesian-principal-geodesic-analysis-in-diffeomorphic-image-registration.pdf

NIPS: Zhang, Miaomiao, and P. Thomas Fletcher. “Probabilistic principal geodesic analysis.” Advances in Neural Information Processing Systems. 2013. 5133-probabilistic-principal-geodesic-analysis.pdf

April 10

Sabuncu, Mert R. “A Universal and Efficient Method to Compute Maps from Image-Based Prediction Models.” Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014. Springer International Publishing, 2014. 353-360.


March 27

Maxime Taquet, Benoıt Scherrer, Jurriaan M. Peters, Sanjay P. Prabhu, and Simon K. Warfield. A Fully Bayesian Inference Framework for Population Studies of the Brain Microstructure. MICCAI 2014. taquet-miccai2014.pdf

Fall 2014

November 18:

Ramesh, Adrian and Danielle will lead:

ICML tutorial on Submodularity in Machine learning - Part I

November 4:

Danielle will lead:

Zhu, Zhang, Liu and Metaxas, Scalable histopathological image analysis via active learning. zhu_miccai2014.pdf

October 21

George will lead:

Herve Lombaert, Darko Zikic, Antonio Criminisi, and Nicholas Ayache. Laplacian Forests: Semantic Image Segmentation by Guided Bagging. miccai-laplacian-forest.pdf

(Update 10/24/2014) Notes are now available. reading-group-random-forests-laplacian-forests-2014-10-21-draft1.pdf

October 7

We will meet in *32-D451*.

Daniel C. Alexander, Darko Zikic, Jiaying Zhang, Hui Zhang, Antonio Criminisi. Image Quality Transfer via Random Forest Regression: Applications in Diffusion MRI. alexander_miccai2014.pdf

September 30

We will meet in *32-G431*.

Xiaoxiao Liu, Marc Niethammer, Roland Kwitt, Matthew McCormick, and Stephen Aylward. Low-Rank to the Rescue – Atlas-based Analyses in the Presence of Pathologies. liu2014_miccai_low_rank_to_the_rescue.pdf


Summer 2014

June 31 (aka July 1)

Rohlfing T1, Sullivan EV, Pfefferbaum A. Regression models of atlas appearance. ipmi2009_rohlfing.pdf

June 24

O. Veksler, Star shape prior for graph-cut image segmentation. ECCV 2008, Lecture Notes in Computer Science Volume 5304, 2008, pp 454-467. starshapeprior.pdf

Spring 2014

May 6

Since we didn’t have dessert, this is my suggestion for next week. I hope you enjoy it. SUNGKYU JUNG, IAN L. DRYDEN and J. S. MARRON. “Analysis of principal nested spheres (PNS)”, Biometrika (2012),

April 29

The reading is split into an amuse-bouche and an entree. I intend on only discussing the entree at length.

Amuse-bouche: To whet your appetite, skim chapter 8 of Boyd’s ADMM monograph for a sampler of applications where ADMM is used (among these are a few I sketched last time: lasso, group lasso, SVM’s):

Entree: Concentrate your fire power on reading the following paper by Yedidia and friends who show how ADMM can be implemented as a message-passing algorithm and how to modify the algorithm to tackle some large nonconvex problems:

Update 4/30/2014 2:28am: Notes by me (George) for last reading group are now up! reading-group-admm-intro-2014-04-15-notes.pdf (lasted updated 4/30 to fix some typos caught by Danielle)

April 15

We begin our series on distributed optimization, with a preliminary focus on a method called ADMM. Please read chapters 1-3 of the following:

A resource webpage that might be helpful:

April 8

Ben Glocker, A Sotiras, N Komodakis, Nikos Paragios. “Deformable Medical Image Registration: Setting the State of the Art with Discrete Methods”, Annual Review Biomedical Engineering, 2011, 13: 219-244. glocker11.pdf

Apr 1

Wang et al, “Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey”, CVIU, 2013. Technical report (42pages), or CVIU paper (18 pages) wang13.pdf

Mar 25

Yang et al, Neighbor-Constrained Segmentation With Level Set Based 3-D Deformable Models, IEEE TMI 2004 yang_neighborconstrainedsegmentation.pdf

Mar 18

Firdaus Janoos, Shantanu Singh, Raghu Machiraju, William M. Wells III, István Ákos Mórocz. State-Space Models of Mental Processes from fMRI. IPMI 2011. janoos-ipmi2011.pdf

And here is the journal paper: janoos_2013.pdf

Mar 11

Harini Eavani, Theodore Satterthwaite, Raquel Gur, Ruben Gur, Christos Davatzikos; Unsupervised Learning of Functional Network Dynamics in Resting State fMRI

Feb 11

Ioannidis, J.P.A. Why most published research findings are false why_most_published_research_findings.pdf (see also this rebuttal paper and Andrew Gelman's counter-rebuttal).

Fall 2013

Dec 9

Probabilistic inference of regularisation in non-rigid registration. Ivor J.A. Simpson, Julia A. Schnabel, Adrian R. Groves, Jesper L.R. Andersson, Mark W. Woolrich. simpson-neuroimage-2012.pdf

Dec 2

Simpson et al, A Bayesian Approach for Spatially Adaptive Regularisation in Non-rigid Registrationbayes_reg.pdf (MICCAI)

Nov 18

We continue our patch series by moving from super-resolution to segmentation.

Wang et al, 2013, Patch-Based Segmentation without Registration: Application to Knee MRI wang_seg_2013.pdf (MLMI-MICCAI)

Nov 4

We will start a series of patch-based methods. For Nov 4th, we’ll look at:

Shi et al, 2013. Cardiac Image Super-Resolution with Global Correspondence Using Multi-Atlas PatchMatch shi_et_al_2013.pdf (MICCAI)

As an optional read, many algorithms use or refer to patchMatch - Barnes, 2009.

Oct 28

We will discuss interesting papers from IPMI and MICCAI. Everyone should come with a couple of papers they are interested in exploring.

Oct 21

We will read Tree-space statistics and approximations for large-scale analysis of anatomical trees, by A. Feragen, M. Owen, J. Petersen, M.M.W. Wille, L.H. Thomsen, A. Dirksen and M. de Bruijne

Oct 7

Matched Signal Detection on Graphs: Theory and Application to Brain Network Classification, by C. Hu, L. Cheng, J. Sepulcre, G. El Fakhri, Y. M. Lu, and Q. Li.


Spring 2013

May 7th.

Chapter 5 of this book: “Handbook of Markov Chain Monte Carlo”, which can be found here

April 23th.

Chapter 27 of this Book: “Bayesian Reasoning and Machine Learning”

April 9th.

Globerson and Roweis, Nightmare at Test Time: Robust Learning by Feature Deletion, ICML, 2006 [globerson_and_roweis_2006.pdf]

March 12

Kayhan and I (George) will talk about a new approach to nonnegative matrix factorization (NMF) that is NOT based on iterative hill-climbing type algorithms that may only reach a local optima; instead, under a separability assumption which is empirically observed in real topic modeling data, NMF for learning topic models can be solved in polynomial time. The paper that we’ll look at is on arXiv, and we’ll focus on everything up to and including Section 3.1 (mid way through page 8–basically we’ll focus on the case where there are true anchor words):

"Learning Topic Models -- Going Beyond SVD"
Sanjeev Arora, Rong Ge, Ankur Moitra
Foundations of Computer Science 2012

Update (3/11/2013 5:10pm). I just put together some preliminary notes going over some key high-level ideas; hopefully this is helpful: reading-group-learning-topic-models-2013-03-12-draft1.pdf

March 5

We met to plan what we’ll read for the spring. Here are the results of the voting:

  • Non-negative matrix factorization (6)
  • Learning with missing features (5)
  • Tutorial on sampling methods (MCMC, HMC, etc) (5)
  • Type I vs Type II sparsity (4): David Wipf and Yi Wu. “Dual-Space Analysis of the Sparse Linear Model” NIPS 2012
  • Video Magnification (4) Initial paper
  • Tutorial on deep learning (4)
  • Compressed sensing (3) Paper?
  • Learning distributions from composition of marginals (3): F. Sanchez-Vega, J. Eisner, L. Younes, and D. Geman. “Learning Multivariate Distributions by Competitive Assembly of Marginals” IEEE PAMI, 2012.
  • Network/time clustering in fmri (3): Identification of Recurrent Patterns in the Activation of Brain Networks
  • ADMM for distribution optimization (3)
  • sym diffeomorphism long. (3)
  • significant for lasso (2)
  • Tutorial on Indian Buffet processes (2)
  • Non-quadratic priors (0)
  • Multidimensional Spectral Hashing (0)
  • Ordered based search (0)

Fall 2012

Nov 20 - Dec

We will read (Mert will lead): A. Criminisi, J. Shotton, and E. Konukoglu, “Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning” The file is big so here’s a link:

Depending on interest, this reading might span several weeks. In our first session, let’s plan to read up to the end of chapter 4, namely the one on regression. On an unrelated note, Ender, one of the co-authors of this paper, will be present.

We also read the following MICCAI paper: E. Konokoglu, B. Glocker, D. Zikic, A. Criminisi, “Neighbourhood Approximation Forests”, MICCAI 2012: neighbourhood_approximation_forests.pdf.

Nov 13

We will continue with MICCAI 2012 papers and read: “Hierarchical Manifold Learning” by Kanwal K. Bhatia, Anil Rao, Anthony N. Price, Robin Wolz, Joseph V. Hajnal, Daniel Rueckert: hierarchicalmanifoldlearning.pdf

Nov 6

We will read this paper: “SVM based significance maps” by Bilwaj Gaonkar, Christos Davatzikos bilwaj2012.pdf

Oct 30

We will read our first paper from the MICCAI 2012 series: “Evaluating segmentation error without ground truth” by Timo Kohlberger, Vivek Singh, Chris Alvino, Claus Bahlmann, Leo Grady: kohlberger2012_gt.pdf

Oct 23

MICCAI Discussion

Decided for next week: “Evaluating segmentation error without ground truth”

To invite for talk — Neighborhood approximation forests, 10

Later reading series - Reading up on random forests (Antonio’s book), 10

Other readings:

Evaluating segmentation error without ground truth, 9

SVM based significance maps, 8

Geodesic information flow, 5

Hierarchical manifold learning, 5

October 16

We will read Chapter 3 of Tom Minka’s thesis on Expectation Propagation minka-thesis.pdf. There is a fair amount of derivation, so hopefully, we can get a clear picture of the algorithm.

Additional references on EP and other approximate inference algorithms can be found at:

October 9 (originally September 25)

We’ll expand on the spatially dependent Pitman-Yor processes we covered a couple of weeks ago, and read:

Soumya Ghosh and Erik B. Sudderth, Nonparametric Learning for Layered Segmentation of Natural Images, CVPR 2012. ghoshsudderth12cvpr.pdf

The supplementary materials cover some detail that we probably won’t have time to cover, but I’ve uploaded them for completeness. ghoshsudderth12cvpr-supplement.pdf

Also relevant is a video lecture on the topic by Erik Sudderth from a NIPS 2011 workshop:

September 19

We will read the original Hierarchical Dirichlet Process paper: Teh, Jordan, Beal, Blei: Hierarchical Dirichlet Processes hierarchical_nonparametric.pdf

Here are some helpful notes written up by Danial: dp_brief_notes.pdf.

September 11

We’ll look at a paper that uses layers approach to image segmentation:

Sudderth and Jordan, Shared Segmentation of Natural Scenes Using Dependent Pitman-Yor Processes, NIPS 2008. sudderth-jordan-nips08.pdf

Summer 2012

September 4

We’ll look at two papers that use/extend SWA, and cover a little more detail on supernode/seed selection:

Alpert, S., Galun, M., Basri, R. and Brandt, A., Image segmentation by probabilistic bottom-up aggregation and cue integration, PAMI 2012. alpert_aggregating_pami_2012.pdf

Goldschmidt, Y., Galun, M., Sharon, E., Basri, R. and Brandt, A. Fast multiscale clustering by integrating collective features, NIPS 2007 goldschmidt2007nips.pdf

Note that the first paper has the same title as their 2007 CVPR submission, so be careful if searching.

August 28

We will look at a paper on tumor segmentation via superpixel hierarchy: corso_et_al.pdf

J. J. Corso, E. Sharon, S. Dube, S. El-Saden, U. Sinha, and A. Yuille. Efficient Multilevel Brain Tumor Segmentation with Integrated Bayesian Model Classification. IEEE Transactions on Medical Imaging, 27(5):629-640, 2008

If you want to read into the SWA algorithm: Hierarchy and adaptivity in segmenting visual scenes, nature04977.pdf

July 24

We will discuss S. C. Zhu’s paper on image segmentation: zhu_pami96.pdf A brief review of the snakes is provided in Sec. 2.1 and in the Appendix of the paper, and active contours are explained in more details in kass_ijcv88.pdf and caselles_ijcv97.pdf.

July 17

We will finish chapter 5 and go over chapter 6.

July 3

This week, we will go over the 5’th chapter (Filtering on Graphs) of the “Discrete Calculus” by Leo Grady.

June 19

We’ll start with Sebastian’s nature paper:

The paper gives intuition for non-negative matrix factorization and compares it to several other approaches.

For reference, the detailed algorithms paper is here:

Future potential papers

To understand the algorithms: A Tutorial on MM Algorithms

Unified Development of Multiplicative Algorithms for Linear and Quadratic Nonnegative Matrix Factorization

On the Equivalence of NMF and Spectral Clustering

Understanding the similarities and the differences between probabilistic topic models and NMF will be useful. A potential paper is: Probabilistic Latent Variable Models as Nonnegative Factorizations

Spring 2012

May 15

We’ll read: 4D registration of serial brain’s MR images: a robust measure of changes applied to Alzheimer’s disease. Marco Lorenzi, Nicholas Ayache, Giovanni Frisoni, and Xavier Pennec. MICCAI STIA Workshop, 2010


Apr 24

We’ll be reading on determinantal point processes. More precisely, we will read chapter 2 of Alex’ thesis:

A. Kulesza, Learning with Determinantal Point Processes, thesis draft. alex_thesis_draft.pdf

Additional material. Conference papers:

A. Kulesza, and B. Taskar, Structured Determinantal Point Processes, Neural Information Processing Systems Conference (NIPS), Vancouver, BC, December 2010. sdpp_nips10-1.pdf

k-DPPs: Fixed-Size Determinantal Point Processes, A. Kulesza, and B. Taskar. International Conference on Machine Learning (ICML), Bellevue, WA, June 2011. kdpps_icml11-1.pdf

Learning Determinantal Point Processes, A. Kulesza, and B. Taskar. Conference on Uncertainty in Artificial Intelligence (UAI), Barcelona, Spain, July 2011. ldpps_uai11.pdf

Math DPP surveys:

Ben Hough, Manjunath Krishnapur, Yuval Peres, Bálint Virág, Determinantal Processes and Independence, Probability Surveys, 2006. determinantal_processes_and_independence_-_hough.pdf

Alexei Borodin, Determinantal point processes, 2009. determinantal_point_processes_-_borodin.pdf

Apr 10

Switching gears, we’ll read - Vounou M, Nichols TE, Montana G; Alzheimer’s Disease Neuroimaging Initiative. Discovering genetic associations with high-dimensional neuroimaging phenotypes: A sparse reduced-rank regression approach. Neuroimage, 53(3):1147-59, 2010 vounou_et_al_2010_-_neuroimage.pdf.

Mar 20

We will read Dan Feldman’s NIPS paper. Here’s an updated PDF that includes supplemental material: nips11coresets-supplemental.pdf

Here’s the NIPS talk:

Update (3/19/2012, 10:17pm) Preliminary notes that I (George) typed up for last week’s discussion are here: reading_group_2012_03_13_notes.pdf Unfortunately, from what I can tell, some constants from the lecture notes don’t quite match constants in literature cited at times. I tried to reconcile these discrepancies. The main message is *not* any different though; basically the k-center coreset construction algorithm has a horrendous running time in terms of the number of clusters k.

Mar 13

We will finish reading the survey paper from last week, emphasizing clustering (section 6) and the high-dimensional setting (section 7).

Update (3/13/2012, 2:34am) Preliminary notes that I (George) typed up for last week’s discussion are here: reading_group_2012_03_06_notes.pdf Unfortunately, these may not be terribly helpful for this week’s reading as the clustering material is quite different...

Mar 6

In preparation for Dan Feldman’s coresets talk in April, we’ll be reading a (dated) survey on the basics of coresets from computational geometry. Basically the idea is that we want to approximate some interesting metric of a bunch of data points by using a hopefully substantially smaller subsample (called a coreset) of the data points. The reading:

P.K. Agarwal, S. Har-Peled, and K.R. Varadarajan. “Geometric Approximation via Coresets”. (2005) coresets_survey_2005.pdf

As a preview for what’s to come, here’s Dan’s paper at NIPS 2011 on how to build a coreset for a (huge) training dataset of a mixture model while retaining an $\epsilon$-approximation to the data likelihood; the result is that you can run, say, the EM algorithm for Gaussian mixture models on the coreset instead (this is NOT part of the reading for Mar 6 although I’d recommend looking over the abstract to see what coresets can achieve now):

D. Feldman, M. Faulkner, and A. Krause. “Scalabe Training of Mixture Models via Coresets”. (2011) coresets_nips11.pdf

Feb 28

We’ll be reading a paper on a model and algorithm for Gaussian process classification:

Urtasun, R. and Darrell, T.: Discriminative Gaussian Process Latent Variable Model for Classification. ICML ‘07


Feb 21

We will read Mackay’s tutorial on Gaussian processes: mackay-gaussian-processes.pdf

Thanks to Iman for finding an updated version: mackay-gaussian-processes-frombook.pdf

We’ll focus on the first 6 sections.

Another useful reference is Williams’ tutorial: williams-gaussian-process.pdf (only Sections 3 and 4 are relevant to this week’s discussion).

Feb 14

We will read the first 4 sections of “A Riemannian Framework for Tensor Computing” riemannian_framework_for_tensor_computing_2005.pdf.

Jan 31

We will be reading the paper “Smooth relevance vector machine: a smoothness prior extension of the RVM” by Alexander Schmolck and Richard Everson (Mach Learn (2007) 68: 107–135) smoothrvm.pdf

We’ll discuss and compare it to papers we covered earlier, Mert’s MICCAI sabuncumiccai2011.pdf and journal paper (coming soon) as well as Tipping’s original work on RVM tippingjmlr2001_sparsebayesianlearningandtherelevancevectormachine.pdf.

Jan 24

Spatially regularized SVM for the detection of brain areas associated with stroke outcome. Remi Cuingnet, Charlotte Rosso, Stephane Lehericy, Didier Dormont, Habib Benali, Yves Samson1, and Olivier Colliot. MICCAI 2010. miccai2010_cuingnet.pdf

Jan 17

UPDATE: Annotations added. I recommend reading most of the introduction for context and background, so I’ve left it without annotations. In the Methods section, I’ve highlighted titles of subsections that are pertinent to the results I intend to cover. Similarly, in the Results section, I’ve highlighted subsections I intend to cover and struck through everything we won’t discuss. Despite this, there is still a fairly large amount of text to read, some of which is somewhat technical (in a neuroscientific sense). Therefore, I recommend initially focusing on the results figures, which are highly illustrative and well described in their respective captions, then referring back to the text.

Annotated PDF: j_neurophys-2011-yeo-annot-print.pdf

We plan to read parts of Thomas’ (and others’) paper on a large scale study of resting state functional connectivity in healthy subjects. The discussion should mostly be centred around particularly interesting results that could serve as motivation for future work in our group. Once I’ve read all of the paper in detail, I’ll try to narrow down the results sections so that the amount of material is reasonable.

B. T. Thomas Yeo, Fenna M. Krienen, Jorge Sepulcre, Mert R. Sabuncu, Danial Lashkari, Marisa Hollinshead, Joshua L. Roffman, Jordan W. Smoller, Lilla Zöllei, Jonathan R. Polimeni, Bruce Fischl, Hesheng Liu and Randy L. Buckner; The organization of the human cerebral cortex estimated by intrinsic functional connectivity j_neurophys-2011-yeo.pdf

Fall 2011

Dec 7

met-mlbayes.pdf We’re going back in time and covering a tutorial in Bayesian inference in Machine Learning by Tipping. It offers a nice build-up from least squares regression to the RVM. The Models in Section 4 are very related to the RVoxM readings from the previous 2 weeks.

rvmpath.pdf I put together a quick list of papers and references for learning the RVM and RVoxM. If you are interested in this topic, take a look at the file for more information/paths to take.

For the curios, heretipping.pdf‘s some results from a quick (potentially buggy) implementation. I tried to keep the colors as those Tipping’s Figure 5, although I had to guess some of the parameters so the curves are slightly different. The new blue line shows the (-log) joint that we were wondering about, and the red is the (-log) marginal likelihood from the paper (these later two curves were rescaled). Although they differ slightly, all three estimations of lambda (validation, joint, marginal) seem ~equally decent.

Dec 1

Note special time this week: Thursday, Dec 1, 4pm
We’ll go into the hyperparameter estimation derivations from the Relevance Voxel Machine. I’ve emailed a draft of the journal RVoxM paper to V-Golland. We will concentrate on Sections II, III and VI (Theory and Appendix).

For the interested, more reading into RVM:

Michel Tipping maintains a website on “Sparse Bayesian Models (& the RVM)”:
One of the papers, “Sparse Bayesian learning and the relevance vector machine” tipping01a.pdf has relevant derivations in sec 1, 2, 5. (see for corrections)
Fletcher fletcher_-_rvm_explained.pdf also dives right into the theory (much less discussion on motivation/application/etc)
See 2004-wipf-ieeesigproc.pdf for another perspective on sparsity

Nov 22

Note special time this week: Tuesday, Nov 22, 4pm

We’ll be reading Mert’s MICCAI paper - Sabuncu and Leemput, The Relevance Voxel Machine (RVoxM): A Bayesian Method for Image-based Prediction, MICCAI, 2011 - sabuncumiccai2011.pdf

Nov 16

We will read “Connectivity-Informed fMRI Activation Detection” by Bernard Ng, Rafeef Abugharbieh, Gael Varoquaux, Jean Baptiste Poline and Bertrand Thirion. ng_connectivityinformedfmri_miccai11.pdf

Nov 2 and 9

We’ll read up on RKHSs. We’ll start with this tutorial: daume04rkhs.pdf Sections 1-5 are straightforward and should be mostly review, so we’ll just be discussing Section 6. There are a few typos:

At the end of 6.2, the first sentence of the last paragraph is more clear as “The property of reproducing kernels that we need is <f,K(x,\cdot)> = f(\cdot).”

The right-hand side of the equation in 6.4.1 should read $\lambda\phi(x)$ (instead of x^\prime).

The notes that George posted are also relevant; here are the notes and the blurb: On an unrelated note, here are some notes I hacked up about basics of Reproducing Kernel Hilbert Spaces and Kernel Ridge Regression: reading_group_prep_rkhs_2011_10_14.pdf This is NOT related to the 10/19 reading; it’s related to the 10/12 reading. This could be useful for when we return to talking about kernel methods. Update 10/13: A few minor typos have been corrected. Update 10/14: The proof for the closed-form solution for kernel ridge regression has been extended to allow for possibly non-invertible kernel matrices.

Oct 26

We will go through the details of INLA and discuss applications. Please read Section 4 in addition to the first part of the paper.

Here’s a write-up on the Laplace’s approximation from 6.437: laplace.pdf

Oct 19

We will read about integrated nested Laplace approximations (it’s a 74-page beast, but references begin on page 32 followed by discussions that happened after publication): Håvard Rue, Sara Martino, and Nicolas Chopin. “Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations.” JRSS 2009. inla.pdf

  • Update 10/18: Just read through the end of section 3; this amounts to ~13.3 pages of reading.
  • Update 10/19 1:38am: Notes I (George) will use today at reading group. It’s high-level and almost surely has bugs: reading_group_inla_notes_2011_10_19_0137.pdf

Oct 12

Nicolas Duchateau, Mathieu De Craene, Gemma Piella, and Alejandro F. Frangi. Characterizing Pathological Deviations from Normality using Constrained Manifold-Learning. miccai 2011. duchateau-miccai2011.pdf

Sep 14

Rómer Rosales and Brendan Frey. Learning Generative Models of Affinity Matrices: UAI 2003 rosales_frey_similarity.pdf

Summer 2011

August 23

Firdaus Janoos, Shantanu Singh, Raghu Machiraju, William M. Wells III, István Ákos Mórocz. State-Space Models of Mental Processes from fMRI. IPMI 2011. janoos-ipmi2011.pdf

August 16

Fani Deligianni, Gael Varoquaux, Bertrand Thirion, Emma Robinson, David J Sharp, A David Edwards, and Daniel Rueckert. A Probabilistic Framework to Infer Brain Functional Connectivity from Anatomical Connections. IPMI 2011. deligianni2011.pdf

August 9

Gaël Varoquaux, Alexandre Gramfort, Fabian Pedregosa, Vincent Michel, Bertrand Thirion. “Multi-subject Dictionary Learning to Segment an Atlas of Brain Spontaneous Activity.” IPMI 2011. ipmi2011-multi-subject-dict-learning-fmri.pdf

August 2

We’ll cover Stanley Durrleman, Marcel Prastawa, Guido Gerig, Sarang C. Joshi. “Optimal Data-Driven Sparse Parameterization of Diffeomorphisms for Population Analysis.” IPMI 2011. ipmi2011-sparse-diffeomorphisms.pdf

July 26

We will continue covering Wainwright’s notes on sparse linear models (i.e. same reading as July 12 and 19). In particular, we will blow up the plot on Page 92, work through the proof for Theorem 5.3 and Example 5.7, and finish discussion about pairwise incoherence and the restricted isometry property (RIP). As an illustrative example, I (George) will also discuss Exercise 5.2 at the end of the chapter (showing that pairwise incoherence implies the restricted nullspace property, without using RIP results). Non-trivial probabilistic results that are used in Example 5.7 and Exercise 5.2c will be briefly discussed.

Definition 5.3. and Theorem 5.3 are in hard-to-read shaded boxes, which are reproduced here: reading_group_july_26.pdf

Page 92 plot, blown up (1.7MB):

Update (July 24, 1:48pm): Notes from the previous reading group are now available: reading_group_2011_07_19_notes.pdf

Update (July 25, 11:43am): Preliminary supplemental notes for this week’s reading group are now available: reading_group_2011_07_26_notes_draft2.pdf

July 19

We will read the notes on sparse linear models, focusing on the conditions, their relationships, and the proofs. Try to come up with or keep in mind a roadmap of where to go from here as you read.

July 12

We will cover Martin Wainwright’s notes on sparse linear models up to and including section 5.3.2.


The gray boxes are kind of hard to read, so they’re reproduced in this supplement:


June 21st

We will continue the discussion on CCA. In addition to the June 14th reading, see

I. Rustandi Thesis, Predictive fMRI Analysis for Multiple Subjects and Multiple Studies - Section 3.3. cmu-cca-ch3.pdf (see section 3.3)

June 14th

D. R. Hardoon , S. Szedmak and J. Shawe-Taylor, Canonical correlation analysis; An overview with application to learning methods, T.R. 2003hardoon-03.pdf

Further reading

F.R. Bach and M.I. Jordan, A Probabilistic Interpretation of Canonical Correlation Analysis, T.R. 2005 bachjordan05.pdf

H. Hotelling, Relations Between Two Sets of Variates, Biometrika, 1936hotelling36.pdf

Spring 2011

May 16

Original Infomax paper. Bell and Sejnowski, Neural Computation, 1995. infomax.pdf

Cardoso ‘97 paper that connects Infomax to Max. Likelihood: cardoso1997_infomax_ml.pdf

May 9

We’ll talk about ICA. The paper will be:

ICA Tutorial (with sections devoted to Infomax ICA):

  • Independent Component Analysis: Algorithms and Applications, Aapo Hyvärinen and Erkki Oja, 2000. icatutorial.pdf

May 2

We’ll talk about an ICA method for evaluating fMRI and genetic data. The paper will be:

  • J.Liu et al, Combining fMRI and SNP data to investigate connections between brain function and genetics using parallel ICA, HBM 2009. 2009_liu-hbm.pdf

A relevant review paper (optional):

  • Calhoun et al. A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data, NeuroImage, 2009 calhoun2009.pdf

Apr 11 (no meeting Apr 4, we will have a talk instead)

We will be discussing “Joint Bayesian Cortical Sulci Recognition and Spatial Normalization” by Perrot et al. from IPMI 2009. perrot_ipmi_09.pdf

Mar 28 (no meeting March 21, spring break)

Gael Varoquaux, Alexandre Gramfort, Jean-Baptiste Poline, Bertrand Thirion; Brain covariance selection: better individual functional connectivity models using population prior. NIPS 2011 varoquax-fmri-nips10.pdf


Mar 14

We’ll look at “Fitting a graph to vector data” fitting_a_graph_to_vector_data.pdf, which gives an alternate approach to K-nearest neighbors and epsilon radius for creating the initial graph.

Feb 28 (no meeting on Feb 21 -- Presidents' Day)

We will look more closely at the derivation that leads to the eigenvector problem in one of the references in the paper we read last time. Specifically, we will discuss theorems 14.2.1 and 14.4.2 in Multivariate Analysis by Mardia, Kent and Bibby. multivariate_analysis.pdf

Proof that the SVD is the best low rank approximation and the exercise in Mardia svdproof.pdf

Remark: Given the discussion during reading group, it seems that a Cholesky decomposition would work. In particular, for real, positive semi-definite B, using Cholesky decomposition would produce lower-triangular, real-valued L such that:

B = L * L^T

As an example, if B is full-rank, then the columns of L would all be independent forming a basis (which may not be orthonormal; although we could say that the original data points also need not be orthonormal).

More information:

Feb 14

Actually, we will discuss Tenebaum, J.B., de Silva, V., and Langford, J.C., A global geometric framework for nonlinear dimensionality reduction, Science, 290, p 2319-2323, 2000. isomap.pdf

Contrary to the first notice, we will NOT discuss: R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. PNAS, 2005. geometric-diffusion1.pdf

This paper demonstrates gradient-based implementation of MDS (known as Sammon mapping):

JOHN W. SAMMON, A Nonlinear Mapping for Data Structure Analysis. IEEE TRANSACTIONS ON COMPUTERS, VOL. C-18, NO. 5, MAY 1969 mds-numerical.pdf

Feb 7

We will discuss a view of spectral embedding via diffusion processes. Please read carefully the first paper and go over the second paper. Both present similar approaches to the problem.

A wandom walk view of spectral segmentation. Meila and Shi, AIstats 2001. meila-aistats-2001.pdf.

R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. PNAS, 2005. geometric-diffusion1.pdf

U. Luxburg. A tutorial on spectral clustering. Statistics and Computing 2007. luxburg_spectralclustering_tutorial.pdf.

Jan 31

J. Shi, J. Malik: Normalized Cuts and Image Segmentation shimalik-normalizedcuts.pdf

We’ll mainly be focusing on section 2 - i.e. why it is that minimizing the normalized cut turns out to be a generalized eigenvalue problem.

We might also want to discuss how we can interpret this by forming a Markov chain on the graph, so I’ve added a relevant paper meila-aistats-2001.pdf.

Jan 25

We’ll be continuing Sharpnack’s paper on graph labeling, focusing on Theorem 1 and its proof:

J. Sharpnack, A. Singh; Identifying graph-structured activation patterns in networks sharpnack-graph-nips10.pdf

A video of the conference talk is here: (the talk doesn’t go into detail on the proofs, but does give a little bit of intuition for the terms in the theorem).

We’ll also look briefly at these papers on other spectral methods: shimalik-normalizedcuts.pdf ng-spectralclustering.pdf belkin-laplacianeigenmaps.pdf

Jan 18

We’ll be reading Sharpnack’s paper on graph labeling:

J. Sharpnack, A. Singh; Identifying graph-structured activation patterns in networks sharpnack-graph-nips10.pdf

A video of the conference talk is here:

January 11

We will take another look at the paper by M. Nielsen, L. Florack and R. Deriche nielsen_regscalespace_jmathimage1997.pdf through Section 5.

Fall 2010

Dec 2

We will clean up our discussion of the demons algorithms. The leftover items is the second-order optimization and the relationship between regularization and smoothing.

1) For the ESM method, we will look at section 3 of this paper:

Insight Into Efficient Image Registration Techniques and the Demons Algorithm. Tom Vercauteren, Xavier Pennec, Ezio Malis, Aymeric Perchant, and Nicholas Ayache. IPMI 2007. insighteffregdemons-ipmi07-vercauteren.pdf

2) For the regularization, we will look at these papers:

Nov 17

Diffeomorphic demons: Efficient non-parametric image registration. Tom Vercauteren, Xavier Pennec, Aymeric Perchant, Nicholas Ayache. NeuroImage, 2009. vercauteren-2009.pdf

The original demons paper: J.-P. Thirion, Image matching as a diffusion process: an analogy with Maxwell’s demons. Medical Image Analysis (1998), volume 2, number 3. thirion98.pdf

Nov 10

From MICCAI, 2010: L. Risser, F.-X. Vialard, R. Wolz, D. Holm, D. Rueckert; Simultaneous Fine and Coarse Diffeomorphic Registration: Application to Atrophy Measurement in Alzheimer’s Disease; MICCAI 2010 miccai2010_risser.pdf

Nov 3

From MICCAI, 2010: P. Risholm, S. Pieper, E. Samset and W. M. Wells; Summarizing and visualizing uncertainty in non-rigid registration miccai2010_risholm.pdf

Background paper: P. Risholm et al. Bayesian Estimation of Deformation and Elastic Parameters in Non-rigid Registration wibr2010_risholm.pdf

Oct 27

From MICCAI, 2010: G. Varoquaux, F. Baronnet, A. Kleinschmidt, P. Fillard, and B. Thirion; Detection of Brain Functional-Connectivity Difference in Post-stroke Patients Using Group-Level Covariance Modeling varoquaux_grouplevelcovariance_miccai10.pdf

Oct 20

We will read Chapter 22 “Analysis of variance” (ANOVA) of Andrew Gelman’s “Data Analysis Using Regression and Multilevel/Hierarchical Models” book gelman.pdf. In particular, we will just be looking at pages 487-501 in the book (PDF pages 513-527). Update 10/15/10 9:04PM: The high quality PDF of just Chapter 22 is now available gelman-ch22.pdf.

Notes by George: reading-group-anova-2010-10-20-notes.pdf

Oct 13

We will read Bernardo and Smith’s coverage of reference priors. The relevant section from the book is here: bernardo_smith_referencepriors.pdf.

  • Pages 298-316 (PDF pages 1-19) cover the basics of 1-dimensional reference priors,
  • pages 316-320 (PDF pages 19-23) cover restricted reference priors (from a restricted family of distributions),
  • pages 320-333 (PDF pages 23-36) cover reference priors with nuisance parameters,
  • and pages 333-339 (PDF pages 36-42) cover the multidimensional case.

Summer 2010

Aug 17-31

The next three meetings will provide an overview of Multilevel Regression and ANOVA using the book, “Data analysis using regression and multilevel/hierarchical models” by Andrew Gelman. The chapters posted below come from the publisher’s website and have a few formatting issues. A short summary of these issues has been made format_errors.pdf

The material for the first 2 weeks comes from Part 2A gelman_pt2a.pdf

Aug 17th: Please read chapter 12, pgs.251-276, as an introduction to multilevel linear models. Also, please read pgs.244-247 from chapter 11, which attempts to explain the difference between “random” and “fixed” effects.

Aug 24th and 31st (pick one to attend): Please read chapter 13, pgs.279-297, which covers some more complex multilevel linear models with varying slopes and “non-nested” data.

Sep 7th: Multilevel regression applied to fMRI. Please read woolrich_behrens_multilevel.pdf

Aug 3

Gerber et al. Manifold modeling for brain population analysis. Medical Image Analysis 14 (2010) 643–653. gerber-media-2010.pdf

Aug 10

Jia et al. ABSORB: Atlas building by self-organized registration and bundling. NeuroImage 51 (2010) 1057–1070. jia_neuroimage_2010.pdf

July 13 - July 27

We will start a 3-4 week overview of Neural Networks and Deep Belief Networks. The rough schedule is as follows:

July 13th: We will introduce the standard Neural Network setup, as presented in Bishop’s “Pattern Recognition and Machine Learning” bishop_patternrecognitionandmachinelearning_ch5.pdf. Please read Sections 5.1 and 5.2. In addition, please skim 5.3.1 and 5.3.2, as we may get started on Back-Propagation.

July 20th: We will talk about back propagation. Please read Section 5.3 of Bishop and the original Back-Propogation paper by Rumelhart rumelhart_backpropagatingerrors_nature86.pdf). If time permits, we may go over computing the Hessian (Section 5.4 of Bishop).

July 27th: We will discuss Hinton’s learning algorithm for Deep Belief networks hinton_fastlearningdeepbelief_neuralcomp06.pdf.

Spring 2010

May 17

We will finish the LP paper from Mar 29.

May 10

We will discuss the meanfield solvers for MRFs.

Background from Wanmei’s thesis: meanfield.pdf

Exerts from course notes: variational.pdf

Apr 26

Graph cuts for energy minimization; Min Cut Max Flow:

Yuri Boykov, Olga Veksler, and Ramin Zabih. Fast Approximate Energy Minimization via Graph Cuts. IEEE PAMI 23(11), 2001. boykov2001.pdf

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein. Min Cut Max Flow and the Ford-Fulkerson method. Introduction to Algorithms, Second Edition minmax.pdf

Apr 12

We will discuss primal-dual algorithms for solving linear programs. The first chapter of “Primal-Dual Interior-Point Methods” by Stephen Wright provides a good overview of the subject: wright_primaldual_1997.pdf

The tutorial on convex optimization by Haitham Hindi is a more complete overview of duality and optimality conditions: hindi_tutorialconvexopt.pdf

Apr 5

We will discuss primal-dual formulation in more detail. See pages 215-227 and 232-234 of Convex Optimization by Boyd and Vandenberghe: boyd_duality.pdf

A PDF of the full book is available at Book pages 215-227,232-234 correspond to PDF pages 229-241,246-248.

We will also try to finish the labeling from last time.

Mar 29

Nikos Komodakis and Georgios Tziritas: Approximate Labeling via Graph Cuts Based on Linear Programming. in PAMI 2007 komodakis07linprog.pdf

we will stick to the PAMI paper, but a concise listing of the most important ideas is in: Nikos Komodakis et al. ICCV 2005 komodakis_iccv2005.pdf External Link

Feb 22

Olivier Commowick, Simon K. Warfield, and Gregoire Malandain. Using Frankenstein’s Creature Paradigm to Build a Patient Specific Atlas; MICCAI 2009 miccai2009_commowick.pdf

Feb 16 (monday schedule for classes)

We continued the discussion of the paper from the previous week.

Feb 8

Nematollah Batmanghelich, Ben Taskar, and Christos Davatzikos. A general and unifying framework for feature construction in image-based patter classification; IPMI 2009 ipmi2009_batmanghelich.pdf

Jan 26

Davis, B.C., Fletcher, P.T., Bullitt, E., Joshi, S. Population Shape Regression From Random Design Data. ICCV 2007. davis-iccv2007.pdf

Jan 19

T. Rohlfing et al. Regression models for atlas appearance. IPMI 2009. ipmi2009_rohlfing.pdf

Fall 2009

Nov 17

Ou; DRAMMS: Deformable registration via Attribute Matching and mutual-saliency weighting; IPMI 2009 ipmi2009_ou.pdf

Oct 27

Criminisi; Decision Forests with Long-Range Spatial Context for Organ Localization in CT Volumes, MICCAI/PMMIA 2009 criminisipmmia2009_decisionforests.pdf

Additional reading:

Breimann 01 Random Forests: breiman01randomforests.pdf

Oct 13

Murgasova, A spatio-temporaral atlas of the growing brain for fMRI studies, MICCAI/IADB 2009 murgasova09development.pdf

Oct 6

D. R. Hardoon , S. Szedmak and J. Shawe-Taylor, Canonical correlation analysis; An overview with application to learning methods, T.R. 2003hardoon-03.pdf

Further reading

H. Hotelling, Relations Between Two Sets of Variates, Biometrika, 1936hotelling36.pdf

F.R. Bach and M.I. Jordan, A Probabilistic Interpretation of Canonical Correlation Analysis, T.R. 2005 bachjordan05.pdf

Sep 29

We will continue discussing the Elastic Net paper by Zou and Hastie: elastic_net_zou_hastie_2005.pdf

Sep 15

We will discuss the Elastic Net paper by Zou and Hastie: elastic_net_zou_hastie_2005.pdf

Sep 1

Will finish the sparse PCA paper.

Summer 2009

Aug 25

We will discuss the Sparse PCA paper by Zou, Hastie, and Tibshirani: sparsepca.pdf

The published version of the paper (which we’ll discuss) is here: spca_jcgs.pdf

Aug 18

We continued discussing the IBP papers.

Aug 11

Indian Buffet Process: we will discuss

Griffith and Ghahramani, Infinite latent feature models and the Indian buffet process, NIPS 2006. ibp-1.pdf

There are two other important papers:

IBP and the hierarchical Beta processes. hbp-2.pdf

Stick-breaking Construction for the IBP. stickbreakingibp.pdf

July 21 and 28

We will discuss

Discriminative Shape Alignment. Marco Loog and Marleen de Bruijne. IPMI 2009. loog-ipmi2009.pdf

Thomas will lead the discussion.

Proof of complex representations: procrusts-complex-representation.pdf

Two tutorials found by Tammy: procrustes-ch-04.pdf procrustes_tutorial.pdf

June 16

We will be discussing Stochastic Tractography Friman, Bayesian Stochastic Tractography. This paper references the work by Behrens et. al. Behrens, Propagation of Uncertainty. Since both papers are fairly straightforward, if time permits, we will talk about both models.

June 9

We discussed the PAMI paper during our Reading Group; the MICCAI paper was listed for further reading.

Finsler Active Contours, J. Melonakos, E. Pichon, S. Angenent, A. Tannenbaum, IEEE PAMI 2008.

Finsler Tractography for White Matter Connectivity Analysis of the Cingulum Bundle J. Melonakos, V. Mohan, M. Niethammer, K. Smith, M. Kubicki, A. Tannenbaum, MICCAI 2007.

June 2

We used the MICCAI paper as the basis of the discussion; the thesis has much more detail.

MCMC Curve Sampling for Image Segmentation, Fan, A., J. Fisher, W. Wells, J. Levitt, A. Willsky, MICCAI 2007.

Curve Sampling and Geometric Conditional Simulation, Fan, A., MIT PhD Thesis, Nov. 2007.

Spring 2009

May 12

Dirichlet Processes (contd.) We plan to read and understand the variational approximations from Kurihara et al.

Kurihara, Welling, Teh: Collapsed Variational Dirichlet Process Mixture Models cvdp.pdf.

May 5

Dirichlet Processes (contd.) We are continuing with the hierarchical Dirichlet processes from Teh et al.

April 21 (no meeting April 7, 14)

Dirichlet Processes. We plan to read and understand Teh et al. But for the first session, we will spend time going through the basics of the nonparametric methods from the tutorial dirichlet_processes.pdf.

Here is some brief notes on DPs: dp_brief_notes.pdf.

Teh, Jordan, Beal, Blei: Hierarchical Dirichlet Processes hierarchical_nonparametric.pdf

March 31 (no meeting March 24 -- springbreak)

We will finish the shapes paper from last time.

March 17

E. Klassen, A. Srivastava, W. Mio, and S. Joshi, Analysis of Planar Shapes Using Geodesic Paths in Shape Spaces, PAMI 2004 klassensrivastavamiojoshipami04.pdf

Feb 24

We will start by finishing the paper from last time. The homework is to derive the coefficients from the next level of splines from the coefficients of the previous level. We will then discuss the thin-plate splines.

Principal Warps: Thin-Plate Splines and the Decomposition of Deformations. FRED L. BOOKSTEIN. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. VOL.II. NO. 6. JUNE 1989. bookstein-89.pdf

Wanmei and Mert will lead the discussion.

Feb 10

We will finish the registration paper and discuss multi-level splines in a bit more detail. Michal will lead the discussion on the registration paper.

Scattered Data Interpolation with Multilevel B-Splines. Seungyong Lee, George Wolberg, and Sung Yong Shin. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 3, NO. 3, JULY–SEPTEMBER 1997. bsplines-2.pdf

Archana will lead the discussion on splines.

Feb 3

Nonrigid Registration Using Free-Form Deformations: Application to Breast MR Images. D. Rueckert, L. I. Sonoda, C. Hayes, D. L. G. Hill, M. O. Leach, and D. J. Hawkes. IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 18, NO. 8, AUGUST 1999. rueckert-ffd.pdf

These are several background papers on splines:

Surface Fitting with Hierarchical Splines DAVID R. FORSEY and RICHARD H. BARTELS. ACM Transactions on Graphics, Vol. 14, No. 2, April 1995, Pages 134-161. bsplines-1.pdf

Scattered Data Interpolation with Multilevel B-Splines. Seungyong Lee, George Wolberg, and Sung Yong Shin. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. 3, NO. 3, JULY–SEPTEMBER 1997. bsplines-2.pdf

Principal Warps: Thin-Plate Splines and the Decomposition of Deformations. FRED L. BOOKSTEIN. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE. VOL.II. NO. 6. JUNE 1989. bookstein-89.pdf

Fall 2008

Dec 16

K. K. Bhatia, P. Aljabar, J. P. Boardman, L. Srinivasan, M. Murgasova, S. J. Counsell, M. A. Rutherford, J. Hajnal, A. D. Edwards, and D. Rueckert, Groupwise Combined Segmentation and Registration for Atlas Construction, MICCAI 2007 bahtiamiccai07.pdf

Dec 12 (note unnusual day, still 12-1:30)

We will go over section V of the same paper.

Dec 2

We will work on details of geometry of the cost function and Lemma 2 in the sparse bayesian learning paper.

Nov 25 -- no meeting (Thanksgiving week)

Nov 18

David P. Wipf and Bhaskar D. Rao. Sparse Bayesian Learning for Basis Selection. IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2004. 2004-wipf-ieeesigproc.pdf has some theoretical explanation about why the sparse Bayesian learning is indeed sparse.

Section II shows the formulation, which is the same as Tipping’s ARD paper, and an EM algorithm to optimize the objective function.

Section III and IV are the theoretical analysis of why the model favors sparse solution. In particular, IV A proposes a very interesting proof.

Nov 11 -- no meeting, Veterans' day

Nov 4

We will continue with the currents.

Oct 21

S. Durrleman, X. Pennec, A. Trouve, P. Thompson, N. Ayache. Inferring brain variability from diffeomorphic deformations of currents: An integrative approach durrleman_currents.pdf

M. Vaillant and J. Glaunes” Surface Matching via Currents vaillant_currents.pdf

Oct 14

S. Durrleman, X. Pennec, A. Trouve, P. Thompson, N. Ayache. Inferring brain variability from diffeomorphic deformations of currents: An integrative approach durrleman_currents.pdf

Oct 7

We are going to read Tipping’s paper on Sparse Bayesian Learning
tipping2003_fastmarginallikelihoodmaximisationforsparcemachinemodels.pdf. It is closely related to some other learning schemes.

2001-tipping-jmlr.pdf lays out the fundamental concepts for sparse Bayesian learning, and it may assist you to understand the above paper.

2004-wipf-ieeesigproc.pdf has some theoretical explanation about why the sparse Bayesian learning is indeed sparse.

Sep 23

Using the logarithm of odds to define a vector space on probabilistic atlases. Kilian M. Pohl , John Fisher , Sylvain Bouix , Martha Shenton , Robert W. McCarley , W. Eric L. Grimson , Ron Kikinis , William M. Wells Medical Image Analysis 2007. pohl-media-2007.pdf

Sep 16

Label Space: A Multi-object Shape Representation. James Malcolm, Yogesh Rathi, and Allen Tannenbaum. MICCAI 2008. malcolm-miccai08.pdf

Summer 2008

Aug 27 (just this time, 11-12:30)

Salima Makni, Philippe Ciuciu, Jérôme Idier, and Jean-Baptiste Poline. Joint Detection-Estimation of Brain Activity in Functional MRI: A Multichannel Deconvolution Solution. IEEE Trans. Sig. Proc., 53(9):3488-502, Sept. 2005. makni-tsp.pdf

Aug 20

Davis, B.C., Fletcher, P.T., Bullitt, E., Joshi, S. Population Shape Regression From Random Design Data. ICCV 2007. davis-iccv2007.pdf

Aug 13

Yonggang Shi, Zhuowen Tu, Allan L. Reiss, Rebecca A. Dutton, Agatha D. Lee, Albert M. Galaburda, Ivo D. Dinov, Paul M. Thompson, Arthur W. Toga: Joint Sulci Detection Using Graphical Models and Boosted Priors. IPMI 2007. shi-ipmi07.pdf

Aug 6

We will finish the GLM paper this time.

July 30

Friston, L., et al.: Assessing the significance of focal activations using their spatial extent. Human Brain Mapping, 1994. friston_focalactivations.pdf

July 23

Friston, K., et al.: Statistical parametric maps in functional imaging: A general linear approach. Human Brain Mapping, 1995.friston_glm.pdf

Worsley, KJ, Friston, KJ.: Analysis of fMRI Time-Series Revisited—Again. Neuroimage, 1995. worsley_fmriagain.pdf

Myers, RH, Montgomery, DC. A tutorial on Generalized Linear Models. Journal of Quality Technology, 1997. myers_tutorialonglm.pdf

July 16

We will finish the paper from last week.

July 9

Joshua E. Cates, P. Thomas Fletcher, Martin Andreas Styner, Martha Elizabeth Shenton, Ross T. Whitaker: Shape Modeling and Analysis with Entropy-Based Particle Systems. IPMI 2007. cates-ipmi07.pdf

July 1 (Tuesday, 12-1:30, just this one time)

We will continue to discuss the LDDMM paper. The goal is to find a geometric interpretation of the first term in the sum in eq. 4, finish the theorem and talk about the metric (Sec 6).

June 18

We will discuss the computation behind LDDMM beg-lddmm.pdf. It will be an easier start than understanding the more theoretical underpinnings of the other LDDMM papers.

Note that the first lemma is the hardest part of the paper, but things get a lot easier after that.

June 4 (12:30-2pm)

We will discuss the Nystrom Method, which can be used to approximate the eigendecomposition of large matrices. An application of this technique to Spectral Clustering is presented in Fowlkes et al. fowlkes_spectralgrouping_nystrom.pdf

Spring 2008

May 13

We will discuss Rao’s paper on the uniqueness of the decomposition into a Gaussian and a non-Gausian part. New paper 1966-rao-sankhyasera.pdf.

May 6

We will discuss Rao’s paper on the uniqueness of the decomposition into a Gaussian and a non-Gausian part 1969-rao-annmathstatist.pdf.

Apr 28 (Monday, 12:30-1:30)

We will continue with the ICA paper.

Apr 22

We will discuss Beckmann et al.’s Probabilistic ICA for fMRI paper beckmann2003.pdf

Apr 15

We will continue with the LDA discussion.

Apr 9 (Wednesday, 2:30pm)

We will discuss Blei et al.’s Latent Dirichlet Allocation paper bleingjordan2003.pdf

We will also discuss this.

Apr 2 (Wednesday, 2:30pm)

We will discuss Lilla Zollei’s paper. A Marginalized MAP Approach and EM Optimization for Pair-Wise Registration: 2007-zollei-ipmi.pdf

Mar 18

We will continue on the Wisharts paper; the homework is to get more comfortable with that particular distribution.

Mar 11

Bing Jian, Baba C. Vemuri: Multi-fiber Reconstruction from Diffusion MRI Using Mixture of Wisharts and Sparse Deconvolution. IPMI 2007. jian-vemuri-ipmi07.pdf

Anastasia will lead the discussion.

Feb 26

Schapire. Boosting overview schapire_msri.pdf


Additional papers:

Jerome Friedman, Trevor Hastie, Robert Tibshirani. Additive Logistic Regression: a Statistical View of Boosting (1998). friedman98additive.pdf

Torralba, Murphy and Freeman. Sharing visual features for multiclass and multiview object detection. sharing.pdf

Feb 19

Boosting Image Retrieval (Tieu, Viola, IJCV 2004) tieu_boostingimageretrieval.pdf

Feb 12

We will continue the discussion about the paper from last meeting.

Feb 1 (Friday)

Chris McIntosh and Ghassan Hamarneh. Is a Single Energy Functional Sufficient? Adaptive Energy Functionals and Automatic Initialization. MICCAI 2007. mcintosh-miccai2007.pdf

Fall 2007

Jan 22 (Tuesday)

Shaohua Kevin Zhou and Dorin Comaniciu. Shape Regression Machine. IPMI 2007. zhou-ipmi2007.pdf

Jan 15 (Tuesday)

We will discuss the two simple examples defined in last meeting: a closed curve and an open curve. Please work through the examples and come with beatiful matlab figures of the embedding.

We will also talk about wavelets. The first paper is what we already looked at; the other two are longer, more detailed versions.

R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Multiscale methods. PNAS, 2005. geometric_diffusion2.pdf

Additional papers on the topic: Diffusion wavelets and their use in spectral clustering: nadler06.pdf coifmanmaggoni06.pdf

Dec 10 (Mon, 9-10am)

Laplacian-Eigenmaps by Belkin and Niyogilaplacianeigenmaps.pdf

Further papers on discrete Laplace-Beltrami Operators (overview): xu-04-discretelaplace.pdf wardetzky-07-discretelbo.pdf

Dec 5 (Wed, 2:30-3:30 since Polina is away at NIPS on Monday)

We will finish the diffusion map discussion and will talk about the second paper. The week after that, we will come back to the operators in the first paper.

Nov 26

We will start by discussing the first paper and go on to the second one. You only need to read the first paper for this meeting, but we will end up reading both by the end of this series.

R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. PNAS, 2005. geometric-diffusion1.pdf

R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Multiscale methods. PNAS, 2005. geometric_diffusion2.pdf

Additional papers on the topic: Diffusion wavelets and their use in spectral clustering: nadler06.pdf coifmanmaggoni06.pdf

Nov 19

We will finish the fast diff. paper. Mert and Thomas will lead.

Nov 14 (Wednesday, since Monday is Veterans Day)

Fast diffeomorphic registration. Thomas will lead the discussion.


Oct 17 (Wednesday, to work around MMBIA)

We will finish Mahony’s paper.

Oct 10 (Wednesday since Monday, Oct 8 is Columbus Day)

Lie groups tutorial.

Oct 1

Continuing on the basic differential geometric notions required for understanding Mahony’s paper, we will review 5th chapter of:

Wolfgang Kuhnel and Bruce Hunt, Differential Geometry: Curves - Surfaces - Manifolds, AMS, Second Edition, 2005.

Please check the discussion for mote details.


Sep 24

Danial will present: Mahony, R. Manton, J. H., ``The Geometry of the Newton Method on Non-Compact Lie Groups,’’ JOURNAL OF GLOBAL OPTIMIZATION, 2002, VOL 23; NUMBER 3, pages 309–327. mahony.pdf


Sep 17

J.-P. Thirion, Image matching as a diffusion process: an analogy with Maxwell’s demons. Medical Image Analysis (1998), volume 2, number 3. Biz will present. thirion98.pdf

Sep 12

Tom Vercauteren, Xavier Pennec, Aymeric Perchant, Nicholas Ayache. Non-parametric Diffeomorphic Image Registration with the Demons Algorithm. MICCAI 2007.


Summer 2007

Aug 29 (Wed)

Min Cut Max Flow and the Ford-Fulkerson method.

Introduction to Algorithms, Second Edition Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein.


Aug 16 (Thu)

Graph cuts for energy minimization. Koen will lead the discussion.

Yuri Boykov, Olga Veksler, and Ramin Zabih. Fast Approximate Energy Minimization via Graph Cuts. IEEE PAMI 23(11), 2001.


July 25 (Wed)

Finish the BP paper.

July 19 (Thu)

Belief propagation

Our main reference is: Constructing free-energy approximations and generalized belief propagation algorithms: J. Yedidia, et al. yedidiafreemanweiss2005.pdf

The basic mathematical motivation of belief propagation: A. M. Aji and J. McEliece gendislaw.pdf

BP is used for learning problems on different types of graphs: Bayesian Networks, Markov Random Fields, Junction graphs, and Factor graphs. To see examples: Y. Weiss and W. T. Freeman max-product_optimality.pdf and F. R. Kschischang, et al. sum-product.pdf

And a couple of other popular review papers by the same authors as of our main paper: genbp.pdf and understandingbp.pdf.

June 27 (Wed)

William T. Freeman, Thouis R. Jones, and Egon C. Pasztor, Example-based super-resolution, IEEE Computer Graphics and Applications, March/April, 2002. cgasres.pdf

W. T. Freeman, E. C. Pasztor, O. T. Carmichael Learning Low-Level Vision International Journal of Computer Vision, 40(1), pp. 25-47, 2000. MERL-TR2000-05. tr2000-05.pdf

June 20 (Wed)

We will continue our discussion on projection pursuit and its connection to Infomax ICA.

Original Infomax paper:

InfoMax ICA algorithm and its connection to projection pursuit. An information-maximization approach to blind separation and blind deconvolution. Bell and Sejnowski, Neural Computation, 1995. infomax.pdf

ICA Tutorial (with sections devoted to Infomax ICA):

Independent Component Analysis: Algorithms and Applications, Aapo Hyvärinen and Erkki Oja, 2000. icatutorial.pdf

June 6 (Wed)

Projection Pursuit:

Please read the first 17 pages of the following paper.

M. C. Jones; Robin Sibson, What is Projection Pursuit?, Journal of the Royal Statistical Society. Series A (General), Vol. 150, No. 1. (1987), pp. 1-37. jones87.pdf

A much longer, more detailed review paper is the below.

Peter Huber,Projection Pursuit,The annals of Statistics, Vol 13, No2 (June 1985), pp. 435-475. huber-1.pdf

Spring 2007

May 11

MM algorithms: Hunter and Lange, A Tutorial on MM Algorithms, Am. Stat. 58(1):30-7, Feb. 2004. Clear Version mm_tutorial.pdf

Hunter and Lange, A Tutorial on MM Algorithms, Am. Stat. 58(1):30-7, Feb. 2004. lange_04_amstat.pdf

Jacobson and Fessler, An Expanded Theoretical Treatment of Iteration-Dependent Majorize-Minimize Algorithms, preprint. jacobson,tip.pdf

May 4

“Information Theoretic Coclustering,” by Inderjit S. Dhillon, Subramanyam Mallela, and Dharmendra S. Modha p89-dhillon.pdf

Apr 27

“A Log-Euclidean Polyaffine Framework for Locally Rigid or Affine Registration” by Vincent Arsigny, Olivier Commowick, Xavier Pennec, and Nicholas Ayache logeuclidean_wbir.pdf

Apr 20

Finish the CG.

Apr 13

Conjugate gradient algorithm.

Jonathan Richard Shewchuk. An Introduction to the Conjugate Gradient Method Without the Agonizing Pain. This is a nice (although long) tutorial paper: painless-conjugate-gradient.pdf

A (shorter) section from the numerical recipes book: c10-6.pdf

Mar 2

We will finish the paper.

Feb 23

We will continue with the same paper. Please read Section 4 and the appendices. It’s heavy reading; you might want to start early.

Feb 16

Danial: The following paper makes a nice connection between exponential-family-mixture-model and distance-measure-based clustering methods.

A. Banerjee et al. Clustering with Bregman Divergence. J Mach Learn Res 6 (2005).

Please read the first three sections for the first meeting. We will go through the key concepts: Bregman divergence, and information in more detail and try to understand the Bregman hard clustering algorithm.


Fall 2006

December 12

Mert will lead the discussion on registration of fMRI.

B. Thirion et al. Improving Sensitivity and Reliability of fMRI Group Studies through High Level Combination of Individual Subjects Results. MMBIA 2006.


December 5

Serdar will lead the discussion on the multi-modal (not in a classical sense) nature of atlases:

Daniel J. Blezek and James V. Miller. Atlas Stratification. MICCAI 2006.


November 21

More Sobolev Active Contours this week...

November 14

Sobolev Active Contours: sobolevactivecontours.pdf


November 7

We will continue our asymptotic quest. Let’s hope this process converges to the true value of the parameters!

Here is my last version of summary: Reading: summarydoob.pdf

October 31

We will continue the discussion of Wilks’ paper on the asymptotics of LR. In order to make sense of the main assumption of equation (3), we have to go through the reference to Doob’s paper which is a seminal derivation of many basic theorems.

Reading: doob1948.pdf


October 24

Reading: leemput_miccai2006.pdf


October 16

Papoulis (pp. 275–278) is a good introductory note on asymptotics of hypothesis testing (photocopies outside Polina’s office). See also his statistics chapter (pp. 241–282) for hypothesis testing in general. The original papers for the asymptotics of the likelihood ratio are:




September 19

We will continue the paper from last week.

We also made a note that we need to look into asympotic statistics results (mentioned in the paper) in the future. Kinh might take a lead on that.

Here’s the original DCM paper, for people who want more detail:


September 12

Note the special time: 10:00am

Penny W.D. et al. (2004). Modelling functional integration: a comparison of structural equation and dynamic causal models.


We will focus on the structural equation model this week.

September 5

Note the special time: 4pm

We will continue our discussion on effective connectivity in neuroimaging. We will focus on the second half of the first paper from last time (Friston 1994). If time is allowed, we will discuss the third paper from last time as well (Friston 1997).

August 29

We will discuss functional and effective connectivity in neuroimaging. The first paper is the most general paper. The second paper is a book chapter version of the first paper on functional connectivity and goes a bit deeper. The third paper deals with more exotic topic in effective connectivity. We will mainly discuss the first paper, so if you have limited time, the first one is the paper to read.

Friston, K.J. (1994). Functional and effective connectivity in neuroimaging: A synthesis. Human Brain Mapping, 2, 56-78.


Friston, K.J, Büchel, C (1993). Functional Connectivity: Eigenimages and multivariate analyses.


Friston, K.J., Buchel, C., Fink, G.R., Morris, J., Rolls, E., and Dolan, R. (1997). Psychophysiological and modulatory interactions in Neuroimaging. NeuroImage, 6, 218-229.


August 22

We will start the meetings with two papers that Wanmei claims use the same model in two somewhat unrelated applications. Both papers consider the fundamental problem of evidence integration from independent sources.

We will discuss the details of the models and the relationship between them. If you have time to read just one paper, the first one is lighter.

Genovese, C. R., Noll, D. C. and Eddy, W. F. (1997). Estimating Test-Retest Reliability in fMRI I: Statistical Methodology, Magnetic Resonance in Medicine, 38, 497–507.


Simultaneous Truth and Performance Level Estimation (STAPLE): An Algorithm for the Validation of Image Segmentation. Warfield S, Zou KH, Wells WM. IEEE Trans Med Imag 2004; 23:903-921.


Spring 2006

May 10

We will discuss ICA for fMRI time series analysis. The longer paper discusses the method in details, and will be the basis for our discussion. The shorter one is a nice overview.



May 3

We will discuss the Dirichlet process mixture model, as presented in Teh 04, , which develops a variant of the DP mixture for grouped data. Other papers (all cited by Teh) are provided here for those interested in a deeper theoretical background: Ferguson 73 and Antoniak 74 are the seminal papers (with fairly technical measure-theoretic proofs), while Sethuraman 94, followed by Ishwaran and James 01 and Ishwaran and Zarepour 02, are more constructive. Of these, I’d recommend Sethuraman for clarity and brevity. – John

Yee Whye Teh et al.
Hierarchical Dirichlet Processes

Thomas Ferguson
A Bayesian Analysis of Some Nonparametric Problems

Charles Antoniak
Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems

Jayaram Sethuraman
A Constructive Definition of Dirichlet Priors

Hemant Ishwaran and Lancelot James
Gibbs Sampling Methods for Stick-Breaking Priors

Hemant Ishwaran and Mahmoud Zarepour
Exact and approximate sum-representations for the Dirichlet process

April 26

We will continue the discussion from the last time. I posted some questions in the Discussion section. Feel free to add comments and more questions. – Polina


April 19

We will discuss the Information Bottleneck Method and its uses in determining the number of clusters in fMRI data. The second paper describes the information bottleneck approach and is a background reading for the first paper.

Bertrand Thirion, Olivier Faugeras. Feature Detection in fMRI Data: The Information Bottleneck Approach. thirionmedia.pdf

Naftali Tishby, Fernando C. Pereira, William Bialek. The Information Bottleneck Method
Uncut Version:

April 13

I (Thomas) am posting the writeups for Apr 12. Gheorghe has also kindly provided his short writeup on EM. Happy Reading!!


April 12

I (Thomas) will be presenting the papers for this week.

Andrew Ng Michael Jordan and Yair Weiss
On Spectral Clustering: An analysis and an algorithm

Optional Papers: I will talk about mixture fitting, but I will not be completely following Michael Collins’ derivations. However, the tutorial is still nice if you have no experience with mixture fitting or EM. We will NOT go over the theorems about convergence, i.e. we will not go beyond section 3.3. The idea is to give everyone an intuitive feel about the concept of mixture fitting, not limiting oneself to using only gaussians or the EM technique.

Michael Collins

More Variants of Spectral Clustering:
Jianbo Shi and Jitendra Malik
Mikhail Belkin and Partha Niyogi
Marina Meila and Jianbo Shi

April 5

We will take a two week break and resume our discussion of FDR at the next meeting.

We’ll start with Efron, B. and Tibshirani, R.
Empirical Bayes Methods and False Discovery Rates for Microarrays
Genetic Epidemiology 23:70-86, 2002.

And then move onto the Genovese et al. paper on thresholding maps in neuroimaging using FDR nicholsfdr.pdf (listed below).

A comparison of several similar methods (FDR, pFDR, PER, PFP) can be found in
K. Manly, D. Nettleton, and J.T.G. Hwang
Genomics, Prior Probability, and Statistical Tests of Multiple Hypotheses
Genome Research 14: 997-1001, 2004 manlyetal.pdf
I might touch on some of these different options during the discussion.

The PFP paper was
Controlling the Proportion of False Positives in Multiple Dependent Tests
R. L. Fernando, D. Nettletonb, B. R. Southey, J. C. M. Dekkers, M. F. Rothschild, and M. Soller
Genetics, Vol. 166, 611-619, January 2004


March 15

Benjamini, Y. and Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B 57 289-300, 1995. benjaminifdr.pdf

Christopher R. Genovese, Nicole A. Lazar, Thomas Nichols. Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate. NeuroImage 15:870-878, 2002. nicholsfdr.pdf

Yoav Benjamini and Daniel Yekutieli. THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY. The Annals of Statistics 2001, Vol. 29, No. 4, 1165-1188.


I think it might be worth switching to one of the pFDR papers for the extensions of the FDR. I’m reading through some of them, and will post suggestions if one of them strikes me. In particular, I think they have a nicer treatment of independence, and it will be nice to see some other approaches to FDR. -Ray


March 8

We will continue with the permutation testing.

We also finshed the proof that the sample mean and the sample variance are independent for Gaussian iid case.

See Discussion for additional notes.


March 1

Thomas E. Nichols and Andrew P. Holmes.
Nonparametric Permutation Tests For Functional Neuroimaging: A Primer with Examples.
Human Brain Mapping 15:1-25(2001).

February 22

Wanmei Ou will define the general setup of the fMRI detection problem, preparing us for the series of papers in fMRI analysis.

Notes for the group meeting.



February 15

Michael Siracusa will lead a discussion on a classical hypothesis testing, in preparation for some papers in fMRI and DTI statistics.

Notes for the group meeting. hyptest-faq.pdf


Fall 2005

December 13

Duygu Tosun and Jerry L. Prince.
Cortical Surface Alignment Using Geometry Driven Multispectral Optical Flow,
Information Processing in Medical Imaging (IPMI), Colorado, USA, July 11-15, 2005.

Additional papers: Duygu Tosun, Maryam E. Rettmann, Jerry L. Prince.
Mapping techniques for aligning sulci across multiple brains.
Medical Image Analysis 8 (2004) 295-309.

Xiao Han, Dzung L. Pham, Duygu Tosun, Maryam E. Rettmann, Chenyang Xu, and Jerry L. Prince.
CRUISE: Cortical reconstruction using implicit surface evolution.
NeuroImage 23 (2004) 997-1012.

Xiao Han, Chenyang Xu, and Jerry L. Prince.
A Topology Preserving Level Set Method for Geometric Deformable Models.
PAMI, VOL. 25, NO. 6, JUNE 2003.

Also on the list for future reading: Thompson, P. Toga, A.W.
A surface-based technique for warping three-dimensional images of the brain.
IEEE Transactions on Medical Imaging, Volume 15, Issue 4, 402 - 417, 1996.

Additional papers:

Xianfeng Gu; Yalin Wang; Chan, T.F.; Thompson, P.M.; Shing-Tung Yau;
Genus zero surface conformal mapping and its application to brain surface mapping.
IEEE Transactions on Medical Imaging, Volume 23, Issue 8, 949 - 958, 2004.

December 6

Bruce Fischl, Martin I. Sereno and Anders M. Dale.
Cortical Surface-Based Analysis: II: Inflation, Flattening, and a Surface-Based Coordinate System.
Neuroimage. 9(2):195-207. 1999.

Bruce Fischl, Martin I. Sereno, Roger B.H. Tootell, Anders M. Dale.
High-resolution inter-subject averaging and a coordinate system for the cortical surface.
Human Brain Mapping, Volume 8, Issue 4, Pages 272 - 284, 1999. fischl99a.pdf.gz

Additional papers:

Anders M. Dale, Bruce Fischl and Martin I. Sereno.
Cortical Surface-Based Analysis: I. Segmentation and Surface Reconstruction.
Neuroimage, 9(2):179-194, 1999. dale99.pdf.gz

November 29

We will continue the discussion on the permutation tests. The future meetings will be shifted by one week.

November 22

Timothy B. Terriberry, Sarang C. Joshi, and Guido Gerig.
Hypothesis Testing with Nonlinear Shape Models.
IPMI 2005, LNCS 3565, pp. 15-26, 2005.

Additional papers:

Blair, R.C., Higgins, J.J., Karniski, W., Kromrey, J.D.
A study of multivariate permutation tests which may replace Hotelling T2 test in prescribed circumstances.
Multivariate Behavioral Research 29 (1994) 141-164.


Pesarin, Fortunato
Multivariate permutation tests : with applications in biostatistics.

I have the book.

November 15

GE Christensen, RD Rabbit, MI Miller.
Deformable Templates Using Large Deformation Kinematics.
IEEE Transactions on Image Processing, 5(10), 1996, pp. 1435-1447.

Additional papers:

Ain A. Sonin.
Fundamental Laws of Motion for Particles, Material Volumes, and Control Volumes, 2001.

On Choosing and Using Control Volumes: Six Ways of Applying the Integral Mass Conservation Theorem to a Simple Problem.

November 8

We will also finish the discussion on using prior examples to bias registration.

D. M. Blei, A. Y. Ng, and M. I. Jordan.
Latent Dirichlet allocation.
Journal of Machine Learning Research, 3, 993-1022, 2003.

Additional papers: Thomas Hofmann.
Probabilistic Latent Semantic Analysis.
UAI 1999.

Josef Sivic, Bryan C. Russell, Alexei A. Efros, Andrew Zisserman, William T. Freeman.
Discovering objects and their location in images.
ICCV 2005.

Brian Russell’s RQE paper.

November 1

Mert R. Sabuncu and Peter J. Ramadge.
Gradient based optimization of an EMST registration function.
IEEE Conference on Acoustics, Speech and Signal Processing, Philadelphia, March 2005.

Mert R. Sabuncu and Peter J. Ramadge.
Graph theoretic image registration using prior examples.
European Signal Processing Conference 2005, Antalya, Turkey, September 2005.

Additional papers:

B. Ma, A.O. Hero, J.D. Gorman and O. Michel.
Image Registration with Minimum Spanning Tree Algorithm.
IEEE International Conf. on Image Processing, vol.1, pp.481-484, Vancouver, BC, Canada, Sept. 2000.

October 25

A. O. Hero, B. Ma, O. Michel and J. Gorman.
Applications of entropic spanning graphs.
IEEE Signal Proc. Magazine (Special Issue on Mathematics in Imaging), Vol 19, No. 5, pp 85-95, Sept. 2002.

Additional papers:

Beirlant, J., Dudewicz, E. J., Gyorfi, L., and van der Meulen, E. C.
Nonparametric entropy estimation: An overview.
International Journal of the Mathematical Statistics Sciences, 6, 17-39, 2001.

October 18

No meeting, many of us are at ICCV.

October 11

P.D. Grünwald.
A Tutorial Introduction to the Minimum Description Principle.

Also use material from Septermber 27 meeting.

A minimum description length approach to statistical shape modeling.
Davies, R.H.; Twining, C.J.; Cootes, T.F.; Waterton, J.C.; Taylor, C.J.
IEEE Transactions on Medical Imaging, 21(5):525 - 537, 2002.

October 4

Erik Learned-Miller.
Data Driven Image Models through Continuous Joint Alignment.
to appear in IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2005.

Additional papers:

Lilla Zollei, Erik Learned-Miller, Eric Grimson and William Wells.
Efficient population registration of 3D data.
Workshop on Computer Vision for Biomedical Image Applications: Current Techniques and Future Trends, at the International Confernece of Computer Vision (ICCV), 2005.

September 27

Connection between the code length and entropy - Section 5.4 (and around) in Cover and Tomas.

J. Rissanen.
An Introduction to the MDL Principle.

Additional papers on MDL:

Rissanen, J.
Stochastic Complexity and Modeling.
Annals of Statistics, Vol 14, 1080-1100, 1986.

P.D. Grünwald.
A Tutorial Introduction to the Minimum Description Principle.

Additional papers on BIC and AIC:

Schwarz, G. (1978).
Estimating the dimension of a model. Annals of Statistics, 6, 461-464.

Akaike, H. (1974).
A new look at the statistical model identification.
IEEE Transactions on Automatic Control, AC-19, 716-723. akaike74.pdf.gz

September 20

A Unified Information-Theoretic Approach to Groupwise Non-rigid Registration and Model Building.
Carole J. Twining, Tim Cootes, Stephen Marsland, Vladimir Petrovic, Roy Schestowitz and Chris J. Taylor.
Information Processing in Medical Imaging: 19th International Conference, IPMI 2005, Glenwood Springs, CO, USA, July 10-15, 2005.

Additional papers:

Carole J. Twining, Stephen Marsland, and Chris Taylor.
Groupwise Non-Rigid Registration: The Minimum Description Length Approach.
BMVC 2004.

Carole Twining and Stephen Marsland.
A Unified Information-Theoretic Approach to the Correspondence Problem in Image Registration.
International Conference on Pattern Recognition, Cambridge, U.K. 2004.

September 13

First meeting, general intros.Internal Link

polina_s_reading_group.txt · Last modified: 2022/11/12 11:02
Recent changes RSS feed Creative Commons License Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki