Reviewing the latest research in machine learning, intelligent systems, systems and software engineering. Your lecturers are Prof. Dr. Nils Bertschinger, Prof. Dr. Matthias Kaschube and Prof. Dr. Visvanathan Ramesh. ( QIS/LSF )

For any questions please contact us teaching@ccc.cs.uni-frankfurt.de

General

Bachelors students are required to give a presentation only, Masters also need to hand in a report about their topic (~ 5-10 pages), weighting for grade 50/50. Presentations will be around 45 minutes plus discussion with the class. Course language is English. We will meet every week and presence is mandatory. Either choose one of the topics below and search for literature for yourself (papers, book-chapters, etc.), or choose a paper from here or request one from any professor above. Registration is mandatory and will be passed to the examination-office.

Presentation Dates

• 27.10.2016 – Decisions on topics and dates
• 03.11.2016 – no talk
• 10.11.2016 – Lars P.: SVMs
• 17.11.2016 – Andres F.: Music classification
• 24.11.2016 – Philipp T.: Goal-driven deep Learning – sensory cortex
• 01.12.2016 – Margarita M.: Twitter opinion mining/text processing
• 08.12.2016 – Hans-Joachim H.: Probabilistic topic modeling
• 15.12.2016 – Michael W.: Deep learning (nature)
• 12.01.2017 – Jiawei H.: Bayesian analysis of GARCH and stochastic volatility
• 19.01.2017 – Merali P.:Deep face recognition
• 26.01.2017 – Julia: Variational inference
• 02.02.2017 – Tobias K: Avoiding pathologies of deep architectures
• 09.02.2017 –
• 16.02.2017 –
• 23.02.2017 –

Topics

• General introduction: Review papers of ML
• Deep Learning: Modern neural networks
• Classics:
  • Neural networks
  • Standard algorithms of ML, e.g. EM
• Applications
  • Neuroscience
  • Finance/Economics
  • Medicine
  • Computer vision (e.g. face recognition)
  • Robotics (e.g. autonomous cars)
  • Recommendation systems (network models)
  • Natural language processing
• Systems/Architectures
  • Integration of subsystems
  • Platforms/Probabilistic programming (e.g. Tensorflow)
  • Big Data
• Theoretical background:
  • Probability theory: Bayesian decision theory
  • Neural networks: Universal approximation

List of papers

* General philosophy
  Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models
  http://www.annualreviews.org/doi/pdf/10.1146/annurev-statistics-022513-115657
  Model-based machine learning
  https://www.microsoft.com/en-us/research/wp-content/uploads/2016/05/Bishop-MBML-2012.pdf

* Classics:
  - Neural networks
    A Sociological Study of the Official History of the Perceptrons Controversy
    http://www.jstor.org/stable/pdf/285702.pdf
  - Standard algorithms
    Maximum Likelihood from Incomplete Data via the EM Algorithm
    https://www.jstor.org/stable/pdf/2984875.pdf
    Linear Dimensionality Reduction: Survey, Insights, and Generalizations
    http://www.jmlr.org/papers/volume16/cunningham15a/cunningham15a.pdf
    Variational Inference: A Review for Statisticians
    https://arxiv.org/pdf/1601.00670.pdf
    - Sampling algorithms
      Sampling methods (chapter 11)
      Bishop, PRML, Springer 2006
      MCMC Using Hamiltonian Dynamics
      http://www.mcmchandbook.net/HandbookChapter5.pdf
      Elliptical slice sampling
      http://www.jmlr.org/proceedings/papers/v9/murray10a/murray10a.pdf
      
* Applications
  - Ethics
    The social dilemma of autonomous vehicles
    http://science.sciencemag.org/content/352/6293/1573
  - Finance/Economics
    - Volatility modeling:
      Bayesian analysis of GARCH and stochastic volatility: modeling leverage, jumps and heavy-tails for financial time series
      https://stat.duke.edu/research/BEST/BEST2009/NakajimaBEST2009honorablemention.pdf
      Generalized Wishart processes
      https://dslpitt.org/uai/papers/11/p736-wilson.pdf
    - Macroeconomics/Econometrics:
      Large Bayesian Vector Autoregressions
      http://onlinelibrary.wiley.com/doi/10.1002/jae.1137/pdf
  - Natural language processing
    Probabilistic topic models
    http://www.cs.columbia.edu/~blei/papers/Blei2012.pdf
    TOPIC MODELS
    http://www.cs.columbia.edu/~blei/papers/BleiLafferty2009.pdf
  - Robotics
    Particle Filters in Robotics
    http://robots.stanford.edu/papers/thrun.pf-in-robotics-uai02.html
  - Biology
    Spiking neurons can discover predictive features by aggregate-label learning
    http://science.sciencemag.org/content/351/6277/aab4113.long
    Using goal-driven deep learning models to understand sensory cortex
    http://www.nature.com/neuro/journal/v19/n3/full/nn.4244.html
  - Computer Vision
    DeepFace: Closing the Gap to Human-Level Performance in Face Verification
    https://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf
    Deep Face Recognition
    https://www.robots.ox.ac.uk/~vgg/publications/2015/Parkhi15/parkhi15.pdf

* Systems/Architectures
  - Probabilistic programming
    - Stan (http://mc-stan.org)
      Stan: A probabilistic programming language for Bayesian inference and optimization
      http://www.stat.columbia.edu/~gelman/research/published/stan_jebs_2.pdf
    Black-Box Stochastic Variational Inference in Five Lines of Python
    http://people.seas.harvard.edu/~dduvenaud/papers/blackbox.pdf
    Probabilistic machine learning and artificial intelligence 
    http://www.nature.com/nature/journal/v521/n7553/full/nature14541.html
  - Deep learning
    Marginal Space Deep Learning: Efficient Architecture for Detection in Volumetric Image Data
    http://comaniciu.net/Papers/MarginalSpaceDeepLearning_MICCAI15.pdf
  Deep Boltzmann Machines 
  http://www.utstat.toronto.edu/~rsalakhu/papers/dbm.pdf  
  Recurrent Models of Visual Attention
  https://arxiv.org/abs/1406.6247

* Theoretical background
  - Deep learning
    Avoiding pathologies of deep architectures
    http://www.jmlr.org/proceedings/papers/v33/duvenaud14.pdf
    On Random Weights and Unsupervised Feature Learning
    http://www.robotics.stanford.edu/~ang/papers/nipsdlufl10-RandomWeights.pdf
    Dropout as a Bayesian Approximation: Insights and Applications
    http://mlg.eng.cam.ac.uk/yarin/PDFs/Dropout_as_a_Bayesian_approximation.pdf
    A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
    https://arxiv.org/abs/1512.06293

* Tutorials and reviews 
  - Deep learning
    Deep Learning in Neural Networks: An Overview 
    https://arxiv.org/abs/1404.7828
    Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey
    https://arxiv.org/abs/1512.03131
    Deep Learning 
    http://www.nature.com/nature/journal/v521/n7553/pdf/nature14539.pdf