Undergraduate Teaching 2022-23

Engineering Tripos Part IIB, 4F10: Deep Learning & Structured Data, 2022-23

Engineering Tripos Part IIB, 4F10: Deep Learning & Structured Data, 2022-23

Not logged in. More information may be available... Login via Raven / direct.

PDF versionPDF version

Module leader

Prof M Gales

Lecturer

Prof M Gales and Dr Miguel Hernandez-Lobato

Timing and Structure

Michaelmas term. 14 lectures + 2 examples classes. Assessment: 100% exam

Prerequisites

Part IIA Modules 3F1, 3F3 and 3F8 advisable

Aims

The aims of the course are to:

  • This module aims to teach the basic concepts of deep learning and forms of structure that can be used for generative and discriminative models. In addition, the use of models for classifying structured data, such as speech and language, will be discussed

Objectives

As specific objectives, by the end of the course students should be able to:

  • Understand the basic principles of pattern classification and deep learning;
  • Understand generative and discriminative models for structured data;
  • Understand the application of deep-learning to structured data;
  • Be able to apply pattern processing techniques to practical applications.

Content

Introduction (1L)

Links with 3F8 and 4F13. General machine learning, examples of structured data, DNA, vision, speech and language processing.

Decision Boundaries and Probability of Error (1L)

Definition of a decision boundary and forms that result from Gaussian class-conditional probability density functions. Calculation of probability of Error.

Graphical Models and Conditional Indpendence (1L)

Graphical models and Bayesian networks. Simple inference examples.

Latent Variable and Sequence Models (3L)

Gaussian mixture models and factor analysis; hidden Markov models and expectation maximisation.

Deep Learning (2L)

Generative and discriminative deep models. Forms of network and activation functions. Convolutional neural networks, mixture-density neural networks. Optimisation approaches (first/second order methods, adaptive learning rates) and initialisation.

Deep Learning for Sequences (2L)

Recurrent neural networks, and long-short-term memory models. Variants of RNN including bidirectional RNNs. Use in generative and discriminative models.

Ensemble Methods (1L)

Ensemble methods:  random forests, bagging, boosting and model combination.

Support Vector Machines (2L)

Maximum margin classifiers, handling non-separable data, training SVMs, non-linear SVMs, kernel functions. Multi-class SVMs.

Kernels over Structured Data (1L)

String kernels, graph kernels and Fisher kernels.

Booklists

Please refer to the Booklist for Part IIB Courses for references to this module, this can be found on the associated Moodle course.

Examination Guidelines

Please refer to Form & conduct of the examinations.

UK-SPEC

This syllabus contributes to the following areas of the UK-SPEC standard:

Toggle display of UK-SPEC areas.

GT1

Develop transferable skills that will be of value in a wide range of situations. These are exemplified by the Qualifications and Curriculum Authority Higher Level Key Skills and include problem solving, communication, and working with others, as well as the effective use of general IT facilities and information retrieval skills. They also include planning self-learning and improving performance, as the foundation for lifelong learning/CPD.

IA1

Apply appropriate quantitative science and engineering tools to the analysis of problems.

IA2

Demonstrate creative and innovative ability in the synthesis of solutions and in formulating designs.

KU1

Demonstrate knowledge and understanding of essential facts, concepts, theories and principles of their engineering discipline, and its underpinning science and mathematics.

KU2

Have an appreciation of the wider multidisciplinary engineering context and its underlying principles.

E1

Ability to use fundamental knowledge to investigate new and emerging technologies.

E2

Ability to extract data pertinent to an unfamiliar problem, and apply its solution using computer based engineering tools when appropriate.

E3

Ability to apply mathematical and computer based models for solving problems in engineering, and the ability to assess the limitations of particular cases.

P1

A thorough understanding of current practice and its limitations and some appreciation of likely new developments.

P3

Understanding of contexts in which engineering knowledge can be applied (e.g. operations and management, technology, development, etc).

P8

Ability to apply engineering techniques taking account of a range of commercial and industrial constraints.

US1

A comprehensive understanding of the scientific principles of own specialisation and related disciplines.

US2

A comprehensive knowledge and understanding of mathematical and computer models relevant to the engineering discipline, and an appreciation of their limitations.

 
Last modified: 24/05/2022 12:53