Digital Twins: Enhancing Model-based Design with AR, VR and MR (Online)

Overview

Digital Twins: Enhancing Model-based Design with Augmented Reality, Virtual Reality, and Mixed Reality 

Background  

The idea of "digital twins" originated with NASA. Digital twins were then adopted into the manufacturing industry as a conceptual version of the Product Lifecycle Management (PLM). However, the core idea behind digital twins is still the same, i.e., a virtual model that incorporates all the necessary information about a physical ecosystem to solve a particular problem. 

Engineering systems have always used abstraction techniques to model complex problems. But the digital twin takes this idea further by allowing you to model a problem and simulate it. A variety of machine learning and deep learning techniques, collectively referred to as artificial intelligence (AI), play a part in the simulation aspects of the digital twin. AI helps to simulate scenarios via the digital twin but also to make autonomous decisions. Further, we could also use augmented reality (AR), virtual reality (VR), and other strategies for modelling engineering problems.  

Collectively, the techniques described above are referred to as 'model-based design.' Model-based design help engineers and scientists to design and implement complex dynamic systems using a set of virtual (digital) modelling technologies. As a result, you can iterate your design through fast, repeatable tests. In addition, you can automate the end-to-end lifecycle of your project by connecting virtual replicas of physical components in a digital space.  Once the system is modelled as a twin, various existing and new engineering problems can be modelled and simulated, such as predictive maintenance, and anomaly detection. 

In this course, we study model-based design under the framework of the digital twin and its advanced modelling techniques like AR, VR, and others. The course is based on MATLAB (MathWorks) and Unity – but prior knowledge of these systems is not needed. 

Course Overview 

The course is targeted to aspiring and seasoned simulation engineers that want to develop digital twin models of engineering components and incorporate these models into AR-VR-MR technologies. We will focus on the role of digital twin technology in model-based engineering practice, simulation of components. We will use tools like MATLAB and Simulink, and Unity (for AR/VR).  

Prior knowledge of Engineering in any discipline is preferred, but prior knowledge of MATLAB or Unity is not mandatory. However, some knowledge of coding (in any language) would be beneficial. The code covers code walkthroughs and demonstrations in some cases.  

Core areas covered by the course include digital twins in engineering practice, physical modelling, data-driven modelling, interoperability of simulation and simulators, Augmented reality and Virtual Reality for engineering practices.  

Intended Audience 

  • Targeted to aspiring and seasoned simulation engineers  
  • Prior knowledge of Engineering in any discipline is preferred  
  • Prior knowledge of MATLAB or Unity is not mandatory. 
  • Some knowledge of coding (in any language) would be beneficial.  
  • No other specific academic pre-requisites to enrol on this course   

Programme details

The following terminology will be used in the course modules (see below):  

Model-based design: A set of technologies and techniques that help engineers and scientists to design and implement complex, dynamic, end-to-end systems using a set of virtual (digital) modelling technologies.  Collectively, these technologies can simulate and model physical objects and processes in multiple industries.  

A digital twin is a digital representation that functions as a shadow/twin of a physical object or process. Digital twins are designed to model and simulate a process to understand it and predict its behaviour. Digital twin originates from engineering and is related to model-based systems engineering (MBSE) and surrogate modelling. The usage of digital twins is now more mainstream in software development, especially for IoT (Internet of Things). Digital twins can be combined with AR and VR to model physical processes.   

Virtual Reality (VR) creates an immersive experience through VR devices like headsets and simulates a three-dimensional world. VR is used in instructional content and educational material for field workers, in industries such as oil and gas, defence, and aviation.  

Augmented Reality (AR) overlays digital information on a physical world. Typically, AR uses conventional devices like mobile phones. Pokemon GO is an example of AR usage.  

Mixed Reality (MR) allows the manipulation of both physical and digital objects in an immersive world. HoloLens is an example of mixed reality.  

Structure of course 

Introduction and concepts

Digital Twin in Model-Based Engineering 

  • Application examples in operations optimisation, control system design, and predictive maintenance 
  • Cloud & deployment in production systems 

Introduction to MATLAB & Simulink 

  • Analyse data for a digital twin  
  • Run a Simulink model and interpret results 

Digital Twin Simulation 

  • Approaches to simulation: first principle, componentised, data-driven, simulators 
  • Generating synthetic data from simulations 

Introduction to Simscape 

  • Create a model for a simple component 
  • Simulate an electrical fault in a motor 
  • The role Digital twins in Model Based Systems Engineering (MBSE) recap 

Introduction to Data-Driven modelling 

Digital Twin & AI 

  • Fit an ML (machine learning) model to create a surrogate model 
  • Detection 

Digital Twin & VR-AR-MR 

  • Playback, co-sim, & integration workflows  
  • ROS (Robot Operating System) & ROS Toolbox 

Co-simulation with Unreal Engine 4 (UE4) 

Future trends in Industrial Digital Twin

Guided next steps: Simscape Onramp 

Case studies  

Case studies for digital twin, AR/VR, and model-based design.

Implementing solutions for scenarios (via Unity ARCore and MARS)  

Using the Unity ARCore and MARS toolkits, in this session, you will visualize designs and bring them to life in augmented reality by: 

  • Interacting with 3D digital twinned objects.  
  • Scaling simulations for teamwork and training.  
  • Develop prototypes using augmented reality and the Unity platform  

Conclusion  

We recap the key ideas in the course and summarize the main insights. 

Dates, Times and Delivery

This course will run over six live online sessions on Fridays, Saturdays and Mondays.

Session dates: Friday 19, Saturday 20, Monday 22, Friday 26, Saturday 27 and Monday 29 November 2021

Sessions will be 15:00 to 18:00 UK time (in some cases, the sessions will extend to 7 pm) and delivered online via Microsoft Teams.

A world clock, and time zone converter can be found here: https://bit.ly/3bSPu6D

No attendance at Oxford is required and you do not need to purchase any software.

Certification

Participants who attend the full course will receive a University of Oxford electronic certificate of attendance. 

The certificate will show your name, the course title and the dates of the course you attended.

You will be required to attend all of the live sessions on the course in order to be considered for an attendance certificate. 

Fees

Description Costs
Course fee £695.00

Payment

All courses are VAT exempt.

Register immediately online 

Click the “book now” button on this webpage. Payment by credit or debit card is required.

Request an invoice

If you require an invoice for your company or personal records, please complete an online application form. The Course Administrator will then email you an invoice. Payment is accepted online, by credit/debit card, or by bank transfer. Please do not send card or bank details via email.

Tutors

Dr Lars Kunze

Course Director

Lars Kunze is a Departmental Lecturer in Robotics in the Oxford Robotics Institute (ORI) and the Department of Engineering Science at the University of Oxford. At ORI, he leads the Cognitive Robotics Group (CRG).

Lars is also a Stipendiary Lecturer in Computer Science at Christ Church, a Programme Fellow of the Assuring Autonomy International Programme (AAIP), and an Editor of the German Journal of Artificial Intelligence (KI Journal, Springer).

His areas of expertise lie in the fields of robotics and artificial intelligence (AI). His goal is to enable robots to understand their surroundings, to act autonomously, and to explain their own behaviour in meaningful human terms. To this end, his research concerns the design and development of fundamental AI techniques for autonomous robot systems. He focusses on the combination of knowledge representation, reasoning, machine learning, and robot perception; motivated by applications in complex, real-world environments.

Lars studied Cognitive Science (BSc, 2006) and Computer Science (MSc, 2008) at the University of Osnabrück, Germany, and partly at the University of Edinburgh, UK.

He received his PhD (Dr. rer. nat.) from the Technical University of Munich, Germany, in 2014. During his PhD, Lars worked on methods for naive physics and common sense reasoning in the context of everyday robot manipulation. He contributed to several national, European and international projects including RoboHow, RoboEarth, and the PR2 Beta program.

In May 2013, Lars was appointed as a Research Fellow in the Intelligent Robotics Lab at the School of Computer Science at Birmingham University.  Here he worked on qualitative spatio-temporal models for perception planning and knowledge-enabled perception, contributing to the European research projects STRANDS and ALOOF.

He was a visiting researcher in the JSK Lab at the University of Tokyo, Japan (Summer 2011) and the Human-Robot Interaction Laboratory at Tufts University, US (Spring 2015).

Ajit Jaokar

Course Director

Based in London, Ajit's work work spans research, entrepreneurship and academia relating to artificial intelligence (AI) and the internet of things (IoT). 

He is the course director of the course: Artificial Intelligence: Cloud and Edge Implementations. Besides this, he also conducts the University of Oxford courses: Developing AI Applications and Computer Vision.

Ajit works as a data scientist through his company feynlabs - focusing on building innovative early stage AI prototypes for domains such as cybersecurity, robotics and healthcare.

Besides the University of Oxford, Ajit has also conducted AI courses in the London School of Economics (LSE), Universidad Politécnica de Madrid (UPM) and as part of the The Future Society at the Harvard Kennedy School of Government.

He is also currently working on a book to teach AI using mathematical foundations at high school level. 

Ajit was listed in the top 30 influencers for IoT for 2017 along with Amazon, Bosch, Cisco, Forrester and Gartner by the German insurance company Munich Re.

Ajit publishes extensively on KDnuggets and Data Science Central.

He was recently included in top 16 influencers (Data Science Central), Top 100 blogs (KDnuggets), Top 50 (IoT central), and 19th among the top 50 twitter IoT influencers (IoT Institute). 

His PhD research is based on AI and Affective Computing (how AI interprets emotion).

Mr Phil Chacko

Tutor

Phil is Group Product Manager for XR at Unity. Phil has been working to make it easier for creators to tell stories in XR, currently overseeing XR platform integrations, frameworks, interaction workflows, simulation-based MARS workflows, and runtime authoring features.
Prior to Unity, Phil worked at Netflix and Audi/Volkswagen.

Francesco Ciriello

Tutor

Lecturer in Engineering Education, King’s College London

Francesco is a Lecturer in the Department of Engineering at King’s College London, where he teaches interdisciplinary design and mechatronics. He previously worked in the Education Group at MathWorks and provided consultancy services to educators and researchers on the use of MATLAB & Simulink. Francesco has broad expertise in Simulation and Artificial Intelligence, with application to Robotics & Control systems, signal processing and IoT. He also holds a PhD in Engineering from the University of Cambridge for his work in experimental fluid dynamics and a MEng in Civil Engineering from Imperial College London. 

Dirk Hartmann

Tutor

Dirk Hartmann is a distinguished scientist, intrapreneur, and thought leader in the field of Simulation and Digital Twins.  

Among many distinctions, he has been awarded the prestigious Wernervon-Siemens Top Innovator by the Siemens CEO and CTO for generating novel products and services cross product lines through his innovations.  

In his career he took several leading roles in research, innovation, and development including a lead of a 2-digit million Siemens R&D program and the technical leadership for the Simulation & Digital Twin field at Siemens Technology.  

He is a passionate mentor, teacher, and supervisor for the next generation of innovators and experts exploring jointly promising Digital Twin solutions for the industry.  

Beyond this, he is a member of several high-level international conference committees and associations like EU-MATH driving industrial mathematics both on a national, European, and international level. 

Dr David McKee

Tutor

Dr David Mc Kee, Chair of the Open Source, Standards, and Platform Stacks at the Digital Twin Consortium / CEO, CTO and founder, Slingshot Simulations

Dr David McKee is the CEO, CTO and founder at Slingshot Simulations, an enterprise fellow at the Royal Academy of Engineering, and chairs the technology working groups at the OMG Digital Twin Consortium. As CTO at Slingshot since 2019 David leads the company’s work on Digital Twins working across Cloud, IoT, and machine learning platforms.

At the Digital Twin Consortium David jointly lead the work on standardising a Digital Twin definition and continues to lead the Technology, Terminology, and Taxonomy working group. He is also responsible for leading the Open Source Initiative and a collaborative effort including Microsoft, Bentley Systems, DELL, and NTT to define a reference architecture for building Digital Twin Systems.

Before forming Slingshot David was a senior researcher at the University of Leeds building these systems for partners including the likes of Jaguar Land Rover and AliCloud.

Ms Ayşe Mutlu

Tutor

Ayşe Mutlu is a data scientist working on Azure AI and devops technologies. Based in London, Ayşe’s work involves building and deploying Machine Learning and Deep Learning models using the Microsoft Azure framework (Azure DevOps and Azure Pipelines).

She enjoys coding in Python and contributing to Open Source Initiatives in Python.

Mr Keith Myers

Tutor

Keith Myers is an artist, professional speaker, and the creative director of AVimmerse, who have produced immersive content for clients including: The NHS; BBC Arts; The Wildlife Trust; Liverpool John Moores University; and Gymshark. 

He also teaches immersive media production techniques and has worked with institutions in the UK, including The University of Manchester, and Manchester Metropolitan University.  In addition, he has taken an active role in Virtual Reality Labs across the North-West region for the last 4 years, where he has shared production techniquesdesign approaches, and led teams of artists to fulfil briefs.   

His tech talks focus on immersive production, critical analysis of a new medium, and immersive storytelling techniques. He has spoken at Leeds Digital Festival, Tech Week Humber, and the annual AR/VR conference in Manchester.   

Social channels include: 

Mr David Menard

Tutor

With over 10 years of mixed reality and real-time development experience, David Menard is an industry-leader in virtual reality (VR) and augmented reality (AR) when it comes to enterprise applications.
At Unity Technologies, David oversees the technical developmentof Unity Reflect, which enables AEC (Architecture, Construction, Engineering) companies to create real-time experiences in augmented reality and virtual reality.
Prior to joining Unity, David led mixed-reality R&D efforts at the enterprise software giant, Autodesk.

Mr Harry Powell

Tutor

Harry Powell is a data scientist who led the advanced analytics businesses at Jaguar Land Rover, an automotive manufacturer, Barclays, a bank and Betfair, a mobile sports-betting company.

As Director of Data and Analytics at JLR, Harry’s team is transforming the business by using data and digital technologies to create new products, optimise performance and change how colleagues think. Over the past 3 years at JLR the team has delivered audited EBIT of £500 million and >50x ROI.

At JLR, Harry's team pioneered the use of graph data structures to build models of supply chain, vehicle quality/fault diagnosis and plant scheduling. This enabled JLR to adapt to changing market conditions, including navigating the 2021 semiconductor crisis. Graph turns out to be a very promising approach to building a digital twin of the firm.

Harry has degrees in Physics (Bristol), Economics (UCL) and Business (Oxford).

Robbie Stevens

Tutor

Aero Technology Lead - Alpine F1 Team 

 

Robbie is the leader of the Aerodynamics Technology group at Alpine F1 Team. He joined the Team in 2016. His work includes the development of physics based aerodynamic models, diagnostic tools, data analysis and future methodology. In addition, he is also responsible for a number of technical and academic partnerships.  

Prior to joining the Team, Robbie was a Post-Doctoral Research Associate at the Cambridge University Engineering Department and a Clare College Cambridge Research Associate conducting research in high Mach number flows. 

Robbie received his Ph.D. from Cambridge University in 2015. His Ph.D. research involved the development of a reduced-order theoretical model to describe flapping-wing flight (at small bird/insect scales).

Robbie is also a chartered member of the Institute of Mechanical Engineers and the author of several published works in Aerodynamics and Fluid Mechanics.

 

Course aims

What you will learn 

  • Understand the main engineering applications in which digital twin technology is being used 
  • Explain the value of digital twin technology and model-based design in engineering practice 
  • Explain the difference between a simulation and a simulator 
  • Interpret the results of a MATLAB analysis 
  • Interpret the results of a Simulink simulation 
  • Explain different workflows to interface a digital twin to AR-VR-MR software, including playback, co-simulation, and integration.  
  • Implement a physical model for a component given a schematic representation 
  • Implement a data-driven model for a component given an experimental data set 
  • Playback the results of a simulation into AR-VR-MR software  
  • Run co-simulations between Simulink and AR-VR-MR software (UE4) 
  • Integrate a physical model in Simulink into AR-VR-MR software 
  • Model an engineering component using Simscape 
  • Optimise a design parameter in Simulink 
  • Critique the difference between white, grey, and black-box modelling approaches 
  • Specify a deployment pipeline for a digital twin in a production system 
  • Identify fidelity-performance trade-offs for simulations and real-time deployment  
  • Critique the role of emerging AR-VR-MR technologies in model-based engineering design 
  • Design an engineering model of a digital twin given a set of specifications and demonstrate integration within a 3D graphics simulator 
  • How to use AR and VR for modelling 
  • Fundamentals of augmented reality, virtual reality, and mixed reality  
  • Building professional AR/VR applications  
  • The AR/VR landscape for tools, technologies, and services

 

Application

If you would like to discuss your application or any part of the application process before applying, please click Contact Us at the top of this page.

IT requirements

This course is delivered online using Microsoft Teams. You will be required to follow and implement the instructions we send you to fully access Microsoft Teams on the University of Oxford's secure IT network.

This course is delivered online; to participate you will need regular access to the Internet and a computer meeting our recommended Minimum computer specification.

It is advised to use headphones with working speakers and microphone.