Fundamental Numerical Methods for Parameter Estimation
Instructor: Prof. Dr. Uwe Naumann
Affiliation: Software and Tools for Computational Engineering (STCE), RWTH Aachen University, Germany
Date and Venue
Monday 3rd – Thursday 6th, February 2025
Room 507, 5th Floor, TGGS Building, KMUTNB
Abstract
Numerical simulation is a widely used tool for understanding phenomena in virtually all areas of science, engineering, economics, finance, …
Numerical solutions for mathematical models of reality are implemented as computer programs. In most cases, free parameters are introduced to account for uncertainty. The latter results from abstraction, simplification, and approximation due to modeling and numerical algorithms.
The desired values of the free parameters are typically unknown. Given estimates often suffer from substantial inaccuracy. Observations of reality can be used to calibrate the free parameters with the objective of improved representation of reality by its numerical simulations.
Numerical methods for parameter estimation aim to determine values for the free parameters such that the discrepancy between the given observations and the corresponding simulated results of the model is minimized. Both algebraic and differential models are widely used. General-purpose nonlinear programming as well as often more efficient custom regression methods can be employed.
This course presents a gentle introduction to fundamental numerical methods for parameter estimation. A simple subset of the programming language C++ is used for reference implementations as well as for numerous interactive hands-on exercise sessions. Initially, all concepts are introduced for the scalar case allowing for geometric illustration and hence easier understanding. Generalization for the non-scalar case follows quite naturally. A particular focus is on differentiability of the numerical simulation programs to enable the use of derivative-based nonlinear programming and regression methods as well as to facilitate robustness analysis of
the solutions obtained. Algorithmic differentiation (e.g., known from backpropagation for training deep artificial neural networks) is introduced as the method of choice.
Prerequisites
Participants should bring a computer running VirtualBox (www.virtualbox.org). A preconfigured Linux image with required software will be available for download prior to the course. Installation validation instructions will also be provided.
An image of a simple Linux system with all the required software preinstalled is available, along with instructions for validating the correctness of the installation before the course begins. You can find the details at the following link:
Agenda
The course is delivered as a lecture with built-in interactive tutorials. Morning (9am-noon) and afternoon (1pm-4pm) sessions are spread over four days (Monday to Thursday). A hands-on programming test is planned for Friday morning.
Monday (Scalar Case)
- Morning: Introduction, Story in a Nutshell, Scripting for Numerics with C++
- Afternoon: Algebraic and differential models, explicit and implicit Euler methods, sensitivity analysis
Tuesday (Scalar Case)
- Morning: Normal equation method for linear regression, gradient descent, first-order tangent mode of AD
- Afternoon: Newton method for nonlinear programming, second-order tangent mode of AD
Wednesday (Scalar Case)
- Morning: Householder method for linear regression, normal equation method for nonlinear regression
- Afternoon: Householder method for nonlinear regression, robustness analysis
Thursday (Outlook to Non-scalar Case)
- Morning: Differential models, Euler methods, normal equation methods
- Afternoon: Gradient descent, first-order and second-order adjoint modes of AD, Newton method
Friday
- Morning: Programming test
- Afternoon (optional): Research and development at STCE@RWTH
