Enric Fontdecaba : March 13, 2002

Introduction to Automatic Differentiation.

Abstract

The derivatives of a function gives to us a lot of information on the behavior of the function. In addition, efficient optimization algorithms need derivatives. However, the main interest of engineers/scientist is to develope computer models for their problems (that is for the functions).
Until recently, the most common solutions to obtain derivatives of models/functions were:

a) to force the engineer/scientist to code the derivatives explicitely.

But derivatives are difficult to code (so time consuming and error prone).

b) to use finite differences.

That is an easy solution but with an important penalty in computational cost and numerical precision.

Nowadays there are several tools called Automatic (or Algorithmic) Differentiators that, using as input the routines to evaluate the function, build up routines that evaluate the derivatives very efficiently and without loosing numerical precision.

In this presentation we will introduce the basic concepts of Automatic Differentiation, the main questions to answer are:
* What is Automatic Differentiation?
* Which are the principles AD is based on?
* How can it be implemented?
* Current Trends.