Lecture Notes

$\newcommand{\vect}[1]{{\mathbf{#1}} }$ $\renewcommand{\vec}[1]{{\overrightarrow{#1}} }$ $\newcommand{\norm}[1]{{\left|\left|#1\right|\right|} }$ $\newcommand{\D}{{{\mathcal D}} }$ $\newcommand{\C}{{{\mathcal C}} }$ $\newcommand{\R}{{\mathbb{R}} }$ $\newcommand{\Rtwo}{{\mathbb{R}^2} }$ $\newcommand{\Rthree}{{\mathbb{R}^3} }$ $\newcommand{\V}{{\mathbb{V}} }$ $\newcommand{\nhat}{{\vect{\hat{n}}} }$ $\newcommand{\vi}{{\vect{i}} }$ $\newcommand{\vj}{{\vect{j}} }$ $\newcommand{\vk}{{\vect{k}} }$ $\newcommand{\f}{{\vect{f}} }$ $\newcommand{\g}{{\vect{g}} }$ $\newcommand{\vz}{{\vect{0}} }$ $\newcommand{\Du}{{D_{\vect{u}}} }$ $\DeclareMathOperator{\sgn}{sgn}$ $\DeclareMathOperator{\dist}{dist}$

Week 7: Directional Derivative and Gradient

Sujeet Akula

Directional Derivative

When considering the derivative of a function of several variables, we realize that it is of course related to the rate of change of the function. When we were learning of functions of a single variable, the derivative is clearly the rate of change along the axis of the independent variable where it is evaluated. However, the realm of functions of several variables is more rich. We can now ask the question of the rate of change of a function at some point in $\R^m$, along the direction of an arbitrary vector. This is both a common and important question. Let $f(x,y)$ be a function of two variables and let $(a,b)$ be a given point. Any line that goes through this point will be of the form \begin{equation} \vect{r}(t) = (a,b)_O + t(h,k)_O . \end{equation} Here, $(h,k)_O$ is the vector giving the direction of the line through the point $(a,b)$. In scalar parametric form, this is written as $x(t) = a + th$ and $y(t) = b+tk$. We can then construct the single variable function \begin{equation} F(t) = f(x(t),y(t)) = f(a+th,b+tk) . \end{equation} Using the chain rule, we can compute \begin{equation} F'(t) = hf_x(a+th,b+tk) + kf_y(a+th,b+tk) . \end{equation} If we evaluate this at zero, \begin{equation} F'(0) = hf_x(a,b) + kf_y(a,b) , \end{equation} we have the rate of change of $f$ at the point $(a,b)$ along the direction of the vector $(h,k)_O$. Note that since the vector is only for a direction, we take it to be a unit vector, i.e. $h^2 + k^2 = 1$.

Definition Let $f:\D\to\R$ be a function which is defined and continuously differentiable on a subset $\D$ in $\Rtwo$. Let the point $(a,b)$ be in $\D$. Let $\vect{u}=(h,k)_O$ be a unit vector. Then, the directional derivative of $f$ at the point $(a,b)$ in the direction of $\vect{u}$ is \begin{equation} \Du f(a,b) = hf_x(a,b) + kf_y(a,b) . \end{equation} In 3D, let $g:\D\to\R$ be a function defined and continuously differentiable on a subset $\D$ of $\Rthree$. Let the point $(a,b,c)$ be in $\D$. Let $\vect{u} = (h,k,l)_O$ be a unit vector. Then, the directional derivative of $g$ at the point $(a,b,c)$ in the direction of $\vect{u}$ is \begin{equation} \Du g(a,b,c) = hf_x(a,b,c) + kf_y(a,b,c) +lf_z(a,b,c) . \end{equation} Note that the directional derivative is a generalization the partial derivative: if we choose $\vect{u}=(1,0)_O$, then $\Du f$ is the same as $f_x$. In other sense, it is practically equivalent to the partial derivative: we can obtain $\Du f$ by first rotating the axes so that the new $x$-axis is along $\vect{u}$.

Example Compute the directional derivative of $f$ at the point $(0,\pi/4)$ in the direction of the vector $(1,1)_O$, given that $$f(x,y) = e^x\cos y + x\tan y .$$ First, we have to compute the unit vector along the desired direction: $$\vect{u} = \left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right)_O .$$ Next, $$f_x(x,y) = e^x\cos y + \tan y \Rightarrow f_x(0,\pi/4) = \frac{1}{\sqrt{2}} + 1 ,$$ $$f_y(x,y) = -e^x\sin y + x(1+\tan^2 y) \Rightarrow f_y(0,\pi/4) = -\frac{1}{\sqrt{2}} .$$ So, $$ \Du f(0,\pi/4) = \frac{1}{\sqrt{2}}\left(\frac{1}{\sqrt{2}}+1\right) - \frac{1}{\sqrt{2}}\frac{1}{\sqrt{2}} = \frac{1}{\sqrt{2}} . $$

Gradient

The gradient is intimately related to the directional derivative. Suppose we have a vector function such that the components of vector are the partial derivatives of $f$: $(f_x,f_y)$. This vector function is called the gradient of $f$, and is denoted as $\nabla f$: \begin{equation} \nabla f = (f_x,f_y) . \end{equation} Or, in 3D, \begin{equation} \nabla f = (f_x,f_y,f_z) . \end{equation} I have omitted the subscript of the vector giving its base point, but it is naturally based at the point where the gradient is evaluated. We can write the directional derivative in terms of the gradient using the dot product in this way: \begin{equation} \Du f(u_0) = \nabla f(u_0)\cdot\vect{u} , \end{equation} where $u_0$ is a point in $\Rtwo$ or $\Rthree$, and $\vect{u}$ is a unit vector. The geometric meaning is as follows: the norm of the gradient of a function at a point, is the largest value of the directional derivative at that point. And, the direction of the gradient is the direction of 'steepest ascent' of the function. Specifically, \begin{equation} \max\limits_\vect{u} \Du f(u_0) = ||\nabla f(u_0)|| . \end{equation} This says that if we consider a function $f$ at a given point $u_0$ and compute the directional derivative along every direction, the largest one is equal to the norm of the gradient of $f$ at $u_0$.

Example Suppose that the height above sea level is given by $$f(x,y) = 18 - 6\frac{x^2+xy+2y^2}{1+x^2+y^2} .$$ And, suppose that you are situated at the point $(1,2)$. In which direction would you have to walk to ascend in the steepest way and what is the maximal directional derivative? \\ The first question asked can be answered by giving the direction of the gradient of $f$ at the desired point, and the second question is given by the norm. So, we first compute the gradient of $f$ at this point: \begin{gather*} f_x(x,y) = 6\frac{x^2y+2x(y^2-1)-y(y^2+1)}{(1+x^2+y^2)^2} , \\ f_y(x,y) = -6\frac{x^3+2x^y+x(1-y^2) +4y}{(1+x^2+y^2)^2} . \end{gather*} Which gives that $$ \nabla f(1,2) = -\frac{1}{3}(1,5) .$$ Therefore, the maximal directional derivative is $||\nabla f||(1,2) = \frac{\sqrt{26}}{3}$, and the unit vector giving the direction of steepest ascent is $$\frac{\nabla f}{||\nabla f||}(1,2) = -\frac{1}{\sqrt{26}}(1,5) .$$

---