[pytoch] calculate the derivative of sin (x) by automatic differentiation

1. Problem analysis

1.1 Problem description

In fact, the problem is very simple , This blog is a memo . Our need is ：

picture source ： Li Mu ：《 Hands-on deep learning PyTorch edition 》

1.2 Solutions

We use pytorch The automatic differentiation mechanism of , That's what I wrote for the first time ：

import torch

x = torch.arange(-5, 5, 0.01, requires_grad=True)
y = torch.sin(x)
y.backward()
print("f(x)=sin(x) The derivative of is ：{}".format(x.grad))

It turned out to be wrong ：

The reason is that y It's a vector , But in the use of y.backward() This method , if backward No incoming from gradient When this parameter , Only y It's scalar . The fact was x and y When it's all vectors ,y Yes x The derivative is a matrix （ Jacobian matrix ）, I have some introductions in this blog ： Vector to vector derivation , Get the Jacobian matrix .

The solution is quite simple , The essential idea is ： Vector y Convert to scalar , Then use scalar y Conduct y.backward().

So how to switch ？ In fact, we just need to do the vector y Sum each item of the , A scalar derivative of a vector is a vector , A vector x=[x1, x1, …, xn], vector y=[sin(x1), sin(x2), …, sin(xn)], We're going to vector y Sum each term of , We get scalars y1=sin(x1)+sin(x2)+…+sin(xn), Finally, use scalar y1 For vectors x Derivation , You can get the vector d(y1)=[cos(x1), cos(x2), …, cos(xn)], The code is as follows ：

import torch

x = torch.arange(-5, 5, 0.1, requires_grad=True)
y = torch.sin(x)
y1 = y.sum()
y1.backward()
print("f(x)=sin(x) The derivative of is ：\n{}".format(x.grad))

2. Code implementation

def get_derivative_tensor(x):
    x = torch.tensor(x, requires_grad=True)
    y1 = torch.sin(x)
    y1.sum().backward()
    return x.grad

x = np.arange(-5, 5, 0.01)
y = np.sin(x)

plt.plot(x, y, label='sin(x)')
dx = get_derivative_tensor(x)
plt.plot(x, dx.numpy(), label='dx')
plt.xlabel('x')
plt.ylabel('f(x)')
plt.legend()
plt.show()

END ：）

In fact, I have been thinking about whether to write a summary for such a simple thing , Later, I felt that I should not despise the problems I didn't know .“ My teacher also said , I know that my husband was born in my year ？” Empathy , I am pursuing progress , It doesn't matter whether the problem itself is difficult , If you disdain to sum up because the problem is simple , Have been muddling along , In the long run, it is difficult to make progress .