Tani's Programming Memo

Deep Learning Step7 Computational Graphs in Python  The python code below shows forward propagation and back propagation for computational graphs that we learned. # weakref is from asyncio import SendfileNotAvailableError import weakref import numpy as np import contextlib class Config: enable_backprop = True @contextlib.contextmanager def using_config(name, value): old_value = getattr(Config, name) setattr(Config, name, value) try: yield finally: setattr(Config, name, old_value) def no_grad(): return using_config('enable_backprop', False) class Variable: __array_priority__ = 200 def __init__(self, data, name=None): if data is not None: if not isinstance(data, np.ndarray): raise TypeError('{} is not supported'.format(type(data))) # input data of np.ndarray self.data = data self.name = name # grad data of np.ndarray self.grad = N...

Deep Learning Step6   Step 6 Computational Graphs About Computational Graphs  There are many computational graphs in AI. They make it easy for us to understand visually how AI model is made. Computational graphs below are shown in this step.  ・Additional Nodes ・Multiplicated Nodes ・Separated Nodes ・Repeated Nodes ・Sum Nodes ・MatMul Nodes About  additional Nodes  Additional nodes add up 2 nodes like X and Y in Fig.6.1. Where L means Loss function.  Fig.6.1 Forward Propagation of additional nodes  On the other hand, in back propagation of it, additional nodes doesn't change the differentiation value from the upper value of the differentiation. Fig.6.2 Back Propagation of additional nodes About  Multiplicated Nodes   Multiplicated nodes make multiplication of X and Y like in Fig.6.3. Fig.6.3 Forward Propagation of multiplicated nodes  The back propagation will be like in Fig.6.4. ∂Z/∂X is equal to Y if Z is X × Y, ∂Z/∂Y is equa...

Deep Learning Step5   Step 5 Back Propagation   Why We Need Back Propagation? In supervised learning systems, Back Propagation gets minimum value of loss function. Loss function is used to decide optimized hyper parameters which will be described later.   Differentiation of Composition Function  First, we need to know how to do differentiation of composition.  For example, the following functions are ready.   Input Data : x  Function A(), B() ,C()  Output each : a = A(X) , b = B(a) , c = C(b)   Fig.5.1 Forward Propagation  If x is put into A(), you will get a. As well as a is put into B and b into C. This way is called Forward Propagation(Fig.5.1).  If we want value of dC(b)/dx ( c = C(b) = C(B(A(X))) => dc/dx = dC(b)/dx = dC(B(A(X)))/dx) , we can solve this equation like the below. Fig.5.2 Backward Propagation  This data flow in Fig.5.2 is called back propagation because the value of differentiation of dc/dc is g...

 Introduction Artificial Intelligence, called AI, is used in various fields today such as automotive, retail and farming industry. AI model is designed by many scientists with AI tools such as PyTorch, TensorFlow and etc. These tools make it easy for us to design AI model which can't be seen inside the tools like a black box. For this reason, it is hard for us to renew AI models that we made. In here, to modify AI model easily or learn how AI works, we're gonna learn to create Deep Learning code like back propagation, forward propagation, loss function, Neural Network. These articles are useful for the person who has a college level of mathematics and learn Python language. Development Environment VS code : latest version  Python : 3.x Numpy : latest version Step 1 about Variables First of all, we' re gonna learn variables as a programing language which are used to store information to be referenced and manipulated in a computer program like in the box below.  ...