Deep Learning Step1 ~ Step4 Introduction and Functions
Introduction
Artificial Intelligence, called AI, is used in various fields today such as automotive, retail and farming industry. AI model is designed by many scientists with AI tools such as PyTorch, TensorFlow and etc. These tools make it easy for us to design AI model which can't be seen inside the tools like a black box. For this reason, it is hard for us to renew AI models that we made. In here, to modify AI model easily or learn how AI works, we're gonna learn to create Deep Learning code like back propagation, forward propagation, loss function, Neural Network. These articles are useful for the person who has a college level of mathematics and learn Python language.
Development Environment
Step 1 about Variables
First of all, we' re gonna learn variables as a programing language which are used to store information to be referenced and manipulated in a computer program like in the box below.
Fig. 1.1. Concept of variable.
To set up variables for input data for Deep Learning, we're gonna prepare for Variable Class which receives numpy.ndarray type variables. Numpy is the fundamental package for scientific computing with Python. In this time, Variable class has only one property "data".
class Variable:
def __init__(self,data):
#data property
self.data = data
import numpy as np
data = np.array(1.0)
x=Variable(data)
print(x.data)
#1.0
x.data=np.array([[1,2,3],[4,5,6]])
print(x.data)
#[[1 2 3]
# [4 5 6]]
print(x.data.ndim)
#2
where __init__(self,data) means the constructure which initializes class properties, self is equivalent to Variable class.
Step 2 about Function
A function is a block code which we can reuse and recall multiple times. Here, we create a function for setting up making AI model. In the image below , the function has x*x(x squared) as a formulation and x as an input data. If x is 3 and you put 3 into the function, you will get 9 as output. the function is shown as below.
Fig. 2.1. Example of function.In a programming of python, function will be defined by 'def', name of function is 'forward', and arguments of the function is 'x'. As you see below, 'forward' function is called two times. Thus we can reuse 'forward' function again.
def forward(x):
return x ** 2
print(forward(3))
#9
print(forward(5))
#25
We will make many functions about AI model later. To create class for functions will be helpful for us to make functions. The codes below show us function class in python. class Variable:
def __init__(self,data):
#data property
self.data = data
class Function:
def __call__(self, input):
x = input.data
output = self.forward(x)
return output
def forward(self,x):
raise NotImplementedError()
#How to extend class in python => class childClass(parentClass)
class Square(Function):
def forward(self,x):
return x ** 2
x = Variable(np.array(10))
#NotImplementedError() occurs when we directly access to forward method.
#f = Function()
#Squrare = x^2
f = Square()
y = f(x)
# class 'numpy.int64'
print(type(y))
#100 10 * 10
print(y)
Where __call__ function in Function class can be called like below. Tani in the below code is the instance of Human class. By using Tani(), we can call __call__ function.
# Human Class
class Human:
# constructor
def __init__(self, name):
self.name = name
# __call__ method
def __call__(self):
print('My name is ' + self.name + '.')
Tani = Human('Satoshi') #instance
Tani() # __call__ method
Step 3 Connection of Function
Now that we can call functions we can move move to the next step but before we will first explain what a composition function is. The composition function is a function which includes functions as arguments.
Fig. 3.1. Expression of composition function.First, we set 2 into x in Fig 3.1. Next, x is put into function F(x) , F(x) into G(F(x)) and as well as Z(G(F(x))). Z(G(F(x))) can be expressed as (Z 〇 G 〇 F )(x). This expression can avoid many '()'. For example, A(B(C(D(E(x))))) is so complicated for us. But (A 〇 B 〇 C 〇 D 〇 E)(x) is so simple and clear(Fig.3.1). The codes below show us how to make composition function in python.
import numpy as np
class Variable:
def __init__(self,data):
self.data = data
class Function:
def __call__(self, input):
x = input.data
y = self.forward(x)
output = Variable(y)
return output
def forward(self,x):
raise NotImplementedError()
class Square(Function):
#x is Variable type
def forward(self,x):
return x ** 2
class Exp(Function):
#x is Variable type
def forward(self,x):
return np.exp(x)
x = Variable(np.array(0.5))
A=Square()
B=Exp()
C=Square()
# x^2
a=A(x)
#exp(x^2)
b=B(a)
#(exp(x^2))^2
c=C(b)
#1.648721270700128
print(c.data)
#composite function
c=C(B(A(x)))
#1.648721270700128
print(c.data)
Step 4 about Differentiation
What is differentiation? It is a gradient of function at the argument value. If the gradient is zero, it will be the smallest value of the function. Let me show the example. In Fig.4.1, y=x^2(x squared) is shown. y=2x-1 and y=-2x-1 are derivative functions of y=x^2 at x=1 and x=2 each. As you see in this graph, at x=0 is the smallest value of y=x^2. And at x=0, gradient is zero because the derivative function is 0 which is equal to 0x+0.
Fig. 4.1. Example of differentiation of y=x^2(x squared).
Why do we need to obtain the smallest value of a function? In AI model, a lot of parameters called hyper parameter is used. To optimize AI model means that hyper parameters should also be optimized. This detail will be shown later in the blog. The codes below show us how to do differentiation function in python.
import numpy as np
class Variable:
def __init__(self,data):
self.data = data
class Function:
def __call__(self, input):
x = input.data
y = self.forward(x)
output = Variable(y)
return output
def forward(self,x):
raise NotImplementedError()
class Square(Function):
def forward(self,x):
return x ** 2
class Exp(Function):
def forward(self,x):
return np.exp(x)
def numerical_diff(f,x,eps=1e-4):
x0 = Variable(x.data-eps)
x1 = Variable(x.data+eps)
return (f(x1).data-f(x0).data)/(2*eps)
def f(x):
A = Square()
B = Exp()
C = Square()
return A(B(C(x)))
x = Variable(np.array(2.0))
f0 = Square()
#4
print(numerical_diff(f0,x))
x = Variable(np.array(0.5))
#1.648721270700128
print(f(x).data)
# function is one of object in python. f can be inserted into other function as index.
#3.2974426293330694
print(numerical_diff(f,x))
Comments
Post a Comment