[Pytorch] 기초 강의 day1

Pytorch 기초 정리

torch

td = torch.linspace(0,3,4)
print(td)
print(torch.exp(td))
print(torch.max(td))
print(torch.log(td))

td2 = torch.tensor([[1,2,3],[4,5,7]], dtype=torch.float32)
print('\n', torch.max(td2, dim=1))
print('\n', torch.max(td2, dim=1)[0])
print(torch.max(td2, dim=1)[1])

tensor([0., 1., 2., 3.])
tensor([ 1.0000, 2.7183, 7.3891, 20.0855])
tensor(3.)
tensor([ -inf, 0.0000, 0.6931, 1.0986])

torch.return_types.max(
values=tensor([3., 7.]),
indices=tensor([2, 2]))

tensor([3., 7.])
tensor([2, 2])

linspace(start, end, step) : start ~ end 사이의 수를 step으로 쪼개 list에 담음
torch.max는 max value와 index를 동시에 반환.
- torch.max( )[0]은 max value
- torch.max( )[1]은 index
Tensor 차원
- 텐서의 차원을 변경하는 view나 텐서를 결정하는 stack, cat, 차원을 교환하는 t, transpose도 사용된다.
- view는 numpy의 reshape와 유사
- cat은 다르길이의 텐서를 하나로 묶을 때 사용
- transpose는 행렬의 전치 외에도 차원의 순서를 변경할 때에 사용
선형대수
- dot : 벡터 내적
- mv : 행렬과 벡터의 곱
- mm : 행렬과 행렬의 곱
- matmul : 인수의 종류에 따라서 자동으로 dot, mv, mm을 선택

import torch

t1 = torch.tensor([1,2,3,4,5,6])
print(t1.size())
t1 = t1.view(2,3)
print('\n', t1.size())
print('\n', t1)

t2 = torch.tensor([3,2,4,8,7,9]).view(2,3)
print('\n', t2)

t3 = torch.tensor([[1,2],[3,4]])
print('\n', t3)
print('\n', t3.size())
print('\n', t3.view(-1))
print('\n', t3.view(-1).size())
print('\n', t3.view(1, -1))

torch.Size([6])

torch.Size([2, 3])

tensor([[1, 2, 3],
[4, 5, 6]])

tensor([[3, 2, 4],
[8, 7, 9]])

tensor([[1, 2],
[3, 4]])

torch.Size([2, 2])

tensor([1, 2, 3, 4])

torch.Size([4])

tensor([[1, 2, 3, 4]])

import torch

t1 = torch.tensor([[1,2],[5,6]])
t2 = torch.tensor([[10,20,30],[50,60,70]])

t3 = torch.cat([t1,t2], dim=1)
print(t1)
print('\n', t2)
print('\n', t3)

t4 = torch.tensor([[1,2,3],[4,5,6]], dtype=torch.float32)
t5 = torch.FloatTensor([[1,2,3],[4,5,6]])
print('\n', type(t4))
print('\n', type(t5))

t6 = torch.stack([t4,t5,t4,t5], dim=0)
print('\n', t6)

tensor([[1, 2],
[5, 6]])

tensor([[10, 20, 30],
[50, 60, 70]])

tensor([[ 1, 2, 10, 20, 30],
[ 5, 6, 50, 60, 70]])

< class 'torch.Tensor'>
< class 'torch.Tensor'>

tensor([[[1., 2., 3.],
[4., 5., 6.]],
[[1., 2., 3.],
[4., 5., 6.]],
[[1., 2., 3.],
[4., 5., 6.]],
[[1., 2., 3.],
[4., 5., 6.]]])

import torch

t1 = torch.tensor([[1,2,3],[4,5,6]])
print(t1)
print('\n', t1[:,:2])
print('\n', t1[t1>4])

t1[:,2] = 20
print('\n', t1)

t1[t1 > 4] = -5
print('\n', t1)


t2 = torch.tensor([[1,2,3],[4,5,6]])
t3 = torch.tensor([[7,8,9],[10,11,12]])
t4 = torch.cat([t2,t3], dim=0)
print(t4)

s1, s2, s3, s4 = torch.chunk(t4, 4, dim=0)
print('\n', s1)
print('\n', s4)

sy1, sy2, sy3 = torch.chunk(t4, 3, dim=1)
print('\n', sy1)

tensor([[1, 2, 3],
[4, 5, 6]])

tensor([[1, 2],
[4, 5]])

tensor([5, 6])

tensor([[ 1, 2, 20],
[ 4, 5, 20]])

tensor([[ 1, 2, -5],
[ 4, -5, -5]])

tensor([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]])

tensor([[1, 2, 3]])

tensor([[10, 11, 12]])

tensor([[ 1],
[ 4],
[ 7],
[10]])

chunk : dim 방향으로 다 자른다

import torch
import torch.nn.init as init

t1 = init.uniform_(torch.FloatTensor(3,4))
print(t1.size())
print('\n', t1)

t2 = init.normal_(torch.FloatTensor(3,4), std=0.2)
print('\n', t2)

t3 = init.constant_(torch.FloatTensor(3,4), 777.77)
print('\n', t3)

t4 = torch.FloatTensor(torch.randn(3,4))
print('\n', t4)

torch.Size([3, 4])

tensor([[0.9186, 0.6311, 0.0660, 0.8973],
[0.3389, 0.6315, 0.8841, 0.8725],
[0.9504, 0.6670, 0.5082, 0.4055]])

tensor([[ 0.5132, -0.2440, 0.1335, 0.0399],
[ 0.1909, -0.3425, -0.0221, 0.0064],
[ 0.1837, 0.1882, 0.0792, 0.2777]])

tensor([[777.7700, 777.7700, 777.7700, 777.7700],
[777.7700, 777.7700, 777.7700, 777.7700],
[777.7700, 777.7700, 777.7700, 777.7700]])

tensor([[-1.0429, 0.1216, -1.7910, -0.5905],
[ 1.1782, -1.1505, -1.5967, -0.3488],
[ 0.4220, -1.9529, -0.2291, -0.6608]])

import torch

t1 = torch.arange(10)
print(t1)
t2 = torch.zeros(10,2)
print('\n', t2)

t3 = torch.zeros(2,1,2,1,2)
print('\n', t3.size())

t4 = torch.squeeze(t3)
print('\n', t4.size())
print('\n', torch.squeeze(t3, dim=1).size())

t5 = torch.zeros(2,3)
print('\n', t5.size())
t6 = torch.unsqueeze(t5, dim=0)
print('\n', t6.size())
print('\n', torch.unsqueeze(t5, dim=1).size())
print('\n', torch.unsqueeze(t5, dim=2).size())

tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

tensor([[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.],
[0., 0.]])

torch.Size([2, 1, 2, 1, 2])
torch.Size([2, 2, 2])
torch.Size([2, 2, 1, 2])
torch.Size([2, 3])
torch.Size([1, 2, 3])
torch.Size([2, 1, 3])
torch.Size([2, 3, 1])

import torch

x = torch.tensor([2., 3.], requires_grad=True)
w = x ** 2
y = 2 * w + 6

target = torch.tensor([3., 4.])
loss = torch.sum(torch.square(y - target))

loss.backward()
print('x.gradient: ', x.grad)

x.gradient: tensor([176., 480.])

미분 대상 변수는 requires_grad=True가 필수!

import torch

x = torch.tensor([2., 3.], requires_grad=True)
w = x ** 2
y = 2 * w + 6

target = torch.tensor([3., 4.])
loss = torch.sum(torch.square(y - target))

loss.backward()
print('x.gradient: ', x.grad)

a = torch.tensor(2.0, requires_grad=True)
epochs = 20
for epoch in range(epochs):
    y = 2 * a
    y.backward()
    print('a로 미분한 값: ', a.grad)
    #optimizer.zero_grad()

x.gradient: tensor([176., 480.])

a로 미분한 값: tensor(2.)
a로 미분한 값: tensor(4.)
a로 미분한 값: tensor(6.)
a로 미분한 값: tensor(8.)
a로 미분한 값: tensor(10.)
a로 미분한 값: tensor(12.)
a로 미분한 값: tensor(14.)
a로 미분한 값: tensor(16.)
a로 미분한 값: tensor(18.)
a로 미분한 값: tensor(20.)
a로 미분한 값: tensor(22.)
a로 미분한 값: tensor(24.)
a로 미분한 값: tensor(26.)
a로 미분한 값: tensor(28.)
a로 미분한 값: tensor(30.)
a로 미분한 값: tensor(32.)
a로 미분한 값: tensor(34.)
a로 미분한 값: tensor(36.)
a로 미분한 값: tensor(38.)
a로 미분한 값: tensor(40.)

미분대상.backward()
미분변수.grad()
epoch가 진행되는 동안 미분 값 누적
방지를 위해 optimizer.zero_grad()로 미분 값 누적 초기화

import torch
import torch.optim as optim

x_train = torch.FloatTensor([[1], [2], [3]])
y_train = torch.FloatTensor([[2], [4], [6]])

w = torch.zeros((1,1), requires_grad=True)
b = torch.zeros(1, requires_grad=True)

optimizer = optim.SGD([w,b], lr=0.01)

for epoch in range(2001):
    hypothesis = torch.mm(x_train, w) + b
    loss = torch.mean((hypothesis - y_train)**2) # mse

    optimizer.zero_grad()
    loss.backward()
    optimizer.step() # 미분된 값을 수정해주는 fn

    if epoch % 100 == 0:
        print('epoch:{} w:{:.3f} b:{:.3f} loss:{:.6f}'\
              .format(epoch, w.item(), b.item(), loss.item()))

epoch:0 w:0.187 b:0.080 loss:18.666666
epoch:100 w:1.746 b:0.578 loss:0.048171
epoch:200 w:1.800 b:0.454 loss:0.029767
epoch:300 w:1.843 b:0.357 loss:0.018394
epoch:400 w:1.876 b:0.281 loss:0.011366
epoch:500 w:1.903 b:0.221 loss:0.007024
epoch:600 w:1.924 b:0.174 loss:0.004340
epoch:700 w:1.940 b:0.136 loss:0.002682
epoch:800 w:1.953 b:0.107 loss:0.001657
epoch:900 w:1.963 b:0.084 loss:0.001024
epoch:1000 w:1.971 b:0.066 loss:0.000633
epoch:1100 w:1.977 b:0.052 loss:0.000391
epoch:1200 w:1.982 b:0.041 loss:0.000242
epoch:1300 w:1.986 b:0.032 loss:0.000149
epoch:1400 w:1.989 b:0.025 loss:0.000092
epoch:1500 w:1.991 b:0.020 loss:0.000057
epoch:1600 w:1.993 b:0.016 loss:0.000035
epoch:1700 w:1.995 b:0.012 loss:0.000022
epoch:1800 w:1.996 b:0.010 loss:0.000013
epoch:1900 w:1.997 b:0.008 loss:0.000008
epoch:2000 w:1.997 b:0.006 loss:0.000005

optimizer = optim.SGD(수정해야 하는 변수, lr=0.01)
optimizer.step() : 미분된 값을 통해 변수 수정
.item() : 해당 변수의 값을 가져옴

import torch
import torch.optim as optim

x1_train = torch.FloatTensor([[77],[93],[88],[96],[73]])
x2_train = torch.FloatTensor([[80],[88],[91],[99],[55]])
x3_train = torch.FloatTensor([[75],[93],[90],[70],[100]])

y_train = torch.FloatTensor([[153],[175],[180],[192],[132]])

w1 = torch.zeros((1,1), requires_grad=True)
w2 = torch.zeros((1,1), requires_grad=True)
w3 = torch.zeros((1,1), requires_grad=True)
b = torch.zeros(1, requires_grad=True)

optimizer = optim.SGD([w1, w2, w3, b], lr=1e-5)

for epoch in range(1001):
    hypothesis = torch.mm(x1_train, w1) + torch.mm(x2_train, w2) + \
        torch.mm(x3_train, w3) + b
    loss = torch.mean((hypothesis - y_train)**2)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if epoch % 100 == 0:
        print('epoch:{} w1:{:.3f} w2:{:.3f} w3:{:.3f} b:{:.3f} loss:{:.6f}'\
                .format(epoch, w1.item(), w2.item(), w3.item(), \
                        b.item(), loss.item()))

epoch:0 w1:0.288 w2:0.281 w3:0.282 b:0.003 loss:28144.400391
epoch:100 w1:0.721 w2:0.828 w3:0.428 b:0.007 loss:46.224365
epoch:200 w1:0.752 w2:0.925 w3:0.303 b:0.007 loss:19.033472
epoch:300 w1:0.769 w2:0.977 w3:0.234 b:0.006 loss:10.972854
epoch:400 w1:0.779 w2:1.005 w3:0.197 b:0.006 loss:8.581089
epoch:500 w1:0.786 w2:1.020 w3:0.176 b:0.006 loss:7.869298
epoch:600 w1:0.790 w2:1.028 w3:0.164 b:0.006 loss:7.655379
epoch:700 w1:0.793 w2:1.031 w3:0.158 b:0.006 loss:7.589105
epoch:800 w1:0.795 w2:1.033 w3:0.154 b:0.006 loss:7.566663
epoch:900 w1:0.797 w2:1.033 w3:0.152 b:0.006 loss:7.557281
epoch:1000 w1:0.799 w2:1.033 w3:0.150 b:0.006 loss:7.551833

import torch
import torch.nn as nn
import torch.nn.init as init
import torch.optim as optim

x_train = init.uniform_(torch.Tensor(1000,1), -10, 10)
y_train = init.normal_(torch.Tensor(1000,1), std=0.2)

y_train = y_train + 2*x_train + 3

model = nn.Linear(1,1)  # 특성의 개수, 출력의 개수
loss_func = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01) # [w,b]

for epoch in range(500):
    hypothesis = model(x_train) # hypothesis = torch.mm(x_train, w) + b
    loss = loss_func(hypothesis, y_train)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if epoch % 50 == 0:
        print(loss.item())

149.63551330566406
1.4559900760650635
0.22867046296596527
0.06568844616413116
0.044045306742191315
0.041171204298734665
0.04078955203294754
0.04073886573314667
0.040732137858867645
0.040731243789196014

import torch
import torch.nn as nn

class CustomLinear(nn.Module):
    def __init__(self, input_size, output_size):
        super().__init__()
        self.w = nn.Parameter(torch.FloatTensor(torch.randn(input_size, output_size)), requires_grad=True)
        self.b = nn.Parameter(torch.FloatTensor(torch.randn(output_size)), requires_grad=True)

    def forward(self, x):
        y = torch.mm(x, self.w) + self.b
        return y

x_train = torch.FloatTensor(torch.randn(16, 10))
model = CustomLinear(10, 5)
hypothesis = model(x_train)
print(hypothesis)

hypothesis2 = model.forward(x_train)
print('\n', hypothesis)

tensor([[-2.4298, 1.3318, -3.7886, 1.8110, 2.1733],
[-1.2886, 1.0198, -2.6385, 3.9295, -0.8019],
[ 4.7981, 0.9384, 2.5878, -2.5623, -1.7327],
[-4.5052, 4.5540, -0.3263, 7.3198, 2.5599],
[ 0.9926, 1.0921, 1.6087, -0.6630, -0.9247],
[ 0.7960, -6.9522, 0.5479, -7.7465, -4.8559],
[ 1.9127, -2.5866, 2.2322, 3.4475, 0.1282],
[-1.2118, 4.6634, -1.9484, 3.3507, 0.6541],
[-1.2995, 3.0331, 1.7786, -4.6596, 1.9005],
[-2.5205, 7.7069, -4.2370, -1.4448, -1.1270],
[ 6.6704, -0.1321, 3.0728, -5.1529, -1.8744],
[-1.8656, -2.6338, 4.0059, 3.9025, 2.3784],
[-2.7870, 0.3020, -0.8925, 3.1671, 1.3450],
[ 0.7843, 5.0509, 0.8108, 1.3060, 0.6556],
[ 2.9843, 0.0305, 4.8794, 0.5420, -5.4174],
[ 1.0737, 2.0368, 5.0380, 2.7313, 1.5146]], grad_fn=)

hypothesis = model(x_train) 와 hypothesis2 = model.forward(x_train)의 결과 값은 같다

import torch
import torch.nn as nn

class CustomLinear(nn.Module):
    def __init__(self, input_size, output_size):
        super(CustomLinear, self).__init__()
        self.linear = nn.Linear(input_size, output_size)

    def forward(self, x):
        y = self.linear(x)
        return y

x = torch.FloatTensor(torch.randn(16, 10))
model = CustomLinear(10, 5)
hypothesis = model(x)
target = 100
loss = (target - hypothesis.sum())
loss.backward()

print(loss)

tensor(98.7330, grad_fn=< RsubBackward1>)

저작자표시

'프로그래머 > Pytorch' 카테고리의 다른 글

ants/bees classification CNN model에서 배울 수 있는 소소한 팁들(python) (0)	2020.06.22
[pytorch] linear regression class를 nn.Sequential로 간단하게 표현 (0)	2020.06.16
[Pytorch] VGG CIFAR-10에 적용 및 정리 (2)	2020.06.07
[Pytorch] VGG 구현 및 정리 (2)	2020.06.07
[Pytorch] MNIST 간단한 CNN 구현 및 정리 (0)	2020.06.07

적자생존의 법칙

[Pytorch] 기초 강의 day1

Pytorch 기초 정리

torch

'프로그래머 > Pytorch' 카테고리의 다른 글

티스토리툴바

[Pytorch] 기초 강의 day1

Pytorch 기초 정리

torch

'프로그래머 > Pytorch' 카테고리의 다른 글

'프로그래머/Pytorch' Related Articles

티스토리툴바