인공지능 #7-2 | 심층 신경망 DNN, ReLU, 옵티마이저

7. 딥러닝¶

데이터 준비하기: 패션 MNIST

In [1]:

from tensorflow import keras
(train_input, train_target), (test_input, test_target) = keras.datasets.fashion_mnist.load_data()

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz
29515/29515 [==============================] - 0s 0us/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz
26421880/26421880 [==============================] - 1s 0us/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz
5148/5148 [==============================] - 0s 0s/step
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz
4422102/4422102 [==============================] - 0s 0us/step

In [2]:

# 데이터의 크기
# 60000개의 샘플, 각 샘플은 28*28
print(train_input.shape, train_target.shape)
# 10000개의 샘플, 각 샘플은 28*28
print(test_input.shape, test_target.shape)

(60000, 28, 28) (60000,)
(10000, 28, 28) (10000,)

In [3]:

# 샘플 확인하기
import matplotlib.pyplot as plt
fig, axs = plt.subplots(1, 10, figsize=(10, 10))
for i in range(10):
  axs[i].imshow(train_input[i], cmap="gray_r")
  axs[i].axis("off")
plt.show()

In [4]:

# 타깃값은?
print([train_target[i] for i in range(10)])

[9, 0, 0, 3, 0, 2, 7, 2, 5, 5]

In [5]:

# 타겟과 타겟별 샘플 수
import numpy as np
print(np.unique(train_target, return_counts=True))

(array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint8), array([6000, 6000, 6000, 6000, 6000, 6000, 6000, 6000, 6000, 6000],
      dtype=int64))

7-2. 심층 신경망¶

데이터 불러오기

In [17]:

from tensorflow import keras
(train_input, train_target), (test_input, test_target) = keras.datasets.fashion_mnist.load_data()

데이터 전처리하기

In [18]:

from sklearn.model_selection import train_test_split
train_scaled = train_input / 255.0
train_scaled = train_scaled.reshape(-1, 28*28)
train_scaled, val_scaled, train_target, val_target = train_test_split(
    train_scaled, train_target, test_size=0.2, random_state=42
)

인공 신경망 모델에 층 추가하기

• 이때 추가되는 층을 은닉층 hidden layer라고 한다
• 은닉층이 있는 인공 신경망을 DNN Deep Neural Network라고 한다
• hidden layer에 적용하는 활성화 함수는 output layer에 비해 자유롭다(ex. ReLU)

In [19]:

# 은닉층 설정하기
dense1 = keras.layers.Dense(100, activation="sigmoid", input_shape=(784,))
dense2 = keras.layers.Dense(10, activation="softmax")

심층 신경망 만들기 Deep Neural Network

• 딥러닝의 강력한 성능은 층을 추가하여 입력 데이터에 대한 연속적인 학습을 하는 능력에서 온다

In [20]:

model = keras.Sequential([dense1, dense2]) # 순서대로 층 추가하기. 마지막이 출력층임

In [21]:

model.summary()

Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_1 (Dense)             (None, 100)               78500     
                                                                 
 dense_2 (Dense)             (None, 10)                1010      
                                                                 
=================================================================
Total params: 79,510
Trainable params: 79,510
Non-trainable params: 0
_________________________________________________________________

In [25]:

# 층을 추가하는 다른 방법
model = keras.Sequential([
    keras.layers.Dense(100, 
                       activation="sigmoid",
                       input_shape=(784,),
                       name="hidden"),
    keras.layers.Dense(10, 
                       activation="softmax",
                       name="output")
    ],
    name="패션 MNIST 모델"
    )

In [26]:

model.summary()

Model: "패션 MNIST 모델"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 hidden (Dense)              (None, 100)               78500     
                                                                 
 output (Dense)              (None, 10)                1010      
                                                                 
=================================================================
Total params: 79,510
Trainable params: 79,510
Non-trainable params: 0
_________________________________________________________________

In [27]:

# 또다른 방법
model = keras.Sequential()
model.add(keras.layers.Dense(100, activation="sigmoid", input_shape=(784,)))
model.add(keras.layers.Dense(10, activation="softmax"))
model.summary()

Model: "sequential_3"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_4 (Dense)             (None, 100)               78500     
                                                                 
 dense_5 (Dense)             (None, 10)                1010      
                                                                 
=================================================================
Total params: 79,510
Trainable params: 79,510
Non-trainable params: 0
_________________________________________________________________

모델 훈련시키기

In [28]:

model.compile(
    loss="sparse_categorical_crossentropy",
    metrics="accuracy"
)
model.fit(train_scaled, train_target, epochs=5)

Epoch 1/5
1500/1500 [==============================] - 4s 2ms/step - loss: 0.5669 - accuracy: 0.8066
Epoch 2/5
1500/1500 [==============================] - 3s 2ms/step - loss: 0.4109 - accuracy: 0.8514
Epoch 3/5
1500/1500 [==============================] - 3s 2ms/step - loss: 0.3758 - accuracy: 0.8639
Epoch 4/5
1500/1500 [==============================] - 4s 2ms/step - loss: 0.3535 - accuracy: 0.8721
Epoch 5/5
1500/1500 [==============================] - 3s 2ms/step - loss: 0.3360 - accuracy: 0.8786

Out[28]:

<keras.callbacks.History at 0x1ad7f139690>

ReLU 함수 = max(0, z)

• 이미지 분류에서 뛰어난 성능을 보임

In [29]:

model = keras.Sequential()
model.add(keras.layers.Flatten(input_shape=(28, 28)))
model.add(keras.layers.Dense(100, activation="relu"))
model.add(keras.layers.Dense(10, activation="softmax"))
model.summary()

Model: "sequential_4"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 flatten (Flatten)           (None, 784)               0         
                                                                 
 dense_6 (Dense)             (None, 100)               78500     
                                                                 
 dense_7 (Dense)             (None, 10)                1010      
                                                                 
=================================================================
Total params: 79,510
Trainable params: 79,510
Non-trainable params: 0
_________________________________________________________________

옵티마이저

• 옵티마이저: 신경망의 가중치와 절편을 학습하기 위한 알고리즘
• ex. SGD, 네스테로프 모멘텀, RMSprop, Adam
• 자세한 내용은 <핸즈온 머신러닝 2판> 참조