[Kaggle] Contrast Cancer, CT/MRI 조영제 투약 여부 분석 (Deep learning with Keras)

CT(Iamages using Contrast Classification CNN with Keras(tensorflow))¶

• RI / CT 를 찍게 되면 조영제(Contrast)를 사용하게 되는데 사용한 그림인지 아닌지 구분하는 코드.

In [66]:

import dicom # dicom file handling library 
import numpy as np # matrix tools
import matplotlib.pyplot as plt # for basic plots
import seaborn as sns # for nicer plots
import pandas as pd
import re
from skimage.io import imread
import os
import keras

Using TensorFlow backend.

In [67]:

%matplotlib inline

In [6]:

os.listdir('data/dicom_dir')[:5]

Out[6]:

['ID_0000_AGE_0060_CONTRAST_1_CT.dcm',
 'ID_0001_AGE_0069_CONTRAST_1_CT.dcm',
 'ID_0002_AGE_0074_CONTRAST_1_CT.dcm',
 'ID_0003_AGE_0075_CONTRAST_1_CT.dcm',
 'ID_0004_AGE_0056_CONTRAST_1_CT.dcm']

In [164]:

PathDicom = 'data/dicom_dir'
lstFilesDCM = []
for dirName, subdirList, fileList in os.walk(PathDicom):
    for filename in fileList:
        if ".dcm" in filename.lower():
            lstFilesDCM.append(os.path.join(PathDicom,filename))

In [165]:

len(lstFilesDCM)

Out[165]:

100

Get ref file¶

In [166]:

RefDs = dicom.read_file(lstFilesDCM[0])

• Load dimensions based on the number of rows, columns, and slices (along the Z axis)

In [167]:

RefDs.pixel_array.shape

Out[167]:

(512, 512)

In [168]:

ConstPixelDims = (len(lstFilesDCM), int(RefDs.Rows), int(RefDs.Columns))

In [169]:

ConstPixelDims

Out[169]:

(100, 512, 512)

In [172]:

# The array is sized based on 'ConstPixelDims'
ArrayDicom = numpy.zeros(ConstPixelDims, dtype=RefDs.pixel_array.dtype)
print(ArrayDicom.shape)
# loop through all the DICOM files
for filenameDCM in lstFilesDCM:
    # read the file
    ds = dicom.read_file(filenameDCM)
    # store the raw image data
    ArrayDicom[lstFilesDCM.index(filenameDCM), :, :] = ds.pixel_array  

(100, 512, 512)

In [173]:

pyplot.imshow(ArrayDicom[20,:,:])

Out[173]:

<matplotlib.image.AxesImage at 0x2508338ef98>

In [174]:

check_contrast = re.compile(r'ID_([\d]+)_AGE_\d+_CONTRAST_([\d])_CT.dcm')

In [175]:

label = []
id_list = []
for image in lstFilesDCM:
    id_list.append(check_contrast.findall(image)[0][0])
    label.append(check_contrast.findall(image)[0][1])
label_list = pd.DataFrame(label,id_list)

In [176]:

len(ArrayDicom)

Out[176]:

100

Split Data to Train, Test¶

In [177]:

from sklearn.model_selection import train_test_split

In [178]:

ArrayDicom = np.expand_dims(ArrayDicom,0)

In [179]:

ArrayDicom = ArrayDicom.reshape((100,512,512,1))

In [214]:

ArrayDicom = ArrayDicom[:,::3,::3]

In [241]:

X_train, X_test, y_train, y_test = train_test_split(ArrayDicom, label_list, test_size=0.2, random_state=0)

In [242]:

X_train.shape

Out[242]:

(80, 86, 86, 1)

In [243]:

n_train, width, height,depth = X_train.shape
n_test,_,_,_ = X_test.shape

In [244]:

input_shape = (width,height,depth)
input_shape

Out[244]:

(86, 86, 1)

In [245]:

input_train = X_train.reshape((n_train, width,height,depth))
input_train.shape
input_train.astype('float32')
input_train = input_train / np.max(input_train)
input_train.max()

Out[245]:

1.0

In [246]:

input_test = X_test.reshape(n_test, *input_shape)
input_test.astype('float32')
input_test = input_test / np.max(input_test)

In [247]:

output_train = keras.utils.to_categorical(y_train, 2)
output_test = keras.utils.to_categorical(y_test, 2)
output_train[5]

Out[247]:

array([ 0.,  1.])

Network¶

In [248]:

from keras.models import Sequential
from keras.layers import Dense, Flatten
from keras.optimizers import Adam
from keras.layers import Conv2D, MaxPooling2D

In [249]:

model2 = Sequential()
model2.add(Conv2D(50, (5, 5), activation='relu', input_shape=input_shape))
 # 32개의 4x4 Filter 를 이용하여 Convolutional Network생성
model2.add(MaxPooling2D(pool_size=(4, 4), strides=(2,2))) # 3x3 Maxpooling 
model2.add(Conv2D(30, (4, 4), activation='relu', input_shape=input_shape))
model2.add(Conv2D(50, (3, 3), activation='relu', input_shape=input_shape))
model2.add(MaxPooling2D(pool_size=(2, 2))) # 2x2 Maxpooling 
model2.add(Flatten()) # 쭉풀어서 Fully Connected Neural Network를 만든다. 
model2.add(Dense(2, activation='softmax'))

In [250]:

model2.summary()

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_18 (Conv2D)           (None, 82, 82, 50)        1300      
_________________________________________________________________
max_pooling2d_15 (MaxPooling (None, 40, 40, 50)        0         
_________________________________________________________________
conv2d_19 (Conv2D)           (None, 37, 37, 30)        24030     
_________________________________________________________________
conv2d_20 (Conv2D)           (None, 35, 35, 50)        13550     
_________________________________________________________________
max_pooling2d_16 (MaxPooling (None, 17, 17, 50)        0         
_________________________________________________________________
flatten_8 (Flatten)          (None, 14450)             0         
_________________________________________________________________
dense_8 (Dense)              (None, 2)                 28902     
=================================================================
Total params: 67,782
Trainable params: 67,782
Non-trainable params: 0
_________________________________________________________________

In [251]:

batch_size = 30
epochs = 50

In [252]:

model2.compile(loss='categorical_crossentropy',
              optimizer=Adam(),
              metrics=['accuracy'])

In [253]:

history = model2.fit(input_train, output_train,
                    batch_size=batch_size,
                    epochs=epochs,
                    verbose=1,
                    validation_data=(input_test, output_test))

Train on 80 samples, validate on 20 samples
Epoch 1/50
80/80 [==============================] - 0s - loss: 0.6804 - acc: 0.5875 - val_loss: 0.6709 - val_acc: 0.5000
Epoch 2/50
80/80 [==============================] - 0s - loss: 0.6784 - acc: 0.5000 - val_loss: 0.6505 - val_acc: 0.7000
Epoch 3/50
80/80 [==============================] - 0s - loss: 0.6676 - acc: 0.6125 - val_loss: 0.6544 - val_acc: 0.7000
Epoch 4/50
80/80 [==============================] - 0s - loss: 0.6688 - acc: 0.6125 - val_loss: 0.6491 - val_acc: 0.7000
Epoch 5/50
80/80 [==============================] - 0s - loss: 0.6645 - acc: 0.6125 - val_loss: 0.6452 - val_acc: 0.7000
Epoch 6/50
80/80 [==============================] - 0s - loss: 0.6592 - acc: 0.6125 - val_loss: 0.6333 - val_acc: 0.7000
Epoch 7/50
80/80 [==============================] - 0s - loss: 0.6620 - acc: 0.6125 - val_loss: 0.6301 - val_acc: 0.7000
Epoch 8/50
80/80 [==============================] - 0s - loss: 0.6586 - acc: 0.6125 - val_loss: 0.6244 - val_acc: 0.7000
Epoch 9/50
80/80 [==============================] - 0s - loss: 0.6562 - acc: 0.6125 - val_loss: 0.6274 - val_acc: 0.7000
Epoch 10/50
80/80 [==============================] - 0s - loss: 0.6543 - acc: 0.6125 - val_loss: 0.6287 - val_acc: 0.7000
Epoch 11/50
80/80 [==============================] - 0s - loss: 0.6527 - acc: 0.6125 - val_loss: 0.6232 - val_acc: 0.7000
Epoch 12/50
80/80 [==============================] - 0s - loss: 0.6500 - acc: 0.6125 - val_loss: 0.6192 - val_acc: 0.7000
Epoch 13/50
80/80 [==============================] - 0s - loss: 0.6500 - acc: 0.6125 - val_loss: 0.6144 - val_acc: 0.7000
Epoch 14/50
80/80 [==============================] - 0s - loss: 0.6453 - acc: 0.6125 - val_loss: 0.6158 - val_acc: 0.7000
Epoch 15/50
80/80 [==============================] - 0s - loss: 0.6409 - acc: 0.6125 - val_loss: 0.6168 - val_acc: 0.7000
Epoch 16/50
80/80 [==============================] - 0s - loss: 0.6336 - acc: 0.6125 - val_loss: 0.6032 - val_acc: 0.7000
Epoch 17/50
80/80 [==============================] - 0s - loss: 0.6267 - acc: 0.6125 - val_loss: 0.5981 - val_acc: 0.7000
Epoch 18/50
80/80 [==============================] - 0s - loss: 0.6197 - acc: 0.6250 - val_loss: 0.5988 - val_acc: 0.7500
Epoch 19/50
80/80 [==============================] - 0s - loss: 0.6261 - acc: 0.6875 - val_loss: 0.6077 - val_acc: 0.6500
Epoch 20/50
80/80 [==============================] - 0s - loss: 0.6104 - acc: 0.6875 - val_loss: 0.5787 - val_acc: 0.7000
Epoch 21/50
80/80 [==============================] - 0s - loss: 0.5985 - acc: 0.6500 - val_loss: 0.5803 - val_acc: 0.7500
Epoch 22/50
80/80 [==============================] - 0s - loss: 0.5764 - acc: 0.7375 - val_loss: 0.5935 - val_acc: 0.7500
Epoch 23/50
80/80 [==============================] - 0s - loss: 0.5696 - acc: 0.7375 - val_loss: 0.5805 - val_acc: 0.8000
Epoch 24/50
80/80 [==============================] - 0s - loss: 0.5587 - acc: 0.7625 - val_loss: 0.5759 - val_acc: 0.8000
Epoch 25/50
80/80 [==============================] - 0s - loss: 0.5365 - acc: 0.7750 - val_loss: 0.5957 - val_acc: 0.7000
Epoch 26/50
80/80 [==============================] - 0s - loss: 0.5671 - acc: 0.7125 - val_loss: 0.5918 - val_acc: 0.7500
Epoch 27/50
80/80 [==============================] - 0s - loss: 0.5228 - acc: 0.7500 - val_loss: 0.6300 - val_acc: 0.7000
Epoch 28/50
80/80 [==============================] - 0s - loss: 0.5434 - acc: 0.7625 - val_loss: 0.6020 - val_acc: 0.7500
Epoch 29/50
80/80 [==============================] - 0s - loss: 0.5044 - acc: 0.7625 - val_loss: 0.6299 - val_acc: 0.6000
Epoch 30/50
80/80 [==============================] - 0s - loss: 0.4963 - acc: 0.7750 - val_loss: 0.6210 - val_acc: 0.6500
Epoch 31/50
80/80 [==============================] - 0s - loss: 0.4775 - acc: 0.7750 - val_loss: 0.6398 - val_acc: 0.7000
Epoch 32/50
80/80 [==============================] - 0s - loss: 0.4628 - acc: 0.7625 - val_loss: 0.6732 - val_acc: 0.6500
Epoch 33/50
80/80 [==============================] - 0s - loss: 0.4422 - acc: 0.8125 - val_loss: 0.6640 - val_acc: 0.7000
Epoch 34/50
80/80 [==============================] - 0s - loss: 0.4406 - acc: 0.7875 - val_loss: 0.6981 - val_acc: 0.7500
Epoch 35/50
80/80 [==============================] - 0s - loss: 0.4312 - acc: 0.8000 - val_loss: 0.7485 - val_acc: 0.6500
Epoch 36/50
80/80 [==============================] - 0s - loss: 0.4497 - acc: 0.7875 - val_loss: 0.7435 - val_acc: 0.6000
Epoch 37/50
80/80 [==============================] - 0s - loss: 0.4048 - acc: 0.8250 - val_loss: 0.7923 - val_acc: 0.6500
Epoch 38/50
80/80 [==============================] - 0s - loss: 0.4131 - acc: 0.8125 - val_loss: 0.7760 - val_acc: 0.6500
Epoch 39/50
80/80 [==============================] - 0s - loss: 0.3962 - acc: 0.8375 - val_loss: 0.7888 - val_acc: 0.7000
Epoch 40/50
80/80 [==============================] - 0s - loss: 0.3929 - acc: 0.8000 - val_loss: 0.7834 - val_acc: 0.6000
Epoch 41/50
80/80 [==============================] - 0s - loss: 0.4009 - acc: 0.8375 - val_loss: 0.7900 - val_acc: 0.6000
Epoch 42/50
80/80 [==============================] - 0s - loss: 0.4081 - acc: 0.8000 - val_loss: 0.8280 - val_acc: 0.6500
Epoch 43/50
80/80 [==============================] - 0s - loss: 0.3873 - acc: 0.8125 - val_loss: 0.8015 - val_acc: 0.6000
Epoch 44/50
80/80 [==============================] - 0s - loss: 0.3701 - acc: 0.8375 - val_loss: 0.8327 - val_acc: 0.7000
Epoch 45/50
80/80 [==============================] - 0s - loss: 0.3677 - acc: 0.8000 - val_loss: 0.8558 - val_acc: 0.7000
Epoch 46/50
80/80 [==============================] - 0s - loss: 0.4077 - acc: 0.8000 - val_loss: 0.8423 - val_acc: 0.6000
Epoch 47/50
80/80 [==============================] - 0s - loss: 0.3431 - acc: 0.8750 - val_loss: 0.9074 - val_acc: 0.6500
Epoch 48/50
80/80 [==============================] - 0s - loss: 0.3677 - acc: 0.8125 - val_loss: 0.8807 - val_acc: 0.6500
Epoch 49/50
80/80 [==============================] - 0s - loss: 0.3253 - acc: 0.8500 - val_loss: 0.8682 - val_acc: 0.6000
Epoch 50/50
80/80 [==============================] - 0s - loss: 0.3417 - acc: 0.8500 - val_loss: 0.8745 - val_acc: 0.6000

In [254]:

score = model2.evaluate(input_test, output_test, verbose=0)
score

Out[254]:

[0.87448054552078247, 0.60000002384185791]