隨著大數據時代的到來和深度學習領域軟硬體的成熟，讓開發出相關應用不再是極為困難的任務。TensorFlow、Keras、PyTorch等主流框架受到全球開發者的歡迎。然而現有深度學習工具的固有缺陷，卻是使用者難以克服的障礙。傳統上高度複雜的多層次演算法形成類似於一個不可打開的黑盒子，最終形成的最佳模型也無法詳細地拆解分析，導致使用者無法自己有效的改進模型。由於訓練數據存在著偏見(bias)，再加上模型無法提煉出明確的規律，所以深度學習模型結果一定混雜了部分錯誤的知識，導致產生「系統性錯誤」，以至於無法達到使用者要求的正確率。本篇論文目的在探討現有深度學習工具之分析改進和應用，並提出方法解決現有深度學習的系統性缺陷。我們探討從原始模型預測錯誤的資料中學習次級模型(secondary models)，進而以模型集合(model set)提升準確率的深度學習工具改善架構和三種學習方法。前兩種方法是針對所有錯誤的資料去學習次級模型，第三種方法是針對最容易互相混淆的類別去學習次級模型。我們將所提出的架構套用在廣受歡迎的四個不同深度學習工具VGGNet、AlexNet、GoogLeNet、和ResNet上，並以土耳其伊茲密爾的一家超市收集的9種不同的海產品、COVID-19陽性病例的胸部X光圖像資料庫和被診斷為腦瘤的病人的腦部X光圖像組成的不同應用領域資料集進行實驗，測試所提方法的普遍性和準確率，同時比較資料量大小與訓練和預測時間的關係。實驗結果顯示我們的方法確實能有效提升各種深度學習工具的整體準確率，而且增加模型的數量並不會使得訓練和預測時間也跟著爆炸性成長。

With the arrival of big data and the maturity of deep learning software and hardware, the development of related applications is no longer an extremely difficult task. Mainstream frameworks such as TensorFlow, Keras, and PyTorch are popular among developers around the world. However, there are intrinsic problems with existing deep learning tools which cause difficulty in applying them. Traditional highly complex multi-level algorithms form a black box that cannot be opened, and the final model cannot be disassembled and analyzed in detail, making it impossible for users to improve the model effectively on their own. Because of the bias in the training data and the inability of the model to extract explicit laws, the results of the deep learning model are often mixed with some erroneous knowledge, resulting in "systematic errors". The purpose of this thesis is to explore the analytical applications of popular deep learning tools in order to overcome the shortcomings of existing methods. We have proposed three new approaches to improve the accuracy of any existing deep learning tool by extending the original model with secondary models learned from the erroneous data. The first two methods are to learn secondary models from all the erroneous data. The third method is to generate secondary models from the most confusing categories. We tested the accuracy and effectiveness of the proposed methods on four popular deep learning tools and compared the training/prediction time with respect to data size using three different datasets, including a dataset consisting of nine different seafood products collected from a supermarket in Izmir, Turkey, a database of chest X-ray images of COVID-19 positive cases, and brain X-ray images of patients diagnosed with brain tumors. The experimental results show that our methods are effective in improving the overall accuracy, and the additional models does not cause the training/prediction time to explode as well.

第一章 緒論　　1
第一節 研究背景與動機 1
第二節 研究目的與方法 2
第三節 研究成果 2
第四節 論文架構 3
第二章 相關研究與技術　　5
第一節 AlexNet　　5
第二節 VGGNet　　6
第三節 ResNet　　7
第四節 GoogLeNet　　8
第五節 集成學習(Ensemble Learning)　　9
第六節 遷移學習(Transfer Learning)　　10
第七節 自動編碼器(AutoEncoder)　　11
第八節 PyTorch　　12
第三章 效能可調式模型學習問題解決策略與方法　　13
第一節 現有方法缺失整理　　13
第二節 效能可調式模型集合學習策略與研究議題　　14
第三節 錯誤模式學習法架構與策略　　23
第一項 錯誤模式學習法之研究動機　　23
第二項 錯誤模式學習法架構　　23
第四節 Softmax指數鑑別法架構與策略　　27
第一項 Softmax指數鑑別法之研究動機　　27
第二項 Softmax指數鑑別法架構　　27
第五節 漸進式錯誤糾正法架構與策略　　33
第一項 漸進式錯誤糾正法之研究動機　　33
第二項 漸進式錯誤糾正法架構　　34
第四章 系統實作與效能評估　　41
第一節 實驗環境　　41
第二節 實驗資料　　42
第三節 實驗結果　　43
第一項 錯誤模式學習法(EPLM)實驗結果　　43
第二項 Softmax指數鑑別法(SIIM)實驗結果　　45
第三項 漸進式錯誤糾正法(IECM)實驗結果　　47
第四項 學習方法效能比較　　48
第五項 模型訓練和預測時間比較　　52
第四節 實驗總結　　58
第五章 結論與未來工作　　59
第一節 結論　　59
第二節 未來工作　　60

[1]韓揚銘, 深不可知!? 深度學習發展及運用現況。檢自https://mic.iii.org.tw/industry.aspx?id=355&list=75
[2]深度學習 – 維基百科。檢自https://zh.wikipedia.org/zh-tw/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0
[3]陳華夫, AI人工智慧「深度學習」的缺陷、補救、及我親身的見證。檢自https://www.daphago.com/wordpress/2020/01/15/33/
[4]A. Krizhevsky, I. Sutskever and G. E. Hinton, “ImageNet classification with deep convolutional neural networks.” Neural Information Processing Systems 25 (NIPS 2012), pp. 1106–1114, 2012.
[5]Joey S, Creating AlexNet on Tensorflow from Scratch. Part 2: Creating AlexNet. 檢自https://joeyism.medium.com/creating-alexnet-on-tensorflow-from-scratch-part-2-creating-alexnet-e0cd948d7b04
[6]K. Simonyan and A. Zisserman, (2015) “Very Deep Convolutional Networks for Large-Scale Image Recognition.” The 3rd International Conference on Learning Representations (ICLR2015), 2015
[7]「深度學習卷積神經網絡」一切皆可盤的深不見底的VggNet模型。檢自https://kknews.cc/tech/ejvk59z.html
[8]He Kaiming, Zhang Xiangyu, Ren Shaoqing and Sun Jian. “Deep Residual Learning for Image Recognition.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.770-778, 2016
[9]作者 wangxiaobei2019 博客园 pytorch_5.11 殘差網路—RestNet https://www.cnblogs.com/wangxiaobei2019/p/13330474.html
[10]作者 馬大哈先生 台部落 CNN相關要點介紹（二）——ResNet（殘差網絡）解析 https://www.twblogs.net/a/5c3a09b4bd9eee35b21db28c
[11]作者 linitachi iT邦幫忙 ResNet-從旁邊來囉 https://ithelp.ithome.com.tw/articles/10239805
[12]C. Szegedy et al., "Going deeper with convolutions," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1-9, doi: 10.1109/CVPR.2015.7298594.
[13]Sagi, O and Rokach, L. “Ensemble learning: A survey.”, WIREs Data Min. Knowl. Discov. 2018, 8, e1249
[14]遷移學習 – 維基百科。檢自https://zh.wikipedia.org/wiki/%E8%BF%81%E7%A7%BB%E5%AD%A6%E4%B9%A0 - cite_note-3
[15]Pratik Bhavsar, An Ultimate Guide To Transfer Learning In NLP.檢自https://www.topbots.com/transfer-learning-in-nlp/
[16]自動編碼器 – 維基百科。檢自https://zh.wikipedia.org/wiki/%E8%87%AA%E7%BC%96%E7%A0%81%E5%99%A8
[17]Yu-Ru Tsai, What are Autoencoders? 檢自https://medium.com/ai-academy-taiwan/what-are-autoencoders-175b474d74d1
[18]PyTorch – 維基百科。檢自https://zh.wikipedia.org/wiki/PyTorch
[19]Softmax函式 – 維基百科。檢自https://zh.wikipedia.org/wiki/Softmax%E5%87%BD%E6%95%B0
[20]A Large-Scale Dataset for Segmentation and Classification. https://www.kaggle.com/crowww/a-large-scale-fish-dataset?fbclid=IwAR0E0gXAXKMKIQBNYfmtOZgK0B_OcVqPqfDZBN943ATH3OraLguvb_NEhRU
[21]COVID-19 Radiography Dataset. https://www.kaggle.com/preetviradiya/covid19-radiography-dataset?select=metadata.csv&fbclid=IwAR1lygydwkzwYUZOd_1xFkZYRw7YraEppILjp3V9caZ17VJiGUKvyiDAOtI
[22]Brain Tumor Dataset. https://www.kaggle.com/preetviradiya/brian-tumor-dataset?fbclid=IwAR3Quz78nJOS1E96TcwKVcJkAOHyReMpCeOiMZzJqsLW0cEMp8g1oQ_ED8s
[23]A. Bagaskara and M. Suryanegara, "Evaluation of VGG-16 and VGG-19 Deep Learning Architecture for Classifying Dementia People," 2021 4th International Conference of Computer and Informatics Engineering (IC2IE), 2021, pp. 1-4, doi: 10.1109/IC2IE53219.2021.9649132.
[24]A. Ikuta et al., "Automatic Classification of Hemp and Cotton in Digital Macro Photography using VGG-16 for Nondestructive Paper Analysis," 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), 2019, pp. 496-498, doi: 10.1109/GCCE46687.2019.9015416.
[25]A. Budhiman, S. Suyanto and A. Arifianto, "Melanoma Cancer Classification Using ResNet with Data Augmentation," 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), 2019, pp. 17-20, doi: 10.1109/ISRITI48646.2019.9034624
[26]P. Aswathy, Siddhartha and D. Mishra, "Deep GoogLeNet Features for Visual Object Tracking," 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS), 2018, pp. 60-66, doi: 10.1109/ICIINFS.2018.8721317.
[27]Jinde Shubham, Ensemble Learning — Bagging and Boosting, 檢自https://becominghuman.ai/ensemble-learning-bagging-and-boosting-d20f38be9b1e
[28]S. -Y. Lin, C. -C. Chiang, Z. -S. Hung and Y. -H. Zou, "A Dynamic Data-Driven Fine-Tuning Approach for Stacked Auto-Encoder Neural Network," 2017 IEEE 14th International Conference on e-Business Engineering (ICEBE), 2017, pp. 226-231, doi: 10.1109/ICEBE.2017.43.
[29]M. H. Yapet al., "Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks," in IEEE Journal of Biomedical and Health Informatics, vol. 22, no. 4, pp. 1218-1226, July 2018, doi: 10.1109/JBHI.2017.2731873.
[30]D. Zhang, J. Lv and Z. Cheng, "An Approach Focusing on the Convolutional Layer Characteristics of the VGG Network for Vehicle Tracking," in IEEE Access, vol. 8, pp. 112827-112839, 2020, doi: 10.1109/ACCESS.2020.3003229.
[31]Z. Xiao, Y. Gong, Y. Long, D. Li, X. Wang and H. Liu, "Airport Detection Based on a Multiscale Fusion Feature for Optical Remote Sensing Images," in IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 9, pp. 1469-1473, Sept. 2017, doi: 10.1109/LGRS.2017.2712638.
[32]Tejas Mohan Ayyar, A practical experiment for comparing LeNet, AlexNet, VGG and ResNet models with their advantages and disadvantages., 檢自https://tejasmohanayyar.medium.com/a-practical-experiment-for-comparing-lenet-alexnet-vgg-and-resnet-models-with-their-advantages-d932fb7c7d17
[33]作者 DarK Kaggle Fish prediction-0.89 accuracy https://www.kaggle.com/nocopyrights/fish-prediction-0-89-accuracy
[34]作者 Gerry Kaggle Lung Disease F1 score ~91% https://www.kaggle.com/gpiosenka/lung-disease-f1-score-91
[35]作者 N. Nikonov Kaggle Brain_Tumor_Detection https://www.kaggle.com/nikitanikonov/brain-tumor-detection