隨著人工智慧的成熟，並在中西醫有越來越多的應用，有越來越多的技術在輔助醫師，而舌診在中醫望診中站著重要的一環，通過對舌象的細微觀察，以中醫獨特的一套系統去分析，將舌頭與臟腑聯繫瞭解病變所在，再依據體質與症狀的分類進行適當的治療。

因此本論文嘗試以深度學習的方式分兩部分研究舌象，第一部分為以本論文所提出卷積神經網路模型來依據舌象辨識中醫體質量表中的平和質與否，在只保留舌體之圖片的實驗中CNN模型的準確率達80.3％；而經稍加修改的VGG16模型準確率達96.5％。第二部分為使用YOLOv4模型偵測齒痕舌與非平滑舌，將有舌頭的圖片分粗略及精細兩種版本，以亮度、色度、對比、銳度四種屬性增強進行資料擴充後進行訓練，實驗中判斷非平滑舌的精細版本準確率達90％；判斷齒痕舌的粗略版本準確率達92％。

本論文兩部分的研究中，資料集內兩種標籤的數量雖不平衡，但實驗結果表明仍有一定的表現，說明不平衡的舌頭資料集還是可以在不特別處理的情況下透過深度學習有一些作用。

As the maturity of the artificial intelligence growing, there are more and more applications in both Modern and Traditional Chinese Medicine assisting doctors by the technologies of artificial intelligence. Meanwhile, tongue diagnosis plays an important role of the “Seeing Diagnosis”, which is one of the “Four Diagnosis of Traditional Chinese Medicine”. Through the subtle observation of tongue and analyzing it with a unique system of connecting tongue with viscera to understand location of the disease, and give an appropriate medical treatment according to the body constitution and symptoms.

Therefore, this thesis attempts to study the tongue image in two parts using deep learning. The first part is to use the convolutional neural network model proposed in this thesis to identify the “gentleness/plain body constitution” in the Body Constitutions Questionnaire (BCQ) based on the tongue image. In the experiment that image only retaining the tongue, the accuracy of the CNN model reached 80.3%; and the accuracy of the slightly modified VGG16 model reached 96.5%. The second part is to use the YOLOv4 model to detect tongue with tooth marks. The images with tongue are divided into rough and fine two versions of dataset and both are enhanced by brightness, chroma, contrast, and sharpness four attributes for data expansion before training. In the experiment, the accuracy of the fine version reached 90%; and the accuracy of the rough version reached 92%.

In the two parts of in this thesis, although the amounts of two labels in the dataset are unbalanced, the experimental results show that there is still a certain performance, indicating that the unbalanced tongue dataset can still have some effects through deep learning without special processing.

第一章 緒論 1
1-1 研究背景 1
1-2 研究動機與目的 3
1-3 論文架構 4
第二章 相關研究與文獻探討 5
2-1 相關文獻 5
2-1-1 舌頭影像與物件偵測 5
2-1-2 自動化舌診系統 5
2-2 中醫舌象診斷 6
2-2-1 舌象常見理論 6
2-2-2 舌診的內容 10
2-3 中醫體質量表 12
2-4 卷積神經網路 14
2-5 物件偵測模型YOLO 18
2-5-1 YOLO 18
2-5-2 YOLOv2 24
2-5-3 YOLOv3 29
2-5-4 YOLOv4 34
2-6 評估方式 39
2-6-1 混淆矩陣 39
2-6-2 mAP 41
2-6-3 IOU 42
第三章 研究方法 43
3-1 研究設備與環境 43
3-2 資料集 44
3-2-1 舌象辨識的資料集 44
3-2-2 齒痕舌辨識的資料集 46
3-3 舌象辨識 48
3-3-1 資料處理 48
3-3-2 數據選擇 53
3-3-3 模型訓練 55
3-4 齒痕舌辨識 59
3-4-1 資料處理 60
3-4-2 模型訓練 65
第四章 實驗設計與結果討論 68
4-1 舌象辨識結果 68
4-1-1 CNN模型的各項結果 69
4-1-2 VGG16模型的各項結果 71
4-2 齒痕舌辨識結果 74
4-2-1 精細版本資料集之各項結果 77
4-2-2 粗略版本資料集之各項結果 81
4-3 問題討論 85
4-3-1 舌象辨識結果討論 85
4-3-2 齒痕舌辨識實驗過程 86
4-3-3 齒痕舌辨識結果討論 91
第五章 結論與未來展望 96
5-1 結論 96
5-2 未來展望 97
參考文獻 98

[1] Weng, H., Li, L., Lei, H., Luo, Z., Li, C., & Li, S. (2021). A weakly supervised
tooth‐mark and crack detection method in tongue image. Concurrency and
Computation: Practice and Experience, e6262. https://doi.org/10.1002/cpe.6262
[2] C. Li, D. Zhang and S. Chen, "Research about Tongue Image of Traditional
Chinese Medicine(TCM) Based on Artificial Intelligence Technology," 2020
IEEE 5th Information Technology and Mechatronics Engineering Conference
(ITOEC), 2020, pp. 633-636, doi: 10.1109/ITOEC49072.2020.9141557.
[3] 林俊汶. (2014). 自動化舌診系統舌部輪廓提取之優化方法. 中山大學資訊工
程學系研究所學位論文, 1-70.
[4] 王節宣. (2011). 舌診影像自動診斷系統之研製. 朝陽科技大學資訊與通訊系
碩士班學位論文, 1-42.
[5] 王彥暉。 (2015)。 臨床實用舌象圖譜。 新北市：合記圖書出版社。
[6] Lin, J. D., Lin, J. S., Chen, L. L., Chang, C. H., Huang, Y. C., & Su, Y. C. (2012).
BCQs: a body constitution questionnaire to assess stasis in traditional Chinese
medicine. European Journal of Integrative Medicine, 4(4), e379-e391.
[7] 林幼淳. (2016). 兩種中醫體質量表之一致性評估. 中國醫藥大學公共衛生學
系碩士班學位論文, 1-99.
[8] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied
to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-
2324, Nov. 1998, doi: 10.1109/5.726791.
[9] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You Only Look Once:
Unified, Real-Time Object Detection," 2016 IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), 2016, pp. 779-788, doi:
10.1109/CVPR.2016.91.
[10] A. Neubeck and L. Van Gool, "Efficient Non-Maximum Suppression," 18th
International Conference on Pattern Recognition (ICPR'06), 2006, pp. 850-855,
doi: 10.1109/ICPR.2006.479.
99
[11] Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger.
In Proceedings of the IEEE conference on computer vision and pattern
recognition (pp. 7263-7271).
[12] Farhadi, A., & Redmon, J. (2018, April). Yolov3: An incremental improvement.
In Computer Vision and Pattern Recognition (pp. 1804-02767).
[13] Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed
and accuracy of object detection. arXiv preprint arXiv:2004.10934.
[14] Qu, P., Zhang, H., Zhuo, L., Zhang, J., & Chen, G. (2017, August). Automatic
tongue image segmentation for traditional chinese medicine using deep neural
network. In International Conference on Intelligent Computing (pp. 247-259).
Springer, Cham.
[15] Wang, C. Y., Bochkovskiy, A., & Liao, H. Y. M. (2021). Scaled-yolov4: Scaling
cross stage partial network. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition (pp. 13029-13038).
[16] 羅倫謙，蔣依吾。 (2019)。 現代臨床望舌。 台北市：知音出版社。
[17] Confusion matrix. (2021, July 4). In Wikipedia.
https://en.wikipedia.org/wiki/Confusion_matrix
[18] S. Ren, K. He, R. Girshick and J. Sun, "Faster R-CNN: Towards Real-Time
Object Detection with Region Proposal Networks," in IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 1 June
2017, doi: 10.1109/TPAMI.2016.2577031.
[19] https://github.com/AlexeyAB/darknet/wiki/YOLOv4-model-zoo
[20] https://github.com/AlexeyAB/darknet
[21] 詹鈞婷. (2019). 基於深度學習之心臟血管鈣化偵測系統. 國立中央大學資
訊工程學系碩士論文, 1-90.
[22] https://github.com/hai-h-nguyen/Yolo2Pascal-annotation-conversion