機器學習已經以各種不同的方式實施到各種系統中，以使其更加優化和準確。在這項研究中，機器學習和支付系統相結合，創建了一種基於情緒識別的新型支付系統。這項研究創建一個情緒識別模型，以下情緒認識的準確率為 83.74%：中性、快樂、驚訝、悲傷、憤怒、厭惡和恐懼。我們將情感識別模型置於安全支付系統的核心。與使用機器學習的其他支付方式相比，這種方法允許我們建立一個動態和準確的支付系統。

Machine learning has already been implemented in a variety of different ways into a variety of systems to make them more optimal and accurate. In this study, machine learning and payment systems are combined to create a novel payment system based on emotions and the recognition of those emotions. This study was able to create an emotion recognition model with 83.74% accuracy of the following emotions: neutral, happy, surprise, sadness, anger, disgust, and fear. The emotion recognition model was then placed at the core of a secure payment system. This approach allowed for a dynamic and accurate payment system when compared to other payment methods using machine learning.

Acknowledgement i
Abstract ii
摘要 iii
Table of Contents iv
List of Figures vii
List of Tables ix
Chapter 1. Introduction 1
1.1 Research Background and Motivation 1
1.2 Research Objectives 2
1.3 Research Contributions 3
1.4 Organization of Thesis 3
Chapter 2. Related Works 5
2.1 Traditional Payment Systems 5
2.2 Modern Payment Systems 8
2.2.1 Contactless Smartcard 8
2.2.2 Mobile Payment Apps 10
2.3 Disadvantages with Typical User Experiences using Traditional and Modern Payment Systems 13
2.4 Facial Recognition 18
2.5 Other Implementations of Emotion Recognition 21
Chapter 3. Proposed Method 24
3.1 Neural Network Design 27
3.2 Dataset 29
3.2.1 FER Dataset 30
3.2.2 FER+ Dataset 31
3.3 Using the model 32
3.4 Registration for the Payment System 35
3.5 Payment in Store 36
Chapter 4. Implementation 38
4.1 Neural Network 38
4.1.1 Outputting the Trained Model 43
4.1.2 The Trained Model 45
4.2 Payment System Implementation 49
4.2.1 In-store Application 49
4.2.2 Users Server 55
4.2.3 Emotion Recognition Server 55
Chapter 5. Conclusion and Future Works 58
References 61

[1] S. Kemp. "6 In 10 People Around The World Now Use The Internet." Kepios. https://datareportal.com/reports/6-in-10-people-around-the-world-now-use-the-internet (accessed May 30th, 2021).
[2] S. Processing. "Credit Card Statistics." https://shiftprocessing.com/credit-card/ (accessed May 29th, 2021).
[3] D. Robertson, "Payment Cards Projected Worldwide," The Nilson Report, no. 1140, p. 12, 2018. [Online]. Available: https://nilsonreport.com/upload/issues/1140_0321.pdf.
[4] VISA. "What are contactless payments?" https://usa.visa.com/pay-with-visa/contactless-payments/contactless-payments.html (accessed May 30th, 2021).
[5] Mastercard. "Contactless Payments." https://www.mastercard.us/en-us/personal/ways-to-pay/contactless.html (accessed May 30th, 2021).
[6] iPass. "iPass." iPass. https://www.i-pass.com.tw/en (accessed May 26th, 2021).
[7] iPass. "Adult Card." iPass. https://www.i-pass.com.tw/en/Page/StandardAdult (accessed May 26th, 2021).
[8] Apple. "Apple Pay." https://www.apple.com/apple-pay/ (accessed May 4, 2021).
[9] N. Kerris, "Apple Announces Apple Pay," T. Muller, Ed., ed. Apple Newsroom: Apple, 2014.
[10] C. M. Kopp. "The Most Popular Mobile Payment Apps." Dotdash. https://www.investopedia.com/articles/personal-finance/040616/best-mobile-payment-apps-tap-and-pay.asp (accessed May 25th, 2021).
[11] Apple. "Apple Pay is compatible with these devices." Apple. https://support.apple.com/en-us/HT208531 (accessed May 25th, 2021).
[12] Apple. "Apple Watch Series 3." Apple. https://www.apple.com/apple-watch-series-3/ (accessed May 25th, 2021).
[13] Apple. "Apple Pay participating banks and card issuers in Asia-Pacific." Apple. https://support.apple.com/en-us/HT206638 (accessed May 25th, 2021).
[14] Apple. "Set up Apple Pay." Apple. https://support.apple.com/en-us/HT204506 (accessed May 25th, 2021).
[15] Google. "Google Pay." https://pay.google.com/payments/u/0/home# (accessed May 4th, 2021).
[16] Google. "Taiwan: Supported Payment Methods." Google Pay. https://support.google.com/pay/answer/7404680 (accessed May 25th, 2021).
[17] Venmo. "What is Venmo and how does it work?" Paypal. https://www.paypal.com/us/smarthelp/article/venmo-payments-faq3445 (accessed May 29th, 2021).
[18] T.-K. Chang, "A Secure Operational Model for Mobile Payments," The Scientific World Journal Research Article vol. 2014, p. 14, 2014.
[19] L. P. Corporation. "LINE Pay." https://pay.line.me/portal/global/main (accessed May 4th, 2021).
[20] Samsung. "Samsung Pay." https://www.samsung.com/global/galaxy/samsung-pay/ (accessed May 4th, 2021).
[21] R. d. Best. "Mobile payments worldwide - Statistics & Facts." statista. https://www.statista.com/topics/4872/mobile-payments-worldwide/#:~:text=In%202023%2C%20it%20is%20estimated,cards%2C%20as%20well%20as%20PayPal. (accessed March 3rd, 2021.
[22] A. B. Insaf Adjabi, Abdeldjalil Ouahabi, Abdelmalik Taleb-Ahmed, "Past, Present, and Future of Face Recognition: A Review," A Review, vol. 9, no. Electronics, 1188, 2020, doi: doi:10.3390/electronics9081188.
[23] J. X. Lin-Lin Zhang, Daim Jeong, Tabe Ekouka, Ha-Kyun Kim, "The Effects of Facial Recognition Payment Systems on Intention to Use in China," Journal of Advanced Researches and Reports, vol. 1, no. 1, 2021, doi: http://dx.doi.org/10.21742/jarr.2021.1.05.
[24] A. M. Madeline B. Harms, Gregory L. Wallace, "Facial Emotion Recognition in Autism Spectrum Disorders: A Review of Behavioral and Neuroimaging Studies," Neuropsychol Rev, vol. 20, pp. 290-322, 2010, doi: https://doi.org/10.1007/s11065-010-9138-6.
[25] H. M. Francisco A Pujol, Ana Martínez, "Emotion Recognition to Improve e-Healthcare Systems in Smart Cities," INV - UNICAD - Comunicaciones a Congresos, Conferencias, etc., 2019, doi: 10.1007/978-3-030-30809-4_23.
[26] P. C.-Y. Ghada Zamzmi, Dmitry Goldgof, R Kasturi, Terri Ashmeade, Yu Sun, "A Comprehensive and Context-Sensitive Neonatal Pain Assessment Using Computer Vision," IEEE Transactions on Affective Computing, 2019, doi: 10.1109/TAFFC.2019.2926710.
[27] A. Y. Hua Gao, Jean-Phillippe Thiran, "Detecting emotional stress from facial expressions for driving safety," 2014 IEEE International Conference on Image Processing (ICIP), pp. 5961-5965, 2014, doi: 10.1109/ICIP.2014.7026203.
[28] M. Sambare. "FER-2013." https://www.kaggle.com/msambare/fer2013 (accessed 2021).
[29] P. Giannopoulos, "Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013," Advances in Hybridization of Intelligent Methods. Smart Innovation, Systems and Technologies, vol. 85, 2018.
[30] G. S. Minh-An Quinn, Guilherme Reis, "Real-time Emotion Recognition From Facial Expressions," p. 5. [Online]. Available: http://cs229.stanford.edu/proj2017/final-reports/5243420.pdf
[31] E. Barsoum, C. Zhang, C. C. Ferrer, and Z. Zhang, "Training deep networks for facial expression recognition with crowd-sourced label distribution," in Proceedings of the 18th ACM International Conference on Multimodal Interaction, 2016, pp. 279-283.
[32] P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 8-14 Dec. 2001 2001, vol. 1, pp. I-I, doi: 10.1109/CVPR.2001.990517.
[33] K.-Y. P. Byung Woo Chung, Sun-Young Hwang, "A Fast and Efficient Haar-Like Feature Selection Algorithm for Object Detection," The Journal of Korea Information and Communications Society, vol. 38A, 6, pp. 486-491, 2013, doi: https://doi.org/10.7840/kics.2013.38A.6.486.
[34] K. D. a. S. Sharma, "Review and comparison of face detection algorithms," 7th International Conference on Cloud Computing, Data Science & Engineering - Confluence, pp. 629-633, 2017, doi: 10.1109/CONFLUENCE.2017.7943228.
[35] T. A. M. Saad Albawi, Saad Al-Zawi, "Understanding of a convolutional neural network," International Conference on Engineering and Technology (ICET), pp. 1-6, 2017, doi: 10.1109/ICEngTechnol.2017.8308186.
[36] Keras. "Keras, Deep learning for humans." https://keras.io/ (accessed 2021).
[37] PyTorch. "From Research to Production." https://pytorch.org/ (accessed 2021).
[38] TensorFlow. "An end-to-end open source machine learning platform." https://www.tensorflow.org/ (accessed 2021).
[39] "Welcome to Colaboratory." Google. https://colab.research.google.com/notebooks/intro.ipynb (accessed 2021).
[40] Unsplash. "Photos for everyone." https://unsplash.com/ (accessed 2021).
[41] Flask. "Flask - web development, one drop at a time." https://flask.palletsprojects.com/en/2.0.x/ (accessed 2021).
[42] S. Consortium. "What is SQLite?" https://www.sqlite.org/index.html (accessed 2021).