串流資料服務讓組織對各類事件可以提供立即反應。因為資料價值隨時間遞減，所以低延遲的串流資料分析服務對各類使用者都越來越重要。然而就我們所知，串流資料分析服務建構需要相關工具與程式設計之相當程度專業知識，不存在高度可及與客製化的串流分析服務。有鑑於此，本研究提出一個新的串流分析服務框架SIFTTT，以及相關模組整合、溝通協定與協作模式。透過整合普受歡迎的開放軟體服務和工具，像是IFTTT、Apache Kafka，以及高效能串流平台Apache Spark Streaming等，成功地讓串流資料分析服務的建置與運作可以高度可及與客製化。我們實際建置SIFTTT系統並進行各項正確性、效能與可擴充性實驗。結果證實，我們所提出的架構和方法能夠成功達成高度可及與客製化的低延遲串流資料分析服務。從事件發生到反應啟動的延遲時間不超過4秒鐘。我們同時提供各種模板與指引，讓設備和服務廠商或一般使用者可以很容易地開發新的裝置和串流服務。

With streaming data services, organizations can do some action as soon as possible if needed. Low latency analysis on streaming data becomes increasingly important since the value of some data will decrease as time goes by. However, as far as we found, there is no available streaming service that can be easily accessed and customized by users without knowledge of streaming data tools and programming to do streaming data analysis yet. Therefore, in this work, we created Streaming IFTTT (SIFTTT), a highly accessible and customizable streaming service framework based on IFTTT and Spark Streaming. IFTTT is chosen since IFTTT can connect apps, devices and services from different developers [1], while we chose Spark Streaming because Spark Streaming support low latency analysis and is very scalable. To support SIFTTT, we proposed a new architecture that integrate some open-source tools, such as Apache Spark Streaming and Apache Kafka. Further, we conducted several experiments on SIFTTT architecture to test the correctness, performance, and scalability of the system. Experiments results show that SIFTTT system can work seamlessly and is also very reliable and scalable for low latency analysis and response on streaming data with around 4 seconds delay. Templates and guideline are also provided for device vendors and service developers to provide new services.

Chapter 1: Introduction 1
1.1 Research Background and Motivation 1
1.2 Research Objectives 2
1.3 Research Contributions 3
1.4 Limitations of Research 4
1.5 Organization of Thesis 4
Chapter 2: Related Work 5
2.1 Internet of Things (IoT) 5
2.2 Stream Processing and Services 7
2.3 Stream Processing on Internet of Things (IoT) 8
2.4 IFTTT 9
2.5 Apache Hadoop 11
2.6 Spark Streaming 12
2.7 Kafka 13
2.8 ZooKeeper 15
2.9 Gunicorn 16
2.10 Supervisor 17
2.11 NGINX 18
Chapter 3: System Architecture 19
3.1 The Goals of the System 19
3.2 Conceptual Framework of SIFTTT 20
3.3 Main Architecture of SIFTTT 21
3.4 SIFTTT Request Handler and Coordinator Software Design 24
3.5 Streaming Analysis Software Design 33
3.6 Adapter Software Design 35
3.6.1 Embedded System Hardware Design 36
3.6.2 Embedded System Software Design 40
3.7 Data Flow 42
3.8 Data Communication Protocols and Data Formats 44
Chapter 4: User-Side Flow and Templates 51
4.1 User-Side Flow 51
4.2 Templates and Guidelines of New Pattern Detection Service Module
53
4.3 Template and Guidelines of Adapter 56
Chapter 5: Implementation and Evaluation 59
5.1 Implementation Environment 59
5.2 Correctness Evaluation 60
5.3 Performance Evaluation 68
5.4 Scalability Evaluation 81
5.5 Application Scenario 83
Chapter 6: Conclusions and Future Work 85
6.1 Conclusions 85
6.2 Future Work 86
References 89

[1] James A. Martin and Matthew Finnegan, “What is IFTTT? How to use If This, Then That services,” in COMPUTERWORLD, Jan. 31, 2019. [Online]. Available: https://www.computerworld.com/article/3239304/what-is-ifttt-how-to-use-if-this-then-that-services.html. [Accessed on Dec. 2, 2019].
[2] Barb Edson, “Creating the Internet of Your Things,” Microsoft Corporation, 2015.
[3] Daniel Burrus, “The Internet of Things Is Far Bigger Than Anyone Realizes,” in WIRED, 2014. [Online]. Available: https://www.wired.com/insights/2014/11/the-internet-of-things-bigger/. [Accessed on March 23, 2020].
[4] Paul Fremantle, “A Reference Architecture for The Internet of Things,” in WSO2, Oct., 2015. [Online]. Available: https://wso2.com/whitepapers/a-reference-architecture-for-the-internet-of-things/. [Accessed on March 23, 2020].
[5] Srinath Perera, “What is Stream Processing?,” in WSO2, May 14, 2018. [Online]. Available: https://wso2.com/library/articles/2018/05/what-is-stream-processing/. [Accessed on March 24, 2020].
[6] H. Isah, T. Abughofa, S. Mahfuz, D. Ajerla, F. Zulkernine and S. Khan, "A Survey of Distributed Data Stream Processing Frameworks," in IEEE Access, vol. 7, pp. 154300-154316, 2019.
[7] S. Kamburugamuve, G. Fox, D. Leake, J. Qiu, "Survey of Distributed Stream Processing for Large Stream Sources", for PhD Qualifying Exam at Indiana University, Dec. 14, 2013.
[8] M. D. da Assunção, A. da S. Veith, R. Buyya, "Distributed data stream processing and edge computing: A survey on resource elasticity and future directions", J. Netw. Comput. Appl., vol. 103, pp. 1-17, Feb. 2018.
[9] M. Elkhodr, S. Shahrestani and H. Cheung, "A Smart Home Application Based on the Internet of Things Management Platform," 2015 IEEE International Conference on Data Science and Data Intensive Systems, Sydney, NSW, 2015, pp. 491-496.
[10] A. K. Gupta and R. Johari, "IOT based Electrical Device Surveillance and Control System," 2019 4th International Conference on Internet of Things: Smart Innovation and Usages (IoT-SIU), Ghaziabad, India, 2019, pp. 1-5.
[11] S. Cheepurishetti and Y. Mohana Roopa, "Iot Applications in Smart Cities," 2019 International Conference on Communication and Electronics Systems (ICCES), Coimbatore, India, 2019, pp. 917-920.
[12] A. Zhamanov, Z. Sakhiyeva, R. Suliyev and Z. Kaldykulova, "IoT smart campus review and implementation of IoT applications into education process of university," 2017 13th International Conference on Electronics, Computer and Computation (ICECCO), Abuja, 2017, pp. 1-4.
[13] V. S. A. A. A. Don, S. W. Loke and A. Zaslavsky, "IoT-Aided Charity: An Excess Food Redistribution Framework," 2018 3rd International Conference on Internet of Things: Smart Innovation and Usages (IoT-SIU), Bhimtal, 2018, pp. 1-6.
[14] A. Saxena, K. Shinghal, R. Misra and A. Agarwal, "Automated Enhanced Learning System using IOT," 2019 4th International Conference on Internet of Things: Smart Innovation and Usages (IoT-SIU), Ghaziabad, India, 2019, pp. 1-5.
[15] A. Akbar, F. Carrez, K. Moessner and A. Zoha, "Predicting complex events for pro-active IoT applications," 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT), Milan, 2015, pp. 327-332.
[16] A. Akbar, A. Khan, F. Carrez and K. Moessner, "Predictive Analytics for Complex IoT Data Streams," in IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1571-1582, Oct. 2017.
[17] A. S. Dhillon, S. Majumdar, M. St-Hilaire and A. El-Haraki, "A Mobile Complex Event Processing System for Remote Patient Monitoring," 2018 IEEE International Congress on Internet of Things (ICIOT), San Francisco, CA, 2018, pp. 180-183.
[18] S. Mahfuz, H. Isah, F. Zulkernine and P. Nicholls, "Detecting Irregular Patterns in IoT Streaming Data for Fall Detection," 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, 2018, pp. 588-594.
[19] Vikash, L. Mishra and S. Varma, "Evaluation of Real-Time Stream Processing for Internet of Things Applications," 2019 International Conference on Electrical, Electronics and Computer Engineering (UPCON), ALIGARH, India, 2019, pp. 1-5.
[20] Ferrari L., Valtolina S., Mesiti M., “Developing IoT Spark-Streaming Applications by Means of StreamLoader,” in Ubiquitous Networking. UNet 2018. Lecture Notes in Computer Science, Boudriga N., Alouini MS., Rekhis S., Sabir E., Pollin S., Eds. Hammamet: Springer, Cham, 2018, vol. 11277.
[21] S. Valtolina, L. Ferrari and M. Mesiti, "Ontology-Based Consistent Specification of Sensor Data Acquisition Plans in Cross-Domain IoT Platforms," in IEEE Access, vol. 7, pp. 176141-176169, 2019.
[22] Ingrid, “Thinknx IFTTT Guide,” in thinkx, Oct. 7, 2019 [Online]. Available: http://www.thinknx.com/wiki/doku.php?id=ifttt. [Accessed on Dec. 7, 2019].
[23] Eric Griffith, “The 25 Best IFTTT Applets,” in PCMag, July 22, 2019. [Online]. Available: https://www.pcmag.com/article/315580/the-110-best-ifttt-recipes. [Accessed on Dec. 7, 2019].
[24] djrxr6, “IFTTT and WordPress: Part 2,” June 12, 2016. [Online]. Available: http://www.djrxr6.com.au/ifttt-wordpress-part-2/. [Accessed on Dec 7, 2019].
[25] Philips Hue, “hue.” in IFTTT. [Online]. Available: https://ifttt.com/hue. [Accessed on Dec. 8, 2019].
[26] Apache Hadoop, “Apache Hadoop.” [Online]. Available: https://hadoop.apache.org/. [Accessed on Dec. 8, 2019].
[27] Shubham Sinha, “What is Hadoop? Introduction to Big Data & Hadoop,” in edureka, May 22, 2019. [Online]. Available: https://www.edureka.co/blog/what-is-hadoop/. [Accessed on Dec. 8, 2019]
[28] Apache Spark, “Spark Streaming Programming Guide.” [Online]. Available: https://spark.apache.org/docs/latest/streaming-programming-guide.html. [Accessed on Dec. 8, 2019]
[29] Apache Kafka, “Introduction.” [Online]. Available: https://kafka.apache.org/intro. [Accessed on Dec. 9, 2019]
[30] Confluent, “What is Apache Kafka?” [Online]. Availbale: https://www.confluent.io/what-is-apache-kafka/. [Accessed on Dec. 9, 2019]
[31] AWS, “What is Apache Kafka?” [Online]. Available: https://aws.amazon.com/msk/what-is-kafka/. [Accessed on Dec. 9, 2019]
[32] Stanislav Kozlovski, “Thorough Introduction to Apache Kafka,” in HACKERNOON, Nov. 24, 2017. [Online]. Available: https://hackernoon.com/thorough-introduction-to-apache-kafka-6fbf2989bbc1. [Accessed on Dec. 9, 2019]
[33] Gavin McDonald, “Index,” in Confluence, Aug. 14, 2019. [Online]. Available: https://cwiki.apache.org/confluence/display/ZOOKEEPER/Index. [Accessed on Dec. 9, 2019]
[34] Cloudurable, “Kafka Architecture,” May 11, 2017. [Online]. Available: http://cloudurable.com/blog/kafka-architecture/index.html#:~:targetText=Kafka%20Architecture%3A%20Topics%2C%20Producers%20and,for%20Broker%20Topic%20Partition%20Leaders. [Accessed on Dec. 9, 2019]
[35] Matt Makai, “Green Unicorn (Gunicorn),” in Full Stack Python. [Online]. Available: https://www.fullstackpython.com/green-unicorn-gunicorn.html. [Accessed on Dec. 9, 2019]
[36] Vladislav Supalov, “What Is Gunicorn, and What Does It Do?” in vsupalov, 2019. [Online]. Available: https://vsupalov.com/what-is-gunicorn/. [Accessed on Dec. 9, 2019]
[37] Omed Habib, “A Performance Analysis of Python WSGI Servers: Part 2,” in APPDYNAMICS, May 11, 2016. [Online]. Available: https://www.appdynamics.com/blog/engineering/a-performance-analysis-of-python-wsgi-servers-part-2/. [Accessed on Dec. 9, 2019]
[38] Supervisor. “Introduction,” March 16, 2020. [Online]. Available: http://supervisord.org/introduction.html. [Accessed on March 16, 2020]
[39] NGINX. “Welcome to NGINX Wiki!” [Online]. Available: https://www.nginx.com/resources/wiki/. [Accessed on March 16, 2020]
[40] NGINX. “What is a Reverse Proxy vs. Load Balancer?” [Online]. Available: https://www.nginx.com/resources/glossary/reverse-proxy-vs-load-balancer/. [Accessed on April 4, 2020]