帳號:guest(3.144.12.19)          離開系統
字體大小: 字級放大   字級縮小   預設字形  

詳目顯示

以作者查詢圖書館館藏以作者查詢臺灣博碩士論文系統以作者查詢全國書目勘誤回報
作者:謝名峰
作者(英文):Ming-Feng Hsieh
論文名稱:串流大數據預測式視覺化系統架構方法與實現
論文名稱(英文):Streaming Big Data Predictive Visualization System Architecture, Method and Implementation
指導教授:吳秀陽
指導教授(英文):Shiow-Yang Wu
口試委員:張耀中
孫宗瀛
口試委員(英文):Yao-Chung Chang
Tsung-Ying Sun
學位類別:碩士
校院名稱:國立東華大學
系所名稱:資訊工程學系
學號:610821241
出版年(民國):112
畢業學年度:111
語文別:中文
論文頁數:78
關鍵詞:大數據視覺化資料預測IoTMongoDBPysparkKafkaGoogle Map APIApache Echarts
關鍵詞(英文):Big DataVisualizationData PredictionIoTMongoDBPySparkKafkaGoogle Map APIApache Echarts
相關次數:
  • 推薦推薦:0
  • 點閱點閱:25
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:26
  • 收藏收藏:0
隨著資訊時代的日新月異,大數據系統逐漸邁向巔峰,每天都有數以兆計的資料產生,而我們是否可以從這些制式化的數字中看出彼此之間的關聯性,甚至是整體未來的走向是一個重要的議題。我們可以透過不同的圖形來表現資料的特性尋找出其具有的規律、並分析建立模型,達到預測未來趨勢甚至是提前建立未來資料庫。
目前現有的視覺化系統多侷限在即時亦或是過去的資料形式,而較無法讓使用者了解更多未來可能出現的狀況。另外對於多數一般的使用者其操作是否足夠簡單讓其可以自行操作得到想要的視覺化圖形,是否能夠在提升視覺化的多元性的前題下讓使用者可以用更少的操作難度與耗費時間,卻可以得到更多資料的相關性,甚至是特定數值的區域分布。
本篇論文的目的在探討現有的串流大數據資料視覺化系統應用及擴展性,並提出3種具有預測性資料視覺化的方法達到更加多元化的搜尋及繪圖,並且透過Open Source進行實現,包含了軌跡預測視覺化、預測軌跡密度視覺化、IoT預測資料視覺化,希望能透過這些方法提升使用者對於資料的掌握,對未來可能發生的狀況提前作出決策從而避開風險。另外,希望透過簡化使用者對於系統的操作,讓使用者可以用較簡單的方式達到複合多元的搜尋並完成視覺化。
With the rapid development of the information age, the big data system is gradually reaching its peak, and trillions of data are generated every day. Can we find the correlation between these standardized numbers, and even the future direction. It is an important issue. We can use different graphs to express the characteristics of the data to find out the laws it has, and analyze and build models to predict future trends and even build future databases in advance. At present, the existing visualization systems are mostly limited to the real-time or past data form, and are less able to allow users to understand more possible future situations. In addition, for most common users, whether the operation is simple enough to allow them to obtain the desired visual graphics by themselves, and whether it can be easier to use and time-consuming under the premise of improving the diversity of visualization. But more data correlation, and even the regional distribution of specific values can be obtained.
The purpose of this paper is to discuss the application and scalability of existing big data stream visualization systems, and to propose three predictive data visualization methods to achieve more diversified search and drawing, including Trajectory Prediction Visualization, Predicted Trajectory Density Visualization, IoT Data Prediction Visualization. It is hoped that through these methods, users can improve their grasp of data, and make decisions in advance to avoid possible future situations. In addition, it is hoped that by simplifying the user's operation of the system, the user can achieve complex and diverse searches and complete visualization in a simpler way.
致謝 i
摘要 ii
Abstract iii
目錄 iv
圖目錄 vii
表目錄 ix
第1章 緒論 1
1.1 研究背景與動機 1
1.2 研究目的與方法 2
1.3 研究成果 2
1.4 論文架構 3
第2章 相關工作 5
2.1 物聯網(IoT) 5
2.2 串流資料(Streaming Data) 6
2.3 城市智能交通狀況監測系統 7
2.4 大數據分析可視化系統 9
2.5 相關系統工具 10
2.5.1 Kafka 10
2.5.2 MongoDB 12
2.5.3 PySpark 13
2.5.4 Apache Echarts 14
2.5.5 Apache Http Server 15
第3章 串流大數據預測式視覺化系統架構 17
3.1 研究議題與解決策略 19
3.2 串流大數據預測式視覺化系統架構 19
3.3 視覺化方法 22
3.4 視覺化方法拓展 22
第4章 串流大數據預測式視覺化方法 25
4.1 軌跡預測視覺化策略與實現 25
4.2 密度預測視覺化策略與實現 29
4.3 IoT監測資料預測視覺化策略與實現 32
第5章 系統實作效果與效能評估 35
5.1 實驗環境 35
5.2 實驗資料 36
5.3 實驗結果 37
5.3.1 軌跡預測視覺化效果實驗結果 37
5.3.2 軌跡預測視覺化準確性實驗結果 39
5.3.3 軌跡預測視覺化效能實驗結果 41
5.3.4 密度預測視覺化效果實驗結果 43
5.3.5 密度預測視覺化準確性實驗結果 45
5.3.6 密度預測視覺化效能實驗結果 46
5.3.7 IoT監測資料預測視覺化效果實驗結果 47
5.3.8 IoT監測資料預測視覺化準確性實驗結果 48
5.3.9 IoT監測資料預測視覺化效能實驗結果 50
5.3.10 架構效能實驗結果 54
5.4 實驗總結 55
第6章 結論與未來工作 57
6.1 結論 57
6.2 未來工作 57
參考文獻 59
[1]何謂資料視覺化? – Tableau。檢自https://www.tableau.com/zh-tw/learn/articles/data-visualization (參照日期 2022/03/08)
[2]畫圖做報表就是資料視覺化? 錯! – FineReport。檢自https://www.finereport.com/tw/data-analysis/data-vis01.html (參照日期2022/03/08)
[3]物聯網:定義、應用和風險 – NordVPN。檢自https://nordvpn.com/zh-tw/blog/wulianwang/ (參照日期 2022/05/20)
[4]Daniel Cooley, 2022年將是物聯網發展的轉捩點。檢自
https://www.eettaiwan.com/20220413nt71-2022-marks-an-inflection-point-for-the-internet-of-things/ (參照日期 2022/05/23)
[5]物聯網(IoT)是什麼? – TIBCO。檢自https://www.tibco.com/zh-hant/reference-center/what-is-the-internet-of-things-iot (參照日期 2022/05/23)
[6]什麼是串流資料? - AWS。檢自https://aws.amazon.com/tw/streaming-data/ (參照日期 2022/05/25)
[7]王宜倫, 資料流(Streaming Data)技術於大數據(Big Data)之應用淺析。檢自https://www.syscom.com.tw/ePaper_New_Content.aspx?id=456&EPID=209&TableName=sgEPArticle (參照日期 2022/04/27)
[8]Streaming Data: How it Works, Benefits, and Use Cases - Confluent。檢自https://www.confluent.io/learn/data-streaming/ (參照日期 2022/05/25)
[9]D. Zhang and Y. Jiang, “Design of Urban Intelligent Traffic Congestion Situation Monitoring System Based on Big Data,” 2020 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS), pp. 12-15, 2020.
[10]Mingchen Feng, Jiangbin Zheng, Jinchang Ren, Amir Hussain, Xiuxiu Li, Yue Xi and Qiaoyuan Liu. “Big Data Analytics and Mining for Effective Visualization and Trends Forecasting of Crime Data,” IEEE Access, vol. 7, pp. 106111-106123, 2019.
[11]Chi-Hsuan Huang, Apache Kafka介紹。檢自
https://medium.com/@chihsuan/introduction-to-apache-kafka-1cae693aa85e (參照日期 2022/03/16)
[12]程序員毒硬幣, Java工程師的進階之路-Kafka篇(一)什麼是微服務擴展性和高可用、可擴展性。檢自https://cdmana.com/2021/08/20210826025803703Y.html (參照日期 2022/05/26)
[13]MongoDB 是什麼?5分鐘帶你了解 MongoDB 的5大特性與好處 – GAIA。檢自 https://www.gaia.net/tc/news_detail/2/134/what-is-mongodb (參照日期 2022/03/26)
[14]What is MongoDB? – IntelliPaat。檢自
https://intellipaat.com/blog/what-is-mongodb/ (參照日期 2022/07/14)
[15]Amo Chen, 零經驗也可的 PySpark 教學 - 初體驗。檢自
https://myapollo.com.tw/zh-tw/pyspark-1/ (參照日期 2022/03/08)
[16]特性 – Apache ECharts。檢自
https://echarts.apache.org/zh/index.html (參照日期 2022/03/05)
[17]作者 伍二娃, Echarts 数据可视化 – Wave52 。檢自
https://wuchengran.com/2020/02/27/charts/ (參照日期 2022/03/05)
[18]Apache HTTP Server – Wiki。檢自https://zh.m.wikipedia.org/zh-tw/Apache_HTTP_Server (參照日期 2021/07/20)
[19]甚麼是Apache服務器 – Website Rating。檢自
https://www.websiterating.com/zh-TW/web-hosting/glossary/what-is-apache-server/ (參照日期 2022/07/20)
[20]Daniel Johnson, “What is Data Analysis? Research, Types & Example,” 檢自https://www.guru99.com/what-is-data-analysis.html (參照日期 2022/06/15)
[21]陳昱衡(2020)。以整合式形狀與符號聚合近似法為基礎之時間序列資料分析與行為辨識。國立東華大學資訊工程學系碩士論文,花蓮縣。
[22]作者 geoday - 知乎 “两地之间的距离是怎么算出来的?”。檢自https://www.zhihu.com/question/267206872/answer/2509913336 (參照日期 2022/06/30)
[23]作者Winshining – Github, “Nginx-http-flv-module.” 檢自https://github.com/winshining/nginx-http-flv-module.git (參照日期 2021/10/11)
[24]作者 August - Let’s Write “Google Maps API 學習筆記 - 1:地圖、標記、客製樣式” 檢自https://www.letswrite.tw/google-map-api-marker-custom/ (參照日期 2021/06/23)
[25]作者 Noob TSAI - Noob’s Space “Web 視覺化(二):使用 Google Maps JS API 建立地圖”。檢自https://noob.tw/web-visualization-google-maps/ (參照日期2021/06/23)
[26]作者 Jeff3071 - iT邦幫忙 “Day 7: 從爬蟲到架站 - echart繪圖(2)”。檢自 https://ithelp.ithome.com.tw/articles/10234990 (參考日期 2021/12/22)
[27]作者 雲溪龍 - 編程知識 “Echarts動態顯示MySQL數據庫中的數據”。
檢自https://cdmana.com/2021/12/20211201203241940c.html (參照日期 2021/12/23)
[28]T. Liang, S. Lu and Q. Liu, “Data Visualization System Based on Big Data Analysis,” 2020 International Conference on Robots & Intelligent System (ICRIS), pp. 76-79, 2020.
[29]Enrico Bertini, Juri Buchmüller, Fabian Fischer, Stephan Huber, Thomas Lindemeier, Fabian Maaß, Florian Mansmann, Thomas Ramm, Michael Regenscheit, Christian Rohrdantz, Christian Scheible, Tobias Schreck, Stephan Sellien, Florian Stoffel, Mark Tautzenberger, Matthias Zieker, and Daniel A. Keim, “Visual analytics of terrorist activities related to epidemics,” 2011 IEEE Conference on Visual Analytics Science and Technology (VAST), ,pp.329 – 330, 2011.
[30]Ardi Imawan and Joonho Kwon, “A timeline visualization system for road traffic big data,” 2015 IEEE International Conference on Big Data (Big Data), ,pp.2928 – 2929, 2015.
[31]Maura Riley, Daniel Harelson, Michael Monte, Michael Chong, Meghan O'Hern, Michael Smith, and K. Preston White Jr., “Criminal justice system data analysis and visualization,” 2015 Systems and Information Engineering Design Symposium, pp.295 – 299, 2015.
[32]Syed Mohd Ali, Noopur Gupta, Gopal Krishna Nayak, and Rakesh Kumar Lenka, “Big data visualization: Tools and challenges,” 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I), pp.656 – 660, 2016.
[33]Tingting Liang, “Design and Implementation of Big Data Visual Statistical Analysis Platform,” 2020 2nd International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI), pp.287 – 291, 2020.
[34]Xiaoxia Jia, Peng Cheng, and Jiming Chen, “A data analysis and visualization system for large-scale e-bike data,” 2016 IEEE International Conference on Big Data (Big Data), pp.3998 – 4000, 2016.
[35]Maura Riley, Daniel Harelson, Michael Monte, Michael Chong, Meghan O'Hern, Michael Smith, and K. Preston White Jr.,“Criminal justice system data analysis and visualization,” 2015 Systems and Information Engineering Design Symposium, pp.295 – 299, 2015.
[36]Guanghui Min, Muhui Lin, Zhe Li, and Yuexia Du, “The Trend, Hotspots, Frontier and Path of Big Data Visualization Research in China——Based on the Knowledge Graph Analysis of Citespace5.5.R2,” 2020 International Conference on Culture-oriented Science & Technology (ICCST), pp.118 – 122, 2020.
[37]Gabriel Ferreira, Paulo Alves, and Simone de Almeida, “Platform for real-time data analysis and visualization based on Big Data methods,” 2021 16th Iberian Conference on Information Systems and Technologies (CISTI), pp.1 – 6, 2021.
[38]Jianhui Gao, “Research and Design of Moving Target Detection and Tracking System Based on Web Server,” 2020 5th International Conference on Communication, Image and Signal Processing (CCISP), pp.246 – 250, 2020.
[39]Roberto Tardío, Alejandro Mate, and Juan Trujillo, “An iterative methodology for big data management, analysis and visualization,” 2015 IEEE International Conference on Big Data (Big Data), pp.545 – 550, 2015.
[40]Carson K. Leung; Yubo Chen, Calvin S.H. Hoi, Siyuan Shang, Yan Wen, and Alfredo Cuzzocrea, “Big Data Visualization and Visual Analytics of COVID-19 Data,” 2020 24th International Conference Information Visualisation (IV), pp.415 – 420, 2020.
[41]Jillian Aurisano, James Hwang, Andrew Johnson, Lance Long, Margaret Crofoot,and Tanya Berger-Wolf, “Bringing the Field into the Lab: Large-Scale Visualization of Animal Movement Trajectories within a Virtual Island,” 2019 IEEE 9th Symposium on Large Data Analysis and Visualization (LDAV), pp.83 – 84, 2019.
[42]Carson K. Leung, Yibin Zhang, Calvin S.H. Hoi, Joglas Souza, and Bryan H. Wodi, “Big Data Analysis and Services: Visualization on Smart Data to Support Healthcare Analytics,” 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp.1261 – 1268, 2019.
[43]Yubo Chen, Carson K. Leung, Siyuan Shang, and Qi Wen, “Temporal Data Analytics on COVID-19 Data with Ubiquitous Computing,” 2020 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom), pp.958 – 965, 2020.
[44]Daisuke Mashima, Stephen G. Kobourov, and Yifan Hu, “Visualizing dynamic data with maps,” 2011 IEEE Pacific Visualization Symposium, pp. 155-162, 2011.
 
 
 
 
第一頁 上一頁 下一頁 最後一頁 top
* *