Unstructured Data Processing using Spark for Topics Modelling 
Adjovi Irène Sokegbe1, Ayushi Nainwal2
1Adjovi Irène SOKEGBE*, Computer science and Engineering, Alakh Prakash Goyal Shimla University, Shimla, India.
2Ayushi Nainwal, Computer science and Engineering, Alakh Prakash Goyal Shimla University, Shimla, India.
Manuscript received on June 01, 2020. | Revised Manuscript received on June 08, 2020. | Manuscript published on June 30, 2020. | PP: 1060-1063 | Volume-9 Issue-5, June 2020. | Retrieval Number: E9992069520/2020©BEIESP | DOI: 10.35940/ijeat.E9992.069520
Open Access | Ethics and Policies | Cite | Mendeley
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: Information Technology domain is facing changes day by day. Furthermore, the size of data increases, as well as the demand to process them. There are two types of data: structured and unstructured data. The multiple sources and the variety of data today involve the use of “Big data” instead of data. It is related that 80% of enteUprise’s data is unstructured [1]. However, the procedures to handle unstructured data are more complex than those for structured data. Thus, it becomes necessary to have a clear idea about this type of data and to know how to extract useful information from this data set. In this paper we will study how to retrieve useful information from unstructured data in E-commerce area using data analysis tools: Spark. To solve this issue, first an overview on structured and unstructured data and data analysis is provided, then information retrieval algorithm will be implemented using Spark MLlib tool in order to determine for a set of reviews, negative or positive, which subjects are more discussed by the customers. This study is needed in order to improve business based on customer satisfaction reviews. In that case, Unsupervised Machine Learning Latent Dirichlet Allocation (LDA) algorithm constitutes our model. Finally, the evaluation of the model will be given based on some parameters.
Keywords: Unstructured data, data analysis, Spark MLlib, LDA.