IADIS International Journal on Computer Science and Information Systems

Published by IADIS (International Association for Development of the Information Society) • ISSN (Online): 1646-3692 • ISSN (Print): 1646-3692
100% Open Access
Double-Blind Peer Review
Crossref DOI Persistent IDs
Open Access Peer-Reviewed Foundational Models & Architectures

A Mapreduce Based Framework to Perform Full Model Selection in Very Large Datasets

Angel Díaz Pacheco *
Jesús A. Gonzalez-Bernal *
Carlos A. Reyes-Garcia *
* , Óptica y Electrónica (INAOE), Computer Science Department. , Puebla, 72840, Mexico (Portugal)
* , Óptica y Electrónica (INAOE), Computer Science Department. , Puebla, 72840, Mexico (Portugal)
* , Óptica y Electrónica (INAOE), Computer Science Department. , Puebla, 72840, Mexico (Portugal)

Abstract

The analysis of large amounts of data has become an important task in science and business that led to the emergence of the Big Data paradigm. This paradigm owes its name to data objects too large to be processed by standard hardware and algorithms. Many data analysis tasks involve the use of machine learning techniques. The goal of predictive models consists on achieving the highest possible accuracy to predict new samples, and for this reason there is high interest in selecting the most suitable algorithm for a specific dataset. Selecting the most suitable algorithm together with feature selection and data preparation techniques integrates the Full Model Selection paradigm a nd it has been widely studied in datasets of common size, but poorly explored in the Big Data context. As an effort to explore in this direction, this work proposes a framework adjustable to any population based meta-heuristic methods in order to perform model selection under the MapReduce paradigm.

Keywords

Model Selection MapReduce Big Data
Full-Text PDF Available

Read Complete Peer-Reviewed Manuscript

Includes full econometric models, data tables, policy recommendations, declarations, and citations.

Declarations & Ethics

Funding: This research received academic dissemination support through ESCAP / JournalsHub publishing programs.
Conflicts of Interest: The authors declare no competing financial or institutional interests.
Peer Review: Double-blind peer reviewed by international subject specialists.
License: Creative Commons Attribution 4.0 International (CC BY 4.0).
How to Cite This Article
APA / MLA / BibTeX
Pacheco, et al. (2018). A Mapreduce Based Framework to Perform Full Model Selection in Very Large Datasets. IADIS International Journal on Computer Science and Information Systems, 13(1). https://doi.org/10.33965/ijcsis_2018_v13i1_02
Pacheco, et al. "A Mapreduce Based Framework to Perform Full Model Selection in Very Large Datasets." IADIS International Journal on Computer Science and Information Systems, vol. 13, no. 1, 2018. https://doi.org/10.33965/ijcsis_2018_v13i1_02
Pacheco, et al. "A Mapreduce Based Framework to Perform Full Model Selection in Very Large Datasets." IADIS International Journal on Computer Science and Information Systems 13, no. 1 (2018). https://doi.org/10.33965/ijcsis_2018_v13i1_02