Report Open Access

Oracle R technologies for data analytics and machine learning in hybrid data systems

Bisht, Reshu; Martin Marquez, Manuel; Romero Marin, Antonio; Hornick, Mark

DataCite XML Export

<?xml version='1.0' encoding='utf-8'?>
<resource xmlns:xsi="" xmlns="" xsi:schemaLocation="">
  <identifier identifierType="DOI">10.5281/zenodo.159949</identifier>
      <creatorName>Bisht, Reshu</creatorName>
      <affiliation>CERN openlab Summer Student</affiliation>
      <creatorName>Martin Marquez, Manuel</creatorName>
      <affiliation>Summer Student Supervisor</affiliation>
      <creatorName>Romero Marin, Antonio</creatorName>
      <affiliation>Summer Student Supervisor</affiliation>
      <creatorName>Hornick, Mark</creatorName>
      <affiliation>Summer Student Supervisor</affiliation>
    <title>Oracle R technologies for data analytics and machine learning in hybrid data systems</title>
    <subject>CERN openlab summer student</subject>
    <date dateType="Issued">2016-10-10</date>
  <resourceType resourceTypeGeneral="Text">Report</resourceType>
    <alternateIdentifier alternateIdentifierType="url"></alternateIdentifier>
    <relatedIdentifier relatedIdentifierType="URL" relationType="IsPartOf"></relatedIdentifier>
    <rights rightsURI="">Creative Commons Attribution 4.0 International</rights>
    <rights rightsURI="info:eu-repo/semantics/openAccess">Open Access</rights>
    <description descriptionType="Abstract">&lt;p&gt;Project Specification&lt;/p&gt;

&lt;p&gt;The goal of this Openlab summer student project is to evaluate Oracle R Advanced Analytics for Hadoop (ORAAH) as a platform for CERN Big Data Analytics. It provides R interface to manipulate the data stored in HDFS, relational databases and file systems along with leveraging the capabilities of CRAN R packages. ORAAH also uses both HIVE transparency capabilities and maps HDFS as direct input into Machine Learning algorithms that can run as Map Reduce jobs or inside an Apache Spark container. The objective was to determine if the Cryogenic Valve Degradation analyses and Automatic Detection of Faulty valve can be efficiently done, in terms of CPU usage and time for completion of the job using ORAAH and Apache Spark. ORAAH allows you to run your R queries on Hadoop against data in HDFS, giving you the benefits of using R while taking advantage of the horizontal scalability of Hadoop and MapReduce. ORAAH is designed for parallel reads and writes, has resource management and database connectivity features, so it can be used together with ORE. Apart from this, the components like ORAAH Spark MLlib algorithms make the analysis, classification and prediction problems easier.&lt;/p&gt;

&lt;p&gt;Software Specifications:&lt;/p&gt;

&lt;p&gt; BigDataLite VM 4.5.0&lt;/p&gt;

&lt;p&gt; Oracle R Advanced Analytics for Hadoop 2.6.0&lt;/p&gt;

&lt;p&gt;o ORCH package&lt;/p&gt;

&lt;p&gt; Interaction with HDFS&lt;/p&gt;

&lt;p&gt; Map-Reduce Jobs&lt;/p&gt;

&lt;p&gt; (Spark MLlib container for Support Vector Machine Algorithm)&lt;/p&gt;

&lt;p&gt; Apache Spark 1.6.0&lt;/p&gt;

&lt;p&gt;o Spark SQL&lt;/p&gt;

&lt;p&gt; com.databricks:spark-csv_2.10:1.4.0&lt;/p&gt;

&lt;p&gt;o Spark MLlib&lt;/p&gt;

&lt;p&gt; org.apache.spark.mllib.classification{SVMModel,SVMWithSGD}&lt;/p&gt;

&lt;p&gt; org.apache.spark.mllib.evaluation.BinaryClassificationMetrics&lt;/p&gt;

&lt;p&gt; org.apache.spark.mllib.util.MLUtils&lt;/p&gt;

&lt;p&gt; R 3.3.0&lt;/p&gt;

&lt;p&gt;o caret&lt;/p&gt;

&lt;p&gt;o rpart&lt;/p&gt;

&lt;p&gt;o e1071&lt;/p&gt;

&lt;p&gt;o ggplot2&lt;/p&gt;

&lt;p&gt;o reshape&lt;/p&gt;

&lt;p&gt; &lt;/p&gt;


&lt;p&gt;I present an evaluation of Oracle R Advanced Analytics for Hadoop as a Big Data Analysis platform for advance analytics and machine learning. I have used R as a basic modelling tool as it one of the most powerful statistical and computing languages with a number of predefined functionalities available to allow an easy analysis and testing of data. To provide a comparison and truly judge the performance of ORAAH, Apache Spark has been used to model the same approaches. The performance has been measured on the basis of the time consumed to build the model and the accuracy of the model.&lt;/p&gt;

&lt;p&gt;The task in this project was aimed to study the potential applicability of the aforementioned technologies using real CERN analytics use cases: (a) The degradation analysis of cryogenic valves in LHCb (b) Predict faulty cryogenic valves.&lt;/p&gt;

&lt;p&gt;The above mentioned use cases were duly run and modelled using the technologies mentioned earlier and the results computed provided very promising statistics for future use of scalable services as CERN Big Data Analytics platform.&lt;/p&gt;</description>
All versions This version
Views 7676
Downloads 3030
Data volume 22.6 MB22.6 MB
Unique views 6565
Unique downloads 2727


Cite as