Big Data - Load CSV File & Query the EZ way - HPCC Systems

80 %
20 %
Information about Big Data - Load CSV File & Query the EZ way - HPCC Systems
Technology

Published on July 18, 2014

Author: FujioTurner

Source: slideshare.net

Description

A "How To" to load CSV files into HPCC Systems and query them. You can use this method to migrate your RDBMS data ,MySQL / Oracle / SQL, into HPCC Systems.

HPCC Systems Loading csv Data & Querying By Fujio Turner @myhousehippo

BusinessDevelopmentCustomers 1 20 Non-Indexed Full Data Set http://hpccsystems.com/why-hpcc/benchmarks

Map/Reduce SQL w/ JOINS GraphDB Machine Learning Simple to Complex Queries

“I’m sub-second fast.” “I can query all or part of your data.” Thor Roxie Hard Disk Index(optional) Hard Disk Index(optional) In-memory Index SSD Either/Both Architecture

Data QueryFile Example http://catalog.data.gov/dataset/consumer-complaint-database CSV data sample source

Administrator Web GUI! on Port 8010IP / Url of HPCC install

4. add ,t 5. 1. Upload file*! 2. Distribute to cluster! 3. Name of file in cluster! 4. Most CSV have t! 5. Push to cluster *2GB file size limit through web No limit if uploaded via SOAP Load !! ! ! Data

In Thor Cluster Loaded*optional file rename

Query w/ ECL Com := DATASET(‘~test::complaints’,ComS, CSV(HEADING(1), SEPARATOR([',','t']))); ComS :=RECORD UNSIGNED3 ComplaintID; STRING23 Product; STRING38 State; …………………………. …………………………. STRING31Consumer_disputed; END; Ma; //Output Ma := Com(State = ‘MA’); WHERE `State` = ‘MA’ File Type File Location,! “FROM Table” “USE DATABASE;” “SELECT * ….” Schema

1. Go to playground! 2. Edit ECL! 3. Pick “thor” Cluster! 4. Submit http://www.meetup.com/HPCC-SV/pages/ECL_EXAMPLE__- _CSV_LOAD_and_QUERY Practice

Schema Made EZ http://hpccsystems.com/demos/data-profiling-demo CSV IN Schema OUTClick Take a small part of your CSV data and go to the link below to make an ECL Schema

ECL Guide http://hpccsystems.com/download/docs/ecl-language-reference

http://www.youtube.com/watch?v=8SV43DCUqJg Watch how to install HPCC Systems in 5 Minutes

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

HPCC - Wikipedia, the free encyclopedia

... big data. The HPCC platform includes system ... indexed data files (Roxie). The HPCC platform ... (HPCC) and Big Data Analytics ...
Read more

Comma Separated Value (.csv) - GDAL: GDAL - Geospatial ...

Comma Separated Value (.csv) OGR supports reading and writing primarily non-spatial tabular data stored in text CSV files. ... VSI Virtual File System API ...
Read more

CSV Comma Separated Value File Format - How To ...

The CSV ("Comma Separated Value") file ... Spreadsheet programs will often assume all data in a CSV file is in the OEM's or system ... (and that's a big ...
Read more

A Fast CSV Reader - CodeProject - CodeProject - For those ...

A Fast CSV Reader. Sebastien Lorion ... { // open the file "data.csv" which is a CSV file with headers using ... I have load this data into my sql database ...
Read more

Import CSV File Into MySQL Table - MySQL Tutorial - Learn ...

This tutorial shows you how to use LOAD DATA INFILE statement to import CSV file ... If you load a big CSV file, ... you can load data from other text file ...
Read more

10 Importing, Exporting, Loading, and Unloading Data

10 Importing, Exporting, Loading, and ... text files only. Load data with ... the host operating system's file system. Data Pump Export and ...
Read more

HPCC Systems

Getting Started. If you’ve seen enough of the HPCC Systems platform to know that it’s the right big data solution for your organization, we can help ...
Read more

Hpcc Systems | LinkedIn

View 48 Hpcc Systems posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn.
Read more