“Big Data” and the Challenges for Statisticians

50 %
50 %
Information about “Big Data” and the Challenges for Statisticians
Education

Published on February 18, 2014

Author: hafidztio

Source: slideshare.net

Description

“Big Data” and the Challenges for Statisticians

“Big Data” and the Challenges for Statisticians Setia Pramana Math Department, FMIPA Brawijaya University Malang 7 February 2014

Data Explosion • Interactions of billions of people using computers, GPS devices, cell phones, and medical devices. • online or mobile financial transactions, social media traffic, and GPS coordinates. • “In the next five years, we’ll generate more data as humankind than we generated in the previous 5,000 years”. Eron Kelly, GM Microsoft

Data Explosion • Interactions of billions of people using computers, GPS devices, cell phones, and medical devices. • online or mobile financial transactions, social media traffic, and GPS coordinates. • “In the next five years, we’ll generate more data as humankind than we generated in the previous 5,000 years”. Eron Kelly, GM Microsoft

Data Explosion http://www.csc.com/

Big Data

Big Data • Volume • Velocity • Variety http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data

Big Data • Volume • Velocity • Variety http://www.documentcapture.co.uk A challenge: managing its size  Storing, searching, analyzing, comparing, refining, combining, and visualizing.

Big Data • Veracity: biases, noise and abnormality in data. • Validity: is the data correct and accurate for the intended use? • Volatility: how long is data valid and how long should it be stored?

Data Analysis Evolution http://www.csc.com/

Connecting the Data http://www.csc.com/

Connecting the Data

Connecting the Data • After Haiti’s earthquake (2010), researchers at the Karolinska Institute and Columbia Univ showed that mobile data patterns could be used to understand the movement of refugees and the consequent health risks posed by these movements.

Big Data in Biomedicine From where the big data comes from? • Billions of measurements in the health system: physician diagnose, drug dispense, blood test, x-ray or CT scan, etc.. • Advanced Molecular tech: Microarray, Next generation Sequencing

Big Data in Biomedicine From where the big data comes from?

Microarray • Measure expression of thousands of genes under different conditions. • Thousands of variables -> need special statistics methods

More: http://www.slideshare.net/hafidztio/geneexpression-introduction.

Publicly Available Data

Publicly Available Data ~300 Diseases!

What We Can Do?

Relate Several Data Repository Disease Gene Expression DB Drug Gene Expression DB • New Drug Discovery • Drug Repositioning e.g., Viagra (unexpected)

More.. • Genome Project: • Next generation Sequencing, e.g, Whole Genome Seq: info our 3 billion bp DNA code • And many more……

Data Science • A multidisiplinary science: Statistics, Math, Comp Science, Machine learning, Data Munging/Cleaning, and Data Visualization, Domain expertise. http://drewconway.com Data Scientist: The Sexiest Job of the 21st Century

Data Scientist vs. Statistician http://blog.revolutionanalytics.com

Is Statistician an Endangered Species?

Statisticians Should... • Have strong foundation in statistical theory, methods, and software. • Be expert in R and Python. • Familiarity with data visualization and machine learning techniques. • Know about parallel computing, combining data from disparate sources, and handling textual and streaming data. • Get engaged to the real world. • More innovative ..

R • R users is growing

R • The continued rapid growth in add-on packages. • The near monopoly R has on the latest analytic methods. • Its free price. • The freedom to teach with real-world examples from outside organization.

Visualization: Twitter traffic

Learn From the Expert

Ready? Compete!

Reference

Thank you…….

Add a comment

Related presentations

Related pages

Overcoming Big Data’s Challenges - strategy+business

Overcoming Big Data’s Challenges. s+b Blogs. by David Meer. ... it’s not as simple as just buying some new technology and hiring some statisticians.
Read more

Statisticians’ Place in Big Data - STATTRAK

Statisticians’ Place in Big Data ... to the table when discussing the challenges of Big Data, ... Statisticians have a distinct and essential ...
Read more

Challenges and Opportunities for Statisticians in the Era ...

Challenges and Opportunities for Statisticians in the Era of Big Data Robert N. Rodriguez 2012 President American Statistical Association August 29 2013
Read more

BIG DATA: CHALLENGES AND OPPORTUNITIES

Big Data: Challenges and ... that is going to require participants to embrace their inner statisticians, ... To illustrate some Big Data best ...
Read more

The ASA and Big Data | Amstat News

The ASA and Big Data. ... behind the disconnect between statisticians and big data: ... methods that specifically address big data challenges?
Read more

Discovery with Data: Leveraging Statistics with Computer ...

statistical thinking can help tackle the many Big Data challenges, ... that statisticians could help advance the Administration’s Big Data Initiative.
Read more

STATISTICS FOR BIG DATA: ARE STATISTICIANS READY FOR BIG DATA?

STATISTICS FOR BIG DATA: ARE STATISTICIANS READY FOR BIG DATA? ... big data challenges statisticians to move out of some familiar habits to engage less ...
Read more

Uses of Big Data for Official Statistics: Privacy ...

Uses of Big Data for Official Statistics: Privacy, Incentives, Statistical Challenges, ... Other Roles for Official statisticians in the Use of Big Data ...
Read more

Big Data | Department of Statistics

Big data refers to a collection of data sets so large and complex that it becomes difficult ... Statisticians, ... White Papers with Challenges ...
Read more