Genomics and BigData - case study

50 %
50 %
Information about Genomics and BigData - case study
Science

Published on May 8, 2014

Author: hongiiv

Source: slideshare.net

Description

Genomics and BigData - case study

dÉåçãáÅë=áå=_áÖ=a~í~== 빅데이터=관점에서의=유전체=데이터 Permissions: you are free to blog or live-blog about this presentation as long as you attribute the work to its authors Chang Bum Hong

오늘 할 이야기는.. Genomics에서의 클라우드 필요성 Cloud Computing Cloud in Genomics 사례로 살펴보는 Big Data 분석에 필요한 HPC Cloud의 요소

1,588명의 응답자 가운데 54%가 자신의 게놈을 분석해 보고 싶다고 응답 Nature, 2010

“기초의학연구와 임상의학의 의료 분야에 가장 큰 혜택을 줄것이며, 5-10년후 개인 맞춤의학이 보편화 될것, 하지만 컴퓨팅과 소프트웨어 부족은 큰 걸림돌” Nature, 2010

Data Size

Needs Public Data

Various Software http://seqanswers.com/wiki/Software

Complicate Pipeline

Computing resources Fusaro VA, Patil P, Gafni E, Wall DP, Tonellato PJ (2011) Biomedical Cloud Computing With Amazon Web Services. PLoS Comput Biol 7(8): e1002147

Linux skill Nature Biotech (2006)

To the Clinic - License, HIPPA

To the Clinic - License, HIPPA

To the Clinic - License, HIPPA

Cloud Computing and HPC (Hight Performance Computing)

컴퓨터와 스마트폰 동기화, 사진/음악/동영상 감상, 대용량 파일 공유 우리가 흔히 접하고 생각하는 클라우드

오늘 이야기할 클라우드는…

오늘 이야기할 클라우드는… Journal of Biomedical Informatics (2013)

하드웨어의 가상화뿐만 아니라, 다양한 서비스로의 확대 Amazon Web Services의 서비스 목록

하드웨어의 가상화뿐만 아니라, 다양한 서비스로의 확대 Google의 서비스 목록

하드웨어의 가상화로 부터 얻는 것 원하는 OS를 선택이 가능하다 맞춤 OS 제작이 가능하다

재현성 (reproducibility) 확보 손쉬운 분석 환경 구축 나의 분석 환경 + 데이터 + 스크립트 “나의 분석 환경 + 데이터 + 스크립트” 바로 얼려서 다른 사람들과 공유

하드웨어의 가상화로 부터 얻는 것 운영중인 서버에 디스크 추가 디스크의 내용을 이미지화하여 공유

하드웨어의 가상화로 부터 얻는 것 원하는 사양의 서버를 원하는 만큼 생성이 가능하다

하드웨어의 가상화로 부터 얻는 것 또 다른 스토리지 서비스 Object Storage http://whatis.techtarget.com/reference/Object-storage-Fast-Guide

하드웨어의 가상화로 부터 얻는 것 또 다른 스토리지 서비스 Object Storage

이 모든것이 바로 Cloud Software -IaaS 구축 가능한 오픈소스 클라우드 플랫폼 OpeStack, Eucalyptus, OpenNebula, CloudStack

OpenStack Architecture

HPC Clustering

Cloud in Genomics

Cloud in Genomics Genomics의 구체적 cloud 사례들 서비스 구분 서비스명 서비스 내용 KT IaaS Amazon WebServices PaaS, SaaS 서비스회사들을 적극 지원 public dataset을 통한 간접 지원 ucloud biz Google Cloud 컴퓨팅과 스토리지 지원 ucloud biz NeCTAR Research Cloud OpenStack 기반 연구자를 위한 Private Cloud ucloud biz SaaS DNANexus NGS 데이터 분석 파이프라인 제공 g-Analysis SevenBridge Genomics NGS 데이터 분석 파이프라인 제공 g-Analysis GotCloud NGS 데이터 분석 파이프라인 제공 g-Analysis Globus Genomics Galaxy를 AWS 기반으로 제공 g-Galaxy GenomeSpace Storage 기반의 bioinformatics 서비스 제공 g-Storage PaaS CloudMan 스토리지 기반의 Bioinformatics 툴 지원 SeqWare 유전체 분석 가능한 기반 플랫폼 제공 StarCluster AWS 기반의 HPC Cluster 컴퓨팅 환경 제공 CycleComputing AWS 기반의 HPC Cluster 컴퓨팅 환경 제공 Google Genomics, BigQeury NGS 데이터 분석을 위한 API 제공 g-Insight

사례로 살펴보는 Big Data 분석에 필요한 HPC Cloud의 요소

http://seqware.github.io/docs/

HPC Cloud의 요소 - Sequencers http://seqware.github.io/docs/ LIMS Object Storage High Speed File Transfer IaaS HPC Private Cloud OpenStack… Job Schedule Bioinformatics Linux… Bioinformatics Bioinformatics Hadoop and Database

LIMS for NGS Clarity LIMS from GenoLogics Galaxy LIMS for NGS Bioinformatics (2013)

LIMS for NGS Galaxy LIMS https://bitbucket.org/jelle/galaxy-central-tron-lims/

Data Upload - online

Data Upload - offline Import/Export

HPC Cloud의 요소 - Storage http://seqware.github.io/docs/ LIMS Object Storage High Speed File Transfer IaaS HPC Private Cloud OpenStack… Job Schedule Bioinformatics Linux… Bioinformatics Bioinformatics Hadoop and Database

http://www.genomespace.org/ GenomeSpace is a cloud-based interoperability framework to support integrative genomics analysis through an easy-to-use Web interface.

HPC Cloud의 요소 - HPC http://seqware.github.io/docs/ LIMS Object Storage High Speed File Transfer IaaS HPC Private Cloud OpenStack… Job Schedule Bioinformatics Linux… Bioinformatics Bioinformatics Hadoop and Database

Before Cloud Computing 구매 요청 -> 견적서…->서버실 확보>하드웨어설치-> 소프트웨 어 설정 -> 최종 테스트 ->구형=하드웨어I=구형=데이터

NeCTAR

StarCluster and CloudMan http://star.mit.edu/cluster/ https://wiki.galaxyproject.org/CloudMan/AWS/GettingStarted

HPC Cloud의 요소 - Pipeline http://seqware.github.io/docs/ LIMS Object Storage High Speed File Transfer IaaS HPC Private Cloud OpenStack… Job Schedule Bioinformatics Linux… Bioinformatics Bioinformatics Hadoop and Database

DNANexus and HGSC

DNANexus and HGSC

DNANexus and HGSC

DNANexus and HGSC

HPC Cloud의 요소 - Web Service http://seqware.github.io/docs/ LIMS Object Storage High Speed File Transfer IaaS HPC Private Cloud OpenStack… Job Schedule Bioinformatics Linux… Bioinformatics Bioinformatics Hadoop and Database

Google Genomics https://developers.google.com/genomics

Google Genomics API http://googleresearch.blogspot.co.uk/2014/02/google-joins-global-alliance-for.html Interoperability: One API, Many Apps

Google Genomics Examples API를 이용한 특정 샘플의 특정 read 정보 추출

Google Genomics Examples API와 JavaScript를 이용한 Genome Browser

HPC Cloud의 요소 - Query Engine http://seqware.github.io/docs/ LIMS Object Storage High Speed File Transfer IaaS HPC Private Cloud OpenStack… Job Schedule Bioinformatics Linux… Bioinformatics Bioinformatics Hadoop and Database

Google BigQuery

# Compute the Ti/Tv ratio for BRCA1. SELECT transitions, transversions, transitions/transversions AS titv FROM ( SELECT SUM(IF(mutation IN ('A->G', 'G->A', 'C->T', 'T->C'), INTEGER(num_snps), INTEGER(0))) AS transitions, SUM(IF(mutation IN ('A->C', 'C->A', 'G->T', 'T->G', 'A->T', 'T->A', 'C->G', 'G->C'), INTEGER(num_snps), INTEGER(0))) AS transversions, FROM ( SELECT CONCAT(reference_bases, CONCAT(STRING('->'), alternate_bases)) AS mutation, COUNT(alternate_bases) AS num_snps, FROM [google.com:biggene:1000genomes.variants1kG] WHERE contig = '17' AND position BETWEEN 41196312 AND 41277500 AND vt = 'SNP' GROUP BY mutation ORDER BY mutation)); Google BigQuery with plot result <- query_exec(project = "google.com:biggene", dataset = "1000genomes", query = sql, billing = billing_project) Ti/Tv ratio in BRCA1

# Count the variation for each sample including phenotypic traits SELECT samples.genotype.sample_id AS sample_id, gender, population, super_population, COUNT(samples.genotype.sample_id) AS num_variants_for_sample, SUM(IF(samples.af >= 0.05, INTEGER(1), INTEGER(0))) AS common_variant, SUM(IF(samples.af < 0.05 AND samples.af > 0.005, INTEGER(1), INTEGER(0))) AS middle_variant, SUM(IF(samples.af <= 0.005 AND samples.af > 0.001, INTEGER(1), INTEGER(0))) AS rare_variant, SUM(IF(samples.af <= 0.001, INTEGER(1), INTEGER(0))) AS very_rare_variant, FROM FLATTEN([google.com:biggene:1000genomes.variants1kG], genotype) AS samples JOIN [google.com:biggene:1000genomes.sample_info] p ON samples.genotype.sample_id = p.sample WHERE samples.vt = 'SNP' AND (samples.genotype.first_allele > 0 OR samples.genotype.second_allele > 0) GROUP BY sample_id, gender, population, super_population ORDER BY sample_id; Google BigQuery with R ggplot(result, aes(x = population, y = common_variant, fill = super_population)) + geom_boxplot() + ylab("Count of common variants per sample") + ggtitle("Common Variants (Minimum Allelic Frequency 5%)") Variant type

Google BigQuery with RStudio Markdown/Knit HTML

Publish your R code with git and RPubs https://github.com/ http://www.rpubs.com/

Data storage: $0.026 (per GB/month) Data query: $0.005 / GB 1000 genomes data Data storage: $0.026 (per GB/month) * 5,500 GB = $143 / month = 15,700/월 Data query (allele frequency query): $0.005 / GB * 647 GB = $3.3 = 3,600원 Cost 30초안에 1000genomes의 모든 allele frequency를 구하다.

Conclusion 적어도 지금 이순간까지 종합선물셋트는 없다. 맛없는 과자도 들어있다. 자신의 현재 상황에 맞는 과자를 담아라.

Add a comment

Related presentations

How organisms adapt and survive in different environment.

Aplicación de ANOVA de una vía, modelo efectos fijos, en el problema de una empres...

Teori pemetaan

Teori pemetaan

November 10, 2014

learning how to mapping

Libros: Dra. Elisa Bertha Velázquez Rodríguez

Materi pelatihan gis

Materi pelatihan gis

November 10, 2014

learning GIS

In this talk we describe how the Fourth Paradigm for Data-Intensive Research is pr...

Related pages

AWS Case Studies: Big Data

Genomics Access powerful computing tools to run genomics workloads. ... Take Part in an AWS Usability Study ... AWS customer case studies about big data ...
Read more

Genomic and epigenomic signatures for interpreting complex ...

Big data Opportunities & Challenges in human disease genetics & genomics. ... Case study : Schizophrenia ... Genomic and epigenomic signatures for ...
Read more

Hadoop Genomics: Intel® Technology Improves Genomic Data ...

Case Study: NextBio enhances genomic data analytics with Intel® Distribution for Apache Hadoop* software, enabling growth and superior performance.
Read more

Big Data in Genomics and Cancer Treatment - Hortonworks

Big Data in Genomics and Cancer Treatment. by ... since its first documented case in ... solve the challenge of big datasets in the field of genomics.
Read more

Big Data Genomics

... we made significant changes to support version 0.6.0 of the Big Data Genomics Avro data formats. We also improved performance on core transforms ...
Read more

Genomics 3.0: Big-data in Precision Medicine - Springer

Genomics 3.0: Big-data in Precision Medicine. ... genomics is the study of the whole genome or the DNA ... co-expression and case/control study; ...
Read more

Big Data and Analytics Blaze a Trail for Genomic Medicine ...

... @BigDataExpo #API #Cloud #BigData. ... that genomics research ... business transformation case study examines how Canadian nonprofit SOCAN ...
Read more

Health IT: Healthcare Analytics - Intel

Health IT: Healthcare Analytics. ... and growing every day. From medical records to genomics, ... Case study: Cutting epic and ...
Read more