advertisement

An introduction to Apache Whirr

43 %
57 %
advertisement
Information about An introduction to Apache Whirr

Published on December 15, 2013

Author: mikejf12

Source: slideshare.net

Description

A short introduction to Apache Whirr. What is it and how does it relate to the cloud ?
How can it be used with Hadoop ?
advertisement

Apache Whirr ● What is it ? ● How does it work? ● The Cloud ● Architecture ● Use with Hadoop www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Apache Whirr – What is it ? ● A library based cloud service system ● API libraries for cloud providers ● Choose a configuration file to define a cluster ● High level interaction ● Service based on roles www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Apache Whirr – How does it work ? ● Libraries provided to offer high level API ● Based on JClouds ● Recipe based approach for cloud providers i.e. EC2 ● It is cloud neutral ● Create clusters as you need them for – Dev / test etc www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Apache Whirr – How does it work ? ● Automatically start instances on the Cloud ● Configure and start Hadoop ● Add applications like – Hive – Hbase – Yarn / MapReduce www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Apache Whirr – Why go virtual ? ● Whirr gives independence from Cloud vendor ● Makes it easier to move vendors later ● Save money by only using what you need ● Expand the cluster as demand requires ● Reduce the cluster when possible ● Compress data as much as possible to reduce costs ● Virtual cost < physical cost ( should be ) until – Data sizes in high Tbyte – low Pbyte range www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Apache Whirr – Supported ● What cloud suppliers are available – – ● Amazon EC2 Rackspace Cloud Services What services do they support ? – Cassandra – Hadoop – Zoo Keeper – Hbase – Elastic Search – Voldemort – Hama www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Apache Whirr – Example config Example Whirr configuration - .whirr/credentials whirr.provider=aws-ec2 whirr.identity=your-aws-key whirr.credential=your-aws-secret Whirr Hadoop configuration - hadoop.properties whirr.cluster-name=myhadoopcluster whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker whirr.provider=aws-ec2 Start Whirr whirr launch-cluster --config hadoop.properties www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Contact Us ● Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz ● We offer IT project consultancy ● We are happy to hear about your problems ● You can just pay for those hours that you need ● To solve your problems

Add a comment

Related pages

Apache Whirr - Cloudera

Apache Whirr. There are currently no incompatible changes. << Apache Sqoop ... Introduction to HDFS High Availability; Hardware Configuration for HDFS HA;
Read more

Introducing Apache Whirr | Paul Scott

I was a little disturbed to see that so few people had even heard of Apache Whirr, so I decided to write this as a bit of an introduction. Apache Whirr is ...
Read more

Whirr Installation - Cloudera

Whirr Installation ... Apache Whirr is a set of libraries for running cloud services. ... Introduction to Hadoop Security
Read more

Apache | Paul Scott

I was a little disturbed to see that so few people had even heard of Apache Whirr, so I decided to write this as a bit of an introduction. Apache Whirr is ...
Read more

An introduction to zeppelin - Video Dailymotion

An introduction to Apache Whirr by semtechsolutions. 2 views ... An introduction to Apache Spark MLlib. 01:41 An introduction to Apache Mesos. 02:25
Read more

Clustered Services With Apache Whirr: From Dev Up ...

Clustered Services With Apache Whirr: From Dev Up - PuppetConf '11 ... An introduction to Apache Whirr - Duration: 1:44. by Mike Frampton 142 views.
Read more