Graphite, an introduction

50 %
50 %
Information about Graphite, an introduction
Technology

Published on March 14, 2014

Author: jamesrwu

Source: slideshare.net

Description

An introduction to graphite and why it's so great.

Graphite: An Introduction Scaling real-time monitoring

The purpose today

What is graphite

Why it’s so great

How to graph (It’s really easy!)

How we use graphite

First, a definition

Alerts+Metrics=Monitoring Graphite Cacti Munin Nagios Icinga Both Zenoss Hyperic Zabbix PNP4Nagios Alerting Metrics

What is graphite

About graphite ● Django web application consisting of 3 parts: ○ carbon (relays, caches, aggregates metrics) ○ whisper (graphite’s equivalent of RRD files) ○ Web UI (graph composer, simple dashboard)

Why graphite?

Why graphing? Discover trends and patterns What time of the day do we get the most users? When x happened, what was the effect on y? How many hits am I getting per hour? How does this compare to last week? last month? Predict future events When will we need to add more servers? Databases? Negative feedback Did the release into production fix problem x?

Cacti SUCKS A few reasons: Ancient user interface (no javascript/ajax), terrible workflow, cannot push metrics, no formulas, no graph introspection, cannot push metrics, cannot feed out of sequence metrics, ugly graphs, no API, expose system/os metrics on host via snmp, no graph composer, no custom graphs, predefine metrics, predefine graphs, static polling interval, unscalable, tons of work to create one graph, no 3rd party ecosystem, etc.

Graphite ++

Simple

Powerful

Functions (sum, derivatives, integrals, timeshift, mostDeviant, scale, averages, etc.)

API (Nagios integration, 3rd party custom dashboards)

Scalable

Easy to feed data

Wide ecosystem of 3rd party tools and dashboards http://graphite.readthedocs.org/en/latest/tools.html

Tools

StatsD

Logster

Skyline

Collectd

Dashboards

Graphite --

No poller

No all in one solution

No easy backups

It probably will become business critical

How to graph

There are tons of ways to feed graphite your data

Bash #!/bin/bash timestamp = `date +%s` value = 10 echo "dot.delimited.metric.name $value $timestamp" | nc -w 1 graphite. host.name 2003 Python def send_msg(message, HOST, PORT): sock = socket.create_connection((HOST, PORT)) sock.send(message) sock.close() Python using graphite-pymetrics from metrics import timing @timing("heavy.task") def heavy_task( x, y, z): # do heavy stuff here

Ruby require 'socket' Host = 'somegraphitehost' conn = TCPSocket.new Host, 2003 conn.puts 'Metrics value timestamp' conn.close Java import java.io.DataOutputStream; import java.net.Socket; Socket conn = new Socket("somegraphitehost" , 2003); DataOutputStream dos = new DataOutputStream(conn .getOutputStream()); dos.writeBytes("metrics value timestamp" ); conn.close();

How we use graphite

700K + metrics per minute

A Common Graphite Stack Graphite-web Collectd Poller(s) Applications Carbon Whisper Dashboards Statsd Scripts Nagios

Collectd Agent for system/hardware level metrics Growing repository of plugins for a wide variety of applications: disk i/o, disk space, cpu, memory, mysql, JMX, java, Redis, file sizes, load, etc. https://collectd.org/wiki/index.php/Table_of_Plugins Write your custom plugin in python

Nagios integration You can write Nagios plugins that can alert off of metrics values Nagios can also feed graphite performance data, events (ie: update counter each time email is sent), etc.

What to collect?

Hardware/OS metrics

Load

Disk space

Disk I/O

Network data

Application metrics

How often function x is called

Average value of function x

Average running time of function x

Database/Datastore

performance metrics

number of records with value == ?

number of slow queries

Events

Deployments

send a 1, draw as infinite

Log files

http access logs (2xx, 3xx, 4xx, 5xx)

Application logs Exception counts, results, important events, hits

Final Musings

Treat graphite like ‘Big Data’

You don’t know what metrics you need until you need it

Get Raid 10 SSD’s once you decide to scale

More devopsy

You can start graphing today!

Add a comment

Related presentations

Related pages

An Introduction to Tracking Statistics with Graphite ...

Introduction. There are plenty of reasons why collecting stats about your servers, applications, and traffic is a good idea. Collecting and organizing data ...
Read more

Introduction | Asbury Carbons

Graphite is one of the three familiar naturally occurring forms of the chemical element carbon. The two other varieties are amorphous carbon (not to be ...
Read more

INTRODUCTION TO CARBON

AN INTRODUCTION TO :- CARBON Carbon Carbon ... Graphite is almost the complete contrast to diamond, in that diamond is one of the worlds hardest, ...
Read more

An Introduction to Synthetic Graphite. - Asbury Carbons ...

1 An Introduction to Synthetic Graphite. Prepared by the Technical Services Department of the Asbury Graphite Mills Inc., Albert V. Tamashausky
Read more

Introduction - European Carbon and Graphite Association

Introduction This homepage is intended for current and potential users of the material graphite and carbon. Answers to questions as What are carbon and
Read more

Graphite production further processing - Carbon and Graphite

Graphite production & further processing • Introduction • Graphitising • Graphitisation furnaces • Coating • Castner • Inductively heated
Read more

Radiation creep of graphite. An introduction

Graphite, a class of materials with many unique and unusual properties, shows a remarkably high creep ductility under irradiation. As this behaviour ...
Read more

Graphite Introduction and Overview - MIT Computer Science ...

Graphite Trades Accuracy for Performance •Simulator performance is a major limiting factor – Limits depth and breath of studies, size of benchmarks
Read more

The electronic properties of lamellar compounds of ...

A brief review is given of the electronic properties of the lamellar compounds of graphite, with particular emphasis on the dilute limit. Experimental ...
Read more