Published on March 14, 2014
Graphite: An Introduction Scaling real-time monitoring
The purpose today
What is graphite
Why it’s so great
How to graph (It’s really easy!)
How we use graphite
First, a definition
Alerts+Metrics=Monitoring Graphite Cacti Munin Nagios Icinga Both Zenoss Hyperic Zabbix PNP4Nagios Alerting Metrics
What is graphite
About graphite ● Django web application consisting of 3 parts: ○ carbon (relays, caches, aggregates metrics) ○ whisper (graphite’s equivalent of RRD files) ○ Web UI (graph composer, simple dashboard)
Why graphing? Discover trends and patterns What time of the day do we get the most users? When x happened, what was the effect on y? How many hits am I getting per hour? How does this compare to last week? last month? Predict future events When will we need to add more servers? Databases? Negative feedback Did the release into production fix problem x?
Functions (sum, derivatives, integrals, timeshift, mostDeviant, scale, averages, etc.)
API (Nagios integration, 3rd party custom dashboards)
Easy to feed data
Wide ecosystem of 3rd party tools and dashboards http://graphite.readthedocs.org/en/latest/tools.html
No all in one solution
No easy backups
It probably will become business critical
How to graph
There are tons of ways to feed graphite your data
Bash #!/bin/bash timestamp = `date +%s` value = 10 echo "dot.delimited.metric.name $value $timestamp" | nc -w 1 graphite. host.name 2003 Python def send_msg(message, HOST, PORT): sock = socket.create_connection((HOST, PORT)) sock.send(message) sock.close() Python using graphite-pymetrics from metrics import timing @timing("heavy.task") def heavy_task( x, y, z): # do heavy stuff here
Ruby require 'socket' Host = 'somegraphitehost' conn = TCPSocket.new Host, 2003 conn.puts 'Metrics value timestamp' conn.close Java import java.io.DataOutputStream; import java.net.Socket; Socket conn = new Socket("somegraphitehost" , 2003); DataOutputStream dos = new DataOutputStream(conn .getOutputStream()); dos.writeBytes("metrics value timestamp" ); conn.close();
How we use graphite
700K + metrics per minute
A Common Graphite Stack Graphite-web Collectd Poller(s) Applications Carbon Whisper Dashboards Statsd Scripts Nagios
Collectd Agent for system/hardware level metrics Growing repository of plugins for a wide variety of applications: disk i/o, disk space, cpu, memory, mysql, JMX, java, Redis, file sizes, load, etc. https://collectd.org/wiki/index.php/Table_of_Plugins Write your custom plugin in python
Nagios integration You can write Nagios plugins that can alert off of metrics values Nagios can also feed graphite performance data, events (ie: update counter each time email is sent), etc.
What to collect?
How often function x is called
Average value of function x
Average running time of function x
number of records with value == ?
number of slow queries
send a 1, draw as infinite
http access logs (2xx, 3xx, 4xx, 5xx)
Application logs Exception counts, results, important events, hits
Treat graphite like ‘Big Data’
You don’t know what metrics you need until you need it
Get Raid 10 SSD’s once you decide to scale
You can start graphing today!
Introduction. There are plenty of reasons why collecting stats about your servers, applications, and traffic is a good idea. Collecting and organizing data ...
Graphite is one of the three familiar naturally occurring forms of the chemical element carbon. The two other varieties are amorphous carbon (not to be ...
AN INTRODUCTION TO :- CARBON Carbon Carbon ... Graphite is almost the complete contrast to diamond, in that diamond is one of the worlds hardest, ...
1 An Introduction to Synthetic Graphite. Prepared by the Technical Services Department of the Asbury Graphite Mills Inc., Albert V. Tamashausky
Introduction This homepage is intended for current and potential users of the material graphite and carbon. Answers to questions as What are carbon and
Graphite production & further processing • Introduction • Graphitising • Graphitisation furnaces • Coating • Castner • Inductively heated
Graphite, a class of materials with many unique and unusual properties, shows a remarkably high creep ductility under irradiation. As this behaviour ...
Graphite Trades Accuracy for Performance •Simulator performance is a major limiting factor – Limits depth and breath of studies, size of benchmarks
A brief review is given of the electronic properties of the lamellar compounds of graphite, with particular emphasis on the dilute limit. Experimental ...