Workshop - introduction to

50 %
50 %
Information about Workshop - introduction to

Published on March 9, 2014

Author: itayweinberger



This workshop focuses on hands-on introduction to

For a complete breakdown of the Workshop itself, refer to the project's wiki @ Workshops Technical Review of Worksh op Details an d In stru ction s


About the workshop Th is is a h an ds-on worksh op Lab m aterials are available for th is worksh op Th is worksh op an d m aterials are available on ou r GitH u b repo

Workshop Goals U n derstan d on all of its com pon en ts Be able to deploy Pu sh an d Qu ery data from th e system Draw basic Visu alization s

Lab Details Du rin g th is worksh op we'll be accessin g an on lin e lab Please SSH with th e followin g $ s s h work s Password: password * ac c es s to the abov e lab is whitelis ted


What is $ npm ins tall Featu re packed Data An alytics an d Visu alization Fram ework Written in N ode.JS Scalable to deal with h igh volu m es of data an d realtim e qu eries Secu re an d m u lti-ten an t Em bed an d in tegrate data visu alization s in to existin g sites an d apps Open Sou rce

How does it work? Operates an d m an aged via a set of H TTP API en dpoin ts Data pu sh ed u sin g JSON via's Beacon API Stored in cach in g layer, optim ized for qu ery Qu eries an d Visu alization s provided by Javascript SDK Th at's all in a n u tsh ell!

Developer Friendly By Data Lovers for Developers Ease of u se Exten d Em bed an d In tegrate Site specific au tom ated testin g Rich docu m en tation

IT Friendly M asterless grid Easy in stallation Zero con figu ration Powerfu l CLI Sm all footprin t M on itor, Alerts an d Logs

DBA Friendly Easy Data M odellin g Sch em a-less Flexible Zero down tim e u pdates Com plete con trol


Nodes Everywhere Each n ode is a fu lly fu n ction in g system Com m u n ication is based on Pu b/Su b A n ode n eeds to qu alify for exeu ction Qu alification can vary an d be exten ded Geo prioritization

Dispatch Routing Every action is a m essage All m essages are distribu ted M essages con tain secu rity con text Defau lt `n otm e` h an dlin g logic Backlog processin g of stu ck m essages

Command the Grid U se REPL, CLI or SDK $ li # admin@loc alhos t:8080 > joolaio.s y s tem.lis tnodes (); # admin@loc alhos t:8080 > joolaio.s y s tem.terminate('nodeuid'); Wh en you 're con n ected to on e n ode, you 're con n ected to th e Grid N o action is execu ted on th e local n ode, bu t dispatch ed All n odes su bscribe to con figu ration ch an ges

Node.JS' Event Loop Block M an y con sider as N ode.JS' ach illes h eel, we don 't Th e Grid an d Pu b/Su b approach resolves th e core issu e ELB cou n ters an d m on itors U sin g th e righ t tool for th e job M ade ou r software better

Accessing Via H TTP API en dpoin ts U se SDK, CLI, cU RL or write you r own Sh ipped with Web Adm in con sole Ou t-of-th e-box fu lly fu n ction in g an alytics site Rich docu m en tation

Pushing Data

Pushing Data is Easy joolaio.beac on.ins ert('c ollec tion-purc has es ', { times tamp: new Date(), Play er: { Us ername: 'A play er', Dev ic e: 'Des k top', Brows er: 'Chrome' }, ip: '', Vis its : 1, Vis itDuration: 123, Clic k s : 23, Purhc as es : { Amount: 123 Currenc y : 'USD', Status : 'Proc es s ed' } }});

Data Modelling Describe you r data as a JSON docu m en t(s) Docu m en ts are stored in a Collection Sch em a-less, docu m en ts can be of differen t stru ctu re Su pport for all data types Su pport for Stron g-Typed collection s

Push What You'll Need Store on ly data relatin g to An alytics Separate in to logical collection s Describe every m etric with as m an y attribu tes as possible Pass raw data, don 't aggregate Pu sh an array of docu m en ts wh en possible

Caching Layer Based on leadin g providers, su ch as M on goDB, Cassan dra an d m ore Exten d to su pport addition al In sert, u pdate an d delete directly from cach e Execu te qu eries directly on cach in g layer N orm alized access to u n derlyin g cach e via Beacon API


Understanding Queries Describe wh at you wish to display, rath er th an h ow to get it (SQL) Su pport for Realtim e (WebSocket) Join data from m u ltiple collection s Relative tim efram es Advan ced filters Dim en sion an d M etric tran sform ation s M an y aggregation operators, in clu din g U n iqu e Cou n t

Basic Query Describe th e requ ired dim en sion s an d m etric, n ot h ow to qu ery th em joolaio.query .fetc h({ timeframe: 'las t_month', interv al: 'day s ', dimens ions : ['times tamp', 'dev ic e'], metric s : ['v is its ', { k ey : 'av gv is itduration', name: 'Av g. Vis it Duration', depends On: 'v is itduration', aggregation: 'av g', s uffix : 'ms .' }] }, func tion(err, res ult){ c ons ole.log(res ult.doc uments ); });

Calculated Metrics Com pose virtu al, calcu lated m etrics on -th e-fly metric s : [{ k ey :'av gamountperplay er', name: 'Av g. Amount per Play er', formula: { depends On:[{ k ey : 'av gamount', depends On: 'Purc has e.Amount', aggregation: 'av g' }, { k ey : 'play erc ount', depends On: 'Play er.Us ername', aggregation: 'uc ount' }], run: 'func tion(av gamount, play erc ount) {return av gamount/play erc ount;}' } }]

Query Filters M u ltiple filters - m u tli-level filters - per m etric filters timeframe: 'las t_month', interv al: 'day s ', metric s : [{ k ey : 'play erc ount', depends On: 'Play er.Us ername', aggregation: 'uc ount', filter: [ ['Play er.Brows er', 'eq', 'Chrome']] }], filter: [ ['Play er.Dev ic e', 'eq', 'Des k top'] ]

Query Results Con tain retrieved docu m en ts an d m eta-data describin g th e qu ery an d its tim in gs. Resu lts con tain both raw an d form atted valu es. res ults : [ { v alues :{ times tamp: '2014-03-09T14:03:34.412Z', av gduration: '123 ms .' }, formattedValues : { times tamp: 'Sun Mar 09 2014 15:03:34 GMT+0100 (CET)', av gduration: 123' } }, {...} ]

Realtime Bu ilt-in in to all visu alization s an d in tegral part of qu ery.fetch joolaio.query .fetc h({ //query params , s uc h as timeframe, metric s , etc ... realtime: true }); First execu tion retu rn s en tire tim efram e Followin g packets retu rn on ly delta Data con sisten cy assu ran ce


Plotting Data Part of th e Javascript SDK Easy as copy-paste Sh ape to th e desired look, feel an d beh avior Offers jQu ery plu gin s Rich Docu m en tation an d Playgrou n d

Visualization Types M etric Box Sparklin e Pie Ch art Tim elin e Table

Drawing a Visualization Easy as a copy-pastin g from th e wiki Accepts a Qu ery object an d addition al form attin g option s joolaio.v iz .timeline({ c ontainer: doc ument.getElementBy Id('drawhere'), query : { dimens ions :['times tamp'], metric s : ['v alue'] }); U ses H TM L5, su pports all device types (m obiles, tablets)

Integrate, Embed and Extend

Integrate Design ed an d bu ilt for in tegration Pu sh data from an y data sou rce type wh ile keepin g con trol of logic Su pport for h igh -rate distribu ted, gu aran teed writes JSON collection s provide flexibility an d site specific in tegration Collect data from all parts of th e bu sin ess Leverage to redu ce overh ead of operation al databases

Embed Seam less drop-in visu alization s in you r site an d apps Even t driven com m u n ication with con tain er page Copy-paste from wiki On ly fron t-en d developers are n eeded, redu ce DBA overh ead Con trol th e tin iest detail of look, feel an d flow Respon sive, H TM L5, CSS3 stan dards

Extend M issin g a featu re? Easy to learn in frastru ctu re an d code Follows best practices gu idelin es an d m eth odology Access to sou rce code, issu es, wiki an d kn ow-h ow Con tribu te to open sou rce project Exten d an yth in g from cach in g layer to visu alization s

Endless Possibilities

Analytics-as-a-Service Offer an alytics an d visu alization s to you r en d-u sers Scalable an d redu n dan t to su pport very large operation s M u lti-ten an t an d secu re Easy bran din g an d clien t in tegration

Web Analytics Easy to setu p an d con figu ration Gath er an d join m u ltiple data sou rces in to a sin gle store joolaio.beac on.ins ert('performanc e', window.performanc e); Setu p alerts on specific u ser activity or th resh old

Ad Serving N o n eed to pre-determ in e au dien ces an d segm en ts Leverage split secon d qu ery tim es Apply advan ced filters an d segm en ts in realtim e Determ in e best m atch by u n lim ited param eters Perform fu ll/partial segm en tation on -th e-fly

Infographics Offer "live" in fograph ics Com bin e realtim e calcu lation s in to static con ten t Segm en ted in fograph ics Secu re/m u lti-ten an t in fograph ics

THANK YOU Visit ou r GitH u b to read m ore abou t

Add a comment

Related presentations

Related pages

Draft Technical Meeting-basket - Documents

... Technical Introduction to This workshop focuses on hands-on introduction to For a complete breakdown of the Workshop itself, ...
Read more

Ecuador To T Workshop Technical Session3 - Documents

... Technical Introduction to This workshop focuses on hands-on introduction to For a complete breakdown of the Workshop itself, ...
Read more