Social Stream Analysis Use Cases

50 %
50 %
Information about Social Stream Analysis Use Cases

Published on February 27, 2014




Big  Brother  for  Enterprises:     Social  Media  Stream  Use  Cases   Samisa  Abeysinghe   VP  Developer  Evangelism   26  Feb  2014

About  the  Presenter   ๏  ๏  2   Samisa  Abeysinghe   VP  Developer  Evangelism     Samisa  Abeysinghe,  Vice   President  of  Developer   Evangelism  joined  the  company  in   September  2005.    Prior  to  the   current  role,  Samisa  used  to  be   VP  of  Engineering  and  managed   the  development  of  WSO2   Carbon  based  product  plaOorm.    

About  WSO2   ๏  ๏  ๏  Global  enterprise,  founded  in  2005   by  acknowledged  leaders  in  XML,   web  services    technologies,   standards    and  open  source   Provides  only  open  source   plaOorm-­‐as-­‐a-­‐service  for  private,   public  and  hybrid  cloud   deployments   ๏  ๏  3   All  WSO2  products  are  100%  open   source  and  released  under  the   Apache  License  Version  2.0.   Is  an  AcXve  Member  of  OASIS,   Cloud  Security  Alliance,  OSGi   Alliance,  AMQP  Working  Group,   OpenID  FoundaXon  and  W3C.   Driven  by  InnovaXon   ๏  Launched  first  open  source  API   Management  soluXon  in  2012   ๏  Launched  App  Factory  in  2Q  2013   ๏  Launched  Enterprise  Store  and   first  open  source  Mobile  soluXon   in  4Q  2013  

What  WSO2  Deliver   4  

NSA like Monitoring for Your Enterprise ๏ Analyze volumes of data ๏ Address correlation complexities in analytics ๏ Off line vs Real time operations implications ๏ Some operations got to be in real time, else the value is lost ๏ Summarized data over time (and other) dimensions for analytics A “Big Brother” that keeps an eye on the whole enterprise 5  

Why should I bother? ๏ Deal with high volume (terra bites) of information ๏ In order to make decisions ๏ Real time & Offline ๏ Take action 6  

WSO2 :ONLY COMPLETE & INTEGRATED Platform ๏ Complete and integrated for ๏ Data capture ๏ Analysis: both real-time and batch ๏ Visualization ๏ Action taking business process execution Making data driven intelligence for your enterprise easy 7  

Big Data: Strategy 8  

Responsive and Effective 9  

WSO2 Big Data Analytics Platform for Your Enterprise WSO2  Big  Data  Analy0cs   Pla4orm  for  Your  Enterprise   10  

Key  Elements   Data   CollecXon   Taking   AcXon   Data   Analysis   Data   VisualizaXon   11  

WSO2 Big Data Analytics Platform for Your Enterprise Data Collection 12   Data Analysis Data Visualization & Taking Action

Use  Case  Scenario:   Analyzing  Twiaer  Streams   13  

Social  media  …  !   o  Test   o      Test  

Processing  Social  Media  !!!      

Why  Monitor  Social  Media  ?   ●  To  get  useful  insight  about  the  market     ○  CompeXtor  Analysis   ○  Customer  Trend  and  Interest   ●  IdenXfy  and  Compare  Trends     ○  Monitoring  the  brand   ○  Comparing  the  trends  of  the  public  interest   16  

WSO2  Business  AcXvity  Monitor   (BAM)   ●  Can  capture  data  from  any  applicaXon/service     ●  APIs,  Services  or  source  including  file-­‐based  data   ○  WSO2  has  lightweight  agents  to  provide     ■  raw  data  from  numerous  sources     ■  as  well  as  support  for     ●  gateways  (lightweight  or  heavyweight)     ●  Service  and  API  proxies   ●  Data  imports  from  various  sources  (including  file  based  data)   ●  Any  other  source  (extended  agents)   17  

WSO2  Business  AcXvity  Monitor   (BAM)   18  

WSO2  Complex  Event  Processor   (CEP)   ●  CEP  process  mulXple  event  streams  generated  within  the   enterprise  to  construct  data  abstracXon  and  idenXfy   meaningful  paaerns  among  those  streams   ○  Powered  by  WSO2  Siddhi   ■  ■  Processes  more  than  2.5M  events/sec  on  single  server  (commodity  hardware)   Has  a  powerful  and  extensible  SQL  Like  Siddhi  query  language     ○  Integrates  with  any  enterprise  system  for  event  capture   ■  Support  mulXple  transports  &  event  formats:  HTTP,  JMS  &  SMTP  on  XML,  JSON,  Text  &  Map   ○  Efficient  large  volume  event  publishing  to  enterprise   systems  and  custom  noXficaXons     ■  ■  ■  19   XML,  JSON,  Text  over  HTTP,  JMS  &  SMTP       SMS  Support     MySQL  DB  and  Cassandra  Support  

WSO2  Complex  Event  Processor   (CEP)   20  

E.g.  Usecases   ●  Finding  out  the  trend  of  #Cloud  Twiaer  tags   over  Xme     ●  IdenXfy  the  public  interest  on  cloud  market   leaders  based  on  tweets  on  #Cloud  Twiaer  tags     ●  Comparing  interest  trends  on  cloud  market     ●  NoXfying  via  Email  when  an  interesXng  tweet   was  posted.   21  

SoluXon  Architecture   22  

Twiaer  Agent                //Configuring  Agent                  AgentConfiguraXon  agentConfiguraXon  =  new  AgentConfiguraXon();                  agentConfiguraXon.setTrustStorePassword("/Users/suho/wso2/dev/twiaer-­‐sample/wso2cep-­‐3.0.0/repository/"  +                                  "resources/security/client-­‐truststore.jks");                  agentConfiguraXon.setTrustStore("wso2carbon”);                  Agent  agent  =  new  Agent(agentConfiguraXon);                    //New  Data  publisher  to  BAM                  final  AsyncDataPublisher  asyncDataPublisher  =  new  AsyncDataPublisher("tcp://localhost:7611",                                  "admin",  "admin",  agent);                    //Defining  the  Stream                  String  STREAM_NAME  =  "twiaer_data_stream";                  String  VERSION  =  "1.0.0";                  String  streamDefiniXon  =  "{"  +                                  "  'name':'"  +  STREAM_NAME  +  "',"  +                                  "  'version':'"  +  VERSION  +  "',"  +                                  "  'nickName':  'Twiaer_Data_Stream',"  +                                  "  'descripXon':  'Stream  of  twiaer  data',"  +                                  "  'payloadData':["  +                                  "  {'name':'createdAt','type':'LONG'},"  +                                  "  {'name':'user','type':'STRING'},"  +                                  "  {'name':'tweet','type':'STRING'}"  +                                  "  ]"  +                                  "}";                  asyncDataPublisher.addStreamDefiniXon(streamDefiniXon,  STREAM_NAME,  VERSION);     23  

Visualizing  #Cloud  Twiaer  Social   Stream   24  


Finding  out  the  trend  of  #Cloud   Twiaer  tags  over  Xme   ๏  ๏  26   insert  overwrite  table  Twi@erDataStreamStatsPerMinute  select  count(1)  as   total_tweet_count,  year(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  ))  as  year,  month(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐ MM-­‐dd  HH:mm:ss.SSS'  ))  as  month,day(from_unixXme(cast(created_at/1000  as   BIGINT),'yyyy-­‐MM-­‐dd  HH:mm:ss.SSS'  ))  as  day,hour(from_unixXme(cast(created_at/ 1000  as  BIGINT),'yyyy-­‐MM-­‐dd  HH:mm:ss.SSS'  ))  as  hour,   minute(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd  HH:mm:ss.SSS'  ))   as  minute,concat(substring(from_unixXme(cast(created_at/1000  as  BIGINT),  'yyyy-­‐MM-­‐ dd  HH:mm:ss'),0,16),':00')  as  Xme  from  TwiaerDataStream  group  by   year(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd  HH:mm:ss.SSS'  )),   month(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  )),day(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  )),hour(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  )),minute(from_unixXme(cast(created_at/1000  as  BIGINT),'yyyy-­‐MM-­‐dd   HH:mm:ss.SSS'  )),substring(from_unixXme(cast(created_at/1000  as  BIGINT),  'yyyy-­‐MM-­‐ dd  HH:mm:ss'),0,16);      


IdenXfy  the  public  interest  on   cloud  market  leaders     insert  overwrite  table  CountOnCompanyTwi@erDataStream     select  'IBM'  as  company,  count(1)  as  cnt  from  TwiaerDataStream  where  instr(lower(tweet),   'ibm')  >0  ;     insert  overwrite  table  CountOnCompanyTwi@erDataStream     select  'Google'  as  company,  count(1)  as  cnt  from  TwiaerDataStream  where   instr(lower(tweet),  'google')  >0  ;     insert  overwrite  table  CountOnCompanyTwi@erDataStream     select  'VMware'  as  company,  count(1)  as  cnt  from  TwiaerDataStream  where   instr(lower(tweet),  'vmware')  >0  ;     insert  overwrite  table  CountOnCompanyTwi@erDataStream     select  'Azure'  as  company,  count(1)  as  cnt  from  TwiaerDataStream  where  instr(lower(tweet),   'azure')  >0  ;       28  


Comparing  interest  trends  on   cloud  market   30  


NoXfying  via  Email  when  an   interesXng  tweet  was  posted!   &  Take  AcXon   32  

CEP:  Retweet  NoXficaXon     from  twiaerStream[tweet  contains  'RT  ']     insert  into  reTweetStream;         33  

Demo  ….. 34  

Log  Analysis  Demos  ….. 35  

Business  Model   36  

Call  to  acXon  page   ๏  ๏  Big  Data  Webinar  Series:   hap://­‐bigbrother-­‐webinar-­‐series/     ๏  WSO2  Business  AcXvity  Monitor:   hap://­‐acXvity-­‐monitor/     ๏  WSO2  BAM  Docs:   hap:// +DocumentaXon     ๏  WSO2  Complex  Event  Processor:   hap://­‐event-­‐processor/     ๏  37   Be  your  own  NSA:  hap://     WSO2  CEP  Docs:   hap:// +DocumentaXon    

Contact  us  !  

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Introduction to Stream Analytics | Microsoft Azure

Social media analysis; ... such real-time event-stream data analysis to succeed in the ... for ease of use, Stream Analytics uses a ...
Read more

Real-Time Stream Processing as Game Changer in a Big Data ...

Some use cases where stream processing can ... TIBCO StreamBase is "just" for stream processing / streaming ... social feeds, etc). You use it for ...
Read more

Use Cases |

Use Case 1: Housekeeper does laundry: Actor: Housekeeper: Use Case Overview: It is Wednesday and there is laundry in the laundry room. The housekeeper ...
Read more

Customer Analytics, Big Data Use Case - Datameer

Visit us to learn how Datameer customer analytics can help you use big data to ... social media, customer ... Datameer for customer analytics use cases in ...
Read more

IBM Big Data and Analytics - Case Studies - United States

IBM Big Data and Analytics case ... Educational institutions use ... Resolve customer service calls quickly with automatic root cause analysis ...
Read more

Tutorial - Writing Effective Use Case

Use case, an important concept in use case analysis, ... We will carry on with writing effective use case with the resulting design.
Read more

Concept: Use-Case Model - EPF Wiki

The use-case model may contain packages that are used to structure the model to simplify analysis, ... of the use case. The use-case model serves as a ...
Read more

Hortonworks: Open and Connected Data Platform Industry ...

Hortonworks Connected Data Platforms transform businesses in every industry, ... USE CASES. VIDEO. Data Discovery. WATCH NOW. ... social media streams, ...
Read more

Stream Analytics Documentation - IoT Analytics | Microsoft ...

Learn Internet of Things (IoT) analytics in the cloud with Stream Analytics. Analyze data in real time from applications, sensors, and devices.
Read more