NSA for Enterprises Log Analysis Use Cases

56 %
44 %
Information about NSA for Enterprises Log Analysis Use Cases

Published on February 20, 2014

Author: wso2.org

Source: slideshare.net

Big  Brother  for  Enterprises:     Log  Analysis  Use  Cases   Samisa  Abeysinghe   VP  Developer  Evangelism   19  Feb  2014

About  the  Presenter   ๏  ๏  2   Samisa  Abeysinghe   VP  Developer  Evangelism   samisa@wso2.com     Samisa  Abeysinghe,  Vice   President  of  Developer   Evangelism  joined  the  company  in   September  2005.    Prior  to  the   current  role,  Samisa  used  to  be   VP  of  Engineering  and  managed   the  development  of  WSO2   Carbon  based  product  plaOorm.    

About  WSO2   ๏  ๏  ๏  Global  enterprise,  founded  in  2005   by  acknowledged  leaders  in  XML,   web  services    technologies,   standards    and  open  source   Provides  only  open  source   plaOorm-­‐as-­‐a-­‐service  for  private,   public  and  hybrid  cloud   deployments   ๏  ๏  3   All  WSO2  products  are  100%  open   source  and  released  under  the   Apache  License  Version  2.0.   Is  an  AcXve  Member  of  OASIS,   Cloud  Security  Alliance,  OSGi   Alliance,  AMQP  Working  Group,   OpenID  FoundaXon  and  W3C.   Driven  by  InnovaXon   ๏  Launched  first  open  source  API   Management  soluXon  in  2012   ๏  Launched  App  Factory  in  2Q  2013   ๏  Launched  Enterprise  Store  and   first  open  source  Mobile  soluXon   in  4Q  2013  

What  WSO2  Deliver   4  

NSA like Monitoring for Your Enterprise ๏ Analyze volumes of data ๏ Address correlation complexities in analytics ๏ Off line vs Real time operations implications ๏ Some operations got to be in real time, else the value is lost ๏ Summarized data over time (and other) dimensions for analytics A “Big Brother” that keeps an eye on the whole enterprise 5  

Why should I bother? ๏ Deal with high volume (terra bites) of information ๏ In order to make decisions ๏ Real time & Offline ๏ Take action 6  

WSO2 :ONLY COMPLETE & INTEGRATED Platform ๏ Complete and integrated for ๏ Data capture ๏ Analysis: both real-time and batch ๏ Visualization ๏ Action taking business process execution Making data driven intelligence for your enterprise easy 7  

WSO2 Big Data Analytics Platform for Your Enterprise WSO2  Big  Data  Analy0cs   Pla4orm  for  Your  Enterprise   8  

Key  Elements   Data   CollecXon   Taking   AcXon   Data   Analysis   Data   VisualizaXon   9  

Use  Case  Scenario  1:   Monitor  your  Java  ApplicaXon  System  Logs     with  BAM  &  CEP 10  

& WSO2 CEP 11  

Overview  of  SoluXon     o  Send Log Events to o  o  o  Real time Log Event Processing o  o  With BAM analytics Visualization of Log Data o  12   With CEP Batch Processing of Log Data o  o  Business Activity Monitor (BAM) & Complex Event Processor (CEP) With Gadgets on Dashboards

Log  Event Publishing  (BAM)     13  

Event  Streams  &  AlerXng  (CEP)   WSO2 CEP 14  

LogEvent  Stream   q  q  Meta Data q  clientType {String} Meta Data q  tenantID {String} q  ServerName {String} q  appName {String} q  logTime {Long} q  priority {Long} q  message {String} q  logger {String} q  ip {String} q  instance {String} q  stacktrace {String} 15  

CEP  Query   from LogEvents [priority == "ERROR"] select message, stacktrace, serverName insert into ExceptionStream Email Body   Error Occurred in {{serverName}} – {{message}} {{stacktrace}} 16  

AnalyXcs  &  Batch  Processing     (BAM)   17  

Hive Query   CREATE EXTERNAL TABLE IF NOT EXISTS LogEventInfo (key STRING, tenantID INT,serverName STRING, appName STRING, priority STRING,logTime DOUBLE,logger STRING,message STRING) STORED BY 'org.apache.hadoop.hive.cassandra.CassandraStorageHandler' WITH SERDEPROPERTIES ( "cassandra.host" = "localhost", "cassandra.port" = "9160","cassandra.ks.name" = "EVENT_KS", "cassandra.ks.username" = "admin","cassandra.ks.password" = "admin", "cassandra.cf.name" = "log_0_AS_2014_01_23", "cassandra.columns.mapping" = ":key,payload_tenantID,payload_serverName,payload_appName, payload_priority,payload_logTime,payload_logger,payload_message" ); CREATE EXTERNAL TABLE IF NOT EXISTS Logs(tenantID INT,serverName STRING, appName STRING, priority STRING,logTime DOUBLE,logger STRING,message STRING) STORED BY 'org.wso2.carbon.hadoop.hive.jdbc.storage.JDBCStorageHandler' TBLPROPERTIES ( 'mapred.jdbc.driver.class' = 'com.mysql.jdbc.Driver', 'mapred.jdbc.url' = 'jdbc:mysql://localhost:3306/MYBAMDB', 'mapred.jdbc.username' = 'root','mapred.jdbc.password' = 'root', 'hive.jdbc.update.on.duplicate' = 'true', 'hive.jdbc.table.create.query' = 'CREATE TABLE LogEvent(tenantID INT,serverName VARCHAR(200), appName VARCHAR(200), priority VARCHAR(200),logTime DOUBLE,logger VARCHAR(800),message VARCHAR(3800))'); insert overwrite table Logs select tenantID, serverName, appName, priority, logTime, logger, message from LogEventInfo; select tenantID, serverName, appName, priority, logTime, logger, message from LogEventInfo; 18  

VisualizaXon -­‐  Gadgets   19  

Demo  …..   http://wso2.com/library/demonstrations/2014/02/screencast-monitoring-system-logs-with-wso2-business-activity-monitor/ 20  

Use  Case  Scenario  2:   HTTP  Log  Monitoring  With  WSO2  BAM   21  

HTTPD  Logs  and  Use  Cases     •  Monitor every web request information –  HTTP method –  URI –  Status code •  Monitor request properties –  Message size –  Host / IP address –  Geo location –  Date and time 22  

HTTPD Logs:  Scenarios   •  Publish HTTP logs to BAM from a data agent •  Split log entry for each fields and store in another big data column family •  Resolve IP addresses to geo locations •  Aggregate request for geo locations and store into a relational database (RDB) •  Visualize from gadgets 23  

SoluXon  Architecture     24  

Key  Performance  Indicator  (KPI)    Use  Cases   •  Analysing request count variation for each host •  Analysing request count variation for each hour •  Analysing request message size variation 25  

Final  Dashboard   26  

Demo  …..   http://wso2.com/library/demonstrations/2014/02/screencast-http-log-monitoring-with-wso2-business-activity-monitor/ 27  

Business  Model   28  

Call  to  acXon  page   ๏  ๏  Big  Data  Webinar  Series:   hhp://wso2.com/landing/wso2-­‐bigbrother-­‐webinar-­‐series/     ๏  WSO2  Business  AcXvity  Monitor:   hhp://wso2.com/products/business-­‐acXvity-­‐monitor/     ๏  WSO2  BAM  Docs:   hhp://docs.wso2.org/display/BAM240/WSO2+Business+AcXvity+Monitor +DocumentaXon     ๏  WSO2  Complex  Event  Processor:   hhp://wso2.com/products/complex-­‐event-­‐processor/     ๏  29   Be  your  own  NSA:  hhp://wso2.com/landing/nsa/     WSO2  CEP  Docs:   hhp://docs.wso2.org/display/CEP300/WSO2+Complex+Event+Processor +DocumentaXon    

Contact  us  !  

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

NSA for Enterprises Log Analysis Use Cases - Technology

1. Big Brother for Enterprises: Log Analysis Use Cases Samisa Abeysinghe VP Developer Evangelism 19 Feb 2014
Read more

NSA TOR BROWSER ANALYSIS - Education - documents.mx

NSA TOR BROWSER ANALYSIS Jun 09, 2015 Education ... NSA for Enterprises Log Analysis Use Cases. ToR. TOR. Tor. Tor. Tor. Login or Join. Processing Login ...
Read more

SonicWALL Analyzer Network Security Software | Dell SonicWALL

Easy-to-use, web-based network traffic analysis and reporting tool offering real ... NSA Series Enterprise-class ... with SonicWALL™ Analyzer network ...
Read more

Use Case-Based Requirements

use cases are then valuable during the next steps in your project development ... analysis, or design documents ... Use Case-based Requirements . . . Use ...
Read more

Top 47 Log Management Tools | ProfitBricks Blog

... The following 47 log management tools ... Supports various use cases including log ... McAfee Enterprise Log Manager automates log ...
Read more

Chapter 7 OCUMENTING U SE C ASES - United States

Read more


National Security Agency ... Admiral Michael S. Rogers introduces NSA's dual mission and the employees that ... Terms of Use; Accessibility;
Read more