Published on January 16, 2014
Business Intelligence Overview IT 4713/6713 BI J.G. Zheng Spring 2014
Overview What is business intelligence (BI)? BI process, system components, and technologies BI industry and career 2
Introduction Types of Information Processing Transactional Processing Focus on routine processing: data insertion, modification, deletion, and transmission Analytical Processing Focus on reporting, analysis, transformation, and decision support 3
DIKW Data: raw value elements or facts Information: the result of collecting and organizing data that provides context and meaning Knowledge: the concept of understanding information that provides insight to information, thus useful and actionable http://en.wikipedia.org/wiki/DIKW Analytical Processing Transaction Processing 4
What is Business Intelligence? Business Intelligence is a set of methods, processes, architectures, applications, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decisionmaking (for business operations and growth). - Forrester 5
Business Intelligence (BI) BI is the an umbrella term for a set of methods, processes, applications, and technologies used to gather, provide access to, and analyze data and information support decision making Traditionally it has been also understood as Decision Support System (DSS) – known as data driven DSS. A brief history of DSS: http://dssresources.com/history/dsshistory.html Narrowly speaking, intelligence comes from data (facts). In this sense, BI focuses on analytical processing. Broadly speaking, intelligence, or knowledge, also comes from human experience and tacit knowledge. In this sense, BI is also related to knowledge management. 6
Analytical Processing What is the reason for a decrease of total sales this year? (reasoning) How do advertising activities affect sales of different products bought by different type of customers, in different regions? (synthesizing) Should we invest more on our e-business? (fuzzy question need high level analysis for decision making) 7
Why BI System is Needed Data everywhere Data in separate systems and different sources Problem of spreadmart http://en.wikipedia.org/wiki/Sprea dmart Information overloading too much data and information difficulty of data organization for effective access and retrieval difficult to find useful information (knowledge) from them Data or facts based decision making A gap between data and knowledge (useful information leading to a decision). 8
BI Capabilities Figure from: Business Intelligence, Rajiv Sabherwal, Irma Becerra-Fernandez, John Wiley & Sons, 2011 http://books.google.com/books?id=T-JvPdEcm0oC 9
BI: A General Process Information -> knowledge: The process involves analytical components, such as OLAP, data quality, data profiling, business rule analysis, and data mining Data -> information: The process of determining what data is to be collected and managed and in what context 10
BI System at a Glance Data Integration and Management • • • • • Operational data Data warehouse Data modeling Data governance Data integration Presentation • • • • • Query OLAP Business analytics Data mining Visual analytics Analysis • Reports • Information visualization • Dashboard • Scorecards • Strategy map Applications • • • • • • • • • • • Performance management Benchmarking Market research CRM Strategic management Web site analytics Website Reporting server Application server Portal Excel services Management and Delivery 11
Data Management A special database system called data warehouse or data mart is often used to store enterprise data The purpose of a data warehouse is to organize lots of stable data for ease of analysis and retrieval. Traditional (operational) databases facilitate data management and transaction processing. They have two limitations for data analysis and decision support Performance They are transaction oriented (data insert, update, move, etc.) Not optimized for complex data analysis Usually do not hold historical data Heterogeneity Individual databases usually manage data in very different ways, even in the same organization (not to mention external data sources which may be dramatically different). 12
Data Integration Enterprise level data are coming from multiple different sources, but need to be combined and associated. The need to bring together different data/information Autonomous Distributed Different General processing - ETL Extraction: accessing and extracting the data from the source systems, including database, flat files, spreadsheets, etc. Transformation: data cleanse, change the extracted data to a format and structure that conform to the destination data. Loading: load the data to the destination database, and check for data integrity 13
Analysis Tools Operational reporting Structured and fixed format reports Based on simple and direct queries Usually involves simple descriptive analysis and transformation of data, such as calculating, sorting, filtering, grouping, and formatting OLAP (Online Analytical Processing) A multi-dimensional analysis and reporting application for aggregated data Great for discovering details from large quantities of data Business analytics Business analytics (BA) is the practice of iterative, methodical exploration of an organization’s data with emphasis on statistical analysis. Data mining Data mining techniques are a blend of statistics and mathematics, and artificial intelligence and machine-learning. 14
OLAP Multi-dimensional queries A dimension is a particular way (or an attribute) of describing and categorizing data Such queries are usually arithmetic aggregation operations (sum, average, etc.) on records grouped by multiple dimensions (attributes) at different aggregation levels. OLAP is a function/operation that is optimized to answer queries that are multi-dimensional in nature Example analysis Descriptive and operational report "What is the total sales amount grouped by product line (dimension 1), location (dimension 2), time (dimension 3) and … (other dimensions)?" "Which segment of business provides the most revenue growth?" More open and exploratory analysis 15
OLAP vs. Transactional Report This is the transactional data report with line by line data. A pivot table or crosstab is usually used for OLAP result view (aggregated data) 16
Data Mining Data mining (or, knowledge discovery in database, KDD) Processes and techniques for seeking knowledge (relationship, trends, patterns, etc.) from a large amount of data Non-trivial, non-obvious, and implicit knowledge Extremely large datasets Data mining applications use sophisticated statistical and mathematical techniques to find patterns and relationships among data Classification, clustering, association, estimation, prediction, trending, pattern, etc. 17
Presentation Reports Presenting data and results with limited interactivity Based on simple and direct queries: usually involves simple analysis and transformation of data (sorting, calculating, filtering, filtering, grouping, formatting, etc.) Visualization An essential way for human understanding and sense making In the forms of table, charts, diagrams Visualization can also be part of the analysis process (visual analytics) Dashboard Digital dashboard is a visual and interactive presentation of data to make it easy to read and understand in a short time Why Quickly understand data and respond quickly Ability to identify trends Save time over running multiple reports Gain total visibility of all systems instantly at one place 18
Reporting and Delivery BI reporting is about managing and delivering analysis results to users Live example: https://discoverer.gsu.edu Live example: https://www.itdashboard.gov/ 19
BI Applications Business management Performance management Strategy management Benchmarking Marketing and sales CRM Customer behavior analysis Targeted marketing and sales strategies Customer profiling Collaborative filtering Financial management Logistics Supplier and vendor management Shipping and inventory control Web site management Web analytics Project management Security management Healthcare management Traffic management 20
BI Market In General 17% growth in 2011. 8.7% growth in 2012. ($34.9 billion). Predict growth of 9.7% annually to 2017. Company 2012 Revenue SAP 2,902.5 Oracle 1,952.1 IBM 1,625.6 SAS 1,599.7 Microsoft 1,189.3 Others 3,861.90 Total 13,131.1 2012 Market Share (%) 2011 Revenue 22.1 2,884.0 14.9 1,913.5 12.4 1,478.8 12.2 1,542.9 9.1 1,059.9 29.3 3,416.00 100.0 12,295.1 2011-2012 Growth (%) 0.6 2.0 9.9 3.7 12.2 13.0 6.8
Gartner Magic Quadrant The big four mega-vendors IBM Cognos Microsoft SQL Server Oracle OBIEE (purchased Hyperion) SAP Business Objects http://www.gartner.com/technology/reprints.do?id=1-1DZLPEP&ct=130207 22
BI Careers Typical BI positions BI solution architects and integration specialists Business and BI analysts BI application developers and testers Data warehouse specialists Database analysts, developers and testers Database support specialists BI jobs on Dice.com http://seeker.dice.com/jobsearch/servlet/JobSearch?o p=300&LOCATION_OPTION=2&RADIUS=80.4672&CO UNTRY=1525&DAYSBACK=14&NUM_PER_PAGE=30& N=1525+1811&NUM_PER_PAGE=30&FREE_TEXT=% 22business+intelligence%22
What are employers looking for? Data from BI Congress Survey 2010 24
Critical Knowledge and Skills Technical knowledge Knowledge of database systems and data warehousing technologies Ability to manage database system integration, implementation and testing Ability to manage relational databases and create complex reports Knowledge and ability to implement data and information policies, security requirements, and state and federal regulations Solution development and management Working with business and user requirements Capturing and documenting the business requirements for BI solution Translating business requirements into technical requirements BI project lifecycle and management Business and Customer Skills and Knowledge Effective communication and consultation with business users Understanding of the flow of information throughout the organization Ability to effectively communicate with and get support from technology and business specialists Ability to understand the use of data and information in each organizational units Ability to train business users in information management and interpretation
Sample Role: Business Intelligence Specialist Technical skill requirements Maintain or update business intelligence tools, databases, dashboards, systems, or methods. Provide technical support for existing reports, dashboards, or other tools. Create business intelligence tools or systems, including design of related databases, spreadsheets, or outputs. 26
Sample Role: Business Intelligence Developer Business Intelligence Developer is responsible for designing and developing Business Intelligence solutions for the enterprise. Key functions include designing, developing, testing, debugging, and documenting extract, transform, load (ETL) data processes and data analysis reporting for enterprise-wide data warehouse implementations. Responsibilities include: working closely with business and technical teams to understand, document, design and code ETL processes; working closely with business teams to understand, document and design and code data analysis and reporting needs; translating source mapping documents and reporting requirements into dimensional data models; designing, developing, testing, optimizing and deploying server integration packages and stored procedures to perform all ETL related functions; develop data cubes, reports, data extracts, dashboards or scorecards based on business requirements. The Business Intelligence Report Developer is responsible for developing, deploying and supporting reports, report applications, data warehouses and business intelligence systems.
Sample Role: Business Intelligence Analyst Technical skill requirements Works with business users to obtain data requirements for new analytic applications, design conceptual and logical models for the data warehouse and/or data mart. Develops processes for capturing and maintaining metadata from all data warehousing components. Business skills requirements Generate standard or custom reports summarizing business, financial, or economic data for review by executives, managers, clients, and other stakeholders. Analyze competitive market strategies through analysis of related product, market, or share trends. Collect business intelligence data from available industry reports, public information, field reports, or purchased sources. Maintain library of model documents, templates, or other reusable knowledge assets. 28
Recent SPSU IT Graduates in BI http://www.linkedin.com/pub/tylerclark/3b/2a8/398 http://www.linkedin.com/pub/karen-ppurcell/9/678/843 http://www.linkedin.com/pub/russhorton/64/3ba/462 http://www.linkedin.com/pub/amirshahrokhi/11/a76/1b6 http://www.linkedin.com/in/tanvirspsu 29
The Future Need and Opportunity for BI Data from BI Congress III Survey 2013 30
Where Can You Learn BI in Atlanta? GATech: no major courses known Emory Business: 2-3 courses focused on analytics (using spreadsheets) CS: courses focused on database and data mining GSU Management science: elective general BI and data mining courses CS: focused on data mining and artificial intelligence SPSU The IT department offers introductory courses on business intelligence at both graduate and undergraduate level. Mercer: none KSU: 2 electives on data warehouse and data mining Disclaimer: based on personal online search as of 2012. Please contact the instructor if there are more recent updates. 31
Good Resources History of BI (casual video with good visuals) http://www.youtube.com/watch?v=_1y5jBESLPE Wikipedia http://en.wikipedia.org/wiki/Business_intelligence ACM techpack http://techpack.acm.org/bi/ Business intelligence resources http://www.businessintelligence.com/ The Data Warehousing Institute http://tdwi.org DSS Resources http://dssresources.com/ 32
Business Intelligence ermöglicht Ihnen eine kontinuierliche Verbesserung Ihrer Unternehmensperformance. Softwaregestütze Lösungen integrieren Ihre ...
This video explains What is Business Intelligence
Business Intelligence Tools reviews, comparisons, alternatives and pricing. The best BI solutions for small business to enterprises.
Business intelligence (BI) software is a collection of decision support technologies for the enterprise aimed at enabling knowledge workers such as ...
Business Intelligence: Overview This documentation is geared to beginners who would like a quick introduction to the functions offered by SAP NetWeaver ...
Business Intelligence Overview What is Business Intelligence? Business Intelligence at first glance is a broad array of applications or technologies ...
Microsoft has several business intelligence (BI) tools and applications with BI features, each of which is important to understand as you decide what will ...
Why Read This Report. Business intelligence (BI) is a set of methodologies, processes, architectures, and technologies that transform raw data into ...
This course introduces you to some of the Business Integration tools available in SQL Server, including SSIS, SSRS, and SSAS.
SAP Business Intelligence (BI) Lösungen: Schneller Zugriff auf Analyse Tools, Berichte, Datenvisualisierung und mehr. Für fundierte Entscheidungen...