Published on September 29, 2015
1. NLP + Brandwatch Analytics Deriving insights from social conversations using Natural Language Processing and the Brandwatch Analytics API
2. What we will cover today What do we want to answer? (and why) Our approach to social data Leveraging the Brandwatch API to extract data Deriving insight from personas Identifying key topics of conversation Segmenting on those topics to develop personas Replicating back into Brandwatch What we’re working on next
3. 3 | § Provides an in-‐situ portrait based on exhibited behavior not on elicited feedback § Highly relevant as it can be updated in near-‐real time § Enables research budget to be focused on insights rather than data collection Social intelligence enables new ways of answering traditional business questions and driving data driven actions What are the sort of questions we want to answer? How can a financial services company reach out to cyclists? How can we get small business owners to engage with their cell phone provider online? What is the customer journey for a motorcycle enthusiast?
4. 4 | There are seven stages to the analytical process of developing utilizing personas with social data Our approach to working with social data Extract Develop the dataset Linguistic model Segment Analyze TrackQuery data Prepare § Need to truly understand your data before any analysis § Iterative query/dataset development through virtual ethnography § Use the Brandwatch API to extract the full text mentions Model § Employ Natural Language processing to model how people talk § Use either qualitative methods or clustering algorithms to segment Understand § Through visualization and analysis we can understand thoughts, feelings and preferences § Replicate back into Brandwatch as sub-‐categories to monitor on an ongoing basis
5. 5 | • Provide the basis for a ‘corpus’ in NLP jargon from which to model • We have built a library of functions using python to retrieve and format the data • The output format of the API is in JSON so there is some work to turn it into a table we can read and use Extracted BW data has many use cases, today we will be primarily focused on full text mentions Leveraging the API to extract the dataset Example API function: def get_mentions_query_URL( startdate,enddate,project_id, query_id,access_token,fullText): query_def = "data/mentions” end_date = "endDate=" + end_date + "T00:00:00.000Z” start_date = "startDate=" + start_date + "T00:00:00.000Z" request_URL="https://newapi.brandwatch.com/projects/" +str(project_id) + "/" +query_def if fullText == True: request_URL = request_URL + "/fulltext" request_URL = request_URL + "?" + "queryId=" + str(query_id) + "&" + start_date + "&" + end_date + "&pageSize=5000" + "&access_token=" + access_token return request_URL Read more: blog.tahzoo.com/tech-‐thursday-‐brandy-‐py-‐a-‐python-‐library-‐for-‐brandwatch/ Github: https://github.com/BillmanH/brandy.py/
6. 6 | Linguistic model -‐ Identifying the topics in a conversation pumpkin sugar HEALTHY LIVING PUMPKIN SPICE CONVERSATIONS TEXT ANALYSIS TOPIC MODEL 1 Break down each conversation into the words and sentences to probabilistically assess each word’s relationship with each other word 2 Analyze to uncover the most common “topics” of conversation 3 Run clustering analysis to segment on topics 4 Iterate on topics until we develop a solid segmentation Four steps to targeting personas
7. 7 | ("pumpkin spice latte") NOT ("vue pack" OR "value pack" OR "how to make" OR "win free" OR "latte cake" OR "black friday" OR "pack of" OR "My TL right now iOS7 Hump Day iOS7" OR site:(twitter.com OR kdvr.com OR fox59.com OR news.google.com)) An example: who discusses Pumpkin Spice Lattes? Our query… Excluded because of irrelevant recipes Purposefully broad query to capture full range of conversations Exclude Twitter as it would overwhelm the results and we couldn’t export full text mentions
8. 8 | Do it Yourself Starbucks Nutrition Healthy living Style Urban living restaurants PS recipes Amazing treat Pumpkin Spice ingredients PS Flavor Coffee at home people pumpkin grams squash fall city binary milk love food pumpkin home make spice fat healthy wear place victoire pumpkin time hari spice inch things latte calories recipe fashion food restaurants coffee day babe pie coffee life starbucks sugar food boots park options sugar good ingredients latte green time fall registers recipes style street time recipe back sugar flavor set thing psl data copycat color local pst spice week science flavored keurig feel drink saturated favorite wearing free visit cup great cancer seasonal count find coffee carbs soup dress art trading make home organic year price years today sodium paleo black event restaurant cream made found taste mountain world lattes pos version top restaurant september syrup work chemical food make The topics
9. 9 | DIY Example: “I get annoyed when a recipe calls for pumpkin pie spice. It's not that people use it that annoys me, it's the mere existence of it as a single spice. … I guess I'm just a purist at heart. Since I haven't seen pumpkin pie spice here in France I now need to make our own pumpkin pie spice mixture, and then figure out the right proportions for my dreamboat pumpkin spice latte. Nothing that a Google search won't solve, but annoying nonetheless. And don't worry, when I do I'll be sure to share it with you. Maybe you'll even get some rainbows. Fingers crossed. An example of how this analysis works Treat/Reward Routine Example: “I thought splurging on a venti pumpkin spice latte would make me feel better this morning, (or maybe even the three cups of green tea with lotsa honey in it!) ...but as my ears pop, my nose runs, and my throat feels like somebody took sandpaper to it last nite, I guess it's time to finally suck it up & take some meds ó¾Œ®ó¾ ‚ I blame you! Rodney Deal!! Haha kidding kidding ; ) Below are two pieces of verbatim content that we used in our model. The first post is connected with the DIY (62% relevant) topic and the second with Treat/Reward (73% relevant) 62% 73% DIY TREAT / REWARD PS FLAVOR FALL (SEASON) PSL RECIPES HEALTHY LIVING FILLER/ INFREQUENT WORDS TOPICS:
10. 10 | • K-‐means highlights clusters of conversations based on the topics they discuss • This creates a segmentation that reflects how people discuss a subject • Keys in on the pattern of topics in a conversation We use the k means clustering algorithm to segment the conversations based on the topics in order to create the personas Segmenting on the topics
11. 11 | Urban Living Fall (season) Dessert Starbucks drinks Pumpkin flavor Treat / reward Pumpkin (recipes) DIY Fall (season) Treat / reward Pumpkin (flavor) Dessert StyleDesserts Treat / Reward Being Healthy Urban Living Fall (Season) Starbucks Drinks Fall (season) DIY Desserts Pumpkin spice recipes Treat/ Reward LESS IMPORTANT MORE IMPORTANT Grouping the topics that are core to each segment we can see where differences break down Mapping topics to personas
12. 12 | Plotting continuums to understand the personas Why they like it What it stands for NOVELTYNOSTALGIA GUILTY PLEASURE DAILY RITUAL OPPORTUNISTICTRADITIONAL PERENNIALSEASONAL OFTEN OCCASIONAL EXPECTED EARNED
13. 13 | What we found 22% PSL PAMPERER A pumpkin spice latte is a treat to be savored after it’s earned or after a tough a Monday morning, “What a weekend. Hello, slow Monday. Oh what's that? I should get a pumpkin spice latte? Well, if you insist...” 34% LATTE CHEMIST They make their own lattes in the comfort of their own home or tinker with the official version “Here is an awesome home version of Starbucks Pumpkin Spice Latte. Very simple to make and alot cheaper… personally I like it better because you control the amounts of ingredients you put in it according to your taste.” 38% FALL FANATIC Pumpkin spice is part of what makes fall special for them, a pumpkin spice latte is one part of their fall tradition “Pumpkin Spice Latte at Panera. Oh yeah, I need one of those! Bring on fall! Looking forward to bonfires in my fire pit and my newly refinished fireplace.” 6% PUMPKIN TRADITIONALIST Loves everything pumpkin from pumpkin pie to lattes, fall is just an excuse to get their fix of pumpkin “Are you ready for a Pumpkin Spice Latte!?!?! Or how about a Pumpkin Bar???? Well tomorrow they both will be available!!!!”
14. 14 | Replicating back into Brandwatch PSL PAMPERER “morning treat” OR “Saved my morning” OR ((rough OR bad OR terrible* OR awful OR stressful)) NEAR/4 (morning OR day OR week)) LATTE CHEMIST ((my OR I OR Mine OR “made a”) NEAR/2f (organic OR make OR recipe OR mixture)) OR homemade OR “the perfect” OR ((coffee) NEAR/3 (dessert OR “sweet tooth”) FALL FANATIC (I OR MY) NEAR/3 (“love fall” OR “finally here” OR “the season” OR autumn) OR ((making OR made OR bake OR baked) NEAR/4f (cake OR pie OR pastry)) PUMPKIN TRADITIONALIST ((pumpkin) AND (candle OR products OR cake OR pie)) OR “pumpkin flavor” OR ((“I need a” OR “must have” OR “must get”) NEAR/3f (latte)) We conduct a careful qualitative analysis of persona mentions to translate the topic model into Brandwatch rules • Allows us to visualize and track in Brandwatch • Create each persona as a sub-‐category • Creating the persona rules are iteratively written Hypothetical rules
15. 15 | Custom geo-‐mapping for DMA’s Persona use cases Typing tools Scoring conversation relevance IDENTIFYING TARGET SEGMENTS Commentator DIY PS Flavor Fall (Season) Treat / Reward Focused on others Traditional
16. 16 Next level – what we’re working on now § Ability to use the model to tag incoming mentions in Brandwatch § Determining demographic characteristics from language § Utilizing topics to predict outcomes
17. 17 | Introduction to Tahzoo
18. 18 | What we do: data driven customer experiences Business Oriented From business process to change management, Tahzoo helps enterprises become organizationally ready for transformation Customer Data Centered Understanding customers from their own point of view is the foundation for successful transformation, Tahzoo compiles, curates and analyzes all the data, outside/in and inside out Digital Content Inspired Customers have transformed the way they engage with companies using every channel, device and platform available, Tahzoo addresses engagement through content marketing Technology Enabled Architecting, implementing and integrating the right technical solutions determines the transformative nature of the experience, this is Tahzoo’s core expertise and experience Frictionlessly Delivered Both the customer and the business win when the experience is delivered as efficiently and effectively as possible, whether that’s delivered in-‐house, as a managed service or in the cloud as an integrated technology and marketing operations services model
19. 19 | • Founded in 2010 • Privately Held • 300+ employees • Primary Verticals • Financial Services • Retail • Automotive/Manufacturing • Business Services A fast growing CX agency focusing on digital transformation across a number of verticals Company overview • Washington DC -‐ HQ • Seattle, WA • San Francisco, CA • Richmond, VA • Milton Keynes, UK • Delft, NL • Borlange, SE • Novi Sad, RS Locations
20. 20 | Past & current clients
21. 21 | Technology partners
22. Thank you Bill Harding – Data Scientist firstname.lastname@example.org Colin Rogers – Direction of Content Strategy email@example.com
Natural language processing ... Using almost no information ... An explicit formalization of natural languages semantics without confusions with ...
... and natural language processing to extract data from ... are using Mattermark to save time ... Products. Mattermark Pro; Mattermark API; Mattermark ...
... San Francisco. ... MIT Technology Review EmTech Digital (formerly Digital Summit) ... Choose from six languages and in 13 regions worldwide.
NUVI’s highly visual social media monitoring platform, combined with DataSift’s expertise in processing aggregate and anonymized Facebook topic data ...
Teradata is a leading provider of powerful, ... Data Driven Marketing, ... Corporate Social Responsibility;
Now headquartered in San Francisco, ... the billions of online conversations and ... that uses natural language processing technology to ...
Make informed customer experience management ... Clarabridge built the industry-leading natural language processing and ... Sentiment Analysis; Social ...
Linguistic analysis ... Airbnb’s Verified ID process uses Rosette Name Indexer to match names in multiple languages, ... Basis Technology provides ...