Building Great Products the Lean Startup Way

33 %
67 %
Information about Building Great Products the Lean Startup Way
Business & Mgmt

Published on November 15, 2013

Author: aipmm



Many companies have already embraced Lean Startup principles. It's hard to argue with learning from customer feedback and iterating to achieve product-market fit. But how exactly are you supposed to do that?

This discussion covers the following topics:
* Understanding and prioritizing customer needs
* Determining your product's value proposition
* Principles of good User Experience (UX) design
* Validating your product with user testing
* Product analytics

About the Speaker
Dan Olsen is an entrepreneur and product expert. He started his high-tech career managing submarine design and then worked at Intuit where he led product management for Quicken. Dan also led product management at Friendster. He was CEO and cofounder of YourVersion, a personalized news discovery engine startup that won the People's Choice Award at TechCrunch50. Dan currently consults to companies, usually as interim VP of Product for startups. His former clients include Facebook, Box, Hightail (YouSendIt), Chartboost, Epocrates, and One Medical Group. Dan holds a BS in Electrical Engineering from Northwestern, a Master's in Industrial Engineering from Virginia Tech, and an MBA from Stanford.

Upcoming Events
Reserve your seat for the next AIPMM webinar. Visit:

Want To Certify Your Team?
If you have a product team of 10 or more that you want to certify, visit:

The AIPMM is the hub of all things product management. It is where product professionals go for answers. With members in over 65 countries, it is the worldwide certifying body of product team professionals.

It is the world's largest professional organization of product managers, brand managers, product marketing managers and other product team professionals who are responsible for guiding their organizations, or clients, through a constantly changing business landscape.

AIPMM's certification programs are internationally recognized because they allow product professionals to demonstrate their expertise and provide corporate members an assurance that their product management and marketing teams are operating at a high competency level.


Call For Speakers:

AIPMM  Webinar  Series     ©  AIPMM  2013  

©  AIPMM  2013  

The Hub Of All Things Product Management -  Founded  1998,  AIPMM  has  cer5fied  product  team   professionals  in  nearly  70  countries   -  World’s  largest  professional  associa5on  for  product   management,  product  marke5ng,  and  brand  management   -  Provides  globally  recognized  cer5fica5ons:   •  Cer5fied  Product  Manager   •  Cer5fied  Product  Marke5ng  Manager   •  Agile  Cer5fied  Product  Manager   •  Cer5fied  Innova5on  Leader   •  Cer5fied  Brand  Manager   ©  AIPMM  2013  

Tweet!   Use AIPMM: Product Ninja: #AIPMM #ProductNinja Follow Dan Olsen Hector Del Castillo: AIPMM: @DanOlsen @hmdelcastillo @AIPMM ©  AIPMM  2013  

Par=cipate  and  Win!   • Win  a  copy  of  the  Guide  to  the  Product   Management  and  Marke5ng  Body  of   Knowledge  (ProdBOK®)     ©  AIPMM  2013  

Today’s  Speaker   Moderator:     Hector  Del  Cas5llo,  CPM,  CPMM,  PMP   Presenter:           Dan  Olsen   Product  Consultant  &  Interim  VP  of  Product   Olsen  Solu5ons   @DanOlsen   hXp://   ©  AIPMM  2013  

+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+ Building Great Products the Lean Startup Way (aka How to be a Lean Product Ninja) Dan Olsen Olsen Solutions November 15, 2013 +*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+*+

What  does  “Lean”  mean?   n  Lean  Startup   n  Achieving  product-­‐market  fit   n  Valida5ng  product  with  users   n  Improving  &  itera5ng  your   product  quickly   n  Tes5ng  hypotheses  &  learning   n  Minimizing  waste  of  resources   n  Product-­‐market  fit  =  product   management’s  responsibility     Audience   Poll Copyright  ©  2011  YourVersion  

Why  “Ninja”?   n  High  level  of  exper5se   n  Self-­‐sufficient   n  Uses  wide  array  of  skills   to  get  the  job  done Copyright  ©  2011  YourVersion  

What’s  the  Formula  for   Product-­‐Market  Fit?   n  A  product  that:   n  Meets  customers’  needs   n  Is  beXer  than  other  alterna5ves   n  Is  easy  to  use   n  Has  a  good  value/price   Focus of my talk today Will cover basics. See my talk “How to be a UX Design Army of One” Copyright  ©  2013  Olsen  Solu5ons  

Dan’s  Model  for  the  Causality   Underlying  Product-­‐Market  Fit   Customer   has  needs   Target   Customer   Customer   Needs   Product  is   designed  to   meet  needs   Customer   decides  how   well  product   meets  needs   (sa5sfac5on)   Product   Copyright  ©  2013  Olsen  Solu5ons  

The  Lean  Product  Ninja  Way:   Steps  to  Achieving  Product-­‐Market  Fit   1.  2.  3.  4.  5.  Understand  underserved  customer  needs   Define  your  product’s  value  proposi5on   Build  wireframe/mockup   Get  user  feedback   Iterate   Copyright  ©  2013  Olsen  Solu5ons  

Understanding   Customer   Needs  

  Problem  Space  vs.  Solu5on  Space     n  Problem  Space   n  A  customer  problem,   need,  or  benefit  that  the     product  should  address   n  A  product  requirement     n  Solu5on  Space   n  A  specific   implementa5on  to   address  the  need  or   product  requirement     Example:   n  Ability  to  write  in  space     NASA:  space  pen   n  (zero  gravity)   ($1     M  R&D  cost)   n  Russians:  pencil   Copyright  ©  2013  Olsen  Solu5ons  

Problem  Space  vs.  Solu5on  Space:   Product  Level   Problem  Space   (user  benefit)   Solu5on  Space   (product)   Prepare my taxes Pen and paper Check my taxes File my taxes TurboTax Maximize deductions Reduce audit risk TaxCut Copyright  ©  2013  Olsen  Solu5ons  

Problem  vs.  Solu5on  Space:  Feature  Level   Problem  Space     Help me Solu5on  Space   prepare taxes Save Money Audit Risk Analyzer Tax Return Error Checker Save time preparing taxes Save Time Reduce my audit risk Check my return Empowerment/ Confidence Tax Interview Wizard Tax Data Downloader Save time filing taxes Electronic Tax Return Filing Maximize my tax deductions Tax Deduction Finder Copyright  ©  2013  Olsen  Solu5ons  

Importance  of  User  Need   Priori5zing  Customer  Needs:   Importance  vs.  Sa5sfac5on   High   Opportunity   Compe55ve Market   Not  Worth  Going  Aper   Low   Low   High   User  Sa5sfac5on  with  Current  Alterna5ves   Copyright  ©  2013  Olsen  Solu5ons  

Kano  Model:  User  Needs  &  Sa5sfac5on   User  Sa5sfac5on   Delighter  (wow)   Performance   (more  is  beXer)   Need   not  met   Need   fully  met   Must  Have   Needs  &  features   migrate  over  5me   User  Dissa5sfac5on   Copyright  ©  2013  Olsen  Solu5ons  

Importance  vs.  Sa5sfac5on    Ask  Users  to  Rate  for  Each  Feature   100 Great   95 84 87 90 Bad   85 Importance 98 70 55 80 79 75 80 72 70 86 84 80 75 65 60 55 41 50 40 50 60 70 80 90 100 Satisfaction Recommended  reading:  “What   Customers  Want”  by  Anthony  Ulwick   Copyright  ©  2013  Olsen  Solu5ons  

What  is  Your  Value  Proposi5on?   n  Which  user  benefits  are  you  providing?   n  How  are  you  beXer  than  compe5tors?     Compe=tor  A   Compe=tor  B   Must  Have  Benefit  1   You   Y   Y   Y   Performance  Benefit  1   High   Low   Med   Performance  Benefit  2   Low   High   Low   Performance  Benefit  3   Med   Med   High   Delighter  Benefit  1   Y   -­‐   -­‐   Delighter  Benefit  2   -­‐   -­‐   Y   Copyright  ©  2013  Olsen  Solu5ons  

Building   Wireframes/   Mockups  

Geqng  Customer  Feedback:   Problem  Space  vs.  Solu5on  Space   n  Customers  CAN’T  ar5culate  problem  space   n  Customers  CAN  react  to  solu5on  space   n  That’s  why  you  need  something  tangible  to   show  to  customers  to  get  good  feedback   Copyright  ©  2013  Olsen  Solu5ons  

Interactivity Design  Ar5facts:   Interac5vity  vs.  Fidelity   Interactive Prototype Clickable Wireframe* Hand sketch Clickable Mockup** Static Wireframe* Mockup Fidelity *  Balsamiq:   **  Invision:   Copyright  ©  2013  Olsen  Solu5ons  

Design  Tools:   Fidelity  vs.  Effort   Design Fidelity 3 Photoshop All Product People 2 1 Balsamiq Visual Designer Hand sketch Effort to Create Artifact Copyright  ©  2013  Olsen  Solu5ons  

Why  You  Should  Wireframe   n  In  case  you  don’t  have  UX  designer  (open)   n  To  clarify  and  refine  your  thinking:   n  Informa5on  Architecture,  layout,  naviga5on   n  To  communicate  your  product  ideas  to   others   n  Modern  tools  make  it  easy  and  fast   Audience Poll Copyright  ©  2013  Olsen  Solu5ons  

Learning  from   Customers   Copyright  ©  2013  Olsen  Solu5ons  

What  Are  You  Going  to  Get  Feedback  &   Learnings  About?   Problem  Space   (your  mental  model)   Solu5on  Space   (what  users  can  react  to)   Feature  Set   Customer   Understanding   (needs  &   preferences)   UI  Design   Messaging     Copyright  ©  2013  Olsen  Solu5ons  

“Ramen”  User  Feedback  for  Startups   n  Anyone  can  do  it!   n  Ingredients:   n  Solu5on-­‐space  product/mockup  to  test   n  1  customer  (with  laptop  if  tes5ng  code)   n  1  desk   n  1  person  to  conduct  the  session   n  Pen  and  paper   n  Op5onal  note-­‐taker  and  observers   Copyright  ©  2013  Olsen  Solu5ons  

Typical  Format  for  Customer  Session   5  -­‐  10  min:  Ask  ques5ons  to  understand  user   needs  and  solu5ons  they  currently  use   n  30  -­‐  50  min:  User  feedback   n  n  Show  user  product/mockup   n  Non-­‐directed  as  much  as  possible   n  When  necessary,  direct  user  to  aXempt  to   perform  a  specific  task   n  5  -­‐  10  min:  Wrap-­‐up   n  Answer  any  user  ques5ons  that  came  up   n  Point  out/explain  features  you  want  to  highlight   n  Ask  them  if  they  would  use  the  product   Copyright  ©  2012  Olsen  Solu5ons  

Dos  &  Don’ts  of  User  Feedback  Sessions   n  Do   n  Explain  to  the  user:   Their  feedback  will  help  improve  the  product   n  Not  to  worry  about  hur5ng  your  feelings   n  “Think  Aloud  Protocol”   n  n  n  n  n  Ask  user  to  aXempt  the  task,  then  be  a  fly  on  the  wall   Ask  non-­‐leading,  open-­‐ended  ques5ons   Take  notes  and  review  them  aperwards  for  take-­‐aways   Don’t   n  n  n  n  n  Ask  leading  ques5ons   “Help”  the  user  or  explain  the  UI  (e.g.,  “click  over  here”)   Respond  to  user  frustra5on  or  ques5ons  (un5l  test  is  over)   Get  defensive   Blame  the  user   Copyright  ©  2012  Olsen  Solu5ons  

Iterate  Your   Wireframes  Based   on  Feedback  

Itera5ng  Your  Product  Vector  Based  on   User  Feedback  in  Solu5on  Space   Problem  Space   (your  mental  model)   Help  user   book  travel   Solu5on  Space   (what  users  can  react  to)   Help  user   plan  travel   Mockups  /  Code    Customer  Feedback   Copyright  ©  2013  Olsen  Solu5ons  

Case  Study  on  Product-­‐Market  Fit:   Copyright  ©  2013  Olsen  Solu5ons  

Product-­‐Market  Fit  Case  Study:   n  My  consul5ng  client  (CEO)  had  an  idea   for  a  new  product   n  Team:  me,  CEO,  head  of  marke5ng,   UI  design  consultant   n  Goal:   n  Validate  product-­‐market  fit  quickly,  cheaply   without  wri5ng  a  single  line  of  code   n  Determine  if  there  was  a  business   opportunity  here   Copyright  ©  2013  Olsen  Solu5ons  

Product-­‐Market  Fit  Case  Study:   Developing  Product  Concept   Product  Concept  was  “marke5ng  report”  that   let  consumers  control  the  direct  mail  that  they   receive   n  Concept  was  fuzzy  with  various  components,   so  I  broke  it  into  2  different  “flavors”:   n  n  #1  “Marke5ng  Shield”:  Service  to  reduce/stop   junk  mail   n  #2  “Marke5ng  Saver”:  Opt  in  &  receive  money-­‐ saving  offers   Copyright  ©  2013  Olsen  Solu5ons  

Clustering  Poten5al  User  Benefits  to   Create  Product  Concepts   “Shield” Concept “Saver” Concept Reduce Junk Mail Find out what “they” know about you Save Trees Marketing Report Marketing Score Money Saving Offers Compare Yourself to Others Social Networking Marketing Profile Copyright  ©  2013  Olsen  Solu5ons  

Copyright  ©  2013  Olsen  Solu5ons  

Copyright  ©  2013  Olsen  Solu5ons  

Product-­‐Market  Fit  Case  Study:   Recrui5ng  People   n  Telephone  recruit  of  prospec5ve  customers   n  Wrote  phone-­‐screen  ques5onnaire  to  create  rough   target  customer  segmenta5on   n  Wanted  users  who  work  full-­‐5me  &  use  internet   n  Fit  for  opt-­‐in  offers:  use  coupons,  Costco  membership   n  Fit  for  an5-­‐junk  mail:  use  paper  shredder,  block  caller  ID   Paid  each  person  $75   n  Scheduled  3  groups  of  2  or  3  people  to  discuss   each  product  concept  for  90  minutes   n  Moderated  each  group  through  the  paper   mockups  to  hear  their  feedback   n  Copyright  ©  2013  Olsen  Solu5ons  

Product-­‐Market  Fit  Case  Study:   Findings  on  Concepts  &User  Benefits   “Shield” Concept “Saver” Concept Reduce Junk Mail Find out what “they” know about you Save Trees Marketing Report Marketing Score Marketing Profile Money Saving Offers Compare Yourself to Others Social Networking Legend Strong appeal Some appeal Low appeal Copyright  ©  2013  Olsen  Solu5ons  

Product-­‐Market  Fit  Case  Study:   Learnings  from  Research   n  n  n  n  Learned  that  “Shield”  (an5-­‐junk  mail)  concept  was   stronger  than  “Saver”   People  didn’t  like  many  of  the  “Saver”  concept   components   Learned  users’  concerns  /  ques5ons  about  “Shield”   concept   Refined  “Shield”  concept:   n  n  n  Removed  irrelevant  components   Improved  messaging  to  address  user  concerns  /  ques5ons   Validated  revised  “Shield”  concept  with  quick  2nd   round  of  tests   n  n  No  customer  concerns   Clear  willingness  to  pay   Copyright  ©  2013  Olsen  Solu5ons  

Copyright  ©  2013  Olsen  Solu5ons  

Product-­‐Market  Fit  Case  Study:   Summary   n  4  weeks  from  1st  mee5ng  to  validated   product  concept  with  zero  coding   n  Reasonable  cost   n  1  round  of  itera5on  on  product  concept   n  Iden5fied  compelling  concept  that  users   are  willing  to  pay  $10/month  for   n  Trimmed  away  non-­‐valuable  pieces   n  You  can  achieve  similar  results   Copyright  ©  2013  Olsen  Solu5ons  

Metrics  to  Validate  Product-­‐Market  Fit   n  Survey  results   n  Importance  &  Sa5sfac5on   Net  Promoter  Score   n   n  n  “How  would  you  feel  if  you  could  no  longer  use  Product  X?”   n  n  Very  disappointed,  Somewhat  disappointed,  Not  disappointed   User  behavior   n  n  n  Prospects  sign  up  (high  conversion  rate)   They  keep  using  it  (high  reten5on  rate)   They  use  it  open  (high  frequency  of  use)   Copyright  ©  2013  Olsen  Solu5ons  

Op5miza5on  with  Metrics:   Define  the  Equa5on  of  your  Business    Model:   Adver5sing  Business     Profit  =  Revenue  -­‐  Cost         nique  Visitors    x    Ad  Revenue  per  Visitor   U            mpressions/Visitor    x    Effec5ve  CPM  /  1000   I             isits/Visitor    x    Pageviews/Visit    x    Impressions/PV   V       ew  Visitors  +  Returning  Visitors   N      nvited  Visitors  +  Uninvited  Visitors   I        of  Users  Sending  Invites    x    Invites  Sent/User    x    Invite  Conversion  Rate   # Copyright  ©  2013  Olsen  Solu5ons  

Sample  Signup  Page  Yield  Data   Daily Signup Page Yield vs. Time New Registered Users divided by Unique Visitors to Signup Page 100% Daily Signup Page Yield 90% 80% 70% 60% 50% 40% Started requiring registration 30% 20% 10% Changed messaging Added questions to signup page 0% 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1 0 Copyright  ©  2013  Olsen  Solu5ons  

Iden5fying  the    “Cri5cal  Few”  Metrics   n  n  What  are  the  metrics  for  your  business?   Where  is  current  value  for  each  metric?     How  many  resources  to  “move”  each  metric?   n  n  Developer-­‐hours,  5me,  money   Which  metrics  have  highest  ROI  opportuni5es?   Metric  B   Bad  ROI   Return   Return   Metric  A   Good  ROI   Investment   Metric  C   Great  ROI   Return   n  Investment   Investment   Copyright  ©  2013  Olsen  Solu5ons  

Case  Study:   Op5mizing  Friendster’s  Viral  Loop   % of users sending = 15% invites Active Users % of users who are active Invites per sender = 2.3 Invite Prospective Users Invite click-through rate Click Registration Process Fail Succeed Don’t Click Conversion = 85% rate Users •   Mul5plied  together,  these  metrics  determine  your  viral  ra5o   •   Which  metric  has  highest  ROI  opportunity?   Copyright  ©  2013  Olsen  Solu5ons  

The  Upside  Poten5al  of  a  Metric   ?   100%   100%   85%   15%   0   Registra5on   Process  Yield   Max  possible   improvement   0.15  /  0.85  =  18%   2.3   0   %  of  users  sending   invita5ons   0.85  /  0.15  =  570%   0   Avg  #  of  invites   sent  per  sender   ?  /  2.3  =  ?%   Copyright  ©  2013  Olsen  Solu5ons  

Okay,  so  how  can  we  improve  the  metric?   How  do  we  increase  the  average  number  of   invites  being  sent  out  per  sender?   n  For  each  idea:   n  n  What’s  the  expected  benefit?  (how  much  will  it   improve  the  metric?)   n  What’s  the  expected  cost?  (how  many  engineer-­‐ hours  will  it  take?)   n  You  want  to  iden5fy  highest  ROI  idea   Copyright  ©  2013  Olsen  Solu5ons  

Aper  Launching  Address  Book  Importer…   Copyright  ©  2013  Olsen  Solu5ons  

Aper  Launching  Address  Book  Importer…   Copyright  ©  2013  Olsen  Solu5ons  

Aper  Launching  Address  Book  Importer…   Copyright  ©  2013  Olsen  Solu5ons  

Op5miza5on  through  Itera5on:   Con5nuous  Improvement   Measure   the  metric   Learning   Analyze   the  metric   Gaining  knowledge:   •   Market   •   Customer   •   Domain   •   Usability   Iden5fy  top   opportuni5es   to  improve   Design  &  develop     the  enhancement   Launch  the   enhancement   Copyright  ©  2013  Olsen  Solu5ons  

Lean  Product  Ninja   Cheat  Sheet   Understand  the  problem  space  to  iden5fy   underserved  customer  needs   n  Define  your  value  proposi5on   n  Create  wireframes/mockups   n  Get  customer  feedback   n  Revise  your  feature  set,  UI  design,  and   messaging  to  improve  product-­‐market  fit   n  Launch,  learn,  and  iterate   n  Copyright  ©  2013  Olsen  Solu5ons  

My  Background   n  Educa5on   n  n  n  n  n  BS,  Electrical  Engineering,  Northwestern   MS,  Industrial  Engineering,  Virginia  Tech   MBA,  Stanford   Web  development  and  UI  design   20  years  of  Product  Management  Experience   Managed  submarine  design  for  5  years   n  5  years  at  Intuit,  led  Quicken  Product  Management   n  Led  Product  Management  at  Friendster   n  CEO  &  Cofounder  of  YourVersion,  “Pandora  for  your  news”   n  PM  consultant  to  startups:  Box,  YouSendIt,  Epocrates     My  slides  will  be  posted  to  hXp://   n  Copyright  ©  2013  Olsen  Solu5ons  

Q  &  A   Moderator:     Hector  Del  Cas5llo,  CPM,  CPMM,  PMP   Presenter:           Dan  Olsen   Product  Consultant  &  Interim  VP  of  Product   Olsen  Solu5ons   @DanOlsen     hXp://     hXp://   ©  AIPMM  2013  

Upcoming  Courses       November  26-­‐28,  2013         December  2-­‐3,  2013     Follow  the  links  above  for  more  informa5on,  or  contact     Katerina  Protonatoriou,  kpro  [at]   Athens  Informa5on  Technology  (AIT)   19  km,  Markopoulo  Ave.   GR  -­‐  19002  Peania,  Athens,  Greece   hXp://   ©  AIPMM  2013  

Upcoming  Courses       Zagreb,  Croa5a   Belgrade,  Serbia   Singapore,  Singapore   Washington,  DC       Singapore,  Singapore   Washington,  DC           Follow  the  links  above  for  more  informa5on,  or  visit:     Transform  Your  Career.   ©  AIPMM  2013  

Please  Join  Us  Again!   AIPMM  Webinar  Series:       The  Art  of  a  Handoff Mind  the  Agile  Business  Gap Why  the  Way  You  Collect  the  Voice  of  the  Customer  MaXers    Nov  22nd,  12  pm  ET    Dec  4th,  3  pm  AEDT    Dec  6th,  12  pm  ET   hXp://     Global  Product  Management  Talk:     hXp://     Stay  Informed!   Announcements:                  hXp://   Call  For  Speakers:   LinkedIn:                          hXp://    hXp://   Membership:                hXp://   Cer=fica=on:            hXp://   ©  AIPMM  2013  

#aipmm presentations

Add a comment

Related presentations

Canvas Prints at Affordable Prices make you smile.Visit http://www.shopcanvasprint...

30 Días en Bici en Gijón organiza un recorrido por los comercios históricos de la ...

Con el fin de conocer mejor el rol que juega internet en el proceso de compra en E...

With three established projects across the country and seven more in the pipeline,...

Retailing is not a rocket science, neither it's walk-in-the-park. In this presenta...

What is research??

What is research??

April 2, 2014

Explanatory definitions of research in depth...

Related pages

Building Great Products

My talk “How to Build Great Products the Lean Startup Way” from SVPMA. ... “Building Great Products the Lean Startup Way” Workshop Oct 10th in San ...
Read more

My talk "How to Build Great Products the Lean Startup Way ...

Building Great Products ... My talk “How to Build Great Products the Lean Startup Way” from SVPMA; Slides for “How to be a Lean Product Ninja ...
Read more

Building Great Products the Lean Startup Way - AIPMM

Building Great Products the Lean Startup Way from AIPMM ... submarine design and then worked at Intuit where he led product management for ...
Read more

Building Great Products the Lean Startup Way on Vimeo

This is "Building Great Products the Lean Startup Way" by AIPMM Administration on Vimeo, the home for high quality videos and the people who love them.
Read more

The Ten Principles Of Building Great Products - Forbes

The Ten Principles Of Building Great Products. ... methodologies such as the lean startup approach as well ... a quicker way to building the ...
Read more

The Lean Startup | The Movement That Is Transforming How ...

The Movement That is Transforming How New Products are Built ... The Lean Startup isn't just about how to create a ... and to solving the world's great ...
Read more

Webinar: How to Build Great Products the Lean | General ...

Webinar: How to Build Great Products the Lean Startup Way Seattle. Digital Marketing; Business ...
Read more

The Lean Startup | Methodology

The Lean Startup methodology has as a premise that ... study and solve problems along the way. ... of a startup is to turn ideas into products, ...
Read more