Published on June 6, 2016
1. Real Time Scoring Joseph R. Barr Chief Analytics Officer www.HomeUnion.com HomeUnion (CA BRE Lic # 01526904) operates investment property market places for traditional and IRA based property investing. The company is not a Financial Services, legal or Tax advisorcompany. For any such financial services, legal or tax needs, please seek help from appropriate professionals. The Company is not a lender and the exact terms of the loan will be decided between the lender and the investment property account holder. Any loan related information provided is for informational purposes only.
2. 2 1. Agenda Scoring Real time part of scoring Use cases The business of real time scoring Challenges SLA & latency Quality of service Catching & fixing problems Copyright 2015 HomeUnion, All Rights Reserved KatharineHepburn
3. 3 2. Scoring Definition Scoring is a generic terms that describes a [sic] statistical procedure, a data-driven algorithm, that associates a numerical vector to each entity of a cohorts. A score is the resulting vector. Examples Credit (FICO, Vantage) Fraud (Falcon, ID Score) Bonds (Moody, S&P, Fitch) Colleges (US News) Neighborhoods (HomeUnion) Investments in single-family homes (HomeUnion) Copyright 2015 HomeUnion, All Rights Reserved
4. 4 3. A mathematical formulation, if you must… i) Domain translation, ‘structure-preserving’ morphism ψ: Ω Φ fromthe category of problems Ω (problem domain) into the category of data science Φ. (Specifically,the objectsof the category of data scienceconsistof examples(X,y), modeling frameworksμ, and algorithms α.) ii) A scoreis an instanceof an objectof the category of data science Φ satisfying standard statisticalcriteria (e.g., for goodnessof , predictivepower,etc.) Copyright 2015 HomeUnion, All Rights Reserved SaundersMacLane,Samuel Eilenberg, John von Neumann
5. 5 4. A clarification Credit risk object,Ω: Creditorsperspectiveof borrowersrisk of defaulting ona loan. Data science object, Φ: A mappingof borrowercharacteristicsintofeaturevectors (X,y), one for eachexample, together with statisticalmodelframework,and,an algorithmto implement the model framework.The featurecomponentX is referredtoas ‘borrower’sprofile’. A scorecard:a summary of the algorithmand model, alongwith borrower’sprofileand model output,mostcommonlya numericalvector. E.G. Borrowerprofile:X = (Level of education, Annual income, Number of credit cards,…). y = 0 or 1 depending on whether a borrower defaulted on loan valued at $1,200or more, during the past 7 years. Computationalframework:logistic regression& an algorithm:features, variable selection based on the chi-squared (Wald) method with out of sample validation, (with K-S and ROC.) Copyright 2015 HomeUnion, All Rights Reserved
6. 6 5. Scoring Copyright 2015 HomeUnion, All Rights Reserved Business Problem Solution Vendors Banking Consumer credit Credit limit Terms of credit,etc. Credit score FICO’s Score Tri bureau’s Vantage Banking Mortgage Credit score CoreLogic’s CreditIQ Banking, retail, telecommunication,etc. Originationfraud Fraudscore ID Analytics’ID Score Insurance Auto Insurance Auto Insurance Score Lexis’Attract Internet Hamor Spam Spamfilter (Virtually) every provider Retail Fraudulent transaction Transactionscore FICO’s Falcon
7. 7 6. Is it really necessary? As imperfect as they may be, in this universe we rely on scores to make business decisions. An over-reliance on scores to the exclusion of judgment may (and often) result(s) in bad business.(Why?) Copyright 2015 HomeUnion, All Rights Reserved Ray Bolger
8. 8 7. Humans do lots of things in real time Recognition (places, faces, voices, musical pieces, poetry passages, abstract patterns) Playing music: the violin, guitar, piano, trumpet Bluffing or recognizing one (e.g., in a game with stakes) Baseball: to swing or not to swing (a ball or a strike) The game of life: reading emotion, Calling a bluff in the game of poker: to raise, to call, to fold Copyright 2015 HomeUnion, All Rights Reserved DiMaggio
9. 9 Observe, translate& gather Parse& combine Match, resolve& disambiguate Correlate, associate& infer Abstract, generalize, translate(into solution domain) & calculate Represent & present Synthesize& scrutinize 8. Real time cognitive operations Copyright 2015 HomeUnion, All Rights Reserved
10. 10 9. Watch out for too many cooks Scoring is a lengthy and iterative process having many parts involvingmultiple stakeholders Often lots of opinions from all parts of the business Multiple interpretations with different parts of the business having different interpretations Lack of clarity may result in uncaught errors Copyright 2015 HomeUnion, All Rights Reserved
11. 11 10. Real time scoring (RTS) or the late answer is the wrong one Copyright 2015 HomeUnion, All Rights Reserved
12. 12 11. Real time is relative Nanoseconds (10-9): quantum measurements Microseconds(10-6): High frequency trading Millisecond(10-3) Cellular communication (Sub)second(≈8*10-2 seconds): Commercial applications like credit/debit card transactions, bidding (like real estate) A few seconds:Video buffering Minutes: File transfers (uploading videos on FB) Copyright 2015 HomeUnion, All Rights Reserved
13. 13 12. RTS is a common practice Omicron: consumerbehavior& advertising, servingads, discounts,etc. ID Analytics: ID Score: fraud prevention FICO: Falcon, Visa & MasterCard:instantverification HomeUnion: Right Bid: real estate optimal offer VeriSign: iDefense: online authentication Travelocity, Priceline: pricing (e.g., airlines, hotels) Netflix, Amazon: personalization Google: web search engine & page ranking Copyright 2015 HomeUnion, All Rights Reserved
14. 14 13. Card-not-present fraud score Mechanisms may vary, butit involves a person trying to execute a (fraudulent) transaction with a stolen credit card number (plus name, expiration date, etc.) The transaction logic verifies credit card and user attributes to look for deviation fromnormalcy and ‘unusualbehavior.’ with is translated into a score (Obviously,) prior to the transaction (the “attempt”), neither attributes, nor score exists. The scoredepends on the card, the vendor, the channel, the location, etc. “On the fly,” all within a second, the following events will take place. 1) Dataassociated with transaction is transferred to a central database, parsed, resolved, disambiguated, etc. 2) An algorithm is put to work resulting in a scorecard 3) On-the-fly validation (adaptivealgorithm) Copyright 2015 HomeUnion, All Rights Reserved
15. 15 14. What’s goes into RTS? Infrastructure Data Algorithm Security Governance Quality Customer support Copyright 2015 HomeUnion, All Rights Reserved
16. 16 15. Data flow infrastructure Copyright 2015 HomeUnion, All Rights Reserved
17. 17 16. Real time architecture paradigm Copyright 2016 HomeUnion, All Rights Reserved CLIENT SCORING ENGINE D A T A P E R S I S T E N C E SALES PROPERTY LISTING TRANSACT IONS INVESTOR PERFORM ANCE CLIENT CLIENT CLIENT https:RESTful API IN MEMORY EXECUTION Secured VPN
18. 18 17. Algorithm, the pieces Batch preprocessing Batch data modeling R/T processing & features creation Model pass through Post processing Dishing it out Copyright 2015 HomeUnion, All Rights Reserved
19. 19 18. Quality of service: Service Level Agreement (SLA) SLA governsquality of service,most prominently,latency SLA contracts specify worst-caseperformance whileallowing some flexibility for the occasionaloutlier TypicalSLA will specify something like this: “Response time of at least 98% of requests will be delivered within 1 second; while no more than 3 in a thousand won’t be delivered at all. The remaining will be dished out within 3 seconds.” Copyright 2015 HomeUnion, All Rights Reserved
20. 20 19. Security Multiple layers of security to protect data Additional layer of security to protect PII Encrypted messaging over secure network protocol via, e.g., • Persistent VPN connection • User authentication (for the one-off user) Continual monitoring, (e.g., for unusual patterns, load Rapid escalation process & procedures (in case of a <suspected> breach) Copyright 2015 HomeUnion, All Rights Reserved
21. 21 20. Governance Legal requirements Regulatory parameters Business standards Ethical standards Transparency … In the universe of consumer credit, FCRA requires that creditors mustn’t take person’s race, gender, etc. into considerations… Copyright 2015 HomeUnion, All Rights Reserved
22. 22 21. Customer support Copyright 2015 HomeUnion, All Rights Reserved Call center serving Mission-critical 24/7 (technical & business) Business-critical normal office hours (8 AM to-6 PM)
23. 23 22. The “periodic table” of scoring 1) Text processing (e.g., parsing,tokenizing, normalizing, merging, splitting,etc.) 2) Searching(througha database) 3) Matching& disambiguating 4) Data modeling (structures,semantics,storage& retrieval) 5) Feature(variables, attributes) 6) Traversingthrougha network, a tree, a graph 7) Mathematicaloperations 8) Statistical& machine learning model development lifecycle 9) Adaptive learning (“after the effect”) 10) Learning re-enforcement(“improving”) 11) Voice-to-text 12) Image processing Copyright 2015 HomeUnion, All Rights Reserved
24. 24 23. Resources Infrastructurearchitecture Dataacquisition Dataengineering& datamodeling Statistical (ormachinelearning)modeling Programming Productionengineering Qualityassurance Network management Security Customersupport Copyright 2015 HomeUnion, All Rights Reserved $$$RTS systemsare expensive to build, maintain & operate $$$
25. 25 24. Questions & comments www.homeunion.com email@example.com Copyright 2015 HomeUnion, All Rights Reserved
Joseph R Barr, Ph.D. ... Naveen Jain First Tech Federal Credit Union Pinar Donmez Kabbage, ... Welcome to the Chief Analytics Officer Forum West Coast
Single Family Rental Forum West. ... based advertising and web analytics) ... hold harmless and defend IMN, including our employees, officers, ...
Mr. Wehrle joined the NICB in 2008 as Senior Vice President and Chief Operating Officer and was ... Home Office operations. She ... President of West Coast ...
CISO Executive Summit. ... Tamara Barr leads the information services ... Dr. Eugene Kolker is chief data analytics officer at Seattle Children ...
The scope of SC includes analytics, ... Western Union Digital, USA. ... Dr. Joseph Barr, Chief Analytics Officer, HomeUnion, ...
View Jim Richards’ professional profile on LinkedIn. ... West Coast Anti-Money Laundering Forum; ... Chief Operating Officer & Head of Program ...
Opal Group is a global conference organizer catering ... Chief Nursing Officer ... Family Office & Private Wealth Management Forum West 2016 ...
That's the heart of First Tech Federal Credit Union. ... he founded the West Coast office of Hovde Financial ... Chief Financial Officer and ...