LinkedIn - A Professional Network built with Java Technologies and Agile Practices

50 %
50 %
Information about LinkedIn - A Professional Network built with Java Technologies and Agile...

Published on May 19, 2008

Author: linkedin

Source: slideshare.net

LinkedIn: A Professional Social Network Built with Java™ Technologies and Agile Practices Nick Dellamaggiore, Principal Software Engineer Eishay Smith, Senior Software Engineer

Learn how Java™ and agile practices are employed in building large-scale consumer internet sites like LinkedIn, the world’s largest professional network. 2008 JavaOneSM Conference | java.com.sun/javaone | 2

Agenda LinkedIn(troduction) Agile engineering process Architecture Building LinkedIn with Java™ Questions/Discussion 2008 JavaOneSM Conference | java.com.sun/javaone | 3

2008 JavaOneSM Conference | java.com.sun/javaone | 4

LinkedIn Your professional identity on the web The site The numbers • Launched in 2003 • 22 million members • 3 degree network • 4+ million unique visitors/month • Strong focus on privacy • 40 million page views/day The company • 2 million searches/day • Profitable since 2006 • 250K invitations sent/day • 250 employees • 1 million answers posted • Based in Mountain View, CA • We’re hiring! 2008 JavaOneSM Conference | java.com.sun/javaone | 5

Engineering Process Release often • Short, 2-4 week development cycles All dev tasks are broken down into small, manageable engineering cards Strong focus on testing Minimize meetings, status (standup meetings as needed) 2008 JavaOneSM Conference | java.com.sun/javaone | 6

Engineering Cards 2008 JavaOneSM Conference | java.com.sun/javaone | 7

Testing 6500+ unit and integration tests 500 HtmlUnit tests Large-scale Hudson deployment • 20+ nodes • Continuous integration build/smoke test on SVN commit • Nightly test run (full test suite) 2008 JavaOneSM Conference | java.com.sun/javaone | 8

“Celebrities” Test Network 2008 JavaOneSM Conference | java.com.sun/javaone | 9

Hudson 2008 JavaOneSM Conference | java.com.sun/javaone | 10

Testing… Lessons Learned Test suite too focused on integration • Long running time • Long-running setup/teardown routine • Exacerbated by move to SOA • Tests become redundant due to overlapping integration tests • Better: use mocks (EasyMock works great) 2008 JavaOneSM Conference | java.com.sun/javaone | 11

Architecture 2008 JavaOneSM Conference | java.com.sun/javaone | 12

LinkedIn Architecture: 2003-2005 2008 JavaOneSM Conference | java.com.sun/javaone | 13

LinkedIn Architecture: 2006 2008 JavaOneSM Conference | java.com.sun/javaone | 14

LinkedIn Architecture: Today 2008 JavaOneSM Conference | java.com.sun/javaone | 15

LinkedIn Architecture: Today 2008 JavaOneSM Conference | java.com.sun/javaone | 16

Service-oriented Architecture Benefits Caveats • Each service can scale • Diminished data integrity, independently consistency • Encourages decoupling, reduces • No cross-service transactions, code+deployment dependencies referential integrity or joins • Graceful degradation of • Backward compatibility functionality • “8 Fallacies of Distributed Computing” Takeaways • Not applicable to small or simple sites, startups • Build services with interfaces, POJOs, dependency injection • Be conscious of API design, granularity 2008 JavaOneSM Conference | java.com.sun/javaone | 17

Services 2008 JavaOneSM Conference | java.com.sun/javaone | 18

The Cloud Backend server caching the entire LinkedIn Network Graph cache • Updated via Databus • Persisted to disk on shutdown Transient network cache • Computationally intensive to build • Bound to member session • Sticky load balancing to Cloud Graph operations: • findRoute(m1, m2) • visit(visitor, deg) • visit(visitor, deg, since) 2008 JavaOneSM Conference | java.com.sun/javaone | 19

The Cloud The numbers 22M nodes, 120 million edges 12 GB JVM heap • In-memory caches implemented in C++, accessed via JNI 40 Cloud server instances deployed in production 2008 JavaOneSM Conference | java.com.sun/javaone | 20

LinkedIn News Part of a new trend in LinkedIn Crawling the web for news • Distributed task management • Parsing technologies: StAX, Rome Indexing and searching • Serving the right news to the right people Duplications Duplications Duplications • Real time problem, fast solution • 500 articles batch = 125K compares 2008 JavaOneSM Conference | java.com.sun/javaone | 21

News Service Architecture 2008 JavaOneSM Conference | java.com.sun/javaone | 22

LinkedIn: 99% Pure Java™ Plumbing Services Tier Web Tier Tools Spring Remoting LinkedIn Spring Louis (in-house fwk) Hudson (CI) ActiveMQ Eh-cache Spring MVC Eclipse+Mylyn Quartz Lucene Grails JIRA/Greenhopper HttpClient Jetty DWR JUnit, HtmlUnit 2008 JavaOneSM Conference | java.com.sun/javaone | 23

Why we love Java™ at LinkedIn Static typing is a lifesaver • Huge codebase, 1M+ lines of code • Refactoring with confidence • Code navigation (via IDE) • 20 active branches, compiler helps with merging • Scaling the engineering organization • Currently 50+ engineers, 8 teams… and growing • Excellent talent pool of Java™ engineers to hire from Community 2008 JavaOneSM Conference | java.com.sun/javaone | 24

LinkedIn: A Professional Social Network Built with Java™ Technologies and Agile Practices Nick Dellamaggiore (http://www.linkedin.com/in/nick) Eishay Smith (http://www.linkedin.com/in/eishay) We’re hiring!

Add a comment

Related presentations

Related pages

LinkedIn - A Professional Network built with Java ...

1. Linkedin: A Professional Social Network Built with Java™ Technologies and Agile Practices Nick Dellamaggiore, Principal Software Engineer ...
Read more

LinkedIn: A Social Network Built with Java Technologies ...

... the largest professional social network on the Java ... with Java Technologies and Agile Practices ... LinkedIn Built Its ...
Read more

LinkedIn - A Professional Network built with Java ...

Home Technology LinkedIn - A Professional Network built with Java Technologies and Agile Practices 3. Agenda LinkedIn(troduction)Agile engineering ...
Read more

LinkedIn - A Professional Social Network Built with Java ...

... A Professional Social Network Built with Java ... Community LinkedIn: ProfessionalSocial Network Built ... aOneSM Practices nkedI ...
Read more

LinkedIn - A Professional Network built with Java ...

LinkedIn - A Professional Network built with Java Technologies and Agile Practices
Read more

LinkedIn - A Professional Network built with Java ...

LinkedIn - A Professional Network built with Java Technologies and Agile Practices linkedin
Read more

LinkedIn at JavaOne 2008 | Official LinkedIn Blog

... Social Network Built with Java™ Technologies and ... A Professional Network built with Java Technologies and Agile Practices from LinkedIn.
Read more

Java | LinkedIn

... Senior Agile Java ... Freelance @ Moka Technologies, Expert Java/Spring Certifié, ... and Android apps built with Java. ...
Read more

Mikalai Alimenkou | LinkedIn

View Mikalai Alimenkou’s professional profile on LinkedIn. ... View Mikalai Alimenkou’s professional ... in Agile engineering practices and Java ...
Read more