Published on June 4, 2013
WP4 - semantic structuring andlinking of event-centric datain the social webRaphaël Troncyraphael.firstname.lastname@example.org / @rtroncy
WP4: Original WorkplanTask Title DurationT4.1 Specification of an event model for representingpersonal eventsM1 – M32T4.2 Linking personal events to multimedia content fromthe wider worldM1 – M32T4.3 Linking personal events to knowledge and promotingdiverse and opinionated interactionM1 – M32T4.4 Linking with people and discovering new contacts M1 – M32T4.5 Promoting sustainable interaction between users M1 – M3204/06/2013 - ALIAS Final Meeting - Munich, Germany - 2EURECOM: 76 PMTUM-MMK: 4 PM
What are Events?Events are observable occurrences grouping04/06/2013 - ALIAS Final Meeting - Munich, GermanyExperiences documented by MediaPeople Places Time- 3
User Tasks and ChallengesALIAS Final Meeting - Munich, GermanyGoals: Discover PAST, PRESENT and FUTURE events Live, relive and predict experiences through shared media Enhance search and recommendations mechanisms Support the decision making processChallenges Semantization of Data (events, media, people) Reconciliation of sparse and highly heterogeneous data User-centric design: discovering, deciding and recommending Scalability, Caching and Optimization04/06/2013 - - 5
Scenario on e-tickets Supporting the elderly in finding eventsand purchasing tickets online Enabling the elderly to re-live past events they haveattended with their family and relatives Need data about: Events: http://linkedevents.org/ontology/ (LODE) Media: http://www.w3.org/TR/mediaont-10/ Tickets: http://www.heppnetz.de/ontologies/tio/ns.html (TIO) Meteo: http://purl.org/ns/meteo# PersonalizationLocal event guides are more appropriate!04/06/2013 - ALIAS Final Meeting - Munich, Germany - 6
Events through the web …04/06/2013 - ALIAS Final Meeting - Munich, GermanyEvent DirectoriesMedia PlatformsSocial NetworksAllows opportunistic/serendipitous discoveryLimited social features and Informationincompleteness (low coverage, weak decision)Aids remembering and sharing past experiencesLack of consistent events descriptionsAttendance: popularity and friends whereabouts,attendees’ opinions.Limited discovery mechanism: need to filter outrelevant information.Explore semantic connections to create better overviews of events- 7
There’s a lot of information out there…04/06/2013 - ALIAS Final Meeting - Munich, Germany - 8
A lot of information…http://www.flickr.com/photos/mwparenteau/43203978304/06/2013 - ALIAS Final Meeting - Munich, Germany
ApproachALIAS Final Meeting - Munich, Germany04/06/2013 - Aggregate heterogeneous data sources Unify into one data model using SemanticWeb technologies Enrich with media, social data andbackground knowledge Identify meaningful and/or interestingrelationships between events, media, andpeople- 10
Architecture (generic dashboard)ALIAS Final Meeting - Munich, GermanyReal-time Reconciliation- Tag-based mapping- Instance matchingHeterogeneous data sourcesData CrawlingRDF ConversionRSS UpdateWeb ApplicationsSPARQLREST API (Elda)04/06/2013 - - 11
WP4.1: Data (Event) Model04/06/2013 - ALIAS Final Meeting - Munich, Germany Ontology for Events:LODE: http://linkedevents.org/ontology/ Extension for personal events:Based on WP1 outputs (YOU, TUM-GSING, PME)Taxonomy of event categories Triple store gathering:Events and media descriptions: user’s long-termmemoriesSystem interaction: user’s short-term memory- 12
ALIAS Final Meeting - Munich, GermanyData Modeling – LODE ontology04/06/2013 -LODE is a minimal model that encapsulates the factual properties ofevents: What, Where, When and Who.- 13
Róisín Murphy at Nouveau Casino04/06/2013 - ALIAS Final Meeting - Munich, Germany350591E0-001-005971169-9- 14
Representing Events with04/06/2013 - ALIAS Final Meeting - Munich, Germany - 15
Róisín Murphy at Nouveau Casino04/06/2013 - ALIAS Final Meeting - Munich, Germany - 16
Media explicitly associated with the event04/06/2013 - ALIAS Final Meeting - Munich, Germany2.3+ million imagesover 120.000+ eventsMachine tags“lastfm:events”APIs11.000+ videosover 3000+ events- 17
Representing Media with Media Ontology04/06/2013 - ALIAS Final Meeting - Munich, Germany - 18
ALIAS Final Meeting - Munich, GermanyData Crawling Various API specifications Policy management Requests chaining Different input and output schemas Time-consumingA Restful Service leveraging the commonalities of API specificationsA Web dashboard to easily handle data collection and interlinkinghttp://eventmedia.eurecom.fr/dashboard/04/06/2013 - - 19
Real-Time ReconciliationALIAS Final Meeting - Munich, GermanyParticipantsMediaCost & descriptionPrecise timeagents04/06/2013 -Need for semantic connections between events, media, agents and locations- 20
ALIAS Final Meeting - Munich, GermanyLive UpdateTag = *:eventPer week• 1500 Photos• 130 eventsNumber of Items posted per days of the weekNumber of Items posted during the last days04/06/2013 - - 21
Interlinking04/06/2013 - ALIAS Final Meeting - Munich, GermanyEventMediaMusicBrainzDBpediaGeonamesFoursquareUberblicFreebaseAgent Last.fm Eventful MusicBrainz DBpedia Freebase Uberblic New York TimesLieu Last.fm Eventful Upcoming DBpedia Freebase Foursquare GeonamesÉvènement Last.fm Eventful Upcoming DBpedia Freebase Uberblic- 22
Instance Matching Events/Agents/Locations semantic connectionsSPARQL-based Instance matchingFiltering by the issuance date ChallengesStructural heterogeneityDifferent representations of the same valueUser-generated knowledge featuring human mistakesDifferent granularities between data sources04/06/2013 - ALIAS Final Meeting - Munich, Germany - 23
The events similarity is a mutual agreement of their factual properties Based on top-k dependencies between properties Evaluation: Precision: 96% and Recall: 94% Ground truth of 300 matched instancesALIAS Final Meeting - Munich, GermanyEvents connectionsp1 p2dependencytitle1 title2 0.30place1 place2 0.28title1 agent2 0.26agent1 agent2 0.21description1 title2 0.16Minimal conditions to fetchsimilar events using SPARQL1st levelRefine the results2nd level04/06/2013 - - 24
Agents connections (precision-oriented) Internal reconciliation + DBpedia, Musicbrainz and BBC Label matching + contextual filtering (Cosine) Granularity? (1 to N mappings) Cosine inequality Locations connections (precision-oriented) Internal reconciliation + DBpedia and Foursquare It is straightforward thanks to the consistency of locations descriptions Top-k Google search Geographical distanceALIAS Final Meeting - Munich, GermanyAgents & Locations connectionsCosine (d, a+b) > Cosine (d ,a)04/06/2013 - - 25
About the dataset SPARQL endpoint: http://eventmedia.eurecom.fr/sparql Restful API : http://eventmedia.eurecom.fr/rest/event/ALIAS Final Meeting - Munich, GermanyEvent Agent Location Photos UserLast.fm 61,173 62,577 15,917 1;464,885 130,249Upcoming 19, 276 15 10,269 308,148 3,932Eventful 45,204 8,725 18,763 3,028 321Total 125,650 71,313 44,948 1,760,787 159,263DBpedia MusicBrainz EventfulLast.Fm 13064 34247 2865Eventful 2515 3616 -04/06/2013 - - 26
WP4.2: Linking Events with Media Start with media assets explicitly associatedwith some events Enrich the dataset using large scale multimediaanalysis combined with event properties(time, location, topic)Learning, training, re-training: image and speech!Apply to personal media collections and enrich gradually04/06/2013 - ALIAS Final Meeting - Munich, Germany - 27
User Uploading Behavior Analysis Most of event-related photos are uploaded infive days.04/06/2013 - ALIAS Final Meeting - Munich, Germany - 28
Achievements Devise a generic approach for collecting mediaoriginating from specific events based on theirvisual characteristics, capture date, locationand owner. Propose two alternative approaches forassessing visual similarity between canditatemedia and known event media:Visual matching against known event media (k-NN)Visual event model trained from examples (SVN)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 29
The Framework (Visual Matching) The medias are collected via the Flickr API,thanks to the events’ multiple facets Visual Pruning and Owner Refinement areemployed to improve the performance04/06/2013 - ALIAS Final Meeting - Munich, GermanylocationtimetitleidEventquery bymachine tagqueryby geoqueryby textMediaOwnerRefineResultsPruningMediaMedia- 30
Media Collection• Multimodal features are exploited forcollecting data:• Location• Time• Text04/06/2013 - ALIAS Final Meeting - Munich, GermanyFlickrAPI- 31
Visual Pruning and Owner Refinement04/06/2013 - ALIAS Final Meeting - Munich, Germany - 32
Event Enricher results04/06/2013 - ALIAS Final Meeting - Munich, Germany - 33
Event Enricher results04/06/2013 - ALIAS Final Meeting - Munich, Germany - 34
Automated Event Modeling Framework04/06/2013 - ALIAS Final Meeting - Munich, GermanyEventtagsRank tagsby frequencytag1tag2tag3……….tagNRank Photosby distance to tagsTop NtagsPic1Pic2Pic3PicM ……PositiveSamplesNegativeSamplesTop MPhotosEventModel- 35
ResultsEventID Query Our Algorithm k-NN Pruninglastfm:804783 87.92 88.68 46.98lastfm:1830095 74.81 78.38 80.26lastfm:1858887 61.84 63.41 63.56lastfm:1499065 9.47 90.53 89.94lastfm:1787326 0.00 98.40 92.65lastfm:1351984 96.32 96.32 55.32lastfm:1842684 87.28 87.93 67.86lastfm:2020655 99.21 91.80 71.69lastfm:1301748 93.53 93.53 73.73lastfm:1370837 83.73 85.15 73.83ACMMM10 88.56 91.04 87.56SIGIR2010 0.00 60.19 42.28ACMMM07 25.42 57.62 46.61NICECarnival2011 22.30 76.58 59.10Average Accuracy 69.41 83.31 68.6404/06/2013 - ALIAS Final Meeting - Munich, Germany - 36
WP4.3: Linking Events with Knowledge Add semantic categories to media assets Re-rank and re-organize media content forpersonalized user groupsLanguage detection Enrich multimedia presentation with richknowledge and content from the Semantic WebOpinions: articles from newspapers (e.g. New York Times)Topical diversity04/06/2013 - ALIAS Final Meeting - Munich, Germany - 37
Sharing photos vs event categories04/06/2013 - ALIAS Final Meeting - Munich, Germany - 38
Sharing photos vs number of participants04/06/2013 - ALIAS Final Meeting - Munich, Germany - 39
Sharing photos vs country of participants04/06/2013 - ALIAS Final Meeting - Munich, Germany - 40
WP4.4: People, Social Network and Events User attendance, user profilesPopularity and friends’ whereaboutsSimilar interests, network, recommendations (behaviors) Social networksEventMedia Facebook applicationhttps://www.facebook.com/TheEventMedia(like us!)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 41
Co-participating events in San Francisco04/06/2013 - ALIAS Final Meeting - Munich, Germany - 42
Towards user profiles (LDA, T = 10)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 43
Facebook native event application04/06/2013 - ALIAS Final Meeting - Munich, Germany - 44
Facebook native event application Event model is weak:missing rich categories, the performers and artists, etc.the user cannot reuse existing events Task: design a Facebook application thatallows for creating and sharing events, andclosely interacting with EventMedia Task: crawl public events and related datahosted by Facebook using the Graph API inorder to continuously feed EventMedia04/06/2013 - ALIAS Final Meeting - Munich, Germany - 45
EventMedia Facebook application04/06/2013 - ALIAS Final Meeting - Munich, Germany - 46https://www.facebook.com/TheEventMedia EventMedia meets here.com Results (worldwide) 4763 venues have been interlinked 30011 events organized in those venues 465175 photos shared on social networksduring events at those venues Examples: http://here.com/services/place/528u173z-672bb28e202e4032848cd6a1fe4b3851(560 events, 9462 photos) oneventmedia http://here.com/services/place/040u29fp-70d8361e99d64a4d978f107eaa69b15a(2 events, 542 photos) on eventmedia
04/06/2013 - ALIAS Final Meeting - Munich, Germany - 47
WP4.5: Promoting Sustainable Interaction Developing end-user interfaces for searchingand browsing media contentAllow opportunistic/serendipitous discoveryAllow social feature, notification Support decision making upon attendingevents Module for cross lingual chat or other IMGoogle translation service04/06/2013 - ALIAS Final Meeting - Munich, Germany - 48
Web Application04/06/2013 - ALIAS Final Meeting - Munich, Germany - 49
Web Application Challenges:Modern single-page app on top of semantic technologiesFast, user-friendly and responsive UIDealing with high variability of data Technologies:Backbone.js: MVC framework enables complex andresponsive UIElda Linked Data API maps SPARQL queriesto RESTful URLs04/06/2013 - ALIAS Final Meeting - Munich, Germany - 50
Whats on this evening?ALIAS Final Meeting - Munich, Germany04/06/2013 - - 51
Whats on this evening?ALIAS Final Meeting - Munich, Germany04/06/2013 - - 52
Demo: Live your eventALIAS Final Meeting - Munich, Germany04/06/2013 -http://eventmedia.eurecom.fr- 53
Demo: Parallel Facetted BrowsingALIAS Final Meeting - Munich, Germany04/06/2013 -http://eventmap-ui.appspot.com/- 54
EventMedia has been awarded04/06/2013 - ALIAS Final Meeting - Munich, Germany - 55http://www.youtube.com/watch?v=8dVD0SqCB6s
ALIAS Chat Module (with translation)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 56
WP4: Deliverables (all delivered)04/06/2013 - ALIAS Final Meeting - Munich, GermanyDeliverable Title Due dateD4.1 Module for representing events related to user’s long term andshort term memoryM7 (Jan 2011)D4.2 Module for cross-media linking of personal events to web content,v1M14 (Sep 2011)D4.3 Module for knowledge enrichment of event descriptions M14 (Sep 2011)D4.4 Module for personalized discovery of new contacts on line M30 (Dec 2012)D4.5 Module for retrieval of opinionated content M33 (Mar 2013)D4.6 Module for cross-media linking of personal events to web content,v2M32 (Feb 2013)D4.7 Module for recommendation of topics and activities suitable formaintaining social contactsM32 (Feb 2013)D4.8 Module for cross-lingual chat support M35(May 2013)- 57
WP4 Publications and Impact 26 high quality publications + 2 awards + 3 more under review: RecSys 2013 + Journal of Web Semantics +Semantic Web Journal Semantic Web Challenge 2012 at ISWC 2012: 1st price IESD Challenge at EKAW 2012: 1st price MediaEval Benchmark 2011: 7 participants (18 interested) 2012: 5 participants (25 interested) 2013: ongoing 2 PhD Thesis: Xueliang Liu (03/12/2012): Event-based Social Media Data Mining Houda Khrouf (scheduled for the end of 2013)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 58
Publications 1/41. Raphaël Troncy, Ryan Shaw and Lynda Hardman. LODE: une ontologie pour représenter desévénements dans le web de données (on HAL). In 21th Journées Francophones dIngénierie desConnaissances (IC10), pages 69-80, Nîmes, France, June 8-11, 2010.2. Raphaël Troncy, Ryan Shaw and Lynda Hardman. LODE: une ontologie pour représenter desévénements dans le web de données (on HAL). In 21th Journées Francophones dIngénierie desConnaissances (IC10), pages 69-80, Nîmes, France, June 8-11, 2010.3. Raphaël Troncy, Bartosz Malocha and André Fialho. Linking Events with Media. In the OpenTrack of the Linked Data Triplification Challenge, colocated with the 6th InternationalConference on Semantic Systems (I-SEMANTICS10), Graz, Austria, September 1-3, 2010.4. Raphaël Troncy, André Fialho, Lynda Hardman and Carsten Saathoff. Experiencing Eventsthrough User-Generated Media. In (ISWC10) International Workshop on Consuming Linked Data(COLD10), CEUR Proceedings Vol. 665, Shanghai, China, November 7-11, 2010.5. Xueliang Liu, Raphaël Troncy and Benoît Huet. Finding Media Illustrating Events. In 1st ACMInternational Conference on Multimedia Retrieval (ICMR11), Trento, Italy, April 17-20, 20116. Houda Khrouf and Raphaël Troncy. Réconcilier les événements dans le web de données. In22nd Journées Francophones dIngénierie des Connaissances (IC11), Chambéry, France, May16-20, 2011.7. Martha Larson, Adam Rae, Claire-Helene Demarty, Christoph Kofler, lorian Metze, RaphaëlTroncy, Vasileios Mezaris, Gareth J.F. Jones. Working Notes Proceedings of the MultimediaBenchmark Workshop (MediaEval 2011), CEUR Proceedings Vol. 807, editors, Santa Croce inFossabanda, Pisa, Italy, September 1-2, 2011.04/06/2013 - ALIAS Final Meeting - Munich, Germany - 59
Publications 2/48. Symeon Papadopoulos, Raphaël Troncy, Vasileios Mezaris, Benoît Huet and IoannisKompatsiaris. Social Event Detection at MediaEval 2011: Challenges, Dataset and Evaluation. InMediaEval Benchmarking Initiative for Multimedia Evaluation, CEUR Proceedings Vol. 807, Pisa,Italy, September 1-2, 2011.9. Xueliang Liu, Benoît Huet and Raphaël Troncy. EURECOM @ MediaEval 2011 Social EventDetection Task. In MediaEval Benchmarking Initiative for Multimedia Evaluation, CEURProceedings Vol. 807, Pisa, Italy, September 1-2, 2011.10. Marieke van Erp, Willem Robert van Hage, Laura Hollink, Anthony Jameson and RaphaëlTroncy. Proceedings of the International Workshop on Detection, Representation, andExploitation of Events in the Semantic Web (DeRiVE11), CEUR Proceedings Vol. 779, editors,Bonn, Germany, October 23, 2011.11. Houda Khrouf and Raphaël Troncy. EventMedia Live: Reconciliating Events Descriptions in theWeb of Data. In (ISWC11) 6th International Workshop on Ontology Matching (OM11), Bonn,Germany, October 23-27, 2011.12. Houda Khrouf and Raphaël Troncy. EventMedia: Visualizing Events and Associated Media.Demo Session at the 10th International Semantic Web Conference (ISWC2011), Bonn, Germany,October 23-27, 2011.13. Xueliang Liu, Raphaël Troncy and Benoît Huet. Using Social Media to Identify Events. In (ACMMultimedia) 3rd Workshop on Social Media (WSM11), Scottsdale, Arizona, USA, November 28-December 1st, 2011.04/06/2013 - ALIAS Final Meeting - Munich, Germany - 60
Publications 3/414. Liu, Xueliang; Huet, Benoit. Social event discovery by topic inference. In 13th InternationalWorkshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2012), 23-25 May2012, Dublin City University, Ireland15. Houda Khrouf, Ghislain Atemezing, Thomas Steiner, Giuseppe Rizzo and Raphaël Troncy.Confomaton: A conference enhancer with social media from the cloud. In 9th ExtendedSemantic Web Conference (ESWC 2012), May 27-31, 2012, Heraklion, Crete16. Houda Khrouf, Ghislain Atemezing, Giuseppe Rizzo, Raphaël Troncy and Thomas Steiner.Aggregating social media for enhancing conference experience. In 1st International Workshopon Real-Time Analysis and Mining of Social Streams (RAMSS 2012), June 4, 2012, Dublin,Ireland17. Symeon Papadopoulos, Emmanouil Schinas, Vasileios Mezaris, Raphaël Troncy and IoannisKompatsiaris. Social Event Detection at MediaEval 2012: Challenges, Dataset and Evaluation. InMediaEval Benchmarking Initiative for Multimedia Evaluation, CEUR Proceedings Vol. 927, Pisa,Italy, October 4-5, 2012.18. Sven Buschbeck, Anthony Jameson, Raphaël Troncy, Houda Khrouf, Osma Suominen andAdrian Spirescu. A Demonstrator for Parallel Faceted Browsing. In (EKAW) IntelligentExploration of Semantic Data Workshop (IESD 2012), Winner of the IESD Challenge, Galway,Ireland, October 8-12, 2012.19. Marieke van Erp, Laura Hollink, Willem Robert van Hage, Raphaël Troncy and David A. Shamma.Proceedings of the Second International Workshop on Detection, Representation, andExploitation of Events in the Semantic Web (DeRiVE 2012), CEUR Proceedings Vol. 902, editors,Boston, USA, November 12, 2012.04/06/2013 - ALIAS Final Meeting - Munich, Germany - 61
Publications 4/420. Houda Khrouf, Vuk Milicic and Raphaël Troncy. EventMedia Live: Exploring Events Connectionsin Real-Time to Enhance Content. In 11th International Semantic Web Conference (ISWC 2012),First Prize Winner of the Semantic Web Challenge, Boston, USA, November 11-15, 2012.21. Xueliang Liu and Benoit Huet. Gathering training sample automatically for social event visualmodeling. ACM International Workshop on Socially Aware Multimedia (SAM 2012), inconjunction with ACM Multimedia 2012, 29 October 2012, Nara, Japan22. Symeon Papadopoulos, Emmanouil Schinas, Vasileios Mezaris, Raphaël Troncy and IoannisKompatsiaris. The 2012 Social Event Detection Dataset. In ACM Multimedia Systems (MMSys2013), Dataset Session, Oslo, Norway, February 27-March 1, 201323. Sven Buschbeck, Raphaël Troncy, Anthony Jameson, Houda Khrouf, Adrian Spirescu, OsmaSuominen, Tanja Schneeberger and Eero Hyvönen. Parallel Faceted Browsing. In ACMConference on Human Factors in Computing Systems (CHI 2013), Interactivity Track, Paris,France, April 27-May 2, 201324. Xueliang Liu and Benoit Huet. EventEnricher: a novel way to collect media illustrating events.ICMR 2013, ACM International Conference on Multimedia Retrieval, April 16-20, 2013, Dallas,Texas, USA25. Xueliang Liu and Benoit Huet. Heterogeneous features and model selection for event-basedmedia classification. ICMR 2013, ACM International Conference on Multimedia Retrieval, April16-20, 2013, Dallas, Texas, USA26. Xueliang and Benoit Huet. On the automatic online collection of training data for visual eventmodeling. Multimedia Tools and Applications, February 2013, ISSN: 1380-750104/06/2013 - ALIAS Final Meeting - Munich, Germany - 62
http://www.slideshare.net/troncy04/06/2013 - ALIAS Final Meeting - Munich, Germany - 63
Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...
In this presentation we will describe our experience developing with a highly dyna...
Presentation to the LITA Forum 7th November 2014 Albuquerque, NM
Un recorrido por los cambios que nos generará el wearabletech en el futuro
Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...
Next: Introducing Graph Data. What is Linked Data and the Semantic Web and what is all the hype about? Principally, the Semantic Web is a Web 3.0 web ...
Semantic Web Company is a major provider of services and products built on top of semantic web standards: semantic search, linked data ... web technologies ...
Semantic Web Company and its ... uniquely and uniformly defined structure. With “RDF Data ... Machine learning algorithms benefit from the linking of ...
He defines the Semantic Web as "a web of data that can be processed directly ... to describe the structure of the knowledge we ... Social Semantic Web; Web ...
... of the Web of linked data. Semantic Web technologies enable ... and linking your data. ... Semantic Web technologies will bridge many ...
... Data is about using the Web to connect related data that wasn't previously linked, or using the Web to lower the barriers to linking data ... Semantic ...