advertisement

Social Tags and Music Information Retrieval (Part II)

100 %
0 %
advertisement
Information about Social Tags and Music Information Retrieval (Part II)
Technology

Published on September 12, 2008

Author: plamere

Source: slideshare.net

Description

Part 2 of the slides for the Social Tags and Music Information Retrieval Tutorial - Abstract: Social Tags are free text labels that are applied to items such as artists, playlists and songs. These tags have the potential to have a positive impact on music information retrieval research. In this tutorial we describe the state of the art in commercial and research social tagging systems for music. We explore some of the motivations for tagging. We describe the factors that affect the quantity and quality of collected tags. We present a toolkit that MIR researchers can use to harvest and process tags. We look at how tags are collected and used in current commercial and research systems. We explore some of the issues and problems that are encountered when using tags. We present current MIR-related research centered on social tags and suggest possible areas of exploration for future resear
advertisement

Social Tags and Music Information Retrieval Part II ISMIR 2008 Paul Lamere – Sun Microsystems Inc. Elias Pampalk – Last.fm

ISMIR 2008

Paul Lamere – Sun Microsystems Inc.

Elias Pampalk – Last.fm

Outline What are social tags? Why do people tag? Issues with social tags Other sources of tags (continued) Search, Discovery & Recommendation Data & Tools Future Research Conclusion Discussion

What are social tags?

Why do people tag?

Issues with social tags

Other sources of tags (continued)

Search, Discovery & Recommendation

Data & Tools

Future Research

Conclusion

Discussion

Other sources of tags Autotagging Uses content analysis to automatically apply tags Tags acquired from other sources (social tags, games, web crawling) can be 'learned' New music or unpopular music can be autotagged with the 'learned' tags. Can scale to the long tail Time per million songs: Manual: with 100 people = 3 Years Automatic: with 100 CPUs = 8 Hours Cost per million songs Manual: ~ $10,000,000 Automatic: ~ $100

Uses content analysis to automatically apply tags

Tags acquired from other sources (social tags, games, web crawling) can be 'learned'

New music or unpopular music can be autotagged with the 'learned' tags.

Can scale to the long tail

Time per million songs:

Manual: with 100 people = 3 Years

Automatic: with 100 CPUs = 8 Hours

Cost per million songs

Manual: ~ $10,000,000

Automatic: ~ $100

Other sources of tags: Autotagging How it works Labeled Examples Unknown Examples Machine Learning Model Labeled Examples Feature Extraction Training Tagging

Other sources of tags: Autotagging Ground Truth Standard (genre) classification each item: 1 category/class/tag annotated by a trusted expert Social tags each item: unlimited (weighted) categories/class/tag annotated by an anonymous crowd

Standard (genre) classification

each item: 1 category/class/tag

annotated by a trusted expert

Social tags

each item: unlimited (weighted) categories/class/tag

annotated by an anonymous crowd

Other sources of tags: Autotagging Ground Truth for Classifiers [Craft, Wiggens, Crawford, ISMIR 2007]

Other sources of tags: Autotagging Ground Truth for Classifiers [Paul Lamere, JNMR 2008]

Autotagging: Ground truth & evaluation MIREX 2008: Tag Track Data MajorMiner game (Mandel & Ellis) 2300 audio clips (10 second) from 1400 tracks from 500 artists (artist-filtering used to ensure training and test don't have different sets of artists) 43 tags verified at least 35 times each (drums, guitar, male, rock, ...)

Data

MajorMiner game (Mandel & Ellis)

2300 audio clips (10 second) from 1400 tracks from 500 artists (artist-filtering used to ensure training and test don't have different sets of artists)

43 tags verified at least 35 times each (drums, guitar, male, rock, ...)

Autotagging: Ground truth & evaluation MIREX 2008: Tag Track Tasks Clip -> tag (yes/no) Clip -> list of tags (ranked) Tag -> list of clips (ranked) Standardized evaluation procedures Statistical significance etc http://www.music-ir.org/mirex/2008/index.php/Audio_Tag_Classification

Tasks

Clip -> tag (yes/no)

Clip -> list of tags (ranked)

Tag -> list of clips (ranked)

Standardized evaluation procedures

Statistical significance etc

http://www.music-ir.org/mirex/2008/index.php/Audio_Tag_Classification

Autotagging LabROSA Features: MFCCs + temporal features Learning: Support Vector Machine with a radial basis function kernel Compared ability to learn 'game' tags vs. social tags M. I. Mandel and D. P. W. Ellis. A Web-Based Game for Collecting Music Metadata. In Journal of New Music Research, 2008 (to appear).

Features: MFCCs + temporal features

Learning: Support Vector Machine with a radial basis function kernel

Compared ability to learn 'game' tags vs. social tags

http://majorminer.com/search Other sources of tags: Autotagging LabROSA

T. Bertin-Mahiex, D. Eck, F. Maillet, and P. Lamere. Autotagger: A model for predicting social tags from acoustic features on large music databases. In Journal of New Music Research, 2008 (to appear). Other sources of tags: Autotagging BRAMS, Sun Labs

[Bertin-Mahiex et al. JNMR 2008] Features: MFCCs + Temporal Learning: AdaBoost and FilterBoost Medium scale: 100,000 tracks "Large and Noise" vs. "Small and Clean" There's no data like more data Other sources of tags: Autotagging BRAMS, Sun Labs

Features: MFCCs + Temporal

Learning: AdaBoost and FilterBoost

Medium scale: 100,000 tracks

"Large and Noise" vs. "Small and Clean" There's no data like more data

Other sources of tags: Autotagging BRAMS, Sun Labs

D. Turnbull, L. Barrington, D. Torres, and G. Lanckriet. Towards musical query-by-semantic description using the CAL500 data set. SIGIR 2007. Other sources of tags: Autotagging CAL UCSD

Other sources of tags: Autotagging Challenges Not all tags are can be easily derived from audio Examples: seen live, great lyrics, a wesome, crap, Boston, Montreal, UK Can we identify words that are musically meaningful? Some tags may be too subtle for current systems to distinguish Power Metal vs. Speed Metal Grunge vs. Post-Grunge Dealing with co-occurring tags Dealing with weak labeling – not every song with piano is labeled piano Scale

Not all tags are can be easily derived from audio

Examples: seen live, great lyrics, a wesome, crap, Boston, Montreal, UK

Can we identify words that are musically meaningful?

Some tags may be too subtle for current systems to distinguish

Power Metal vs. Speed Metal

Grunge vs. Post-Grunge

Dealing with co-occurring tags

Dealing with weak labeling – not every song with piano is labeled piano

Scale

Outline What are social tags? Why do people tag? Issues with social tags Other sources of tags Search, Discovery & Recommendation Data & Tools Future Research Conclusion Discussion

What are social tags?

Why do people tag?

Issues with social tags

Other sources of tags

Search, Discovery & Recommendation

Data & Tools

Future Research

Conclusion

Discussion

Search & Discovery

Search & Discovery The Vocabulary Problem When searching – people will often use the wrong words. Should I search for "rnb", "r and b", "r&b", or "rnb" ? With well tagged items – it doesn't matter http://www.last.fm/music/Rihanna/+tags

When searching – people will often use the wrong words.

Should I search for "rnb", "r and b", "r&b", or "rnb" ?

With well tagged items – it doesn't matter

Search & Discovery The Vocabulary Problem What about infrequently tagged items? We can use overall tag overlap to infer synonymy Cluster tags via overlap, tf-idf or other similarity metric Augment query with synonyms Not just for synonyms can deal with spelling errors (“rithm and blues”) can help with multi-lingual tags

What about infrequently tagged items?

We can use overall tag overlap to infer synonymy

Cluster tags

via overlap, tf-idf or other similarity metric

Augment query with synonyms

Not just for synonyms

can deal with spelling errors (“rithm and blues”)

can help with multi-lingual tags

The Vocabulary Problem Using tag clustering to combat synonymy The 'female' cluster of a 2,000 node tag hierarchy

The Vocabulary Problem Multiple languages Similar tags to deutscher hiphop German hip-hop Deutscher Hip Hop Hamburg Eimbush Deutschrap Deutsch Deutschsprachig Hiphop German rap German

Similar tags to deutscher hiphop

German hip-hop

Deutscher Hip Hop

Hamburg

Eimbush

Deutschrap

Deutsch

Deutschsprachig

Hiphop

German rap

German

The Vocabulary Problem Using item clustering to combat polysemy

The Vocabulary Problem Using Latent Semantic Analysis Female Singing Unstructured/Unreliable representation Latent Semantic Model Can address: Synonym, Polysemy Noise Dimensionality reduction

Can address:

Synonym, Polysemy

Noise

Dimensionality reduction

Latent Semantic Analysis Query Latent Semantic Space Search Project query into semantic space Best match may not have any tags that match the query! See: Levy & Sandler, Learning Latent Semantic Models for Music from Social Tags . JNMR 2008. Girl, GirlBand, Grrl, Girly Female Singer Female Female Vocalists Diva Chicks Woman Pop American Idol Pop RnB Rhythm & Blues Girl Pop LSA

Project query into semantic space

Best match may not have any tags that match the query!

Similarity of text Weight the terms Some terms are more meaningful than others: rock vs. shoegaze TF x IDF Distance between term vectors Cosine distance the cosine of the angle between the two vectors Independent of length of vectors Search, Discovery & Recommendation Item Similarity

Similarity of text

Weight the terms

Some terms are more meaningful than others: rock vs. shoegaze

TF x IDF

Distance between term vectors

Cosine distance

the cosine of the angle between the two vectors

Independent of length of vectors

Artist similarity based on tags for Weezer Top Tags Alternative Rock Indie Punk Pop Power pop Geek Rock 90s Metal Indie pop Distinctive Tags Geek Rock Punk-pop College Rock Not Emo Overrated Los Angeles Modern Rock Pop punk California Pop Rock Similar Artists via Tags Green Day Phantom Planet The Offspring Sugarcult Foo Fighters blink-182 Ozma Jimmy Eat World Nada Surf The Ataris Similar Artists via CF The White Stripes Foo Fighters Death Cab For Cutie Beck Radiohead Green Day Coldplay The Beatles The Killers The Smashing Pumpkins Web survey: Tag-based artist similarity scores better than CF-based similarity Search, Discovery & Recommendation Artist Similarity

Artist similarity based on tags for Weezer

Top Tags

Alternative

Rock

Indie

Punk

Pop

Power pop

Geek Rock

90s

Metal

Indie pop

Distinctive Tags

Geek Rock

Punk-pop

College Rock

Not Emo

Overrated

Los Angeles

Modern Rock

Pop punk

California

Pop Rock

Similar Artists via Tags

Green Day

Phantom Planet

The Offspring

Sugarcult

Foo Fighters

blink-182

Ozma

Jimmy Eat World

Nada Surf

The Ataris

Similar Artists via CF

The White Stripes

Foo Fighters

Death Cab For Cutie

Beck

Radiohead

Green Day

Coldplay

The Beatles

The Killers

The Smashing Pumpkins

Tag similarity based on artists Metal Metallica System of a down Iron Maiden Rammstein Slipknot In Flames Korn Pantera Judas Priest Heavy Metal Iron Maiden Judas Priest Black Sabbath Manowar Motorhead Pantera Megadeth Ozzy Osbourne Dio Pop Madonna The Beatles Black Eyed Peas Beach Boys Kelly Clarkson Michael Jackson Gwen Stefani Coldplay U2 Search, Discovery & Recommendation Tag Similarity

Tag similarity based on artists

Metal

Metallica

System of a down

Iron Maiden

Rammstein

Slipknot

In Flames

Korn

Pantera

Judas Priest

Heavy Metal

Iron Maiden

Judas Priest

Black Sabbath

Manowar

Motorhead

Pantera

Megadeth

Ozzy Osbourne

Dio

Pop

Madonna

The Beatles

Black Eyed Peas

Beach Boys

Kelly Clarkson

Michael Jackson

Gwen Stefani

Coldplay

U2

Tag similarity examples Similar tags to relax Relaxing Calm Chill Meditation Spiritual Chill out Soft Dreamy New age Mellow Search, Discovery & Recommendation Tag Similarity Similar tags to deutscher hiphop German hip-hop Deutscher Hip Hop Hamburg Eimbush Deutschrap Deutsch Deutschsprachig Hiphop German rap German Similar tags to turntablism dj abstract hip-hop scratch instrumental hip-hop beats ninja tune ambient breakbeat hip hop instrumental hip hop underground hip hop Similar tags to metal heavy metal death metal thrash metal hard rock progressive metal metalcore power metal melodic death metal gothic metal hardcore

Tag similarity examples

Similar tags to relax

Relaxing

Calm

Chill

Meditation

Spiritual

Chill out

Soft

Dreamy

New age

Mellow

Similar tags to deutscher hiphop

German hip-hop

Deutscher Hip Hop

Hamburg

Eimbush

Deutschrap

Deutsch

Deutschsprachig

Hiphop

German rap

German

Similar tags to turntablism

dj

abstract hip-hop

scratch

instrumental hip-hop

beats

ninja tune

ambient breakbeat

hip hop

instrumental hip hop

underground hip hop

Similar tags to metal

heavy metal

death metal

thrash metal

hard rock

progressive metal

metalcore

power metal

melodic death metal

gothic metal

hardcore

Search, Discovery & Recommendation User Similarity Paul (green) vs Elias (red) http://anthony.liekens.net/

Paul (green) vs Elias (red)

http://anthony.liekens.net/

Discovery & Recommendation Goal: Recreate Ishkur's Guide to EDM

Discovery & Recommendation From Folksonomy to Taxonomy

Metal Discovery & Recommendation From Folksonomy to Taxonomy

Discovery & Recommendation Creating an Artist Hierarchy Attachment order: Artist Popularity

Attachment order: Artist Date Discovery & Recommendation Creating an Artist Hierarchy

Discovery & Recommendation Creating a Personal Artist Hierarchy

Discovery & Recommendation Transparency - Explaura

Discovery & Recommendation Transparency - Explaura

2) Drag a tag to make it bigger or smaller. Discovery & Recommendation Steerable recommendations - Explaura 1) Click to add a tag or artist 3) Receive recommendations that match the tag cloud 4) Get an explanation for each recommendation

Discovery & Recommendation Transparency - Pandora http://flickr.com/photos/libraryman/1225285863/

http://flickr.com/photos/libraryman/1225285863/

Discovery & Recommendation Transparency - Pandora Technology loosing it's "cold" Pandora feels like a smart friend to me. This friend can articulate the reasons I love some of the things I love most (songs) better than I can, but only because I have told it what I like. This is one of my very favorite Prince songs and Pandora knows just why I like it so much. And I didn't know how to say it so well. Makes the technology seem very warm and reflective of my feelings an identity. It's an extension of the user, not a cold, isolating technology. I feel a part of Pandora some times. http://flickr.com/photos/libraryman/1225285863/

Technology loosing it's "cold"

Pandora feels like a smart friend to me. This friend can articulate the reasons I love some of the things I love most (songs) better than I can, but only because I have told it what I like. This is one of my very favorite Prince songs and Pandora knows just why I like it so much. And I didn't know how to say it so well. Makes the technology seem very warm and reflective of my feelings an identity. It's an extension of the user, not a cold, isolating technology. I feel a part of Pandora some times.

http://flickr.com/photos/libraryman/1225285863/

Discovery & Recommendation Transparency – Musical MadLibs Norah Jones - Don’t Know Why Summary generated automatically using SML model: This is soft rock , jazz song that is mellow and sad . It features piano , synthesizer , ambient sounds , and monotone , breathy vocals . It is a song with a slow tempo and with low energy that you might like to listen to while studying . [Turnbull, Liu, Barrington & Lanckriet, ISMIR 2007]

Norah Jones - Don’t Know Why

Summary generated automatically using SML model:

This is soft rock , jazz song that is mellow and sad . It features piano , synthesizer , ambient sounds , and monotone , breathy vocals . It is a song with a slow tempo and with low energy that you might like to listen to while studying .

[Turnbull, Liu, Barrington & Lanckriet, ISMIR 2007]

Discovery & Recommendation Cold start: new users Need 'taste data' for new users Results: Awkward user enrollment Poor recommendations Solution Represent taste 'portably' Taste data can move with the user Several efforts Attention Profile Markup Language - APML OpenTaste

Need 'taste data' for new users

Results:

Awkward user enrollment

Poor recommendations

Solution

Represent taste 'portably'

Taste data can move with the user

Several efforts

Attention Profile Markup Language - APML

OpenTaste

Attention Profile Markup Language <APML> <Profile name=&quot;music&quot;> <ImplicitData> <Concepts> <Concept key=&quot; piano &quot; value =&quot;.81&quot;/> <Concept key=&quot; female vocalists &quot; value =&quot;.72&quot;/> <Concept key=&quot; indie pop &quot; value =&quot;.65&quot;/> <Concept key=&quot; art rock &quot; value =&quot;.60&quot;/> <Concept key=&quot; experimental &quot; value =&quot;.57&quot;/> <Concept key=&quot; great lyricist &quot; value =&quot;.55&quot;/> <Concept key=&quot; noise &quot; value =&quot;.35&quot;/> </Concepts> </ImplicitData> </Profile> </APML> Turn this Into this Then this APML

Discovery & Recommendation Last.fm Products using Tag Data

Search and Discovery Last.fm Playground http://playground.last.fm/multitag

Search and Discovery Last.fm Playground http://playground.last.fm/multitag Multi Tag Search Categories Popular Up-and-coming Free downloads Example queries pop british 90s sad piano catchy funny soundtrack hilarious cover &quot;one hit wonder&quot; 90s american &quot;guilty pleasure&quot; happy sad relaxing “speed metal” More information : Late breaking/demo session Thursday 11.00 - 13.00 Klaas Bosteels

Multi Tag Search

Categories

Popular

Up-and-coming

Free downloads

Example queries

pop british 90s sad piano

catchy funny soundtrack

hilarious cover

&quot;one hit wonder&quot; 90s

american &quot;guilty pleasure&quot;

happy sad

relaxing “speed metal”

Last.fm Playground: Islands of Music Clustering listeners by tags

Last.fm Playground: Islands of Music Clustering listeners by tags 13k randomly sampled Last.fm listeners Each listener represented by a tag cloud 2000 dimensions (tags) -> 120 dimensions (SVD) k-means clustering to extract 400 prototypical listeners Self-organizing map (20x40) http://playground.last.fm/iom

13k randomly sampled Last.fm listeners

Each listener represented by a tag cloud

2000 dimensions (tags) -> 120 dimensions (SVD)

k-means clustering to extract 400 prototypical listeners

Self-organizing map (20x40)

http://playground.last.fm/iom

Search, Discovery & Recommendations Vocabulary problem Item similarity Hierarchical clustering Transparency User cold start and APML Last.fm

Vocabulary problem

Item similarity

Hierarchical clustering

Transparency

User cold start and APML

Last.fm

Outline What are social tags? Why do people tag? Issues with social tags Other sources of tags Search, Discovery & Recommendation Data & Tools Future Research Conclusion Discussion

What are social tags?

Why do people tag?

Issues with social tags

Other sources of tags

Search, Discovery & Recommendation

Data & Tools

Future Research

Conclusion

Discussion

Data and Tools Pointers to all data at SocialMusicResearch.org/data Expert/Survey data CAL -500 http://cosmal.ucsd.edu/cal/projects/AnnRet/AnnRet.php 1700 human generated musical annotations 500 popular western tracks All Music Available through commercial license Genre, Styles and Moods for thousands of artists

Pointers to all data at SocialMusicResearch.org/data

Expert/Survey data

CAL -500

http://cosmal.ucsd.edu/cal/projects/AnnRet/AnnRet.php

1700 human generated musical annotations

500 popular western tracks

All Music

Available through commercial license

Genre, Styles and Moods for thousands of artists

Data and Tools Game Data ListenGame 26,000 annotations 250 songs 120 words 440 unique players Available upon request from Doug Turnbull MajorMiner Available for browsing at: http://majorminer.com/search human tags autotags MIREX 2008

Game Data

ListenGame

26,000 annotations

250 songs

120 words

440 unique players

Available upon request from Doug Turnbull

MajorMiner

Available for browsing at:

http://majorminer.com/search

human tags

autotags

MIREX 2008

Data and Tools Social tags LastFM-ArtistTags2007 Tag data for over 20,000 artists labs.strands.com http://labs.strands.com/music/affinity/ Playlist co-occurrence data Artist-Tag data for 4,000 artists

Social tags

LastFM-ArtistTags2007

Tag data for over 20,000 artists

labs.strands.com

http://labs.strands.com/music/affinity/

Playlist co-occurrence data

Artist-Tag data for 4,000 artists

Data and Tools TagWorks Loads tag data from LastFM-ArtistTags2007 Simple-overlap similarity tf-idf / cosine distance similarity Agglomerative clustering of tags, artists, users Folksonomy -> Taxonomy algorithm tags, artists, users Last.fm crawler for user-tags Written in the Java programming language Uses the Minion search engine: https://minion.dev.java.net/ Available at: SocialMusicResearch.org/code

Loads tag data from LastFM-ArtistTags2007

Simple-overlap similarity

tf-idf / cosine distance similarity

Agglomerative clustering of

tags, artists, users

Folksonomy -> Taxonomy algorithm

tags, artists, users

Last.fm crawler for user-tags

Written in the Java programming language

Uses the Minion search engine: https://minion.dev.java.net/

Available at: SocialMusicResearch.org/code

Data and Tools Last.fm API 1.0 Tags applied by a user Tags a user applied to specific artist/album/track Tag clouds for artist/track or overall Top artists/album/tracks for given tag Creative Commons Attribution-Non Commercial-Share Alike License http://www.audioscrobbler.net/data/webservices/

Tags applied by a user

Tags a user applied to specific artist/album/track

Tag clouds for artist/track or overall

Top artists/album/tracks for given tag

Creative Commons Attribution-Non Commercial-Share Alike License

http://www.audioscrobbler.net/data/webservices/

Data and Tools Last.fm API 1.0 Examples Social Tags for any MP3 (~ 70 lines of Python code) Given MP3 file (without ID3) Use Last.fm Fingerprinter to obtain ID3 info Use ID3 info to obtain tags applied by Last.fm community Tagger age vs vocabulary size (~ 85 lines of Python code) Start with one or more seed users Get friends, of friends, of friends (-> large list of users) For each user get tag applied, and demographic information (age) Analyze data (tagger's vocabulary vs age) http://SocialM usicR esearch.org / code

Social Tags for any MP3 (~ 70 lines of Python code)

Given MP3 file (without ID3)

Use Last.fm Fingerprinter to obtain ID3 info

Use ID3 info to obtain tags applied by Last.fm community

Tagger age vs vocabulary size (~ 85 lines of Python code)

Start with one or more seed users

Get friends, of friends, of friends (-> large list of users)

For each user get tag applied, and demographic information (age)

Analyze data (tagger's vocabulary vs age)

Data and Tools Last.fm API 2.0 user: add , remove or get tags for an artist/a lbum/track get similar tags for given tag get top albums/artists/tracks for given tag tag clouds for artist/track/overall search for a tag tags applied by a user http://www.last.fm/api

user: add , remove or get tags for an artist/a lbum/track

get similar tags for given tag

get top albums/artists/tracks for given tag

tag clouds for artist/track/overall

search for a tag

tags applied by a user

http://www.last.fm/api

 

Outline What are social tags? Why do people tag? Issues with social tags Other sources of tags Search, Discovery & Recommendation Data & Tools Future Research Conclusion Discussion

What are social tags?

Why do people tag?

Issues with social tags

Other sources of tags

Search, Discovery & Recommendation

Data & Tools

Future Research

Conclusion

Discussion

Future Research Librarians, musicologists, anthropologists What are tags? Who tags? Why do people tag? How do people use tags? Is tagging behaviour and tag usage different for different demographics? Can music tags be mapped to taxonomies? Impact on Digital libraries Organizing and discovering music? Musicology Perceiving, describing, categorizing and talking about music? Anthropology Emergence of new subcultures?

What are tags? Who tags? Why do people tag? How do people use tags?

Is tagging behaviour and tag usage different for different demographics?

Can music tags be mapped to taxonomies?

Impact on

Digital libraries

Organizing and discovering music?

Musicology

Perceiving, describing, categorizing and talking about music?

Anthropology

Emergence of new subcultures?

Future Research Can machines play tag? Signal processing & statistical learning Preliminary results look very promising but ... What are the limitations and how far can we push them? Quality in general? What types of tags that cannot be learned? Integration of machine learned tags in human interfaces? Plenty of training data available! And new fun challenges (data sparsity, noise, ...)

Preliminary results look very promising but ...

What are the limitations and how far can we push them?

Quality in general?

What types of tags that cannot be learned?

Integration of machine learned tags in human interfaces?

Plenty of training data available!

And new fun challenges (data sparsity, noise, ...)

Future Research User experience and UI design Tagging How do suggestions impact quality of tags? Can suggestions be improved? Social interaction through tags Can how people communicate with tags be improved? Discovery/browsing/steering recommendations Beyond tag clouds? Are tags a better way to organize playlists?

Tagging

How do suggestions impact quality of tags?

Can suggestions be improved?

Social interaction through tags

Can how people communicate with tags be improved?

Discovery/browsing/steering recommendations

Beyond tag clouds?

Are tags a better way to organize playlists?

Future Research More Open Questions What can we learn about a user from her tagging behaviour? Tag games How do game tags differ from social tags? How can you make games even more fun? Computation of similarity Combination of tags with other data sources? Transparent recommendations Using tags to explain recommendations?

What can we learn about a user from her tagging behaviour?

Tag games

How do game tags differ from social tags?

How can you make games even more fun?

Computation of similarity

Combination of tags with other data sources?

Transparent recommendations

Using tags to explain recommendations?

Future Research More Open Questions (2) Semantic web Connecting information and tags across the web? E.g. Flickr & Last.fm MOAT? Internationalization? Different communities tagging the same item with different languages? (Japanese vs Russian vs English) “World” music? Social tagging of other musical entities? Detecting tag abuse?

Semantic web

Connecting information and tags across the web?

E.g. Flickr & Last.fm

MOAT?

Internationalization?

Different communities tagging the same item with different languages? (Japanese vs Russian vs English)

“World” music?

Social tagging of other musical entities?

Detecting tag abuse?

Future Research Tag Gardening Tools for improving tags Weeding Seeding Landscaping Fertilizing TagCare MOAT

Tools for improving tags

Weeding

Seeding

Landscaping

Fertilizing

TagCare

MOAT

Future Research Learn from Japanese Style Tagging? http://joilab.ito.com/2008/02/nico-nico-douga.html http://www.nicovideo.jp/watch/sm9 User generated content Mash-ups Community Tags as art form Business model

User generated content

Mash-ups

Community

Tags as art form

Business model

Poster Session 2c: Mon 16.00 – 18.00 Ternary Semantic Analysis of Social Tags for Personalized Music Recommendation P. Symeonidis, M. Ruxanda, A. Nanopoulos and Y. Manolopoulos Five Approaches to Collecting Tags for Music D. Turnbull, L. Barrington and G. Lanckriet MoodSwings: A Collaborative Game for Music Mood Label Collection Y. Kim, E. Schmidt and L. Emelle ISMIR 2008

Poster Session 2c: Mon 16.00 – 18.00

Ternary Semantic Analysis of Social Tags for Personalized Music Recommendation

P. Symeonidis, M. Ruxanda, A. Nanopoulos and Y. Manolopoulos

Five Approaches to Collecting Tags for Music

D. Turnbull, L. Barrington and G. Lanckriet

MoodSwings: A Collaborative Game for Music Mood Label Collection

Y. Kim, E. Schmidt and L. Emelle

Poster Session 2c: Mon 16.00 – 18.00 Collective Annotation of Music From Multiple Semantic Categories Z. Duan, L. Lu and C. Zhang Connecting the Dots: Music Metadata Generation, Schemas and Applications (*) N. Corthaut, S. Govaerts, K. Verbert and E. Duval The Quest for Musical Genres: Do the Experts and the Wisdom of Crowds Agree? (*) M. Sordo, O. Celma, M. Blech and E. Guaus ISMIR 2008

Poster Session 2c: Mon 16.00 – 18.00

Collective Annotation of Music From Multiple Semantic Categories

Z. Duan, L. Lu and C. Zhang

Connecting the Dots: Music Metadata Generation, Schemas and Applications (*)

N. Corthaut, S. Govaerts, K. Verbert and E. Duval

The Quest for Musical Genres: Do the Experts and the Wisdom of Crowds Agree? (*)

M. Sordo, O. Celma, M. Blech and E. Guaus

Poster Session 2d: Mon 16.00 – 18.00 A Web of Musical Information Y. Raimond and M. Sandler Uncovering Affinity of Artists to Multiple Genres From Social Behaviour Data C. Baccigalupo, J. Donaldson and E. Plaza Oh Oh Oh Whoah! Towards Automatic Topic Detection in Song Lyrics F. Kleedorfer, P. Knees and T. Pohle ISMIR 2008

Poster Session 2d: Mon 16.00 – 18.00

A Web of Musical Information

Y. Raimond and M. Sandler

Uncovering Affinity of Artists to Multiple Genres From Social Behaviour Data

C. Baccigalupo, J. Donaldson and E. Plaza

Oh Oh Oh Whoah! Towards Automatic Topic Detection in Song Lyrics

F. Kleedorfer, P. Knees and T. Pohle

Poster Session 3a: Tue 11.00 – 13.00 Multi-Label Classification of Music Into Emotions K. Trohidis, G. Tsoumakas, G. Kalliris and I. Vlahavas Poster Session 5a: Wed 11.00 – 13.00 Multiple-Instance Learning for Music Information Retrieval M. Mandel and D. Ellis MIREX Poster Session: Wed 16.00 – 18.00 Audio Tag Classification M. Mandel, T. Bertin-Mahieux, G. Tsoumakas, D. Turnbull & L. Barrington, G. Peeters ISMIR 2008

Poster Session 3a: Tue 11.00 – 13.00

Multi-Label Classification of Music Into Emotions

K. Trohidis, G. Tsoumakas, G. Kalliris and I. Vlahavas

Poster Session 5a: Wed 11.00 – 13.00

Multiple-Instance Learning for Music Information Retrieval

M. Mandel and D. Ellis

MIREX Poster Session: Wed 16.00 – 18.00

Audio Tag Classification

M. Mandel, T. Bertin-Mahieux, G. Tsoumakas, D. Turnbull & L. Barrington, G. Peeters

Late-Breaking / Demo Session: Thu 11.00 – 13.00 Music Retrieval Based on Social Tags: A Case Study K. Bosteels, E. Kerre, and E. Pampalk Herd the Music - A Social Music Annotation Game L. Barrington, D. O’Malley, D. Turnbull, and G. Lanckriet Music and Lyrics: Can Lyrics Improve Emotion Estimation for Music? Daniel C. Wu Jr et al. MOODY: A Web-Based Music Mood Classification and Recommendation System Xiao Hu et al. Creating Transparent, Steerable Recommendations P. Lamere and F. Maillet ISMIR 2008

Late-Breaking / Demo Session: Thu 11.00 – 13.00

Music Retrieval Based on Social Tags: A Case Study

K. Bosteels, E. Kerre, and E. Pampalk

Herd the Music - A Social Music Annotation Game

L. Barrington, D. O’Malley, D. Turnbull, and G. Lanckriet

Music and Lyrics: Can Lyrics Improve Emotion Estimation for Music?

Daniel C. Wu Jr et al.

MOODY: A Web-Based Music Mood Classification and Recommendation System

Xiao Hu et al.

Creating Transparent, Steerable Recommendations

P. Lamere and F. Maillet

From genres to tags: Music information retrieval in the age of social tagging Editors: J.-J. Aucouturier and E. Pampalk Scanning the Dial: The Rapid Recognition of Music Genres R. O. Gjerdingen and D. Perrott Social Tagging and Music Information Retrieval P. Lamere Autotagger a Model for Predicting Social Tags from Acoustic Features T. Bertin- Mahieux, D. Eck, F. Maillet and P. Lamere Learning Latent Semantic Models for Music M. Levy and M. Sandler A Web-Based Game for Collecting Music Metadata, M. Mandel and D. Ellis Journal of New Music Research Special issue to appear in 2008

From genres to tags: Music information retrieval in the age of social tagging

Editors: J.-J. Aucouturier and E. Pampalk

Scanning the Dial: The Rapid Recognition of Music Genres

R. O. Gjerdingen and D. Perrott

Social Tagging and Music Information Retrieval

P. Lamere

Autotagger a Model for Predicting Social Tags from Acoustic Features

T. Bertin- Mahieux, D. Eck, F. Maillet and P. Lamere

Learning Latent Semantic Models for Music

M. Levy and M. Sandler

A Web-Based Game for Collecting Music Metadata,

M. Mandel and D. Ellis

Outline What are social tags? Why do people tag? Issues with social tags Other sources of tags Search, Discovery & Recommendation Data & Tools Future Research Conclusion Discussion

What are social tags?

Why do people tag?

Issues with social tags

Other sources of tags

Search, Discovery & Recommendation

Data & Tools

Future Research

Conclusion

Discussion

Conclusions SocialMusicResearch.org Wiki Slides Data Code Bibliography You can participate Participate

Wiki

Slides

Data

Code

Bibliography

You can participate

The Fundamental Theorem of Music Informatics Music is created by humans for other humans, and humans can bring a tremendous amount of contextual knowledge to bear on anything they do; in fact, they can't avoid it, and they're rarely conscious of it. But computers can never bring much contextual knowledge to bear, often none at all, and never without being specifically programmed to do so. Therefore doing almost anything with music by computers is very difficult ;many problems are essentially intractable . -- Don Byrd, January 2008 (I wrote the above statement, which I described with tongue firmly in cheek as a &quot;theorem&quot; -- a more accurate term would be &quot;axiom&quot; or &quot;dogma&quot; -- for my Spring 2008 graduate seminar, Organization and Searching of Musical Information . It's probably a bit too strongly worded, but I think it really comes close to describing a very basic problem that most music-informatics research has to deal with.) --Don Byrd, September 2008

Music is created by humans for other humans, and humans can bring a tremendous amount of contextual knowledge to bear on anything they do; in fact, they can't avoid it, and they're rarely conscious of it. But computers can never bring much contextual knowledge to bear, often none at all, and never without being specifically programmed to do so. Therefore doing almost anything with music by computers is very difficult ;many problems are essentially intractable . -- Don Byrd, January 2008

Acknowledgments

Outline What are social tags? Why do people tag? Issues with social tags Other sources of tags Search, Discovery & Recommendation Data & Tools Future Research Conclusion Discussion

What are social tags?

Why do people tag?

Issues with social tags

Other sources of tags

Search, Discovery & Recommendation

Data & Tools

Future Research

Conclusion

Discussion

Bibliography http://SocialMusicResearch.org C. Baccigalupo, E. Plaza, J. Donaldson. Uncovering affinity of artists to multiple genres from social behaviour data. ISMIR 2008. L. Barrington, D. O’Malley, D. Turnbull, and G. Lanckriet. Herd the Music - A Social Music Annotation Game. ISMIR 2008. T. Bertin-Mahieux, D. Eck, F. Maillet, and P. Lamere. Autotagger: A model for predicting social tags from acoustic features on large music databases. In Journal of New Music Research, 2008 (to appear). K. Bosteels, E. Kerre, and E. Pampalk. Music Retrieval Based on Social Tags: A Case Study. ISMIR 2008. A. Craft, G. Wiggins, T. Crawford. How Many Beans Make Five? The Consensus Problem in Music-Genre Classification and a New Evaluation Method for Single-Genre Categorisation Systems. ISMIR 2007. N. Corthaut, S. Govaerts, K. Verbert, and E. Duval. Connecting the Dots: Music Metadata Generation, Schemas and Applications. ISMIR 2008.

C. Baccigalupo, E. Plaza, J. Donaldson. Uncovering affinity of artists to multiple genres from social behaviour data. ISMIR 2008.

L. Barrington, D. O’Malley, D. Turnbull, and G. Lanckriet. Herd the Music - A Social Music Annotation Game. ISMIR 2008.

T. Bertin-Mahieux, D. Eck, F. Maillet, and P. Lamere. Autotagger: A model for predicting social tags from acoustic features on large music databases. In Journal of New Music Research, 2008 (to appear).

K. Bosteels, E. Kerre, and E. Pampalk. Music Retrieval Based on Social Tags: A Case Study. ISMIR 2008.

A. Craft, G. Wiggins, T. Crawford. How Many Beans Make Five? The Consensus Problem in Music-Genre Classification and a New Evaluation Method for Single-Genre Categorisation Systems. ISMIR 2007.

N. Corthaut, S. Govaerts, K. Verbert, and E. Duval. Connecting the Dots: Music Metadata Generation, Schemas and Applications. ISMIR 2008.

Z. Duan, L. Lu and C. Zhang. Collective Annotation Of Music From Multiple Semantic Categories. ISMIR 2008. R. Gjerdingen and D. Perrott. Scanning the dial: The Rapid Recognition of Music Genre. In Journal of New Music Research, 2008 (to appear). M. Guy & E. Tonkin. Folksonomies: Tidying up Tags? D-Lib Magazine, January 2006:12(1). X. Hu, M. Bert, & J. S. Downie. Creating a simplified music mood classification ground-truth set. ISMIR 2007. Xiao Hu et al. MOODY: A Web-Based Music Mood Classification and Recommendation System. ISMIR 2008. Y. Kim, E. Schmidt and L. Emelle. MoodSwings: A Collaborative Game For Music Mood Label Collection. ISMIR 2008. F. Kleedorfer, P. Knees and T. Pohle. Oh Oh Oh Whoah! Towards Automatic Topic Detection in Song Lyrics. ISMIR 2008. P. Knees, M. Schedl, T. Pohle, and G. Widmer. An Innovative Three-Dimensional User Interface for Exploring Music Collections Enriched with Meta-Information from the Web. ACM MM 2006.

Z. Duan, L. Lu and C. Zhang. Collective Annotation Of Music From Multiple Semantic Categories. ISMIR 2008.

R. Gjerdingen and D. Perrott. Scanning the dial: The Rapid Recognition of Music Genre. In Journal of New Music Research, 2008 (to appear).

M. Guy & E. Tonkin. Folksonomies: Tidying up Tags? D-Lib Magazine, January 2006:12(1).

X. Hu, M. Bert, & J. S. Downie. Creating a simplified music mood classification ground-truth set. ISMIR 2007.

Xiao Hu et al. MOODY: A Web-Based Music Mood Classification and Recommendation System. ISMIR 2008.

Y. Kim, E. Schmidt and L. Emelle. MoodSwings: A Collaborative Game For Music Mood Label Collection. ISMIR 2008.

F. Kleedorfer, P. Knees and T. Pohle. Oh Oh Oh Whoah! Towards Automatic Topic Detection in Song Lyrics. ISMIR 2008.

P. Knees, M. Schedl, T. Pohle, and G. Widmer. An Innovative Three-Dimensional User Interface for Exploring Music Collections Enriched with Meta-Information from the Web. ACM MM 2006.

P. Lamere. Social Tagging and Music Information Retrieval. Journal of New Music Research 2008 (to appear). P. Lamere and F. Maillet. Creating Transparent, Steerable Recommendations. ISMIR 2008. E. L. M. Law, L. von Ahn, R. B. Dannenberg, and M. Crawford. TagATune: A game for music and sound annotation. ISMIR 2007. M. Levy & M. Sandler. Learning Latent Semantic Models for Music from Social Tags. In Journal of New Music Research, 2008 (to appear). M. I. Mandel and D. P. W. Ellis. A Web-Based Game for Collecting Music Metadata. In Journal of New Music Research, 2008 (to appear). M. I. Mandel and D. P. W. Ellis. Multiple-instance learning for music information retrieval. ISMIR 2008. Nielsen Report: Nielsen soundscan state of the industry. 2008 Convention of the National Association of Recording Merchandisers, 2008. http://www.narm.com/2008Conv/StateoftheIndustry.pdf. E. Peterson. Beneath the metadata: Some philosophical problems with folksonomy. D-Lib Magazine 2006:12(11). http://www.dlib.org/dlib/november06/peterson/11peterson.html

P. Lamere. Social Tagging and Music Information Retrieval. Journal of New Music Research 2008 (to appear).

P. Lamere and F. Maillet. Creating Transparent, Steerable Recommendations. ISMIR 2008.

E. L. M. Law, L. von Ahn, R. B. Dannenberg, and M. Crawford. TagATune: A game for music and sound annotation. ISMIR 2007.

M. Levy & M. Sandler. Learning Latent Semantic Models for Music from Social Tags. In Journal of New Music Research, 2008 (to appear).

M. I. Mandel and D. P. W. Ellis. A Web-Based Game for Collecting Music Metadata. In Journal of New Music Research, 2008 (to appear).

M. I. Mandel and D. P. W. Ellis. Multiple-instance learning for music information retrieval. ISMIR 2008.

Nielsen Report: Nielsen soundscan state of the industry. 2008 Convention of the National Association of Recording Merchandisers, 2008. http://www.narm.com/2008Conv/StateoftheIndustry.pdf.

E. Peterson. Beneath the metadata: Some philosophical problems with folksonomy. D-Lib Magazine 2006:12(11). http://www.dlib.org/dlib/november06/peterson/11peterson.html

E. Pampalk, A. Flexer, and G. Widmer. Hierarchical Organization and Description of Music Collections at the Artist Level. ECDL 2005. E. Pampalk and M. Goto. MusicSun: A New Approach to Artist Recommendation. ISMIR 2007. E. Pampalk and M. Goto. MusicRainbow: A New User Interface to Discover Artists Using Audio-based Similarity and Web-based Labeling. ISMIR 2006. Y. Raimond and M. Sandler. A Web of Musical Information. ISMIR 2008. R. Sinha. Tagging – From Personal to Social. SXSW 2006. http://rashmisinha.com/2006/03/12/my-slides-for-tagging-20-panel-at-sxsw/ M. Sordo, O. Celma and M. Blech. The Quest For Musical Genres: Do the Experts and the Wisdom of Crowds Agree? ISMIR 2008. P. Symeonidis, M. Ruxanda, A. Nanopoulos and Y. Manolopoulos. Ternary Semantic Analysis Of Social Tags For Personalized Music Recommendation. ISMIR 2008. A. E. Thompson. Playing Tag: An Analysis of Vocabulary Patterns and Relationships Within a Popular Music Folksonomy. A Master’s Paper for the M.S. in L.S. degree. April, 2008. http://etd.ils.unc.edu/dspace/bitstream/1901/535/1/abbeythompson.pdf K. Trohidis, G. Tsoumakas, G. Kalliris and I. Vlahavas, Multi-Label Classification of Music Into Emotions. ISMIR 2008.

E. Pampalk, A. Flexer, and G. Widmer. Hierarchical Organization and Description of Music Collections at the Artist Level. ECDL 2005.

E. Pampalk and M. Goto. MusicSun: A New Approach to Artist Recommendation. ISMIR 2007.

E. Pampalk and M. Goto. MusicRainbow: A New User Interface to Discover Artists Using Audio-based Similarity and Web-based Labeling. ISMIR 2006.

Y. Raimond and M. Sandler. A Web of Musical Information. ISMIR 2008.

R. Sinha. Tagging – From Personal to Social. SXSW 2006. http://rashmisinha.com/2006/03/12/my-slides-for-tagging-20-panel-at-sxsw/

M. Sordo, O. Celma and M. Blech. The Quest For Musical Genres: Do the Experts and the Wisdom of Crowds Agree? ISMIR 2008.

P. Symeonidis, M. Ruxanda, A. Nanopoulos and Y. Manolopoulos. Ternary Semantic Analysis Of Social Tags For Personalized Music Recommendation. ISMIR 2008.

A. E. Thompson. Playing Tag: An Analysis of Vocabulary Patterns and Relationships Within a Popular Music Folksonomy. A Master’s Paper for the M.S. in L.S. degree. April, 2008. http://etd.ils.unc.edu/dspace/bitstream/1901/535/1/abbeythompson.pdf

K. Trohidis, G. Tsoumakas, G. Kalliris and I. Vlahavas, Multi-Label Classification of Music Into Emotions. ISMIR 2008.

D. Turnbull, L. Barrington, G. Lanckriet. Five Approaches to Collecting Tags for Music. ISMIR 2008. D. Turnbull, L. Barrington, D. Torres, and G. Lanckriet. Towards musical query-by-semantic description using the CAL500 data set. SIGIR 2007. D. Turnbull, R. Liu, L. Barringon, and G. Lanckriet. A game-based approach for collecting semantic annotations of music. ISMIR 2007. L. von Ahn and L. Dabbish. Labeling images with a computer game. SIGCHI 2004. http://www.cs.cmu.edu/~biglou/ESP.pdf D. Weinberger. How tagging changes peoples relationship to information and each other. Pew Internet & American Life Project 2007. http://www.pewinternet.org/pdfs/PIP_Tagging.pdf K. Weller. Folksonomies and Ontologies. Two New Players in Indexing and Knowledge Representation. 2007. http://www.phil-fak.uni-duesseldorf.de/infowiss/admin/public_dateien/files/35/1204288118weller009_.htm Daniel C. Wu Jr et al. Music and Lyrics: Can Lyrics Improve Emotion Estimation for Music? ISMIR 2008.

D. Turnbull, L. Barrington, G. Lanckriet. Five Approaches to Collecting Tags for Music. ISMIR 2008.

D. Turnbull, L. Barrington, D. Torres, and G. Lanckriet. Towards musical query-by-semantic description using the CAL500 data set. SIGIR 2007.

D. Turnbull, R. Liu, L. Barringon, and G. Lanckriet. A game-based approach for collecting semantic annotations of music. ISMIR 2007.

L. von Ahn and L. Dabbish. Labeling images with a computer game. SIGCHI 2004.

http://www.cs.cmu.edu/~biglou/ESP.pdf

D. Weinberger. How tagging changes peoples relationship to information and each other. Pew Internet & American Life Project 2007. http://www.pewinternet.org/pdfs/PIP_Tagging.pdf

K. Weller. Folksonomies and Ontologies. Two New Players in Indexing and Knowledge Representation. 2007. http://www.phil-fak.uni-duesseldorf.de/infowiss/admin/public_dateien/files/35/1204288118weller009_.htm

Daniel C. Wu Jr et al. Music and Lyrics: Can Lyrics Improve Emotion Estimation for Music? ISMIR 2008.

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Social bookmarking - Wikipedia, the free encyclopedia

A social bookmarking service is a centralized ... with a chosen tag, and include information about the number ... for efficient retrieval; ...
Read more

Hybrid Retrieval Approaches to Geospatial Music Recommendation

Hybrid Retrieval Approaches to Geospatial Music ... listening events for retrieval purposes and (ii) ... for Music Information Retrieval with ...
Read more

Collaborative search engine - Wikipedia, the free encyclopedia

Collaborative search engine; ... Recent work in collaborative filtering and information retrieval has shown that sharing ... Social search; Document retrieval;
Read more

MusicBrainz - The Open Music Encyclopedia

The ultimate source of music information by ... Tag Your Music. MusicBrainz ... Editing introduction; Style guidelines; FAQs; Community. Become a part of ...
Read more

International Journal of Multimedia Information Retrieval ...

The International Journal of Multimedia Information Retrieval ... social voting and ranking ; Music and ... (except in form of an abstract or as part of ...
Read more

Patent US8615565 - Automatic content retrieval based on ...

... on the tags and that (ii) ... to hip-hop music fans during the night, one set of tags may be used ... feed of information related to the tag.
Read more

Featured Content on Myspace

Iggy Azalea Kinda Agrees With Macklemore and Ryan Lewis' "White Privilege II" But which parts ... social networks or start ... Myspace. Just start typing ...
Read more

Applying associative retrieval techniques to alleviate the ...

Applying associative retrieval techniques to ... Information and Engineering Systems, Part II, ... social networks through association retrieval, ...
Read more