MEI NAACL Presentation

50 %
50 %
Information about MEI NAACL Presentation

Published on January 16, 2008

Author: Raulo


Mandarin-English Information (MEI): Investigating Translingual Speech Retrieval Johns Hopkins University Summer Workshop 2000:  Mandarin-English Information (MEI): Investigating Translingual Speech Retrieval Johns Hopkins University Summer Workshop 2000 Presented at the ANLP-NAACL 2000 Embedded Machine Translation Systems Workshop The MEI Team MEI Team:  MEI Team Senior Members Students Helen Meng Chinese University of Hong Kong Erika Grams Advanced Analytic Tools Sanjeev Khudanpur Johns Hopkins University Gina-Anne Levow University of Maryland Douglas Oard University of Maryland Patrick Schone US Department of Defense Hsin-Min Wang Academia Sinica, Taiwan Berlin Chen National Taiwan University Wai-Kit Lo Chinese University of Hong Kong Karen Tang Princeton University Jianqiang Wang University of Maryland Outline:  Outline Audio indexing MEI Project overview Research challenges System architecture Collaboration opportunities Motivation:  Motivation Speech retrieval applications are emerging e.g., source:, Feb 2000 Internet-accessible Radio and Television Stations The Big Picture:  The Big Picture Speech to Speech Translation Translingual Audio Browsing Translingual Audio Search English Query English Audio Select Examine MEI Related Work:  Related Work TREC Spoken Document Retrieval Close coupling of recognition and retrieval TREC Cross-Language Retrieval Close coupling of translation and retrieval TDT-3 Topic Tracking Coupling recognition, translation and retrieval Using speech recognition transcripts The MEI Project:  The MEI Project Closely couple recognition and translation For the purpose of retrieval Using English examples, find Mandarin audio Research Challenges:  Research Challenges Multi-scale audio indexing Multiple feature sets capture more information Multi-scale translation Lexicon and pronunciation are complementary Multi-scale retrieval Combination of evidence can add robustness Multi-scale Mandarin Audio Indexing:  Multi-scale Mandarin Audio Indexing Multi-scale Translation:  Word-scale Dictionary-based [Levow & Oard 00] Parallel corpora [Nie 99] Comparable corpora [Fung 98] Subword-scale [Knight & Graehl 97] Cross-language phonetic mapping /bei2 ai4 er3 lan2/ Kosovo (/ke1-sou3-wo4/, /ke1-sou3-fo2/, /ke1-sou3-fu1/, /ke1-sou3-fu2/) Multi-scale Translation Cross-Language Phonetic Mapping:  Cross-Language Phonetic Mapping Syllabify English spelling e.g. Jiang Zemin, Shandong Province Map English pronunciation to Mandarin Convert phonemes to pinyin e.g. /k ow s ax v ow/ to /ke1-suo3-wo4/ Plan to investigate alternative techniques Rule-based Statistical mapping Multi-scale Retrieval:  Multi-scale Retrieval Word-scale exploits lexical knowledge Enhances precision Subwords can achieve complete coverage Enhances recall Combination of evidence may be best If a good merging strategy can be found Multi-scale Retrieval Techniques:  Multi-scale Retrieval Techniques Subword-scale Syllable lattice matching [Chen, Wang & Lee 00] Overlapping syllable n-grams [Meng et al. 99] Syllable confusion matrix [Meng et al. 99] Word-scale Structured queries [Pirkola 98] Structured translation [Sperer & Oard 00] Merging Strategies:  Merging Strategies Loose coupling Separate retrieval runs Merge ranked lists [Voorhees 95] Tight coupling [Ng 00] Unified indexing of words and subwords Single ranked list Robust Retrieval:  Robust Retrieval Multiple causes Speech recognition errors Translation ambiguity Transliteration ambiguity Possible solutions Weighted n-best indexing [Levow & Oard 00] Syllable lattice indexing [Chen, Wang & Lee 00] Syllable confusion expansion [Meng et al. 99] Structured queries [Pirkola 98] Document expansion [Levow & Oard 00] System Architecture Overview:  System Architecture Overview Word Translation English Example Phonetic Transcription Known Terms Retrieval System Words Syllable n-grams Mandarin Documents Translation Lexicon Corpus Statistics Eval Code Average Precision Relevance Judgments Syllable n-gram Generation The TDT Collections:  The TDT Collections 41 Hours VOA Mandarin Audio 59 Topics 121 Hours Voice of America (VOA) Mandarin Audio APW+NYT English Associated Press (APW) New York Times (NYT) English Newswire 20 Topics Development Test (TDT-2) Evaluation (TDT-3) Mar 98 Oct 98 Dec 98 Jun 98 Jan 98 Four stories per topic in each language Each reporting on some aspect of one event Story Boundaries Known Condition MEI Project Schedule:  MEI Project Schedule Six Weeks at Hopkins: Dec Feb Jun Apr Aug Workshop 00 Planning Meeting First Team Planning Meeting Second Team Planning Meeting Things We Need:  Things We Need Ideas To sharpen our focus Connections To build a community of interest Resources To build on what others have done For More Information:  For More Information MEI Project Translingual Retrieval Speech Retrieval Hopkins Summer Workshop Series Backup Slides:  Backup Slides Background: Chinese:  Background: Chinese Many dialects (e.g., Mandarin and Cantonese) differences in phonetics, vocabularies, syntax… Syllable-based language ~400 base syllables, 4 lexical tones + light tone Syllable structure (CG)V(X) (CG): onset, optional, consonant+medial glide V: nuclear vowel X: coda, glide / alveolar nasal / velar nasal ~ 21 initials, 39 finals Background: Chinese (cont):  Background: Chinese (cont) Characters (written) -> syllables (spoken) Degenerate mapping /hang2/, /hang4/, /heng2/ or /xing2/ /fu4 shu4/ (LDC’s CALLHOME lexicon) Tokenization / Segmentation /zhe4 yi1 wan3 hui4 ru2 chang2 ju3 xing2/ Translating the English Example:  Translating the English Example English Example Documents (words) Named entities and unknown words Words found in translation dictionary Translation Transliteration Mandarin Queries (words and syllable n-grams) Indexed Mandarin Audio (words and syllable n-grams) Information Retrieval Engine Ranked List Detailed Query Processing (1):  White-space separated text with named entity tags. Stopping Stemming Phrase Extraction List of translatable words and phrases Terms that have Mandarin translations Named entities terms with no Mandarin translations Detailed Query Processing (1) Detailed Query Processing (2):  term translation English-Mandarin translation lexicon Named entity parsing Named entity parsing rules for transliteration phonetic expansion English pronunciation lexicon term translation phonetic expansion xxxxx xx xxx xxx zzzzzzz ww xx ww z rrrrrr sss ttttt tt ww www eh n t ih t iy t eh k s t Bag of Mandarin terms Bag of English phone sequences Terms that have Mandarin translations Terms with no Mandarin translations Named entities Northern Ireland Detailed Query Processing (2) Detailed Query Processing (3):  English phone strings to Mandarin syllables Mandarin syllabification rules ASR Insertion prone words ASR substitution/deletion prone words Retain this term? sa sb sc sd sc se s1 s2 s2 s5 s1 s2 s3 Mandarin pronunciation lexicon Bag of Mandarin terms Syllabic expansion yes xxxxx xxx zzzzzzz xx ww z rrrrrr sss ttttt tt ww www Bag of English phone sequences Two bags of Mandarin syllable sequences Smaller bag of Mandarin terms To trash (or downweight) no Detailed Query Processing (3) Detailed Query Processing (4):  Syllable n-gram generation Syllable n-gram generation s0 s1 s1 s2 s2 s3 s3 s4 s3 s5 s5 s6 s5 s7 s7 s7 sa sb sb sc sc sd sc se s1 s2 s2 s3 s2 s5 s5 s1 Syllable n-gram generation xxxxx xxx zzzzzzz xx ww z rrrrrr sss ttttt tt ww www Mandarin pronunciation lexicon syllabic expansion Mandarin syllable sequences from likely recognition errors Mandarin syllable sequences from unknown words Bag of Mandarin lexical terms Three Bags of Mandarin syllable n-grams from different sources Bag of high-confidence Mandarin terms Detailed Query Processing (4) Translating the Mandarin Audio:  Translating the Mandarin Audio English Example Documents (words) Indexed Mandarin Audio (words) Translation Information Retrieval Engine Ranked List Documents in English

Add a comment

Related presentations

Related pages

Seminararbeit by Philipp Kaiser on Prezi

This link expires 10 minutes after you close the presentation; A maximum of 30 users can follow your presentation; Learn more about this feature in the manual;
Read more

1 Natriumchlorid (NaCl) oder einfach Kochsalz Fabian ...

1 Natriumchlorid (NaCl) oder einfach Kochsalz Fabian Gelies Sommersemester 2002 Experimentalvortrag. Veröffentlicht von: Ludo Lang, Geändert vor 13 Jahren
Read more

Importanta sarii de bucatarie ( NaCl) by on Prezi

People invited to a presentation do not ... NaCl denumita popular si "Sare de ... au de obicei o restrictie a consumului de sare.Aceasta mai este ...
Read more

Tao Mei (

Tao Mei ( ... LSTM UC Berkeley, NAACL (2014/12) 71.19 19.40 9.70 ... PowerPoint Presentation Author: Tao Mei
Read more

NAACL-HLT 2012: 7th Workshop on Innovative Use of NLP for ...

The 7th Workshop on Innovative Use of NLP for Building Educational Applications. ... Mei-Hua Chen, Shi-Ting Huang ... Please use the 2012 NAACL-HLT style ...
Read more

COMS E6998 Machine Translation - Google Sites

Project 1-minute presentations today! ... Mei Yang & Katrin ... A comparison of pivot methods for phrase-based statistical machine translation.NAACL 2007.
Read more

PowerPoint Presentation - MetaMouse

Dong-Mei Gao Ye-Qin Yu Xin-Da Zhou Zeng-Chen Ma ... 10min in TRIS/NaCl and sections were incubated for 40 min with ... PowerPoint Presentation
Read more

PowerPoint Presentation - orapreferata - home

Title: PowerPoint Presentation Last modified by: pc Created Date: 1/1/1601 12:00:00 AM Document presentation format: On-screen Show (4:3) Other titles
Read more