Published on January 15, 2008
CyberInfrastructure Applications in Music & Digital Audio: CyberInfrastructure Applications in Music & Digital Audio Stephen David Beck, Ph.D. Director, CCT Laboratory for Creative Arts & Technologies Outline: Outline History of Computation & Music CI requirements for digital media Current Applications Emerging Applications Computation & Music: Computation & Music Music & Digital Audio have been at the forefront of CS research from the beginnings of the Information Age Hiller & Isaacson - UIUC: Hiller & Isaacson - UIUC Explored Markov chains to model music composition Computer-assisted composition Early efforts in Machine Learning Illiac Suite (1955) for string quartet Illiac computer Research @ Bell Labs: Research @ Bell Labs Digital Audio experimentation in 1950s & 1960s lead to fundamental paradigm for DSP and audio synthesis Max Mathews @ Bell Labs Developed music/sound synthesis language (Music I - V) Paradigm of “orchestra/score” Research @ Bell Labs: Research @ Bell Labs Bicycle Built For Two (1961) Music & Voice synthesized First “singing” computer Referenced in 2001 A Space Odyssey Computer Music: Computer Music 1970s GROOVE - real-time digital control of analog synthesizers 1980s FM Synthesis - a computationally inexpensive synthesis system Yamaha DX7 - FM synth, most successful synthesizer of all-time MIDI protocol developed Computer Music: Computer Music Early 1990s DSP-based synthesis and signal-processing on specialized audio cards (ProTools, ISPW, Kyma, Creative) Late 1990s CPU-based real-time synthesis/signal processing programming environments (Max/MSP) Computer Music: Computer Music 2000-pres. Real-time Virtual Instruments Modeling of classic analog synthesizers Virtual physical models Real-time convolution (reverberation) Interactive Performance Environments Moore’s law has effectively “solved” the audio problem Or has it? The Next Steps: The Next Steps Use distributed and grid technologies to leverage underutilized CPUs for audio production and analysis Use high-speed networks for distributed performances Use network communication tools (AccessGrid, Portals) to enhance and develop educational opportunities CI Requirements: CI Requirements Current Applications: Current Applications Access Grid for musical performances In Common: TIME UF Digital Worlds Institute, James Oliverio Created network-metronome to synchronize performances with network latency Distributed Signal Processing Logic Audio Compute nodes are established on idle machines over a 1GB switched network Uses additional CPUs to generate real-time audio and signal processing First (and only) commercial product to do this Current Apps - DART: Current Apps - DART Distributed Audio Rendering using Triana Triana is a Java-based graphic programming environment Program objects are linked together Objects include a range of signal processing algorithms Objects can access web services, or distribute computing across a peer-to-peer network Originally designed for use in Gravitational Wave Astronomy Triana: Triana Schroeder Reverb: Schroeder Reverb Voice Cancel: Voice Cancel Wave Viewer: Wave Viewer DART Applications: DART Applications Complex FFT-based Signal Processing Large FFT-size analysis High window count FFT analysis Multipass FFT analysis Bulk processing of large file sets Using high-speed networks to play sound files on remote computers Music Info Retrieval: Music Info Retrieval A “Grand Challenge” for Music Automatic metadata creation and music analysis for matching and searching algorithms Music Info Retrieval: Music Info Retrieval Goals: Searchable databases using musical descriptors and parameters Query by Humming Beat & Tempo detection Melody extraction Harmonic Analysis Pitch Histograms Pattern matching for analysis of style and content Style Identification & Classification Automatic metadata generation MIR Technologies: MIR Technologies Based upon analysis of audio signals Fast Fourier Transforms Translates signal from time-domain to frequency-domain (spectral analysis) Critical parameters and measurements Zero-crossings Spectral Centroids Base Frequency Tracking Spectral Harmonicity MIR Examples: MIR Examples Manual Analysis Pandora PlayProduction Automatic Computer Analysis M2K Marsyas DART & Alchemist Music Genome Project: Music Genome Project A private venture to establish a taxonomy of “musical” information 400 “parameters” of musical characteristics are identified Human analysis of music creates a meta-data representation of these 400 parameters It takes 20-30 minutes per song to analyze Music from the past 60 years only Six years of work Why? Pandora: Pandora Personalized Internet Radio How does it work?: How does it work? You select a song you like It searches its database for other songs “like” it Those songs whose metadata is similar to the one you want It creates a radio station from this new list of songs Amazon for Music Why Pandora?: Why Pandora? Money!!! Subscription model Free model, with advertising Competition? Satellite Radio Online Music Stores (iTunes) Distinction from competition? Intelligent(?) selection process “Smart Muzak” Spin-offs: Spin-offs EMI PlayProduction: EMI PlayProduction Search tool for EMI catalog Designed for music production in: Advertising & Television Analysis-based MIR: Analysis-based MIR Computer-based analysis and MIR is the “grand challenge” of music Application environments: Marsyas Music To Knowledge (M2K) DART! Testbed Framework (IMIRSEL) International Music Information Retrieval Systems Evaluation Laboratory Analysis-based MIR: Analysis-based MIR Successes Style matching Query by Humming Beat & tempo detection Note detection Score following Left to do? Classical music! Formal analysis Transcription Instrument Identification And so much more… Compositional Uses?: Compositional Uses? Spectral Re-animation or Substitution Create a library of FFT frames Take a source file, generate FFT frames, and then replace each frame with a close match from the library Inverse FFT recreates the original sound but with different FFT frames Alchemist: Alchemist A proposed multimedia analysis framework based on Triana Audio and Video based analysis tools Super-peer-to-peer network distribution Motivation is to use audio analysis to assist with audio/video analysis and extraction of salient meta-data Application areas Audio and Video searching (MVIR)? Why CI for MIR?: Why CI for MIR? Massive data stores required Audio libraries Analysis files are larger than the audio files themselves Distributed Analysis Large libraries of source audio Copyright Issues Grids bring computing to the data, not data to the computing Contact: Contact Stephen David Beck, Ph.D. Center for Computation & Technology Louisiana State University Baton Rouge, LA 70803 225-578-2594 firstname.lastname@example.org
2006cihass_Stephen-Beck-MusicAppsForCI; LibIssues2001; ADABasics_CSC484_Win#391C8F; Internet2AbileneAug2002; admin; LibIssu; icphs-final; Chapter_13-2E ...
2006cihass_Stephen-Beck-MusicAppsForCI; exphil; ch04; ch04; ch04annotated; CCAC; AnnualProgramReviewProgressReportFall2008-Music; AA University Prep w.Area ...