zhangfy slides

67 %
33 %
Information about zhangfy slides
Business-Finance

Published on April 22, 2008

Author: Dorotea

Source: authorstream.com

多媒体信息处理 ( Multimedia Computing ) 张福炎 2000/11/7:  多媒体信息处理 ( Multimedia Computing ) 张福炎 2000/11/7 内 容:  内 容 0 Space of Multimedia Computing 1 Media and Multimedia 2 Media Processing 2.1 Media Food(Value) Chain 2.2 Digital Item Declaration 2.3 Multimedia Content Representation(coding) 2.4 Digital Item Identification and Description 2.5 Content Management and Usage 3 Networked Multimedia Applications 0 多媒体信息处理空间:  以文本为主 以音频为主 以视(音)频为主 离散媒体: text, image,… 连续媒体: audio, video, ... 0 多媒体信息处理空间 多媒体信息处理的总目标 :  多媒体信息处理的总目标 To enable transparent and augmented use of multimedia resources across a wide range of networks and devices. 使能跨越各种不同网络和设备, 透明地、强化地使用多媒体资源。 1 Media and Multimedia:  1 Media and Multimedia 1.1 计算机与媒体:  1.1 计算机与媒体 媒体又称媒介、媒质,指的是用于表示、存储、分发、传输和展现数据(信息) 的手段、方法、工具、设备或装置。 ( In general, one describes medium as a means for representation, storage, distribution, transmission and presentation of data. ) 计算机与媒体:  计算机与媒体 1.2 感觉媒体与表示媒体:  1.2 感觉媒体与表示媒体 感觉媒体是人与人、人与计算机交换信息所使用的媒体; 表示媒体是计算机内部、计算机与计算机之间交换信息所使用的媒体; 多媒体技术所指的媒体是:感觉媒体及其相应的表示媒体。 感觉媒体 表示媒体举例:  感觉媒体 表示媒体举例 表示媒体的时间属性:  表示媒体的时间属性 time-independent ( discrete ) media(离散媒体,值不会随着时间而变化) text , graphics , static image, …. time-dependent ( continuous ) media (连续媒体,值随着时间而变化) speech , music , video , animation, …. 表示媒体的空间属性:  表示媒体的空间属性 1D媒体 单声道speech , music , 2D媒体 双声道music ; text , image , 2D graphics , 3D及多维媒体 3D graphics , 全景图像 , 空间立体声music , 表示媒体的生成属性:  表示媒体的生成属性 自然媒体(使用数字化方法从现实世界获取的媒体) 取样声音 , image , video 合成媒体(使用计算机创建的媒体) 合成语音, MIDI , graphics , animation 多媒体系统中听觉媒体的分类:  多媒体系统中听觉媒体的分类 来源 机外形式 空间属性 媒体名称 语言 1D 波形语音 语言 多维 多声道波形语音 非语言 1D 波形声音 非语言 多维 双声道/3D立体声 语言 1D 合成语音 语言 多维 多声道合成语音 音乐 1D 合成音乐(MIDI音乐) 音乐 多维 多声道合成音乐 多媒体系统中视觉媒体的分类:  多媒体系统中视觉媒体的分类 来源 机外形式 时间 空间属性 媒体名称 文字 静止 2D 取样文本 图画 静止 2D 取样图像(image) 图画 静止 3D 全景图像 图画 活动 2D 视频(video) 图画 活动 3D 临境视频 文字 静止 2D 编码文本(简称文本) 图画 静止 2D 二维图形 图画 静止 3D 三维图形 图画 活动 2D 二维动画 图画 活动 3D 三维动画 自然媒体与合成媒体的区别:  自然媒体与合成媒体的区别 自然媒体 合成媒体 来源 获取 创建 表示方法 取样表示 符号表示 表现能力 真实感强 真实感有限 可编辑性 有限 全部 存储 数据量很大 数据量很小 检索 困难 容易 传输 要求高 要求低 展现 处理容易 处理复杂 1.3 多媒体:  1.3 多媒体 多媒体强调的是使用多种媒体、综合表达信息内容并进行交互式处理的技术。从本质上来说,它具有三种最重要的特性: 1 媒体的多样性,其中至少有一种连续媒体; 2 媒体的集成性(综合性),多种不同媒体综合地表现某个内容,取得更好的效果; 3 处理的交互性,使人们获取和使用信息的过程中具有细粒度的控制和操纵能力。 多媒体是信息技术发展的必然趋势:  多媒体是信息技术发展的必然趋势 计算机 通 信 大众传媒 数值计算 数据处理 图形、图象处理 声音处理 视频处理 邮 政 电 报 电 话 传 真 数据通信 综合业务(N-ISDN) 多媒体 书,刊,报纸 广 播 电 影 电 视 光盘出版物 交互电视 网络出版 支持宽带综合业务的数据通信网络 信息技术走向汇聚和融合 2 Media Processing:  2 Media Processing 2.1 媒体食物(价值)链:  2.1 媒体食物(价值)链 媒体处理内容:  媒体处理内容 对媒体进行的各种处理: 媒体准备(media preparation) 媒体获取/媒体创建 媒体编辑(media editing) 媒体集成/组合(media integration / composition) 结构描述、特征描述、应用描述…. 媒体通信/发送(media communication / delivery) 媒体消费(media consumption) 媒体转换(码),展现, 浏览, 检索,….. 媒体食物(价值)链:  媒体食物(价值)链 多媒体(内容)处理的7要素:  多媒体(内容)处理的7要素 1 数字项目说明(Digital Item Declaration) 2 内容表示(Content Representation) 3 数字项目的标识与描述(Digital Item Identification and Description) 4 内容管理与使用(Content Management and Usage) 5 知识产权管理与保护(Inte11ectual Property Management and Protection) 6 内容发送与终端及网络(Terminals and Networks) 7 事件报告(Event Reporting)。 2.2 Digital Item Declaration:  2.2 Digital Item Declaration Goal:  Goal Establish a uniform and flexible abstraction and interoperable schema for defining digital items. 为定义数字项而建立一个统一、灵活的抽象表示和可互操作的模式。 Digital Item Declaration的需求1:  Digital Item Declaration的需求1 Digital items are open and extensible to any and all media resources types and description schemes.数字项是开放的,可扩展到所有媒体资源类型和描述模式. Composite items can be constructed from other items, without losing the structure and properties of the sub-items. 复合项由子项组合而成,子项的结构和性质不丢失. Multiple composite items may share individual elements.多个复合项可以共享单个元素. Digital Item Declaration的需求:  Digital Item Declaration的需求 An individual element may be referenced by multiple locations within a digital item. 一个元素可以被数字项中的多个地方引用. To enable applications to correctly manipulate and validate digital items.能使应用程序正确操纵和验证数字项. Identification and revision of digital items and their components must be supportable in an open and extensible manner.对数字项及其组件的标识和修改必须以开放和可扩展的方式进行. Digital Item Declaration的需求2:  Digital Item Declaration的需求2 Digital items explicitly define the relationships between elements and their corresponding descriptors.数字项对其元素与相应描述符之间的关系显式地进行定义 Descriptors may be simple statements or full media components.描述符可以是简单语句也可以是整个媒体组件 Descriptors can be described by other descriptors.描述符也可以由其它描述符描述 Anchors must be declarable within a component that allow: descriptors to be associated with a specific point or range within a media resource. linking back from within a media resource to the anchor. 锚(anchor)必须在一个组件中予以说明,从而允许描述符与媒体资源中的一个指定点或指定范围相关联,也允许从媒体资源反向链接到锚。 Current Situation:  Current Situation 目前还没有数字项的标准模型或表示方法: 没有一种能适合任何场合中说明各种各样内容的通用的、灵活的、可以互操作的解决方案, 也没有一个能将任何种类媒体资源同各种类型媒体描述信息链接在一起的统一的方法。 由于数字项的概念是建立在显式地描述媒体数据与描述数据之间关系的想法之上,这就严重限制了标准模型或表示的存在。 举例: MPEG-4的场景说明:  举例: MPEG-4的场景说明 MPEG-4 视听场景(audiovisual scenes) 是由若干媒体对象(media objects)以层次方式组合而成的,最低层的称为初级(primitive) 媒体对象, 如: still images (e.g. as a fixed background), video objects (e.g. a talking person - without the background) audio objects (e.g. the voice associated with that person); text and graphics; talking synthetic heads and associated text used to synthesize the speech and animate the head; synthetic sound 初级媒体对象既可以是自然媒体,也可以是合成媒体,既可以是 2维的,也可以是3维的。 MPEG-4的场景说明:  MPEG-4的场景说明 MPEG-4视听场景的描述可以看作一棵树, 叶节点是初级媒体对象, 中间节点是复合媒体对象(compound media objects),根节点则为场景。使用这种方法, 内容提供商(作者)可以构造出相当复杂的场景, 例如: place media objects anywhere in a given coordinate system; apply transforms to change the geometrical or acoustical appearance of a media object; group primitive media objects in order to form compound media objects; apply streamed data to media objects, in order to modify their attributes (e.g. a sound, a moving texture belonging to an object; animation parameters driving a synthetic face); change, interactively, the user’s viewing and listening points anywhere in the scene. 视听场景的说明:  视听场景的说明 由4个复合媒体对象(人,背景,家具,音视演示)组成。 系统定义了一个场景坐标系, 然后可以指定组成该场景的媒体对象在该坐标系中的位置,并可设定用户观察该场景时所在的坐标位置,这些位置信息都会包含在场景的描述中。 视听场景的说明:  视听场景的说明 视听场景的说明:  视听场景的说明 1 对象是如何组合在一起的。MPEG-4的场景的组织呈树状结构,它是一个无回路的有向图,图中每个节点代表一个媒体对象。树的结构未必是不变的,当增加节点、删除节点或替换节点时,节点的属性(如位置参数)就会发生修改。 2 对象的空间和时间定位。MPEG-4中,每一个视听对象都有其空间和时间范围, 有自己的局部坐标系。局部坐标系是用作在空间和时间范围内操纵该媒体对象的手柄(handle) ,通过指定一个将对象从局部坐标系变换到全局坐标系的坐标变换,可以将媒体对象放置在场景中规定位置处。 3属性值选择。单个的媒体对象和场景描述节点向组合层(composition layer)提供了一组参数,从而使它们的部分行为可以进行控制。例如: 声音的音调,合成对象的颜色,可分级编码的增强信息的起用和停止等。 多媒体内容的展现:  多媒体内容的展现 2.3 Multimedia Content Representation(coding):  2.3 Multimedia Content Representation(coding) Rationale (基本原理):  Rationale (基本原理) Content 是多媒体技术框架中最重要的要素, 它需要进行编码、标识、描述、存储、分发、保护、交易、消费等等。 它以满足一组需求的数字编码方式表示,数字内容的有效表示将会允许开展许多新的服务. Requirements 1:  Requirements 1 Data Types – it shall be possible to represent a large range of data types, both with natural and synthetic origin, as well as any combination of these data types, e.g. still pictures, frame-based as well as arbitrarily shaped video, specific and generic 3D models, graphics, text, natural and synthetic audio, natural and synthetic speech, etc. 数据类型:应能够表示各种不同的数据类型,包括自然的和合成的,也包括两者的组合。如:静止图像,其于帧的视频,任意形状的视频,专用的和通用的3D模型,图形,文本,自然声音和合成声音,自然语音和合成语音等。 Requirements 2:  Requirements 2 Content Variety – it shall be possible to represent any type of content for all the data types considered. 内容种类:上述所有数据类型应能表示任何种类的内容。 Requirements 3:  Requirements 3 Efficiency – it shall be possible to represent content for each of the data types above mentioned in the most bit efficient way for different target qualities; for a multimedia scene composed as a combination of various elements of the data types mentioned above, it shall be possible to selectively choose the coding efficiency/quality for each one of them. 有效性:根据不同的目标质量,上述每一种数据类型都使用最有效的数据位来表示内容。对于由不同数据类型的多种元素组成的多媒体场景,应能对每个元素选择其编码的效率/质量。 Requirements 4:  Requirements 4 Scalability – it shall be possible to represent all the elements in the multimedia scene in a scalable way and with a fine granularity in the dimensions considered relevant, e.g. spatial, temporal, quality. 可分级性:应能以一种可分级的方法来表示多媒体场景中的所有元素,且在所考虑的相关维度如时间、空间、质量上是细粒度可分级的。 Requirements 5:  Requirements 5 Random Access – it shall be possible to randomly access, within a limited time and with fine resolution, at all scalable layers, all elements in the multimedia scene; for a multimedia scene composed as a combination of various elements of the data types mentioned above, it shall be possible to randomly access each one of them. 随机存取:在一定时间内,以比较精细的分辨率,在所有可分级的层次中,对多媒体场景中的所有元素可以进行随机存取;对于由不同数据类型元素组成的多媒体场景,应能随机存取场景中的每一个元素。 Requirements 6:  Requirements 6 Error Resilience – it shall be possible to selectively protect the various elements in a multimedia scene against channel errors with relevant error patterns, e.g. mobile networks, ATM networks or storage media. 出错恢复:对多媒体场景中的各种元素,应能有选择地进行保护,以免受到通道错误的影响,如移动网络、 ATM网络或存储介质错误等。 Requirements 7:  Requirements 7 Interaction – it shall be possible to interact with the various elements in the multimedia scene, with a fine granularity, both in space and time; for a multimedia scene composed as a combination of various elements of the data types mentioned above, it shall be possible to independently interact with each one of them. 交互:无论是时间和空间,应能以细粒度方式与多媒体场景中的各种元素进行交互;对于由不同数据类型元素组成的多媒体场景,应能单独同其中的每个元素进行交互。 Requirements 8:  Requirements 8 Synchronization – it shall be possible to synchronize all the elements in the multimedia scene as well as other relevant data. 同步:应能对多媒体场景中所有元素以及其它相关数据进行同步。 Requirements 9:  Requirements 9 Multiplexing – it shall be possible to multiplex the coded data corresponding to the various elements in the multimedia scene as well as other relevant data. 复用:应能对多媒体场景中不同元素的编码数据以及其它相关数据进行复用。 Current Situation:  Current Situation 编码问题已研究了很多年,已经有了很多有名的标准,涉及许多种媒体, 满足了不同类型的需求. 例如: still pictures: JPEG, JPEG-LS and JPEG2000 frame-based video: H.261, H.263, MPEG-1 and MPEG-2 Video object-based pictures and video: MPEG-4 Visual. audio: MPEG-1, MPEG-2 and MPEG-4 Audio, speech: G.723, G.728 and MPEG-4 Audio. synthetic visual content, VRML and MPEG-4 synthetic audio content , MIDI and MPEG-4 Audio. 尽管这些标准在满足相关需求方面已相当完整,但现有标准并不完全满足某些需求。 例1: 数字声音的编码标准:  例1: 数字声音的编码标准 ITU(CCITT): G.711,721,722,… 编码对象:自然语音 应用领域:通信 ISO/IEC: MPEG-1 audio (11172-3) : 1993 MPEG-2 BC (13818-3) : 1995 MPEG-2 AAC (13818-7) : 1998 MPEG-4 audio (14496-3) : 1999 编码对象:自然声音(全频道):  编码对象:自然声音(全频道) 取样频率(kHz) 声道数目 MPEG-1 audio : 32, 44.1, 48 单,双 MPEG-2 BC : 增加16, 22.05, 24 5.1, 7.1 MPEG-2 AAC : 8-96 48个主声道, 16个LFE, MPEG-4 audio : 8-96 编码对象:合成声音:  编码对象:合成声音 MPEG-1 audio : 无 MPEG-2 BC : 无 MPEG-2 AAC : 无 MPEG-4 audio : 合成语音, 合成音乐 输出码率:  输出码率 码率范围(kbps) 全频道声音 MPEG-1 audio : 32-384 384(I), 256-192(II), 128-112(III) MPEG-2 BC : 8-640 同MPEG-1 MPEG-2 AAC : 8-160/CH 约 64/CH MPEG-4 audio : 2-64/CH 约 64/CH 功能(functionality):  功能(functionality) MPEG-1 audio :对错误的鲁棒性功能 MPEG-2 BC : 对错误的鲁棒性功能 MPEG-2 AAC :对错误的鲁棒性,可分级性功能 MPEG-4 audio : 播放速度控制功能 音调(pitch)变化功能 对错误的鲁棒性功能 可分级性(scaleability)功能 可分级性(scaleability)功能:  可分级性(scaleability)功能 MPEG-1 audio :无 MPEG-2 BC : 无 MPEG-2 AAC : 取样率可分级(SSR) MPEG-4 audio : Bit rate scalability Bandwidth scalability Encoder complexity scalability decoder complexity scalability 编码原理:  编码原理 MPEG-1 audio ,MPEG-2 BC, MPEG-2 AAC : 基于心理声学的感知编码,特别是频率域的听觉掩蔽特性. MPEG-4 audio : 各种编码技术 通用声音编码器对MPEG-2 AAC 作了改进和扩充; 自然语音编码器对LPC, CELP等作了改进和扩充,增加了可分级性功能. Scalabilities in MPEG-4/CELP:  Scalabilities in MPEG-4/CELP 对“对象”的支持:  对“对象”的支持 MPEG-1 audio ,MPEG-2 BC, MPEG-2 AAC : 不支持 MPEG-4 audio: 把声音对象定义为“可听见的语义实体”. 若干声音对象可以组合或混合在一起成为一个复合对象, 称为Audio-Composition Objects:Natural audio,Synthetic audio,Control Operations on objects: Synchronize Decode Compose into compound objects Present Interact MPEG-4 声音的应用:  MPEG-4 声音的应用 例2: JPEG2000的目标:  例2: JPEG2000的目标 1 Superior low bit-rate performance: 例如具有丰富细节的灰度图像在不牺牲其性能时压缩比低于0.25 bpp. 这是优先级最高的目标,网络图像传输和遥感需要此特性. JPEG2000的目标:  JPEG2000的目标 2 Continuous-tone and bi-level compression: 使用相似的系统资源,既能压缩连续色调图像又能压缩2值图像. 可处理各种动态范围的图像 (如每个颜色分量 1 bit to 16 bit). 此类应用有混合文档,带有注释的医学图像,图表及带有2值或近似2值区域、 alpha平面和透明面的计算机合成图像,传真图像等。 JPEG2000的目标:  JPEG2000的目标 3 Lossless and lossy compression: 提供无失真压缩累进式解码。此类应用有医学图像,图像存档,网络应用,印前图像处理。 4 Progressive transmission by pixel accuracy and resolution: 以像素精度或分辨率作为单位的累进式传输。此类应用有WWW,图像存档,打印机等。 JPEG2000的目标:  JPEG2000的目标 5 Fixed-rate, fixed-size, limited workspace memory: 固定速率指的是给定数目的连续像素编码后比特数应小于等于一定值,这就允许解码器在带宽有限的通道上进行实时解码,如遥感图像,运动编码等。 固定大小指的是一幅完整图像的码流的大小应为定值,这就允许存储空间有限的硬件能保存一个完整的码流(不管图像大小), 例如扫描仪,打印机等。 JPEG2000的目标:  JPEG2000的目标 6 Random codestream access and processing: 随机码流存取指:允许用户定义图像中感兴趣的区域( Regions-Of-Interest, ROI) ,并能以较小的失真对它进行随机存取and/or 解码; 随机码流处理包括:rotation, translation, filtering, feature extraction, scaling, etc. JPEG2000的目标:  JPEG2000的目标 7 Robustness to bit-errors: 这在无线通信时很重要。由于解码时码流中的某些部分在决定图像质量方面比其它部分更加重要, 码流的正确设计能帮助后继的错误改正系统以减轻解码错误。 JPEG2000的目标:  JPEG2000的目标 8 Open architecture: 开放的体系结构能为不同类型的图像和应用优化系统性能。做法是开发高灵活性的编码工具或采用描述语言分发和集成新的压缩工具。允许用户选择适合其应用和将来发展的工具,解码器只要实现核心的一组工具和能理解码流的一个语法分析器,在需要时,解码器可以向“源”请求发送一个未知的工具。 JPEG2000的目标:  JPEG2000的目标 9 Sequential build-up capability (real time coding): 能顺序地一趟对图像进行编码和解码,对图像的分量可以交错地也可以顺序地进行处理。压缩和解压缩期间,使用的上下文应限定在合理的像素线数目内。 JPEG2000的目标:  JPEG2000的目标 10 Backwards compatibility with JPEG: It is desirable to provide for backwards compatibility (or easy transcoding) with the current JPEG standards. JPEG2000的目标:  JPEG2000的目标 11 Content-based description: Finding an image in a large database of images is an important problem in image processing. For example, a doctor could request only images from a set that are recognized to have a certain type of tumor. This could have major applicability to the medical, law enforcement and environmental communities, and for image archival applications. Regardless of the techniques used, JPEG 2000 should strive to provide the opportunity for solutions to this problem. JPEG2000的目标:  JPEG2000的目标 12 Protective image security: 保护数字图像安全。方法如: watermarking, labeling, stamping, fingerprinting, encryption, scrambling, etc. Labeling is already implemented in SPIFF and must be easy to transfer back and forth to JPEG 2000 image file. Stamping is a mark set on top of a displayed image that can only be removed by a specific process. Encryption, and scrambling can be applied on the whole image file or limited to part of it (header, directory, image data) to avoid unauthorized use of the image. (Note: The issue of whether JPEG2000 should cover specific solutions to encryption, watermarking, authentication algorithms should be revisited) JPEG2000的目标:  JPEG2000的目标 13 Compatibility with ITU-T recommendations for image exchange: 传真也是JPEG 2000的应用, ITU-T关于图像交换的建议也将用来交换和发送JPEG 2000的编码图像 ,JPEG 2000编码图像也会符合 ITU-T Rec关于灰度图像和彩色图像编码的需求。 JPEG2000的目标:  JPEG2000的目标 14 Interface with MPEG-4: JPEG2000所开发的压缩静止图像的编码工具应当以合适的接口提供给MPEG-4,从而允许交换和集成(与MPEG-4之类的面向语法的编码方案框架),特别是在向/从运动图像嵌入/抽取静止图像时,必须保持知识产权信息IPR。 JPEG2000的目标:  JPEG2000的目标 15 Side channel spatial information (transparency): alpha平面和透明平面之类的边通道空间信息,可用于传输有关图像处理(如显示、打印或编辑等)信息时有用,例如透明平面可用于WWW应用。 JPEG2000的目标:  JPEG2000的目标 16 Object Based Functionality - Object based composition: 具有任意形状(或透明性)的多个对象将允许用户能非常灵活地复合出许多不同的静止图像。 Object based information embedding: 附加在每个对象上的对象描述信息,为数字图像的索引和检索提供了一种有效的方法。 2.4 Digital Item Identification and Description:  2.4 Digital Item Identification and Description Rationale:  Rationale digital item that is identified and described is more manageable and bears a higher value. A digital item that is identified and described enables a large number of possible applications including IPMP, search, filtering, cataloguing, … 经过标识和描述的数据项更便于管理,因而具有更高的使用价值。经过标识和描述的数据项将会使大量的应用成为可能,例如: IPMP、搜索、滤波、分类 ……。 Expected impact :  Expected impact A framework for common identification and description of digital items will enable efficient deployment of business models and applications requiring rights management, automated transaction and billing, monitoring, search, retrieval, cataloguing, … 一个用于数字项的公用标识和描述框架,将会使得需要版权管理、自动交易和支付、监控、搜索、检索、目录服务等功能的商业模型和应用得到更加有效的开展。 Requirements 1(标识什么?):  Requirements 1(标识什么?) Content: which could be a representation of the abstract work(内容的摘要表示) Transactions and/or Contracts: such as transaction number(内容交易和/或合同的标识) Physical and/or Legal Persons: Rights Holders, Licensees, Publishers, Providers, Distributors, Retailers, Consumers(物理人和/或法人的标识) Usage rules: Copy, Pay per view, Pay per listen, …(使用规则的标识) Requirements 2 (对标识的使用):  Requirements 2 (对标识的使用) Access authorization(访问授权): read, change, write, … 与标识符和描述符的交互应是灵活的、多种多样的。例如,根据不同的应用,对它们的存取可以采用简单的、加密的或者嵌入的方式,可以授权也可以不授权。 Requirements 3 (标识的操作模式):  Requirements 3 (标识的操作模式) Systems and processes for identification should ensure persistency and consistency.标识系统和标识方法应保证持久性和一致性。 Systems for description should ensure consistency. Identification and description should be coherent with each other.描述系统应保证一致性。标识与描述应连贯一致。 More over, they should also allow both static and dynamic identification and description schemas e.g. the ID and description of a digital item could be changed when its right holder changes.应既允许静态的标识与描述方案也允许动态的方案,例如当数字项的版权持有者改变时, 其ID和描述也应随之改变。 Requirements 4 (标识的成本):  Requirements 4 (标识的成本) Systems and processes for identification and description should allow a full range of applications and business models from low cost and simple approaches to richer and more sophisticated ones. 用于标识与描述的系统和方法,应允许全范围的应用和商业模型, 从低成本、简单方法到高代价、复杂方法都能支持。 Current Situation 1:  Current Situation 1 所有权标识(辨认)是一项支离破碎的、无穷尽的、动态变化的工作,它涉及法律(legal)和契约(contractual)问题. 目前,所有权标识系统与标准化的标识方法同时存在,一些标识符已成功地应用了多年,但只是针对单媒体类型, 如ISBN(国际标准书号)、ISRC(International Standard Recording Code)、URN(Universal Resource Number全球资源号)、 URI(Universal Resource Identifier全球资源标识符)等。 有些工作正在进行之中,如 ISAN(International Standard Audiovisual Number国际标准音视频号码)、 ISWC(Information System Work Code信息系统工作码)、 DOI(Digital Object Initiative数字对象创新计划)、 cIDf(content ID forum,内容 ID论坛)。 Current Situation 2:  Current Situation 2 Lack of integration between the different schemas of different sectors(不同部门不同方案之间缺少集成) Lack of standard dynamic identification schemas when entities related to the content (digital item) change(当与内容关联的实体变化时,缺乏标准的动态标识方法) Lack of standard methods to differentiate between versions of a digital item when it is revised(缺乏区分不同版本数字项的标准方法) Current Situation 3:  Current Situation 3 Lack of interoperability among identification schemas for the purpose of common applications e.g. automated transaction/billing systems.(在自动交易/支付系统之类的公共应用中,标识方案缺少互操作性) Lack of international standard identification schemas for some entities (i.e. text, images, speech, etc.)(某些实体缺少国际标准的标识方案) Lack of granularity definition/guideline for giving an ID to a digital item(为数字项赋于ID时,缺少粒度大小的定义/指导) Current Situation 4:  Current Situation 4 Lack of scalability in the identification process accommodating from self-publishing to large scale identification.(标识过程中缺少可分级性) Conflicting requirements to identify a digital item depending on the purpose of identification, such as Commerce (search, purchase, acquisition, etc) Rights management 由于标识的目的不同(如商业目的和版权管理目的),引起需求的冲突. Inconsistency of schemas between and within media sectors(在媒体之间或媒体内部标识方案的不一致性) 举例: MPEG-7 (Multimedia Content Description Interface) :  举例: MPEG-7 (Multimedia Content Description Interface) Context(背景):  Context(背景) 世界各地的数字化视听信息越来越多. 人们在使用任何信息之前,首先要找到信息。然而,信息越来越多,检索越来越困难. 目前,检索文本信息有一些办法,WWW上就有许多text-based search engines . 然而,由于视听材料没有通用的可识别的描述,因此无法进行识别. 寻找内容的问题不仅在数据库检索应用中存在,在其它领域也存在,如数字广播的频道越来越多,如何找到感兴趣的频道就是一个例子. What is the MPEG-7 ?:  What is the MPEG-7 ? MPEG-7是一种用于信息检索的内容表示的标准。 MPEG-7 will be a standardized description of various types of multimedia information. This description will be associated with the content itself, to allow fast and efficient searching for material that is of interest to the user. MPEG-7将建立各种类型多媒体信息的标准的描述方法。这种描述与内容关联在一起,支持对用户感兴趣的材料的快速、高效的检索。 所谓“材料”包括:静止的画面、图形、声音、运动视频以及上述元素如何结合成多媒体信息(“剧情”、合成信息), 还包括上述通用数据类型中的特例,如面部表情、人物特点等。 MPEG-7对信息的“描述”:  MPEG-7对信息的“描述” “描述”是依赖于应用的,所以不同用户、不同应用所使用的描述是不同的。也就是说,同一材料可以使用不同类型的特征进行描述。例如, 视频材料的低级抽象是它的shape, size, texture, colour, movement (trajectory) and position (‘where in the scene can the object be found?)等的描述. 音频材料的低级抽象是它的key, mood, tempo, tempo changes, position in sound space等的描述; 而高级的描述也许是给出语义信息如: ‘This is a scene with a barking brown dog on the left and a blue ball that falls down on the right, with the sound of passing cars in the background.’ 文本,图像,音频和视频媒体的 特征描述(元数据)举例:  文本,图像,音频和视频媒体的 特征描述(元数据)举例 MPEG-7对信息的“描述”:  MPEG-7对信息的“描述” 除了内容的特征描述之外, 也许还需要其他类型的有关多媒体数据的信息,如: The form - An example of the form is the coding scheme used (e.g. JPEG, MPEG-2), or the overall data size. This information helps determining whether the material can be ‘read’ by the user. Conditions for accessing the material - This could include copyright information, and price; Classification - This could include parental rating, and content classification into a number of pre-defined categories; Links to other relevant material - The information may help the user speeding up the search. The context - In the case of recorded non-fiction content, it is very important to know the occasion of the recording (e.g. Olympic Games 1996, final of 200 meter hurdles, men) MPEG-7对信息的“描述”:  MPEG-7对信息的“描述” 许多情况下使用文本信息(textual information)进行描述,然而描述的usefulness应尽可能与语言无关. (A very clear example where text comes in handy is in giving names of authors, film, places.) MPEG-7 的描述数据可以与相关的 AV material在一起(same data stream or on the same storage system), 也可分开。在分开的情况下需要有mechanisms 把两者双向link起来。 Scope of MPEG-7:  Scope of MPEG-7 automatic extraction of features (or ‘descriptors’) 很重要,但不在标准的范围之内,原因是不需要具有互操作性. Search engines也不在标准的范围之内. MPEG-7 Objectives:  MPEG-7 Objectives MPEG-7 的目标是: 规定 a standard set of descriptors 用来描述各种类型的多媒体信息, 对描述符及其相互关系的预定义结构(称为Description Schemes)以及用户定义自己的结构的方法进行标准化, 制定一种用于定义新的description schemes的标准化的语言 DDL, (Description Definition Language ), 把描述 (i.e. a set of instantiated Description Schemes) 与内容关联在一起,以便用户快速有效地检索感兴趣的材料, 把描述进行编码表示,以便有效地进行存储和快速访问 . An abstract representation of possible applications using MPEG-7:  An abstract representation of possible applications using MPEG-7 上图是使用MPEG-7的各种可能的应用的抽象表示;所使用的编码器和解码器是可选的;多媒体 Content到用户的流在图中没有具体表示出来; Areas of Interest:  Areas of Interest 许多应用和应用领域都会从MPEG-7 standard得到好处,例如: Digital libraries (image catalogue, musical dictionary,…) Multimedia directory services (e.g. yellow pages) Broadcast media selection (radio channel, TV channel,…) Multimedia editing (personalized electronic news service, media authoring) 可能的应用领域:  可能的应用领域 Education, Journalism (e.g. searching speeches of a certain politician using his name, his voice or his face), Tourist information, Cultural services (history museums, art galleries, etc.), Entertainment (e.g. searching a game, karaoke), Investigation services (human characteristics recognition, forensics), Geographical information systems, Remote sensing (cartography, ecology, natural resources management, etc.), Surveillance (traffic control, surface transportation, non-destructive testing in hostile environments, etc.), Bio-medical applications, Shopping (e.g. searching for clothes that you like), Architecture, real estate, and interior design, Social (e.g. dating services), and Film, Video and Radio archives. 2.5 Content Management and Usage:  2.5 Content Management and Usage 背景:  背景 网络中的内容以及对内容的存取正在以指数规律增长。多媒体技术的目标是使得通过各式各样的网络或设备透明地使用内容成为可能. 有利于内容的搜索、定位、缓冲、存档、路由、发布和使用(searching, locating, caching, archiving, routing, distributing and using content〕的标准显得越来越重要。此外,内容管理还应包括个性化以及用户概要(profile)管理。 Goal::  Goal: the MPEG-21 framework should provide interfaces and protocols that enable creation, manipulation, search, access, storage, delivery, and (re)use of content across the content distribution and consumption value chain; with emphasis on improving the interaction model for users with personalization and content management. MPEG-21框架应能提供一组接口和协议, 沿着内容分发和消费的价值链, 使能建立、操纵、搜索、访问、存储、发送和(重)使用内容,重点是以用户的个性化服务和内容管理来改善交互模型. 情况1:  情况1 Majority of the content lacks identity and descriptions Content that is available currently on the Internet and broadcast networks lacks identity and persistent descriptions. The problem is made worse by the fact that there is no mechanism to ensure that this identity and description information is persistently associated with the content. To enable any kind of management of content, this is a fundamental requirement. 情况2:  情况2 Accessibility of content is becoming widespread to lots of devices (set-top boxes for terrestrial/cable/satellite networks, personal digital assistants, mobile phones, computers, TV etc.) Consumers’ appetite for content and the accessibility of information is increasing at an incredible pace. The access devices with a myriad set of differing capabilities are making their way into consumers’ homes and offices. These access devices have different processing and rendering capabilities and are used in different localities, posing a challenge to service providers to ensure that their content is available and is used/rendered in a meaningful way on these devices. 情况 3:  情况 3 Users are flooded with content presently and this is going to get worse in the future. They need systems that can manage, categorize, and filter content. With the availability of large amounts of content to consumers, consumers are overwhelmed and are looking for solutions that will simply manage and organize content for them. This problem is going to get in the future and will also hinder businesses from getting consumers’ attention among all the noise. 情况 4:  情况 4 Users are becoming more mobile and have a need to access similar information on multiple devices in multiple places. Currently content providers or authors have to author multiple formats of content and deploy them in each of the networks. There is no automated configurable way of delivering and consuming content that scales automatically to different device profiles. 情况 5:  情况 5 Decision making about content selection and acquisition is dependent on the specific service provider and the type of service. Selection of content that fits a user’s needs and profile is currently dependent on the specific network or service being accessed. There is no way to express your preferences and build up a profile that can be used by a non-human entity to acquire content irrespective of service provider and service. 情况 6:  情况 6 Content caching and management is typically not available on most terminals. When it is present, it is quite limited and does not consider the requirements of rights holders or user’s right to privacy. Content caching is an issue that is pervasive across the network. There are solutions that address issues in a specific domain and is not conducive to managing lifetimes and usage rules in a flexible manner. 情况 7:  情况 7 Content is typically packaged for specific services/delivery/devices with little ability to move from one environment to the other or be consumed on different devices. 情况 8:  情况 8 Currently, text is the only type of content that is ubiquitously searchable. For other types of media, search is typically limited to text input. 情况 9:  情况 9 Typically content is stored within user’s control. But, this is changing and hence requiring better systems to manage and secure content on shared environments. 情况 10:  情况 10 Current asset management systems do not communicate with each other; content cannot readily be identified across those asset management systems, or accessed in an easy-to-use, distributed fashion. 情况 11:  情况 11 There is currently no user-available association of services with distributed content – standard “hooks” for services that can be associated with the media. For example, think of a digital item not as a single entity, but as a compound document consisting of multiple representations (e.g. it’s not just a digital still image, it’s a single digital still image available in a variety of forms: in JPG format, in GIF format, in PNG format, in PDF, at 640x480 resolution, at 1200x1600 resolution, etc.) The hooks we’re talking about may be pointers to the alternate representations of the digital item, or it could be a pointer to a service and data from which the alternate form can be created. This is one of many technologies currently being considered for digital libraries to track/implement versioning of digital items. 情况 12:  情况 12 Users currently need to have very explicit format and technology knowledge in order to utilize format-conversion tools. 小结:  小结 In summary, as illustrated above, there are numerous shortcomings throughout the distribution and consumption networks. Where standards do exist, they tend to address a specific problem or domain and are not comprehensive enough to make a difference to the complete solution. 总之, 在分发和消费内容的整个网络过程中, 存在着许多问题和不足, 虽然已经存在一些标准, 但它们一般都用于解决特点问题或者适用于特定领域, 还不是一个完整的解决方案. Requirements 1(内容存储管理):  Requirements 1(内容存储管理) 1 Search, Storage and Retrieval of Content and Descriptions内容和描述的搜寻,存储和检索 · Users should be able to search and locate content of interest:用户应能够搜寻和定位感兴趣的内容,包括通过代理进行搜寻和采集和跨越分布式资产管理系统的搜寻 Including searches and acquisition by agents Search across distributed asset management systems · Users in the network must be able to securely store and retrieve content.网络用户可以安全地存储和检索内容 Requirements 1(1.内容存储管理):  Requirements 1(1.内容存储管理) · Ability to search and locate content at finer levels of detail within a container should be provided. 提供在包容器中能进行细节的精细级搜寻和定位的能力。 · Lifetime control of content within all the nodes on a given network should be supported with configurable policies.通过可配置策略,支持在给定网络的所有节点中对内容的终身控制 · User should be able to identify where all copies of content are located, as well as any usage rights that are in effect.用户应能识别出内容的所有拷贝的所在地,以及其使用权是否有效. Requirements 1(2. 格式和定义):  Requirements 1(2. 格式和定义) 2 Formats and Protocols Serialization formats for digital items should be defined within the framework.在MPEG-21中应为数字项的连载格式进行定义 Requirements 1(3. 归档和编目):  Requirements 1(3. 归档和编目) 3 Archiving and Cataloguing Support for archiving content for later use along with preservation of any associated rights. 支持内容归档,以便以后的使用, 并保留所有相关权利。 Ability to organize and catalogue content within collections should be supported.能对一定集合中的内容进行组织和分类编目的能力。 Requirements 1(4.受控的访问和修改跟踪):  Requirements 1(4.受控的访问和修改跟踪) 4 Controlled Access and Change Tracking Ability to control access and modification of content and associated descriptions by users and groups of users. 能控制用户和用户组对内容及相关描述进行的存取和修改。 Ability to track changes and versions of content with associated descriptions.能跟踪内容及相关描述的变化和版本。 Requirements 2(用户概要管理):  Requirements 2(用户概要管理) Exchanging user profiles for services System should clearly identify what information a user is willing to trade in exchange for services (name, address, credit card, usage history, profile). System should be able to match the profiles of a user’s privacy preferences with service provider’s requirements for service delivery.系统应能清楚识别用户意愿在得到服务时交换哪些信息(名字、地址、信用卡、使用历史、简况等)。在提供服务时,系统应能够将用户的个人选择与服务供应商的需求进行匹配。 Requirements 3(跟踪和性能测量) Tracking and Performance Metrics:  Requirements 3(跟踪和性能测量) Tracking and Performance Metrics 1 Predefined Set of Content Metrics Framework should predefine a set of metrics applicable to content usage. These metrics should be based upon framework wide metric representation model. Some predefined content personalization metrics could include:框架应预定义一组应用于内容使用的计量单位,如内容个性化的计量是: Number of times content was rendered Last time content was rendered How many times was it purchased How long was it used? How many times was it completely listened to, etc? Requirements 3(跟踪和性能测量):  Requirements 3(跟踪和性能测量) 2 Packaging of metric information for exchange Any user should be able to package metric information for exchange with other users in the distribution chain. This user should be allowed to make this content for sale in the framework任何用户都能将计量信息打成包与分配链上的其它用户进行交换。该用户应能允许销售该内容。 Requirements 4(内容和展现的个性化):  Requirements 4(内容和展现的个性化) Arbitrary Organization of content User should be able to add descriptions to an item and or a container that may be used to organize a group of items with related descriptions. 内容的任意组织:用户可在数字项或者包容器上添加新的描述,从而以相关的描述来组织一组数字项。 Automated Organization of content System should allow organization of content based on user preferences and appropriate descriptors associated with the items.内容的自动组织:系统应能根据用户的喜好和与数字项关联的合适的描述符来组织内容 Requirements 5(网络和终端允许使用代理):  Requirements 5(网络和终端允许使用代理) A better management of the complexity of the content management and usage can be achieved by giving the user the possibility to delegate some specific functions (instantiated in a goal) to non-human entities. Examples of goals would be:一个更好的内容管理与使用的方法是用户将一些特定功能交给“非人实体”(no human entities)来完成,例如: routing of content across distributed networks in an intelligent manner. 跨越分布式网络以智能方式确定内容的传送路径。 Requirements 5(网络和终端允许使用代理):  Requirements 5(网络和终端允许使用代理) In the Internet domain, decide on the content set to be served up, based on connection characteristics (bandwidth, network type, reliability, QoS etc.) and profile of the requesting device.在互连网领域,根据连接特性(带宽、网络类型、可靠性、 QoS等)和设备情况决定提供的内容组。 In the broadcast domain, filter and insert content into the multiplex by being aware of the programming information.在广播领域,通过了解节目信息而对内容进行滤波和插入。 Slide122:  make decisions on intelligent ways to cache content within the network for best delivery. 以智能方法决定是否需要在网络中 对内容作高速缓存,以便更好地进行传送 search and locate content based on identifications and descriptions at levels within the content store subject to privacy and rights constraints.在服从私密性和版权的约束下,在内容存储中根据标识和描述在各个级别搜寻和定位内容 negotiate deals with other agents for the purpose of reaching certain assigned goals, including user’s right to privacy.为了达到一定的目的与其它代理进行协商(包括用户的私密权)。 3 Networked Multimedia Appl.:  3 Networked Multimedia Appl. 3.1 信息技术发展的3个定律:  3.1 信息技术发展的3个定律 第一定律:摩尔定律(Moore’s Law) :  第一定律:摩尔定律(Moore’s Law) 摩尔定律:微处理器的速度会每18个月翻一番。这就意味着每五年它的速度会快十倍,每十年会快一百倍。同等价位的微处理器会越变越快,同等速度的微处理器会越变越便宜。同样,内存空间、硬盘空间、LCD以及图像处理速度等也都适用于摩尔定律。 计算最终将无所不在,而且是免费的。 第二定律:吉尔德定律(Gilder’s Law):  第二定律:吉尔德定律(Gilder’s Law) 在未来25年,主干网的带宽将每6个月增加一倍。其增长速度超过摩尔定律预测的CPU增长速度的3倍! 不但计算免费,带宽也会免费(零成本通信)。 光传输能力的增长:  光传输能力的增长 Optical multiplexing extends electronic multiplexing up to Terabits 10Gb/s - Highest electronic multiplexing today OC-48 135Mb/s 565Mb/s 1.7Gb/s OC-192, 2 OC-192, 32 OC-48, 40 OC-48, 96 OC-192, 4 OC-192 = 10 Gb/s OC-48 = 2.5 Gb/s OC-192, 16 第三定律:麦特卡尔夫定律(Metcalfe’s Law):  第三定律:麦特卡尔夫定律(Metcalfe’s Law) 网络价值同网络用户数量的平方成正比 (即N个联结能创造N2的效益)。 · 电话是一个人打给另外一个人,信息是从一到一,得到的效率是1。 · 电视是一个节目有N个人在收看,信息是从一到N,得到的效率是N。 · 在网上,每一个人都能够联到N个人的网站, N个人能看到N个人的信息,所以信息的传送效率是N2。 万人上网,亿份效益 互连网的特性:  互连网的特性 空前的信息容纳能力 高速的信息传递能力 有力的信息组织与检索能力 普遍的可连接性(时间,地点,设备) 多种多样的信息媒体 消除了人们进行信息交流的时空限制,媒体限制,语言限制,.... 万物均联网:  万物均联网 我们已经接近这样一个现实,那就是任何物体与任何物体都可以联通,任何人都可以找到世界上的任何信息,任何人可以和任何人交朋友。这种变化已经远远超出了改善生产效率的层面。 3.2 网络多媒体应用模式:  3.2 网络多媒体应用模式 Networked Multimedia Applications (1) People-people applications:  Networked Multimedia Applications (1) People-people applications 目的: 改善人与人之间的远程通信. 特点:至少有2人以上参与. between 2 individuals, between group of people 分类: 同步方式 异步方式 与CSCW交叉,为高级CSCW提供了条件 Networked Multimedia Applications (1) People-people applications:  Networked Multimedia Applications (1) People-people applications Networked Multimedia Applications (2) People-system applications:  目的: 改善或提供一种创新的人与信息源之间的通信. 根据通信时访问信息服务器的方式,可分为: 交互方式(interactive), 或“提拉”方式(pull) 分配方式(distribution),或“推送”方式(push) Networked Multimedia Applications (2) People-system applications Slide135:  Networked Multimedia Applications (2) People-system applications 多媒体系统的用户模型:  多媒体系统的用户模型 User Requirements (从技术角度):  User Requirements (从技术角度) Interoperability Transparency Robustness Integrity Scalability Flexibility Customization Event management Protection Rights management Tangibility Metrics and interfaces. Universal Access:  Universal Access New classes of pervasive computing devices such as personal digital assistants (PDAs),hand-held computers (HHC), smart phones, automotive computing devices, and wearable computers allow users more ubiquitous access to information than ever. As users are beginning to rely more heavily on pervasive computing devices, there is a growing need for applications to bring multimedia information to the devices. The basic idea of Universal Multimedia Access is to enable client devices with limited communication, processing, storage and display capabilities to access rich multimedia content. Adaptation of multimedia content to pervasive computing devices. 多媒体技术的总目标 :  多媒体技术的总目标 To enable transparent and augmented use of multimedia resources across a wide range of networks and devices. 使能跨越各种不同网络和设备, 透明地、强化地使用多媒体资源。 谢 谢!:  谢 谢!

Add a comment

Comments

giubbini moncler | 25/01/15
I am always browsing online for ideas that can benefit me. Thanks! giubbini moncler http://ovomattino.it/img/index.asp
 | 19/03/15
 | 30/03/15
 | 02/04/15
Georgeres | 21/04/15
Enjoy your blog ) http://anmdcf7i.com my blog

Related presentations

Related pages

Ancient landslide-dam events in the Jishi Gorge, upper ...

E-mail addresses: ghdong@lzu.edu.cn (G. Dong), zhangfy@lzu.edu.cn (F. Zhang). ... slides by radiocarbon dating the soils of shear zones found in ancient
Read more

Effect of pore-water chemistry on undrained shear ...

these hazardous landslides, loess flow slides are the most destruc-tive because of their high mobility, ... (e-mail: zhangfy@lzu.edu.cn) Abstract: ...
Read more

European Journal of Environmental and Civil Engineering

Shallow landslides play an important role in influencing topographic attributes and geotechnical properties. The landslides inventory is mapped using ...
Read more

agricultural information network: Topics by National ...

USDA Global Agricultural Information Network ... zhangfy@mail.dlut.edu.cn ... Call Slides and Discussion Summary, ...
Read more

Ancient landslide-dam events in the Jishi Gorge, upper ...

Fax: +86 931 8912330. E-mail addresses: ghdong@lzu.edu.cn Dong),zhangfy@lzu.edu.cn Zhang) ... We aim ancientland- slides dammed-lakeevent JishiGorge, ...
Read more

Physicochemical and index properties of loess stabilized ...

Tel.: +86 931 8914560; fax: +86 931 8914561. E-mail address: Zhangfy@lzu.edu.cn ... These treatments were applied to different slides (i.e., air-dried ...
Read more

Spatial variation in geotechnical properties and ...

Spatial variation in geotechnical properties and topographic attributes on the different types of shallow landslides in a loess catchment, China
Read more

www.osti.gov

Wireless technology collects real-time information from oil and... Broader source: All U.S. Department of Energy (DOE) Office Webpages (Extended Search ...
Read more

Kyoji Sassa, Paolo Canuti, Yueping Yin (Eds.)-Landslide ...

Kyoji Sassa, Paolo Canuti, Yueping Yin (Eds.)-Landslide Science for a Safer Geoenvironment_ Volume 2_ Methods of Landslide Studies-Springer International ...
Read more