TW201119346A - Media extractor tracks for file format track selection - Google Patents

Abstract

A video coding apparatus may be configured to utilize media extractors in a media extractor track that reference two or more non-consecutive network access layer (NAL) units of a separate track. An example apparatus includes a multiplexer to construct a first track including a video sample comprising NAL units, based on encoded video data, wherein the video sample is included in an access unit, construct a second track including an extractor that identifies at least first one of the NAL units in the video sample of the first track, and wherein the extractor identifies a second NAL unit of the access unit, wherein the first identified NAL unit and the second identified NAL unit are non-consecutive, and include the first track and the second track in a video file conforming at least in part to ISO base media file format. The identified NAL units may be in separate tracks.

Description

201119346 VI. Description of the Invention: TECHNICAL FIELD OF THE INVENTION The present invention relates to the transmission of encoded video material. This application for the following US temporary application claims: September 9th, September, b, US application for temporary application No. 61/243, 〇3〇, September 29, September 曰 Shen Qing U.S. Provisional Application No. 61/244, 827, 〇 〇 i 曰 曰 曰 之 之 之 之 之 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 61 61 61 61 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国 美国295,261, the entire contents of each of which are incorporated herein by reference. [Prior Art] Digital video capabilities can be incorporated into a wide range of devices, including digital TVs, digital live systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital Cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones, video teleconferencing devices and the like. Digital video devices implement video compression techniques, such as those defined by mpeg_2, MPEG-4, ITU-T H.263 or ITU-T H.264/MPEG-4 Part 10 Advanced Video Recording (AVC) and The video compression technology described in the extension of these standards to transmit and receive digital video information more efficiently. Video compression techniques perform spatial prediction and/or temporal prediction to reduce or remove redundancy inherent in video sequences. For block-based video writing, the video frame or slice can be split into giant blocks. Each giant block can be further divided. In-frame code (1) The giant block in the frame or slice makes the code for the space predictor of the adjacent giant block. Inter-frame coding (a megablock in a p-frame or slice can be predicted relative to other reference frames relative to the same frame or block (4) (4). After the video material has been encoded, it can be The Guardian encapsulates the video data for transmission or storage. The G_2 packet is used to reduce the "system" part of the transmission level of many video coding standards. The G_2 transmission level system can be encoded by the 2 2 video encoder or according to different video _ (4) codes. For example, MPEG_4 specifies a coding and decoding method different from MPEG], but the video encoder that implements the MpEG_4 standard technology can still use the MPEG_2 transmission level method. Generally speaking, 'right' The reference to the MPEG-2 system refers to the transmission level of the video material specified by MpEG_2. The transmission level specified by MPEG_2 is also referred to as "MPEG-2 Transport Stream" or simply "Transport Stream" in the present invention. The transport level of the MPEG-2 system also includes the program stream. The transport stream and the program stream usually include different formats for delivering similar data, and the # transport stream includes audio data and video assets. One or more "programs", and the program stream includes a program that includes both audio and video data. Efforts have been made to develop a new video writing standard based on H.264/AVC. Expandable video write code (SVC) for H.264/AVC expandable extension. Another standard is multi-view video write code (mvc), which is a multi-view extension for H.264/AVC. The }2 system specification describes how compressed multimedia (video and audio) data streams can be multiplexed with other data to form a single data stream suitable for digital transmission or storage. J51028.doc 201119346 May Information Technology-Generic Coding of Moving Pictures and Associated Audio: Systems, Recommendation H.222.0; International Organisation for Standardisation, ISO/IEC JTC1/SC29/WG11; Coding of Moving Pictures and Associated Audio" MPEG has recently been designed to be superior to the MVC transmission standard for MPEG-2 systems, and the latest version of this specification is "Study of ISO/IEC 13818-1:2007/FPDAM4 Transport of MVC", MPEG do c. N10572, MPEG of ISO/IEC JTC1/SC29/WG11 (Maui, Hawaii, USA, April 2009). The latest joint draft of MVC is described in JVT-AB204, "Joint Draft 8.0 on Multiview Video Coding" (28th JVT Conference, July 2008), available at http://wftp3.itu.int Obtained at /av-arch/jvt-site/2008_07_Hannover/JVT-AB204.zip. A newer version integrated into the AVC standard is described in JVT-AD007, "Editors" draft revision to ITU-T Rec. H.264 | ISO/IEC 14496-10 Advanced Video Coding - in preparation for ITU-T SG 16 AAP Consent (in integrated form) (30th JVT meeting in Geneva, Switzerland, February 2009), available at http://wftp3.itu.int/av-arch/jvt-site/2009_01_Geneva/JVT-AD007 Obtained at .zip. SUMMARY OF THE INVENTION In general, the present invention describes techniques for using a media extractor to form a media extractor trajectory in a multi-track video data format. The present invention modifies the International Standards Organization (ISO) base media format to utilize an extractor capable of reference to one or more potentially discontinuous network access layer (NAL) singles 151028.doc 201119346. This extractor can exist in any track of the _ISO base media format building. The present invention also describes one of the attributes of the third generation partnership program (the modification of the 3Gp (four) case format to include the frame rate value as the - track selection box selecti〇n b〇x). This is a pure day to describe the use of the extractor to support efficient extraction of MVC operating points with respect to the multiview video code (MVC) extension to the IS0 base media format. In the example, the method for encoding video data includes: constructing a first track by using a source video device based on the encoded video data, the first track comprising a plurality of network access layers (Nal) The video sample of the unit is included in the access unit; the source video device constructs a second track including an extractor, and the extractor identifies the video sample in the first track At least one of the plurality of NAL units, the at least one of the plurality of NAL units including a first identified NAL unit, and wherein the extractor identifies a second NAL unit of the access unit, wherein the The first identified NAL unit and the second identified NAL unit are discontinuous; the first track and the second track are included in a video that is at least partially in accordance with an International Standards Organization (IS) basic media file format In the case of the building; and output the video building case. In another example, an apparatus for encoding video data includes: an encoder configured to encode video material; a multiplexer configured to: construct a video based on the encoded video material a first track, the first track includes a video sample including a plurality of network access layer (NAL) units, wherein the video sample is included in an access unit; and a second track including an extractor is constructed The extractor identifies at least one of the plurality of said units of the first trajectory of the first trajectory, wherein the at least one of the plurality of NA4 包含 includes - the first identified nal unit And wherein the extractor identifies one of the second NAL units of the accessary, wherein the first identified unit and the second identified hidden unit are discontinuous; the first track and the second The manifestation is included in a video slot at least partially in accordance with the International Standards Organization (IS) basic media slot format; and an output interface configured to output the video file. In another example, an apparatus for encoding video data includes: means for constructing based on encoded video data - a component of a first track comprising - a plurality of network access layer (nal) units a video sample, wherein the video sample is included in an access unit; configured to construct a component including a second track of the benefit, the extractor identifying the plurality of NAL units in the video sample of the first trace In at least one of the plurality of NAL units, the at least one of the plurality of NAL units includes a first identified weidan Fan and wherein the extractor identifies the access unit - the second fox unit. The NAL unit and the nal unit are discontinuous; the first trajectory and the second trajectory are included in a video file at least partially conforming to the International Standards Organization (IS) basic media slot format The component in the device; and the component for outputting the video file. In another example, a computer readable storage medium includes instructions that, when executed, cause a processor of the source device to: construct a first trajectory based on the encoded video material, the A track includes - a video sample comprising a plurality of network access layer (NAL) units, wherein the video sample is included in an access unit; constructing a second 151028.doc 201119346 track including an extractor, the extracting Identifying at least one of the plurality of NAL units in the video sample of the first representation, the at least one of the plurality of Nal units including a first identified NAL unit, and wherein the extracting Identifying a second NAL unit of the access unit, wherein the first identified NAL unit and the first identified NAL unit are discontinuous; the first track and the second track are included in one At least partially in accordance with the International Standards Organization (ISO) basic media file format of the video file; and output the video file. In another example, a method for decoding video data includes receiving, by a multiplexer of a destination device, a video file at least partially in accordance with an International Standards Organization (ISO) basic media file format. a first track and a second track, the first track comprising a view sample comprising a plurality of network access layer (NAL) units corresponding to the encoded video data, wherein the video sample is included in an access unit And the second track includes an extractor identifying at least one of the plurality of NAL units of the first track, the at least one of the plurality of NAL units including - the first identified NAL unit, And wherein the extractor identifies the first NAL unit of the access unit, wherein the first identified unit and the first k identified NAL unit are discontinuous; selecting the second track for decoding, and The encoded video data of the first NAL unit and the first NAL unit identified by the extractor of the two tracks is sent to one of the video decoders of the destination device. In another example, an apparatus for decoding video data includes: a video decoder configured to decode video data; and a demultiplexer, 151028.doc -9 - 201119346 configured Receiving a video file at least partially in accordance with the International Standards Organization (7) basic media file format, the video file comprising - a first track and a second track, the first track comprising an image corresponding to the encoded video data a plurality of video samples of a network access layer (NAL) unit, wherein the video sample is included in an access unit, and the second track includes two (four) at least one of the plurality of NAL units of the first track— Extractor, the at least one of the plurality of NAL units includes - the first identified NAL, and wherein the extractor identifies one of the access units, the second unit j, wherein the first NAL unit and the second identified NAL unit are discontinuous; selecting the second track to perform encoded video of the first NAL unit and the second NAL unit identified by the extractor of the second track Data sent to the view decoder. In another example, an apparatus for decoding video data includes: means for receiving, by a demultiplexer of a destination device, a video file at least partially in accordance with an International Standards Organization (ISO) basic media file format The video file includes a first track and a second track. The first track package includes a video sample including a plurality of network access layer (NAL) units corresponding to the encoded video data, and the video sample is obtained. Included in an access unit, and the second track includes an extractor that identifies at least one of the plurality of fox units of the first-permission, the at least one of the plurality of NAL units An identified NAL unit, and wherein the extractor identifies the HAL unit of the access unit, the towel identifying the unit and the identified NAL unit being discontinuous, for selecting the a second trajectory for decoding; and for transmitting the encoded _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ The destination device Component video decoder. In another example, a computer readable storage medium is encoded with instructions that, when executed, cause a processor of a destination device to perform the following operations: upon receiving an at least partial compliance with an international standard organization (is) After the video file of the media file format, the video file includes a first track and a second track. The first track includes a plurality of network access layer (NAL) units corresponding to the encoded video data. a video sample, wherein the video sample is included in an access unit command, and the second track includes an extractor identifying at least one of the plurality of NAL units of the first track, the plurality of NAL units At least one of the first identified NAL units, and wherein the extractor identifies a second NAl unit of the access unit, wherein the first identified NAL unit and the second identified NAL unit are Continuously selecting the second track for decoding; and transmitting the encoded video data of the first NAL unit and the second NAL unit identified by the extractor of the second track of the second path to a video decoder. Details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objectives, and advantages will be apparent from the description and drawings and claims. [Embodiment] The technology of the present invention is generally directed to enhancing the extension of the International Standards Organization (IS) basic media file format and the ISO base media file format. Extensions to the ISO base media file format include, for example, Advanced Video Recording (AVC) file format, expandable video code (SVC) file format, multi-view video code 151028.doc -11 - 201119346 (MVC) file format And the 3rd Generation Partnership Project (3Gpp) file format. In general, the techniques of the present invention can be used to generate extended media extractor tracks in the IS〇 base media file format and/or the ISO base media file format. As described in more detail below, in some examples, such media extractor trajectories may be used to support adaptation in hypertext transfer protocol (Ηττρ) video streams. In some examples, the media extractor forms part of an extension of the ISO base media file format and/or the ISO base media file format (eg, AVC, SVC, MVC, and 3GPP) to extract the entire sample of another track to form a new media extraction. Trajectory. Such techniques may be used by the MPEG-2 (Animation Experts Group) system (i.e., systems that comply with MPEG-2 in terms of transmission level details). MPEG_4 (for example) provides a standard for video coding, but it is generally assumed that a video encoder that complies with the MpEG_4 standard will utilize the MPEG_2 transmission level system. Thus, the techniques of the present invention are applicable to video encoders that conform to: MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, or utilize MPEG-2 transport streams and/or Any other video coding standard that the program streams. The ISO base media file format provides a file that includes one or more tracks. The IS 0 Basic Media Archive Format Standard defines the execution as a sequence of time-series of related samples. The ISO Basic Media File Format Standard defines a sample as the data associated with a single-time stamp and provides a sample of the actual frame (4) for individual frames of video, in decoding order, for series of video frames, or for decoding in audio decoding order. Through the I contraction section. A special implementation called a "hint track" does not contain media material' and contains instructions for encapsulating one or more tracks in a streaming channel. The ISO Basic Media Archives Format Standard states that, in I5l028.doc •12· 201119346 Ding: Traces, the sample defines the formation of one or more stream packets. Month's technology provides the creation of media extractor trajectories. The media extractor track can typically include - or multiple extractors. The extract in the media extractor trajectory is used to identify and extract samples of the other trajectory. In this way, the media extractor in the trajectory of the mediator can be considered as an indicator, and the metrics are taken from another scrambled sample in the solution. Unlike the extractor of svc, for example, the extractor of this month can refer to another---------------------------------------------------------------------------------------------------------- In accordance with the teachings of the present invention, the track of the media extractor track s one or more media extractors and other tracks that do not include the media extractor may be grouped together to form an alternating group. The present invention uses the term "continuous" with respect to NAL units to describe two or more nal units that occur consecutively in the same trajectory. That is, when two NAL units are consecutive, the last bit group of the data in the units is immediately before the first byte of the data of another nal unit in the same track. . In the case where two NAL units in the same access unit are separated by a certain amount of data within the same trajectory, or in the case where one nal unit appears in the trajectory and another NAL unit appears in a different trajectory, it is generally considered The two NALs are "discontinuous". The technique of the present invention provides an extractor that can identify two or more discrete NAL units of an access unit. Moreover, the extractor of the present invention is not limited to SVC, but may generally be included in any other extension of the 1 s 0 base media file format or the IS 0 base media file format (such as 'AVC, SVC or 'VIVC). The extractor of the present invention may also be included in the 3rd Generation Partnership Project (3GPP) file format. The present invention 151028.doc • 13· 201119346 additionally provides an option to modify one of the 3GPP file format trace selection boxes. Using the explicit frame rate as a track media extractor track can be used in the Mvc slot format (for example) to support the extraction of operating points. The server device can provide various operating points in the MpEG_2 transport layer bitstream, each of which corresponds to a respective subset of the view of the multiview video $code video material. That is, the operating point passes through a subset of the view of the bit turbulence. In some instances, the parent-view of the operating point includes video data at the same frame rate. According to the present invention, the operating point can be represented by a media extractor trajectory, which includes reference to other executive video data, or multiple extraction benefits, and is not included in other trajectories. Potential extra samples. The use of this = U can simply include the necessary NAL units needed to decode the operating point to output a subset of the views at a common frame rate. The combination of the extractor trajectory' and the overall representation of the MVC video can form an MV: the representation is played 'early month'. The use of the media extractor trajectory of the present invention can support operating point selection and, for example, switching of operating points having various bit rate rates caused by time adjustable capabilities. The media extractor of the present invention can also form an alternating group or a switching group. That is, in the ISO base media file format t, the traces can be grouped together to form an alternating group. In the example of the assisted base media markup format, the trajectories of the alternating groups form a viable alternative to each other such that at any one time the alternate group of the alternate group of play or stream alternate groups is usually obligatory (eg ) Distinguish from other tracks of alternate groups via attributes such as bit 帛, codec, language, packet size, or other characteristics. 151028.doc -14- 201119346 On. The techniques of the present invention provide grouping of media extractor trajectories, media-containing trajectories, and/or other normal video trajectories to form groups. In the example of obeying MVC, each trajectory may correspond to "each" point. That is, each of the operating points in the MVC can be represented by one of the tracks: a trace (e.g., a media extractor or a tracker that does not include a media extractor). The same-alternative grouping—the falsification is usually selected for progressive downloading' to accommodate the available bandwidth. Similarly, media extractor trajectories and other trajectories can be grouped together to form a switching group of bribe file formats, and can be used for trajectory selection to accommodate bandwidth and decoder capabilities in HTTP streaming applications. The 3Gpp file format provides the definition of the (4) group of tracks. The switch group (4) belongs to the same two-group. According to the 3Gpp (4) format, the tracks in the same-switching group can be used to switch during the session, and the tracks in the different switching groups are not available for switching. 1 is a block diagram showing an example system 1 of an audio/video (A / V) source device 2 for transmitting audio data and video data to an A/V destination device 40. The A/V source device 20 may also be referred to as a "source video device." The system ι can correspond to a video teleconferencing system, a server/client system, a broadcast device/receiving system, or a video device from a source device (such as a/v source device 20) to a destination. Any other system of ground devices (such as Α/ν destination device 4〇). The A/V destination device 40 may also be referred to as a "destination video device" or a "customer device." In some examples, the Α/ν source device 2〇 and the A/V destination device 4〇 can perform two-way information exchange. That is, the Α/ν source device 20 and the A/V destination device 4〇 may be capable of encoding and decoding (and transmitting and receiving) audio and video data. In some examples, the audio encoder % may include a speech coder that is also a vocoder. The A/V source device 20 in the example of FIG. 1 includes an audio source 22 and a video source 24. The audio source 22 can include, for example, a microphone that produces an electrical signal representative of the captured audio material to be encoded by the audio encoder 26. Alternatively, the audio source η may comprise: a storage medium storing previously recorded audio material, an audio data generator such as a computerized synthesizer' or any other source of audio data. The video source 24 can include a video camera that produces video data to be encoded by the video encoder 28: a storage medium encoding the previously recorded video material, a video data generating unit, or any other source of video data. The original audio and video materials may contain analog or digital data. The analog data can be digitized prior to being encoded by the audio encoder 26 and/or the video encoder 28. The audio source 22 can obtain audio (4) from the conversation participant while the conversation participant is talking, and obtain the visual data of the conversation participant depending on (4) 2 (4). In other examples, audio source 22 can include a computer storable medium containing stored a data, and video source 2 includes a computer readable storage medium containing stored video data. In this manner, the techniques described in this disclosure can be applied to streaming live audio and video data or to pre-recorded audio and video data that are archived. The audio frame corresponding to the video frame is usually an audio frame containing audio data, which is captured by the audio source 22 and the video data captured by the video source 24 and contained in the video frame. For example, when a conversation participant typically generates audio data by talking, the audio source 22 captures the audio material, and the video source 24 is simultaneously (ie, when the audio source 22 is capturing the audio material). ) Capture video information about live participants. Therefore, the audio frame can correspond to one or more specific video frames in time. Therefore, the audio frame corresponding to the video frame generally corresponds to the simultaneous capture of the audio data and the video data, and for the sinusoidal situation, the audio frame and the video frame respectively contain the simultaneously captured audio data and video data. In an example, the audio encoder 26 can encode a timestamp in each encoded audio frame, the timestamp indicating the time at which the audio material of the encoded audio frame is recorded, and similarly, the video encoder 28 can be programmed to each other. The timestamp of the encoded videowire t is the time at which the video material of the encoded video frame is recorded. In these examples, the audio frame corresponding to the video frame may include an audio frame containing a time stamp and a video frame containing the same time stamp. The A/V source device 20 can be included in the audio encoder or/the video encoder 28, or can be used by the audio source 22 and the video source 24 to associate the audio data and the video device with the time. Internal clock. In some examples, the audio source 22 can transmit data corresponding to the time at which the audio material was recorded to the tone. The video source 24 can transmit the data corresponding to the time at which the video was recorded to the video code (4). In one example, the audio encoder 26 can be programmed

Kj| - , , t . 1 Beikeshen coded a sequence of identifiers 4 (4) when reading the encoded audio material, the absolute time of recording the audio data, and similarly, = order. Similarly, the relative timing of the actual shot/becaine is related to the time stamp. The mapping, or the technique of the present invention, is generally directed to the transmission of encoded multimedia (e.g., audio and video) data, and the receipt and subsequent interpretation and decoding of transmitted multimedia material. The techniques of the present invention are applicable to a variety of standard and extended video materials (such as scalable video write code (svc), advanced video write code (AVC), OSI base layer or multi-view video code (MVC) data) or The transmission of other video material containing a plurality of views. As shown in the example of Figure 视, video source 24 can provide video encoder 28 with a plurality of views of a scene. Multiple views of the video material can be used to generate 3D video material to be used by a 3D display, such as a glasses-type stereo or an ophthalmic stereoscopic 3D display. The A/V source device 20 can provide a "service" to the A/V destination device. The service typically corresponds to a subset of the available views of the MVC material. For example, multi-view video data can be used for eight views sorted from zero to seven. One service may correspond to stereoscopic video with two views, while another service may correspond to four viewing circles, and yet another service may correspond to all eight views. In general, a service corresponds to any combination of available views (ie, any subset). Services can also correspond to a combination of available views and audio data. A/V source device 20 is capable of providing services corresponding to a subset of views in accordance with the teachings of the present invention. In general, a view is represented by a view identifier also known as "Wew_id". The view identifier usually contains the syntax elements that can be used to identify the view. When encoding the view, the MVC encoder provides the view_id of the view. The view id can be used by the MVC decoder for inter-view predictions' or used by other units for other purposes (e. g., for display). 15I028.doc •18· 201119346 Inter-view prediction is used to refer to the MVC video data of the common faH code-frame as the “frame” as the coded frame of different views. 7 provides an example write code scheme for inter-view prediction. In general, the Mvc video data warp frame can be inter-predicted, and/or reference frames of other views at common time positions. Predictive coding. Therefore, the view for predicting other views is usually decoded before the reference view serves as the reference view. 1 These decoded views can be used for reference when decoding the view with reference content. The decoding order does not have to correspond. In the order of view "d". Because &, the view order index is used to describe the decoding order of the view. The view order index is an index indicating the decoding order of the corresponding view component in the access unit. 'Every: The individual data (information or video) stream is called the elementary stream. The elementary stream is a single digit-coded (possibly compressed) component of the program. For example, the coded video or audio portion of the program can be an elementary stream. The elementary stream can be converted into a packetized elementary stream before the multiplex is a stream or a transport stream (pEsp is in the same type, and the stream ID is used to distinguish the pES packet belonging to one elementary stream from the PES packet belonging to other elementary stream). The basic data unit of the elementary stream is a packetized elementary stream (PES) packet. Therefore, each view of the MVC video data corresponds to a respective elementary stream. Similarly, the audio data corresponds to one or more respective elementary streams. The video sequence encoded by the MVC can be separated into a number of sub-bitstreams, each of which is an elementary stream. Each sub-bitstream can be identified using the MVc view_id subset. The concept of a Mvc view_id subset defines the MVC video sub-bit stream ^ MVC video sub-bitstream contains the unit of view listed in the 151028.doc •19· 201119346 MVC vieW”d subset. The program stream usually contains only The NAL unit is derived from the NAL unit of the elementary stream. It is also designed that no two elementary streams can contain the same view. In the example of Figure 1, the multiplexer 3 receives the elementary stream containing the video data from the video encoder and the audio stream. coding 26 includes an elementary stream of audio material. In some examples, 'video encoder 28 and audio encoder % may each include a packetizer for forming a pES packet from encoded data. In other examples, video encoder 28 And the audio encoder % can each be interfacing with a separate packetizer for forming a PES packet for the encoded data. In other examples, 'multiple JL(4) may include a packet for forming a PES packet from the encoded audio material and the video material. A "program" as used in the present invention may include a combination of audio data and video data 'eg, a subset of audio elementary streams and available views delivered by the service of A/v source device 2(). Each m packet includes a stream identifying the elementary stream to which the ES packet belongs. multiplexer 3. The elementary stream can be combined into a constitutive program stream or a transport stream. The program stream and the transport stream system target are different applications. The two alternative multiplexes. The program stream includes a program of data' and the transport stream may include data of one or more programs. The multi-jade 3G can encode the program stream or the transport stream based on the following: Any or both: the service being provided, the stream will be passed to ^media, the number of programs to be sent' or other considerations. For example, when the video material will be encoded in the storage medium, the multiplexer It may be more likely to form a program stream, and when the video data will be streamed, broadcasted or sent via the network, the multiplexer 3 may be more than 1510.doc 201119346 may be used. The transport stream can make the multiplexer 30 more inclined to use the program stream for the storage and display of the single-program from the digital storage service. Because the program stream is quite sensitive to errors, the program stream is intended for use in an error-free environment. Or less likely to encounter an error in the environment. A program stream contains only the elementary streams that belong to it, and usually contains variable-length packets. In the program flow, the autocorrelation elementary stream is derived: the ES packet is organized into a "pack". The package includes any number of PES packets in any order obtained from any of the package header, the I-select system header, and the autocorrelation elementary stream. The system header contains an overview of the characteristics of the program stream, such as its maximum data rate, associated video and audio elementary stream / ', his timing information' or other information. The decoder can use the information contained in the system to determine if the decoder can decode the program stream. The evening state 30 can use a transport stream for transferring a plurality of programs via a potentially error-prone frequency. The transport stream is for multi-program applications (such as: broadcast = juice multiplex) so that a single transport stream can accommodate many independent rafts. The transport stream contains _i car bugs, pen & a serial transport packets, such transport Each of the packets - I 188 bits and 1 group length. Short fixed length seals & amps = less sensitive to errors. In addition, the packets can be processed by the 188-bit 'Lead-Solomon code'. Improvement of the flow = 7 provides the wrong material (four) protection. For example, t ^ ^ ^ ^ D, the transport stream is more likely to survive the error-prone channel seen in the broadcast environment. The transmission of the three major abilities of the brothers and the ability to carry many simultaneous programs is the lucky one of the two knives. However, the transport stream is more complicated than the program i51028.doc 201119346 w, so it is more difficult to generate and difficult to solve. The first byte of the packet is a sync byte having a value of 0x47 (hex 47, binary "010001U", decimal 71). The single-transport stream can be programmed with many different programs, each of which contains many packetized elementary streams. Xigongyi 3G can use the 13-bit packet identifier (piD) field to distinguish between a transport packet containing data of the elementary stream and a packet carrying data of other elementary streams. The multiplexer is responsible for ensuring that a unique (10) value is granted to each elementary stream. The last byte of the transmitted packet is the continuity count block. The multiplexer 3 increments the value of the continuity count stall between consecutive transport packets belonging to the same elementary stream. The situation enables the decoding of the destination device (such as the A/v destination device 4) or other unit to be able to detect the loss or gain of the transmitted packet and hopefully conceal the error that might otherwise be caused by such an event. The october 30 receives the PES packet ' from the elementary stream of the program of the sfl encoder 26 and the video encoder μ and forms a corresponding network abstraction layer (NAL) unit from the PES packet. In the case of H.264/AVC (Advanced Video Write Code), the coded video segments are organized into NAL units that provide "networks" for applications such as video telephony, storage, broadcast or streaming. Affinity" video said. NAL units can be classified into video codec layer (vcl) nal units and non-VCL NAL units. The VCL unit contains a core compression engine and can contain blocks, megablocks, and/or slice levels. Other NAL units are non-vcl NAL units. The multiplexer 30 may form NAL units, which include a header identifying the program to which the NAL belongs, and a payload, such as audio data, video data, or I5I028 describing the transport stream or program stream to which the NAL unit corresponds. Doc •22· 201119346 Poor material. For example, in H.264/Avc, a NAL unit can include a one-bit header and a variable-sized payload. In one example, the Nal unit header includes a priorityjd element, a temp〇ralJd element, an anCh〇r_Pic_flag element, a view_id element, an n〇n_idr-fiag element, and an inter-view-flag element. In the conventional MVC, the NAL unit defined by Η ” is reserved (except for the first NAL unit including the 4-byte Mvc NAL unit header and the nal unit payload) and the PCT (: slice of the code) Outside the Nal unit) The pri〇rity_id element of the NAL header can be used for a simple single-path bitstream adaptation process. The temporal "d" element can be used to specify the time level of the corresponding NAL unit, and the different time levels of t correspond to different frame rates. The anCh〇r_pic_flag element may indicate whether the picture is an anchor picture or a non-anchor picture. The wrong picture and its subsequent output order (ie, display order) = all pictures may not be in decoding order (ie, bit) The stream order) is decoded correctly in the case of decoding the previous picture, and thus can be used as a random access point. The anchor picture and the non-anchor picture can have different dependencies, both of which are signaled in the sequence parameter set. Other flags are discussed and used in subsequent paragraphs of this section. This erroneous picture may also be referred to as an open G〇p (image group) access point, and also supports closed G〇p when the non-idr_flag element is equal to zero. Access point The n_n"dr_flag element indicates that the picture is an instant decoder re-new (idr) picture or a view IDR (V. job) picture... in general, the job picture and its use in the output order or bit stream order Pictures can be decoded correctly without decoding the previous picture H brother in decoding, person order or display order. The view element contains grammar information that can be used to identify the view, which is available 151028.doc •23- 201119346

The information inside the MVC decoder is stone red. I Ή interaction (eg 'for inter-view predictions') and data interaction outside the decoder (eg, .v J ^ for visualization). The inter_view_flag element specifies whether the corresponding NAL unit county ~ dry and *疋 is used by other views for inter-view prediction. In order to convey the 4-byte nal unit header information that may conform to the basic view of Avr + β丄, C's in the MVC, m \TA τ 0 „ _ 丄 夂 夂 言 言 NAL NAL NAL 兀 兀. ^^(: In the case of the 'base view access unit', the VCL NAL single τ ο ο ο ο ο ο ο ο ο ο ο ο ο ο ο ο ο ο ο ο ο The .264/AVC decoder can ignore the first code ancestor unit. The seam unit including the video data in the payload can include video data of various granularity levels. For example, the seam unit can include the video data block: the giant block, the plural a block of video blocks, a slice of video data, or an entire frame of §fl tribute. Generally, an access unit may include a frame for indicating video data or a NAL unit, and corresponding to the frame. The audio data (in this audio data can be expected) ° access unit usually includes - all NAL single τ ο ' at the moment of the moment - 'time' all the audio data and video data. In the case corresponding to H.264 / AVC In the access unit, the access unit can be included in the time code The picture 'This picture can be rendered as an initial coded picture. Therefore, the access unit can pack all of the video frames at the same time, for example, all view components corresponding to time X. The present invention also encodes pictures of a particular view. (4) "View component". That is, the 'view component contains a coded picture (or frame) of a specific view at a specific time. Because A, the access unit may include all view components of a common time in some instances. The exhaustion order does not necessarily have to be combined with the output 151028.doc.24·201119346 order or display order. The set of consecutive access units can form a coded video sequence that can correspond to a NAL unit bit stream or sub-bit. Streaming Picture Group (GOP) or other independently decodable unit. Like most viewing tfl code standards, H.264/AVC defines error-free bitstreams, syntax, semantics, and decoding processes, either of which follows a certain A specification or level. H.264/AVC does not specify an encoder, but assigns to the encoder to ensure that the resulting bitstream is compliant with the standard for the decoder. In the case of video coding standards, "Fan" corresponds to a subset of algorithms, features or tools and constraints imposed on algorithms, features or tools. For example, as defined by the H.264 standard, "norms" are specified by the 264 standard. A subset of 1 bit. The "level" corresponds to the limit on the resource consumption of the damper (such as decoder memory and calculation). The restrictions are related to the resolution of the image, the bit rate and the mega zone. Block (MB) processing rate. The H.264 standard (for example) confirms that, within the bounds imposed by the syntax of a given specification, the value obtained by the syntax element in the bitstream (such as the designation of the decoded picture) Large variations in the performance of the encoder and decoder are still possible depending on the size. The H.264 standard further confirms that in many applications it is neither practical nor economical to implement a decoder that is capable of handling all hypotheses used in a grammar within a particular specification. Therefore, the H264 standard defines "hierarchy" as a specified set of constraints imposed on the values of the syntax elements in the bitstream. These constraints can be a simple limit on the value. Alternatively, such constraints may take the form of constraints on the arithmetic combination of values (e.g., picture width x picture height\number of pictures decoded per second). The H 264 standard further provides that 'individual implementations can support different levels of each supported specification. 151028.doc -25- 201119346 Compliance with the specification n-like support for all features defined in the specification. For example, as a write code feature, the B picture write code is not supported in the H 264/avc base line specification and is supported in other specifications of H.264/AVC. A decoder that complies with a level should be able to decode any bit stream that does not require resources beyond the limits of the limits in that class. The definition of specifications and levels can help with interpretation capabilities. For example, during video transmission, a pair of specifications and level definitions can negotiate and achieve an overall transmission session. More specifically, in H.264/AVC, the level may define, for example, restrictions on the number of megablocks that need to be processed, the decoded picture buffer (DPB) size, and the coded code. The slice buffer (CPB) size, the vertical motion vector range, the maximum number of motion vectors per two consecutive MBs, and whether the B block can have sub-megablock partitions smaller than 8x8 pixels. In this way, the decoder can determine if the solution is capable of properly decoding the bit stream. A set of parameters typically contains a sequence layer header in the sequence parameter set (sps) that occasionally changes the picture layer header information in the Pixel Picture Parameter Set (PPS). With the set of parameters, this occasional change of information does not need to be repeated for each sequence or picture; therefore, the write efficiency can be improved. In addition, the use of a set of parameters enables out-of-band transmission of header information, thereby avoiding the need for redundant transmissions to achieve error tolerance. In out-of-band transmission, a parameter set NAL unit is transmitted on a different channel than the other speaking unit. The technique of the present invention involves including an extractor in a media extractor trajectory. The extractor of the present invention can refer to another unit in the common file. That is, the rights case may include - a plurality of NALs having a plurality of NALs: 151028.doc -26-201119346 - the second track of the -include-extractor, the extractor identifying the plurality of NALs of the first trace Two or more nal units in a cell. In general, the extractor can act as an indicator such that when the multiplexed (4) encounters the extractor 3, the demultiplexer 38 can fetch the NAL units identified by the extractor from the _trajectory and send the NAL units To video decoding (four). The trajectory including the extractor can be referred to as a media extractor trajectory. The extractor of the present invention may be included in files in accordance with, for example, various file formats of the following: iso basic media file format, expandable video write code (svc) file format, advanced video write code (AVC) file format. Third Generation Partnership Project (3GPP) file format, and/or multi-view video code (Mvc) file format. In general, various tracks of a video file can be used as a switching track. That is, multiplexer 30 can include various trajectories to support various frame rate, display capabilities, and/or decoding capabilities. For example, when a video file conforms to the file format, each track can represent a different MVC operating point. Thus, the demultiplexer 38 can be configured to select one of the trajectories to manipulate the nal unit 7L of the NAL unit identified by the extractor from the selected trajectory from the selected trajectory and discard the data of the other trajectories. That is, when the selected trajectory includes an extractor that references a NAL unit of another trajectory, the demultiplexer 38 can extract the referenced NAL unit while discarding the unreferenced cells of the other trajectories. The multi-XH38 can send the extracted NAL unit to the video decoder. 48 ° By using the extractor in the media extractor track, the techniques of the present invention can be used to achieve time-adjustable capabilities between the various tracks of the video file. In MPEG-1 and MPEG-2, for example, the intrinsic 151028.doc -27· 201119346 time adjustable capability is provided by the (4) code picture. The first track of a video file conforming to MPEG-I or MPEG-2 may include the entire set of I-coded pictures, p-coded pictures, and 8-encoded pictures. The second track of the video file may include one or more extractors that only reference the I-coded picture of the first track and the p-coded picture, omitting references to the B-coded picture. By discarding the B-encoded picture, the video file can achieve a one-and-a-half resolution video representation. MPEG-1 and MPEG_2 also provide a base layer and enhancement layer concept to write two time layers, where the enhancement layer picture can be selected for reference from the base layer or enhancement layer for each prediction direction. As another example, H.264/AVC uses hierarchical B-coded pictures to support time-adjustable capabilities. The first month of the video sequence using H 264/AVC can be called an instant decoder renew (IDR) picture (also known as a key picture). Key pictures are typically coded in regular or irregular time intervals. These key pictures use the previous key picture as a reference to in-frame or inter-frame code for motion compensated prediction. A group of pictures (G〇p) usually includes - a key picture and all pictures that are temporally located between the key picture and the previous key picture. The GOP can be divided into two parts, one for the key picture' and the other for the non-key picture. The non-key picture is hierarchically predicted by two reference pictures, which are pictures of the past and the future with a lower time level. Each picture can be assigned a time identifier value to indicate the hierarchical location of the picture. Therefore, a picture having a time identifier value of up to 1^ can form a video segment having twice the frame rate of the video segment formed by the picture having the time identifier value of N-1. Box rate. Therefore, the technique of the present invention can also be used to include one or more extractors by reference to the 151028.doc -28-201119346, the one or more extractor reference traces having a time identifier value of N-1 Nal unit. The following operations are performed to achieve the time adjustable capability in H.264/AVC: the first track includes all NAL units having a time identifier value of up to N, and the second implementation is as indicated above, the technique of the present invention is applicable To videoconferencing in accordance with at least one of the following: IS〇 Basic Media File Format Expandable Video Recording (SVC) File Format, Advanced Video Recording (AVC) File Format, Third Generation Partnership Program ( 3GPP) file format, and multi-view video code (MVC) file format. The ISO base media file format is designed to contain chronological media information for presentation in a flexible and extensible format that facilitates the interchange, management, editing and presentation of media. 1§〇 The base media file format (ISO/IEC 14496-12:2004) is specified in mpeg_4, Part 12, which defines the general structure of time-based media files. It is used as other file formats in the family (such as the Avc file format (ISO/IEC 14496-15), 3GPP file format, svc file format and MVC block defined to support H.264/MPEG-4 AVC video compression. The basis of the case format). The 3GPP file format and the Mvc file format are extensions of the AVC file format. The IS® base media file format contains the timing, structure, and media information of the chronological sequence of media materials, such as audiovisual presentations. The file structure is object oriented. Files can be broken down very simply into basic objects, and the structure of these objects is implied by their type. Files in accordance with the IS 0 base media standard format are formed into a series of objects called "boxes". The information in the ISO base media file format is contained in these boxes' and no other information exists in the file. This box includes the specific file format 151028.doc •29- 201119346 Any initial signature required. A "box" is an object-oriented building block defined by a _type identifier and a length. Usually, the presentation item is included in a broadcast case and the media appears to be self-contained. The movie container (cinema box) contains media and video relay data, and the audio frame is contained in the media data container and can be attached to other files. The presentation item (action sequence) can be included in several files. 1 The timing and frame (size and size) information is usually in the ISO base media file, and the auxiliary =: qualitatively can use "any format. This presentation item contains The presentation item may be "local" or may be processed. The j-ridge re-flowing delivery machine can have a logical structure, a time structure, and a solid structure, and the L-structure does not need to be face-to-face. The logical structure of the trough case can be a time-parallel trajectory of the set of people's thunder ice also x s trace A right pull main, η ^. The time structure of the file can be that trajectory 3 has a sequence of chronological samples, and the list of matches is mapped to the time table of the overall electricity % # optional gold movie. The sample of the media data of the file itself is separated from the 'Royce's self-documentation... The box required for the time and structure decomposition of the structure is expanded in time. _ The timing of the sample may be confirmed by the film fragment timing _ The indicator can point to the same -I slaughter = where the relationship is located. They are in the middle. (d) or (for example) another slot case referenced by Fox: - the media stream may be contained in an accurate media type dedicated to the media track 'and can be entered by the sample item'. #本项可以包含" (Decoding I5W28.doc -30. 201119346 type required for decoding the stream) and any digitization of the decoder required by the four-character code (for example, "m.... The second "_") can also be used not only for Sung G_4 media but also for the defined sample item format of the media type used by his organization. The support for relay data is usually in two forms. The relay data of the first order can be stored in the proper obstruction, and the fish inter-information data can be synchronized as needed. Second, there may be general support for attaching to a movie or two: = non-chronological relay data. The structure support is general security and, as in the case of the data, the relay data resources are allowed to be stored in the other & In addition, these resources can be named and protected. In the help-based media slot format, the samples are grouped to assign each of the samples in the trace as a member of the -sample group. Samples in the sample group are not required to be contiguous. For example, when the H.264/AVC in the AVC architectural format is presented, the video samples in the time level can be sampled into two groups. The sample group can be represented by the following two data structures: SamPleToGroup box (sbdp) and SampleGr〇upDescr box.

The SampleToGroup box represents the assignment of samples to sample groups. For each sample group item, there may be one instance of the SampleGroupDescription box to describe the nature of the corresponding group. An optional relay data trace can be used to tag each track with a "characteristic of interest" for each track, for which the value of the "characteristic of interest" is different from other members of the group (eg, Its bit rate, screen size or language). Some samples within the trajectory may have special characteristics or may be identified by another 151028.doc •31- 201119346. An example of this feature is the sync point (usually the video box). These points can be identified by a special table in each track. More general, the nature of the dependence between the trajectory samples can also be documented using relay data. The relay data can be structured into a sample format-sequence as a video track. This trace can be referred to as a relay data track. Each relay data sample can be structured into a relay data statement. There are various kinds of statements corresponding to various questions that may be asked about the corresponding private case format sample or its constitutive sample. When delivering media via a streaming protocol, it may be necessary to change the media in a manner that presents the media in the archive. An example of this is when the media is transmitted through Γρ). In the case of the slot, for example, every message of the video: , · 'Bei Dianjian as a file format sample. In the RTP, the packetization rules specific to the codec used by the codec must be adhered to in the coffee package. The service can be configured to calculate the packetization at runtime. However, there is a trajectory that can be placed in the slot for the auxiliary stream. ...expressed as a special trajectory The schematic trajectory contains general instructions for the streaming server on how to form a packet stream for a particular protocol; Because of these instructions: independent of the media, when a new codec is introduced, the server may not need to be modified. In addition, edited: "When you want to cry a ^ ^ 1, 1 Han, two subjects Qiu body can not know the competition servo - π pair file editing, called the segment can be used in the file m (nter) software film one Place on the streaming server before going to the file. Adding an RTPO example as a sigma ~ trajectory 'There is a schematic trajectory format defined for L in the 稽4 Instance Format Specification. 15I028.doc •32· 201119346 3 0? (3 0?? File Format) is a multimedia container format defined by the 3rd Generation Partnership Project (30??) for 3G UMTS multimedia services. It is typically used on 3G mobile phones and other devices with 3G capabilities, but can also be played on some 2G and 4G phones and devices. The 3GPP file format is based on the ISO base media building format. The latest 3GP is specified in 3GPP TS 26.244 "Transparent end-to-end packet switched streaming service (PSS); 3GPP Hie format (3GP)". The 3GPP file format stores the video stream as MPEG-4 Part 2 or H.263 or MPEG-4 Part 10 (AVC/H.264). Because 3GPP specifies the use of sample and template fields in the ISO base media file format and a new box that defines the codec reference, 3GPP allows the use of AMR in the ISO Base Media File Format (MPEG-4 Part 12). Η·263 codec. For MPEG-4 media-specific information stored in 3GP files, the 3GP specification refers to the MP4 and AVC file formats, and the MP4 and AVC file formats are also based on the ISO base media file format. The MP4 and AVC file format specifications describe the use of MPEG-4 content in the ISO base media file format. The extended SVC file format for the AVC file format has a new structure of extractors and layers. The extractor is an indicator that provides information about the location and size of the video code data in the sample that has equal decoding time in another track. This situation allows the trajectory hierarchy to be built directly in the code domain. The extractor trajectory in the SVC is linked to one or more base trajectories, and the extractor trajectory extracts data from one or more basic trajectories at runtime. The extractor is an indicator with a NAL unit header that can be dereferenced by SVC extension. If the trace used for extraction contains video code data at different frame rates, the extractor also contains a solution time offset of 151028.doc -33·201119346 to ensure synchronization between the tracks. At runtime, the extractor must be replaced by the data it points to before being streamed to the video decoder. Since the extractor trajectory in the SVC is structured similarly to the video write code trajectory, the extractor trajectory in the SVC can represent a subset that it needs in different ways. The SVC extractor trajectory contains only instructions on how to extract data from another trajectory. In the SVC file format, there is also a summary tool that aggregates the NAL units within the sample together as a NAL unit, including summarizing the NAL units in a layer into a summary tool. The extractor in the SVC is designed to extract a range of bytes from: a sample or summary tool, or only one entire NAL unit rather than multiple NAL units, especially discontinuous NAL units in the sample. There are many video operating points in the SVC file format. The layers are designed to group samples in one or more of the operating points. The MVC file format also supports extractor trajectories that extract NAL units from different views to form an operating point that is a subset of views that use a certain frame rate. The design of the MVC extractor is similar to the extractor in the SVC file format. However, the use of MVC extractor trajectories to form alternate groups is not supported. To support trajectory selection, MPEG is proposed to the following MPEG proposal: P. Frojdh, A. Norkin, and C. Priddle "File format sub-track selection and switching" {JS0/1EC JTC1/SC29/WG11 MPEG M16665, London, UK) . This proposal attempts to alternate the concept of switching/switching groups in the sub-track level. The mapping sample group is an extension of the sample group. In the mapping sample group, each group item (group item of the sample) has a description of its "groupID", and its 151028.doc -34-201119346 is actually a summary of the NAL units in a view. The mapping to vieW_id after the NAL unit. In other words, each sample group item has a view that it contains in the ScalableNALUMapEntry value. The gr〇uping_type of this sample group item is rscnm". Progressive download is a term used to describe a digital media slot that typically uses a transfer from the server to the client. When starting from the computer, the consumer can start playing the media before the download is complete. The key difference between streaming media and progressive download is how the end user device that is accessing the digital media receives and stores the digital media material. A media player with progressive download capabilities relies on the local buffering of the relay data and digital media files in the header of the complete file when the digital server is downloaded from the web server. The media will start playing when the specified amount of data is available for the local playback device. This specified amount of buffer is embedded in the archive by the producer of the content in the encoder settings and is enhanced by the additional buffer settings imposed by the media player. In 3GPP, HTTp/Tcp/Ip transfers are supported for 3GP files for download and progressive download. In addition, the use of HTTp for video streaming has several advantages, and video streaming services based on Ηττρ are becoming more popular. Some of the advantages of HTTP streaming include the ability to use existing Internet components and protocols' so that no new effort is required to develop new technologies for transmitting video over the network. Other transport protocols, such as the RTP payload format, require: intermediate network devices (eg, 'middle boxes') to be aware of media formats and messaging. Also, 'HTTP streaming can be driven by the client, # avoid many control problems. For example, '$ utilizes all features for optimal performance, Servo 15I028.doc •35· 201119346 The server can track the size and content of unconfirmed packets. The server can also analyze the file structure and reconstruct the client buffer. Status to make RD optimal switching/reduction decisions. In addition, the constraints on the change of the bit stream can be satisfied in order to comply with the negotiated specifications. HTTP does not require new hardware or software implementations on web servers that implement HTTP 1.1. HTTP streaming also provides TCP affinity and firewall traversal. The techniques of the present invention can overcome the problem of bandwidth by, for example, improving the HTTP stream of video data by providing bit rate adaptation. Video compression standards such as ITU-T H.261, H.262, H.263, MPEG-1, MPEG-2, and H.264/MPEG-4 Part 10 utilize motion compensated temporal prediction to reduce time redundancy I. The encoder uses motion compensated predictions from some previously encoded pictures (also referred to herein as frames) to predict the current coded picture based on the motion vector. There are two main picture types in a typical video code. It is a picture written in the frame ("[picture" or "I frame"), a predicted picture ("p picture" or "p frame"), and a bi-predicted picture ("B picture" Or the block of the "B frame" /p picture can be written 29 in the box, or see - other pictures for prediction. In the B picture, the 'block' can be predicted from one or two reference pictures, or can be coded in-frame. These reference pictures can be positioned in chronological order before or after the current picture. Very H.264 write.., as expected, as an example, the B picture uses two lists of reference pictures of the previous code (list Q and list... these two;; each can contain the order of the order between the picks And/or future writing; film. It can be predicted in one of the following ways. Area 3 in the picture is predicted by the action compensation of the "° reference picture, since the clear reference 丨151028.doc -36 - 201119346 The motion compensated prediction, or the motion compensation of the combination of the two modes, and the list 1 reference picture and list! The reference picture of the two sets of Hi in order to obtain the list 主 reference, the main °. 1 A '' Knife from the list 0 to test y = early 1 reference picture to obtain two motion compensated reference area L film will be used to predict the current block. Domain group, small video block can provide better resolution, and It can be used to locate the video frame of the level of the package. - 妒 and ' _ ^ is: the various partitions of the block are treated as video blocks. In addition, the slice can be regarded as a plurality of video blocks, such as giant blocks and / or sub-blocks. Each can be video-simplified - unit 4 that can be decoded independently The frame itself can be decodable | ' or other parts of the frame can be defined as decodable units. The term "coded unit" ""code unit" can refer to any independently decodable video frame. Units, such as the entire frame, the slice of the frame, also known as the sequence of picture groups (G〇P), or another independently decodable unit defined according to the applicable coding techniques. The term megablock refers to According to a data structure ε including a 16 χ 16 pixel two-dimensional pixel array for encoding pictures and/or video data, each pixel includes a chrominance component and an illuminance component. Therefore, the megablocks can each define a two-dimensional array of 8×8 pixels. The four illumination blocks, each containing one is 丨6><Two chrominance blocks of a two-dimensional array of 16 pixels, and a header containing syntax information such as: coded block type (CBP), coding mode (for example, in-frame (1) Encoding mode or inter-frame (Ρ or Β encoding mode), the partition size of the partition of the block coded (for example, 16x16, 16x8, 8x16, 8x8, 8Μ' 4x8 or 4x4), or 151028 .doc -37· 201119346 One or more action vectors for the giant block of code. Video encoder 28, video decoding 48, audio encoding (4), audio decoder private, multiplexer 30, and demultiplexer 38 may each be implemented as any of a variety of suitable encoder or decoder circuits, such as One or more microprocessors, digital signal processors (), special application integrated circuits (ASK:), field programmable gate arrays (FpGA), discrete logic circuits, software: hardware, hard work, or any -combination. Video encoder-like video decoding The mother of the horse state 48 can be included in one or more encoders or decoders, and can be part of the video encoder/decoder (codec). Similarly, each of the audio encoder 26 and the audio decoder can be included in one or more encoders or decoders, either of which can be a combined audio encoder/decoder (c〇DEC) )part. Including Μ encoder 28, video decoding (4), audio coding (4), audio decoder 46, multiplexer 30, and/or demultiplexer, the device may include an integrated circuit, a microprocessor 'and/or - such as Wireless communication device for cellular telephones 0 In accordance with the teachings of the present invention, the multiplexer 3 can combine the Wei units to conform to the ISO base media slot format or its derived format (9). , svc '^ or 3 coffee) the archive of the video archive, and includes - the media extractor track also 'the media extractor track identifies one or more potential discontinuous NAL units and passes the video rights to Output interface 32. The output interface η can include, for example, a transmitter, a transceiver, a device for writing data to a computer readable medium (such as a 'disc player, a magnetic media drive (eg, a floppy disk drive), a universal serial bus ( USB), network interface), or other output 151028.doc -38- 201119346 " face wheel 4 media® 32 will NAL single it or access unit to send media 34 (for example, 锉 4 you think the main % 驷凟Magnetic #轮信号 or carrier transient media), or such as body. ...the computer-readable storage medium of the media, memory or flash drive, the H interface 36, is read from the computer-readable medium 34. The input interface 36 can be, for example, a CD player, a magnetic media drive, a job, a receiver, a unit; or other computer readable media interface. The input interface 36 can provide the NAL early or access unit to the de-streaming or program stream. The multiplexer 38 can transmit the singular PES stream and decapsulate the PES > Take the encoded data, and view the _ _ (Example 2 ES: the indicated audio stream or video flooded two: multiplexer π / (four) sent to the audio decoder 46 or video decoder 48. Optionally, the selected (four) traces of the selected (four) traces of the other (four) of the selected traces of the H-reference (4) are passed to the visual scale 48, discarding 1 The encoded audio data is not encoded by the extractor coder (10) of the selected trajectory, and the decoded =: to:: output 42' and the video solution... decodes the encoded IS = multiple views of the stream - Viewing Continuation 44. Video output 44 may include - a display using several views, such as 'presenting a scene at the same time = wearing a stereoscopic stereoscopic display or an ophthalmic stereoscopic display. See (10) Figure 2 is a multiplexed thin 1λ Block diagram of the example configuration in Figure 2. Benefits Management Unit 6〇, Video Input 15I028.doc •39· 201119346 interface 80, audio input interface 82, multiplex flow output interface fabric, and program specific information table 88 ^ stream management unit 60 includes NAL unit constructor 62, stream identifier (stream ID) search unit 66, track generation unit 64 and Extractor generation unit 68. In the example of Figure 2, video input interface 8 and audio input interface 82 include separate packetizers for the encoded video material and the encoded audio data forming unit. The video and/or audio packetizer can be presented external to the multiplexer 3. For the example of FIG. 2, the video input interface 80 can be opened from the encoded video data received from the video encoder 28 into a PES packet. And the audio input interface 82 can form a pES packet from the encoded audio material received from the audio encoder 26. After the NAL unit constructor 62 constructs the NAL unit, the NAL unit constructor 62 sends the NAL unit to the trajectory generating unit 64. The trajectory generating unit 6 receives the NAL unit and combines the video building including the NAL unit into one or more trajectories of the video file. The trajectory generating unit 64 can further perform the extraction. The generator 68 generates an extractor that constructs one or more media extractor trajectories by the trajectory generating unit 64. When it is determined that - or a plurality of said units belong to a plurality of trajectories rather than repeating the NAL unit between the trajectories The extractor generates a single 兀 68 that can construct an extractor that references the trajectory of the NAL unit. In this manner, the multiplexer 30 can avoid duplication of data between tracks, which can reduce bandwidth consumption when transmitting video files. Discussing the various data structures and components of the extractor. In general, the extractor may include: a trajectory identifier value, which refers to the trajectory of the referenced NAL unit; and - or multiple muscle units Identifier, zhang 151028.doc 201119346 Identify the NAL unit referenced by the extractor. In the second example, the nali open identifier can refer to the range of bits or bytes in the trajectory corresponding to the identified nat A ^ [earlier track identifier value reference. » - ^ . In some examples, the NAL·early 7L identifier may be individually referenced by the parent of the extractor > W - NAL unit, for example, to identify discontinuous NAL units. a _ * 6 _ In the upper case, the NAL 兀 identifier can refer to the NAL unit based on the offset of the time or space position of the k-staker from the media extractor trajectory. The generating unit M may include, in some instances, a media extractor track. That is, the media extractor trajectory can include both the ancestor unit and the extractor. Therefore, in this example, the trajectory generating unit 64 can construct a video file 'the video file and have /, the page only includes the NAL unit: the track and one includes one or more extractors The second track, the one or more benefits refer to all or a subset of the ancestor elements of the first track. Moreover, in some examples, the falsification generating unit 64 may include additional NAL units in the trajectory that are not included in the first trajectory. The technique of the sample invention can be extended to a plurality of tracks. For example, the trajectory generating unit 64 may construct a second track that may refer to the NAL unit of the first track and/or the nal unit of the second track, and may further include a NAL unit that is not included in the first track or the second. . 3 is a block diagram illustrating an example audit file (10) including - a first track having a set of video samples and an extractor having a subset of video samples having a reference first track Two obedience. In the example of Fig. 3, the file 100 includes MOOV|gl() 2 and a media material (salt 7) box 11〇. The phase 102 corresponds to a movie box, which is defined by the helper base media file format. I51028.doc • 41 · 201119346 is a container box. The container box of the container defines the relay data for presentation. ΜΙ^ΑΤ箱1()4 corresponds to the media data box, which is defined by the help basic media building format as a box that can hold the actual data for presentation. In the example of Figure 3, the M00V box 1〇2 includes the complete subset track 1〇4 and the media extractor track 1〇6. The IS® Basic Media Archive format defines the “track” as a chronological sequence of related samples in the ISO base media rights case. The ISO base media file format is step-by-step, and for media data, the track corresponds to the sequence of images or sampled audio. In the example of FIG. 3, the MDAT box includes a j-coded sample η], a P-coded sample 114, a B-coded sample 116, and a B-coded sample 118. The B-coded samples 116 and the B-coded samples 118 are considered to be at different levels of coding. In the example of FIG. 3, the B-coded sample ι6 can be used as a reference to the B-coded sample 118, and thus the 3-coded sample 117 can be at a lower level coding level than the B-coded sample 丨16. Class coding level. The order in which the samples are displayed may differ from the hierarchical order (also referred to as the decoding order) and the order in which the samples are included in the MDAT box. For example, 5, the coded sample 112 may have a display order value of 〇 and a solution-human sequence value of 0. The p-coded sample ii 4 may have a display order value of 2 and a decoding order of 1. The value, B-coded sample 116 may have a display order value of 1 and a decoding order value of 2, and the B-coded sample 118 may have a display order value of 4 and a decoding order value of 3. Track 1 may include additional samples, such as 'samples having a display order value of 3 and a decoding order value of 4. Each of the sample U2, the P-coded sample U4, the B-coded sample 116, and the B-coded sample 丨18 may correspond to various nal 151028.doc -42·201119346 units or accesses. unit. The TSO Basic Media File Format defines a "sample" as all data associated with a single timestamp, such as individual frames for video, a series of video frames in decoding order, or compressed regions in the decoding order of audio. segment. In the example of FIG. 3, the complete subset track 1〇4 includes a reference to the I-coded sample π 2, the P-coded sample 144, the B-coded sample 116, and the B-coded sample 117. . The MDAT box 11 further includes an extractor 12, an extractor 122, and an extractor 124. Thus, the extractors 12A through 124 are included in a movie data box, which will include data samples. In the example of FIG. 3, the extractor 12 refers to the I-coded sample 112, the extractor 122 refers to the P-coded sample 114, and the extractor 124 refers to the B-coded sample 丨18. There may be two or more NAL units corresponding to the coded sample 1 12, the P-coded sample 丨 14 and/or the B-coded sample 118, and the unit may be discontinuous. In accordance with the teachings of the present invention, although there may be two or more discontinuous NAL units in the respective samples, extractors 12A through 124 may still identify each of the NAL units of the respective samples. In the example of FIG. 3, the media extractor trajectory 106 includes relay data for the reference extractor 12, the extractor 122, and the extractor 124. Each of the extractors 120 to 124 may also include a display order value and a decoding order value. For example, the extractor 120 can have a display order value of 〇 and a decoding order value of ,, the extractor 122 can have a display order value of 丨 and a decoding order value of 1, and the extractor 124 can have 2 The display order value is a decoding order value of 2. In some instances, 'display values and/or decoded values may skip certain values' (for example) to match the values of the identified samples. 151028.doc -43· 201119346 The complete subset trajectory 104 and the media extractor trajectory i 〇 6 can form an alternating group 'to make the % multiplexer 38 (Fig. 1) select the complete subset trajectory or media extractor trajectory 106 is decoded by video decoder 48. Regarding the MVC example, the meta-subset trajectory 10 4 may correspond to a first operating point, and the media extractor trajectory 106 may correspond to a second operating point. Regarding the example of 3 (} pp, the complete subset trajectory 104 and the media extractor trajectory 106 may form a switching group. In this way, the complete subset trajectory 104 and the media extractor trajectory 1 〇 6 (for example) in HTTP streaming The application can be used to adapt bandwidth availability and decoder capabilities. When the complete subset trajectory 104 is selected, the demultiplexer 38 can sample the corresponding subset trajectory 104 (eg, the encoded sample 丨12, The p-coded sample 114, the B-coded sample 116, and the B-coded sample 118) are sent to the video decoder 48. When the media extractor track 1〇6 is selected, the demultiplexer 38 can correspond to the media extraction. The sample of track 1 ( 6 (including samples identified by the media extractor corresponding to media extractor trajectory 106) is sent to video decoder 48. Thus, when media extractor trajectory 106 is selected, solution multiplex 38 can The encoded coded sample 112, the p-coded sample η*, and the b-coded sample 118 are sent to the video decoder 48, and the demultiplexer 38 can extract the H12G, the extractor 122, and the extractor 124 for dereference. The self-finishing track 104 is captured by the I code. Sample 丨 2, p-coded sample 114, and B-coded sample 118. Figure 4 is a block diagram illustrating another example of a private case 140 including two distinct extractor trajectories i46, 148. The two extractor trajectories are illustrated in the example 'but in general, a file may include any number of extractor trajectories. In the example of Figure 4, the 'file 14' includes the 〇〇乂 box 142 and the 〇 〇 box J51028. Doc • 44 * 201119346 150. The MOOV box 142 includes a complete subset trajectory 144 and media extractor trajectories 146, 148. The MDAT box 150 includes data samples and extractors for various trajectories, such as 'via' encoded samples 152, Ρ encoded Sample 154, Β encoded sample I%, beta encoded sample 158, and extractors 160 through 168. In the example of Figure 4, 'extractors 160 through 164 correspond to media extractor trajectory 146' and extractor 166 to 168 corresponds to media extractor trajectory 148. In this example, extractor 16 of media extractor trajectory 146 identifies I-coded sample 152 'extractor ι 62 to identify p-coded sample 154, and extractor 164 identifies The encoded sample 156. In this example, The fetcher 166 identifies the I-coded samples 152, and the extractor 162 identifies the p-coded samples 154. The example of Figure 4 demonstrates that two or more extractors of the various media extractor traces refer to the complete subset of the traces An example of the same sample. The media extractor trajectory may represent a time subset of the decodable video stream and an alternate/switching trajectory containing the trajectory of the original full-time resolution bit stream (eg, complete subset trajectory 144). The complete subset track 144 can, for example, represent a video stream of 30 frames per second (Fps). In some instances, by not including a β-picture of a certain level in the sub-bitstream, the sub-rate's pure rate can be halved or reduced to some other fraction. For example, the media extractor track 146 may have a frame rate that is halved relative to the full subset trajectory 144 by not including the B-coded sample 158. For example, the media extractor track Μ can have a frame rate of 15 FPS. With #, the media extraction trajectory 148 has a frame rate that is halved with respect to the media extractor trajectory 146 by omitting the B-coded sample 156 and the 匕 encoded sample 158, and thus has a 7. 5 Frame rate of FPS. 15l028.doc • 45- 201119346 FIG. 5 is a block diagram illustrating another example archive 180 including a subset trace 188 and two media extractor traces 184, 186. The file 182 of the file 包括8 includes a subset track 188, media extractor tracks 184, 186, and the MDAT box 190 includes an I-coded sample 192, a Ρ-coded sample i 94, a B-coded sample. 202, B-coded sample 208, and extractors 198, 20t), 204, 206, and 210. As discussed above, the media extractor trajectory can include an extractor that references a sample of another trajectory. Additionally, the media extractor trajectory can further include additional video samples that are not included in another trajectory. In the example of FIG. 5, subset track 188 includes an I-coded sample 192 and a P-coded sample 丨94. Media extractor trajectory 186 includes extractors 198, 200 and additionally includes B-coded samples 202. Similarly, media extractor track} 84 includes extractors 204, 206, 210' and additionally includes truncated coded samples 2〇8. In the example of FIG. 5, the media extractor exemplifies a coded sample of the video material (the encoded sample 202), and the media extractor trajectory 184 includes an extractor 210, the reference including the encoded sample The sample of the media extractor track 丨 86, i.e., in the example of Fig. 5, the 'extractor 2' refers to the sample 202 encoded by the 。. Thus, media extractor trace 184 can represent a full time resolution bitstream, while media extractor trace 186 and subset archive 188 can represent a subset of the full time resolution bitstream. That is, the media extractor trajectory 186 and the subset trajectory 188 may have a temporal resolution (eg, a lower frame rate) that is lower than the full time resolution represented by the media extractor trajectory "A." According to the techniques of the present invention, The H.264/AVC file format can be modified to include an extractor trajectory that can be extracted as a trajectory containing any compatible time subset of the original full time 15I028.doc -46·201119346 resolution bitstream. Hierarchy B (or p) H.264/AVC for picture writing, assuming there are N time levels, including from time level 0 to k (k) Each sub-bit stream of the sample of <N) can be extracted by defining a corresponding extractor trajectory. Thus, for the same video, there may be N trajectories (including N-1 extractor trajectories) that form a parent/switching group. The extractor can be associated with a time hierarchy level corresponding to the time level 4 of the sample identified by the extractor. For example, the time identifier value of the time level of the specified sample can also be communicated in the extractor. Figures 6A through 6C are block diagrams showing an example of a mdAT box 220 of a file that includes a plurality of media extractor traces for various media extractor tracks. Each of FIGS. 6A-6C depicts a sample 222 that includes a view 0 sample 224A, a view 2 sample 226A, a view 1 sample 228A, a view 4 sample 230A, and a view 3 sample 232A; and a non-tin sample 223 It includes a view 〇 sample 224B, a view 2 sample 226B, a view 1 sample 228B, a view 4 sample 230B, and a view 3 sample 232B. The ellipsis next to the non-anchor sample 223 means no, and additional samples may be included in the MDAT box 22〇. Each of the anchored sample and the non-anchored sample can collectively form a first trajectory of the archive. In one example, in accordance with the teachings of the present invention, the media extractor trajectory for each set of extractors of the archives depicted in Figures 6a through 6C may correspond to independent operating points of video archives that conform to the MVC file format. In this manner, the techniques of the present invention can be used to generate one or more media extractor trajectories corresponding to the operating points of video archives that conform to the MVC file format. 6A-6C depict various media extractor trajectory extractors 24, 244, 250, wherein the extractors 24, 244, 25 〇 will each be included in the 151028.doc -47 - 201119346 MDAT box 220, but out of For clarity purposes, it is not illustrated in a separate diagram. That is, the MDAT box 22A can include each of the extractors 240, 244, 250 when fully combined. Figures 6A through 6C provide an example of a file that includes a track containing a media extractor and real video samples. The various samples can be independently included in different tracks depending on the time level. For each time level, a particular trajectory can contain all video samples as well as extractors for trajectories with lower time levels. The video samples (NAL units) can be separated into different tracks' and the tracks with higher frame rate can have extractors pointing to other tracks. In this way, it is possible to have a movie clip containing samples of only one time level and a movie clip may contain an extracter pointing to other clips. In this case, movie segments of different trajectories but of the same time period can be interleaved in increasing order of time. Figure 6A provides an example of an extractor 240 that includes extractors 242a through 242N corresponding to media extractor trajectories. In this example, extractor 242a references two views 〇 samples 224A of anchor sample 222. Extractor 242n references view 〇 sample 224B of non-anchor sample 223. In general, in the example of Figure 6A, the extractor of extractor set 240 references the corresponding view. sample. Each of the extractors 242A through 242N corresponds to a common media extractor trajectory, which may belong to a switching group and/or an alternating group. The media extractor trajectory can correspond to individual operating points, for example, including views. The operating point. In some real money, for stereoscopic video using MVC code, there may be three operating points, including support for outputting one of two views, and one output for only one view (for example, view only or view 1) The second operation 151028.doc -48- 201119346 2. The third operating point can be the operating point of the output view 1. Depending on the prediction relationship - "the point can include only the VCL · NAL unit in view 1 and the associated All of the elements of the non-VCL NAL unit, view, and view}, or the nal unit of view 1 and the NAL unit of the tinned NAL unit (also referred to as the NAL unit of the component). In this stereoscopic situation, an example of the disclosed technique It is provided that the other two operating points can be represented by two extractor tracks. These two extractor tracks can form a switching group, and together with the original video track, these three tracks can form an alternating group. The present month provides a technique for modifying the MVC file format to include the MVC media extractor trajectory. In general, Mvc video trajectories including MVC media extractor trajectories having the same number of output views can be characterized as a switching group. by All of the operating points represented by the track of the archive may belong to an alternating group of mvc video presentations. The view of each of the anchor sample 222 and the non-anchor sample 223 may form a complete subset trajectory, for example, including all available views. Operating points. For example, as shown with respect to extractors 246A through 246N in Figure 6B, the extractor may reference successive portions of the sample. In Example + of Figure 6β, extractor 246A refers to view 〇 sample 224A and refers to view 2 sample 226a The data structure representing the extractor 246A may specify the byte range, the start view and the end view of the identified view, the number of start and subsequent views, or other tables* of the series-continuous view identified by the extractor. The extractor set (10) may correspond to another media extractor trajectory, which in turn may correspond to a separate MVC operating point. For example, as with respect to extractor 254A, 256A in Figure 6C, 151028.doc - 49- 201119346 shows that the two extractors can also refer to two parts of the same book (for example, two discontinuous views). For example, the extractor sample 252A includes a reference view. Extractor 254A of 224A and View 2 samples 226A, and extractor 254B of reference view 4 sample 230A. Thus, the samples represented by extractor samples 252A may correspond to extractor samples that reference discontinuous view samples. Similarly, in In the example of Figure 6C, the extractor sample 252N includes a reference view 〇 sample 224B and an extractor 256A of the view 2 sample 226B, and an extractor 256B of the reference view 4 sample 230B. The extraction may also be defined with respect to anchored or non-anchored samples. The extractor with respect to the anchor sample definition and the extractor for the non-anchor sample definition may refer to different views. The above-mentioned MVC media extractor trajectory of the ISO basic media file format or the Mvc file format may be an example of a relay data track, which may be implemented to have similar extraction functionality and may be used to indicate normal Alternation of video tracks and/or switching of tracks. In an example using the MVC file format, a full bit stream may be included in the trajectory, and all other possible operating points may be represented by an extractor trajectory, each of which may be a signal, such as The number of views output, the view identifier value of the view for output, the bandwidth required for transmission, and the frame rate. Figure 7 is a conceptual diagram illustrating an example MVC prediction pattern. In the example of Fig. 7, eight views (with view ID rs〇 to "S7") are illustrated, and 12 time positions ("τ〇" to "Tu") of each view are illustrated. that is,. Each column in Figure 7 corresponds to a -view' and each row indicates a temporal position. 151028.doc •50- 201119346 Although M VC has a so-called base view decoded by H.264/AVC decoder, and stereo view pairs can also be supported by MVC, MVC has the advantage of supporting more than two views. As an example of 3D video input and decoding of this 3D video represented by multiple views. A renderer with a client side of the Mvc decoder can expect 3 video content with multiple views. The tin fix component and the non-tin fix component in the view can have different view dependencies. For example, in view S2, the view component of the view component view _ is in the view component. However, the non-marketing view component in view S2 is not dependent on the view components in other views. / In Figure 7, a shaded box comprising letters is used to indicate each column and each frame in Figure 7 to specify that the corresponding frame is broken in the frame (i.e., I frame), or - The code is inter-frame coded (ie, is a p-frame) or written between frames in multiple directions (ie, as a (four) box). In general, the prediction is indicated by an arrow, where the arrow points to the frame using the arrow to start the object for predictive reference. For example, from the time of view $◦ at position T0! The p-frame at the time position τ〇 of the prediction view S2 is predicted. Like early-view video coding, the frame of the multi-view video coded video sequence can be predictively coded with respect to frames at different time positions. For example, the b frame at the time position T1 of the view SO has a self-view time position τ o at the m frame pointing to its arrow ', thereby indicating the prediction from the (four) frame b Sfl box, but 'other' in the multi-view video coding In this case, the frame can be predicted between views. That is, the view component can use the view component in other views for reference. In Mvc, for example, as in the view in another view, the pieces are inter-view predictions. The potential view inter-parameter 151028.doc -51 - 201119346 is used to communicate in the Sequence Parameter Set (SPS) MVC extension and can be modified by reference to the picture list construction process, which enables flexible ordering of predictive or inter-view prediction references. Table 1 below provides an example of a set of MVC extended sequence parameters. Table 1 seq_parameter_set_mvc_extension() { c descriptor num_views a minus 1 0 ue(v) for( i = 0; i <= num_views_minusl; i++) view—id[ i ] 0 ue(v) for( i = 1; i <= num a views-minusl; i++ ) { num_anchor_refs_10[ i ] 0 ue(v) for( j = 0; j < num_anchor-refs—10[ i ]; j++) anchor_ref_10[ i ][ j ] 0 ue(v) num-anchor-refs-11 [ i ] 0 ue(v) for( j = 0; j < num—anchor—refs—11 [ i ]; j++) anchor_ref_ll[ i ][ j ] 0 ue(v) } for( i = 1; i <= num_views_minusl; i++ ) { num_non_anchor_refs_10[ i ] 0 ue(v) for( j = 0; j < num a non_anchor_refs_10[ i ]; j++ ) non-anchor-ref_10[ i ][ j ] 0 ue(v) num_non_anchor_refs_ll[ i ] 0 ue(v) for( j = 0; j < num a non_anchor-refs_ll [ i ]; j++ ) non-anchor_ref_l 1 [ i ] [ j ] 0 ue(v) } num—Ievel_values a signalled a minus 1 0 ue(v) for(i = 0; i <= num_level_values_signalled_minusl; i++) { level_idc[ i ] 0 u(8) num—applicable—ops_minusl[ i ] 0 ue(v) for( j = 0; j <= num_applicable_ops_minusl[ i ]; j++ ) { applicable_op—temporal—id[ i ][j ] 0 u(3) applicable_op a num—target a views_minusl[ i ][j ] 0 ue(v) for( k = 0 ; k <= appIicable_op_num_target_views_minusl[i][j];k++) applicable_op_target_view_id[ i ][ j ][ k ] 0 ue(v) applicable-op-num_views_minusl [ i ][j ] 0 ue(v) } } } -52- 151028.doc 201119346 Figure 7 provides various examples of inter-view prediction. In the example of FIG. 7, the frame of the view S1 is illustrated as being predicted from the frame at different time positions of the view, and between the frames in the frame at the same time position from the views S0 and S2. For example, the b frame of the view s 1 at the time position T 系 is from the B frame at the time positions T0 and T2 of the view S 1 and the b frame at the time position T1 of the view 8 〇 and S 2 . Each of the predictions is made. In the example of Figure 7, the capital letter r B" and the lowercase letter "b" are intended to indicate different hierarchical relationships between frames rather than different encoding methods. In general, the uppercase "B" frame is relatively higher than the lowercase rb" frame prediction layer. That is, in the example of Fig. 7, the "B" frame is encoded by referring to the "B" frame. Additional hierarchy levels can be added to have additional bi-directionally encoded frames that can be referenced to the "b" frame of Figure 7. Figure 7 also uses different levels of shading to account for changes in the predicted hierarchy, where a larger amount of shadow (i.e., relatively deeper) frame prediction hierarchy is higher than those with less shadows (i.e., relatively giga). frame. For example, all the I frames in Figure 7 are illustrated by full shading and the frames have a slightly shallower shadow, and the frames (and lowercase b frames) have various levels of shadow relative to each other, but always Shallower than the shadow of the ρ frame and the I frame. In general, the prediction hierarchy is related to the view order index. The relationship between the predictions is that the higher frame of the frame should be decoded in the frame with a relatively lower decoding level, so that the class is relatively higher. The frame can be used as a reference frame during the period when the decoding level is relatively low. The view order index is an index indicating the decoding order of the view components in the access. As specified in H 4/AVC Appendix H (MVC Amendment), the view order index 151028.doc -53· 201119346 is implicit in the SPS MVC extension. In the SPS, for each index z, the corresponding view is id. The decoding of the view component should follow the ascending order of the view order index. If all views are rendered, the view order index is in a sequential order from 〇 to num_views_minus_l. In this way, the frame used as the reference frame can be decoded before decoding the frame referenced by the reference frame. The view order index is an index indicating the decoding order of the view components in the access unit. For each view order index z, the corresponding view is id. The decoding of the view component follows the ascending order of the view order index. If all views are rendered, the set of view order indexes contains a set of consecutive sorts from zero to less than the full number of views. For certain frames at equal level levels, the order of decoding may not be important relative to each other. For example, the I frame of the view S0 at the time position τ 用作 is used as the reference frame of the p frame of the view S2 at the time position T0, and the p frame of the view S2 at the time position T0 is used as the view S4 again. The reference frame of the p-frame at the time position το. Therefore, the I frame of the view s at the time position τ〇 should be decoded before the p frame at the time position T0 of the view S2, and the frame of the view S2 at the time position Τ0 should be at the time position of the view S4. The P frame at Ding was decoded before. However, between views s and S3, the order of decoding is not important, since views 81 and 83 do not rely on each other for prediction, but only for predictions from higher prediction levels. Furthermore, view S1 can be decoded prior to view S4 as long as view S1 is decoded after view 8 and illusion. In this way, hierarchical ordering can be used to describe views 8〇 to S7. Making the notation SA> SB means that the view SA should be decoded before the view is dimmed. In the example of Fig. 7] 5] 028.doc -54·201119346 S0>S2>S4>S6>S7. Further, regarding Fig. 7, the actual 'S2> S3, S4 > S3, S4 > S5, and S6 > S5. In this case, using the notation, for example, the S0>S1, S2>S1 view does not violate these requirements - the decoding order is possible. Therefore, many different decoding orders are possible with only certain limitations. Two example decoding orders are presented below, but it should be understood that many other decoding orders are possible. In one of the examples illustrated in Table 2 below, the view is decoded as quickly as possible. Table 2 View ID so S1 S2 S3 S4 S5 S6 View Order Index 0 2 1 4 3 6 5 〇/ 7 The example in Table 2 confirms that view S1 can be decoded immediately after decoded views S0 and S2; in decoded view 82 After 84, view S3 can be decoded immediately; and after decoded view S4 & S6, view s5 can be decoded immediately. Table 3 below provides another example decoding order, wherein the decoding order enables any view that is used as a reference for another view to be decoded before being used as a reference for any other view. Table 3 View ID so S1 S2 S3 S4 S5 S6 C7 View Order Index 0 5 1 6 2 7 3 〇/ 4 Example of Table 3 Confirmation 'In the example of Figure 7, the frames si, s3, 85 and 37 are Does not act as a reference frame for any other view frame, and thus views S1, S3, S5, and S7 are used as frames for their views (ie, views SO, S2, S4, and S6) decoding. Views S1, S3, s5, and S7 can be decoded in any order relative to each other. Therefore, in the example of Table 3, 151028.doc - 55 · 201119346, the view S7 is decoded before each of the views SI, S3, and S5. For the sake of clarity, there may be a hierarchical relationship between the frame of each view and the temporal position of the frame of each view. With respect to the example of Fig. 7, the frames at the time position T0 from other views are used to predict the intra-view or inter-view prediction time position T0. Similarly, the frame at the time position T8 from other views is used to predict the intra-view or inter-view prediction time position Τ8. Therefore, with respect to the time hierarchy, the time positions Ding and Τ8 are at the apex of the time horizon. In the example of FIG. 7, because the reference time position is different from the frame of the time frame β coded by the frame, the frame at the time position Τ4 is lower than the time position of the frame at the time level. . The time position and the frame of the frame are lower than the time frame Τ4 at the time level. Finally, the frames at the time positions ΤΙ, Τ3, Τ5, and Τ7 are at the time level ± below the time position Τ 2 and Τ 6 frames. In MVC, a subset of the overall bitstream can be extracted to form a sub-bitstream that is still MVC compliant. There are many possible sub-bitstreams that a particular application can need based on, for example, the services provided by the server, the capacity of the decoders of multiple clients, support and capabilities, and/or one or one user. The preference of the end. For example, 1 client may need only three graphs' and there may be two scenarios. In the instance, the client can _ smooth the viewing experience, and may prefer the view with = S2, while the other __ user may need to adjust the circle and prefer the vieW_id value S0, 52 and the view . If the first instance of Table 9 - id is sorted, then the view of the order index value in these two instances 15I028.doc -56 - 201119346 are (〇, i, 2} and AI 4} respectively. Note that this Both of the sub-bitstreams can be decoded into independent MVC bitstreams and can be supported simultaneously. There can be many Mvc sub-bitstreams that can be decoded by the MVC decoder. In theory, the following two properties are satisfied. Any combination of these may be decoded by an MVC decoder conforming to a certain specification or level: (1) sorting the view components in each access unit in a view order: bowing, and (7) for each view in the combination The dependent views are also included in the combination. With respect to the techniques of the present invention, media extractor trajectories and/or pure video sample trajectories may be used to represent various MVC sub-bitstreams. Each of these trajectories may correspond to An MVC operating point., Figures 8 through 21 are block diagrams illustrating various examples of data structures of media extractors and other supporting data structures that may be used in accordance with the teachings of the present invention. As discussed in detail below, Figure 8 Figure 22 of the various media The volume extractor includes various features. In general, any of the media extractors of Figures 8 through 21 can be included in a media extractor track of a file to identify a sample of the file's coded code, the file conforming to iso The base media file format or an extension to the 18” base media file format. In general, the media extractor can extract one or more global samples from the reference track. Figures 8 through 12 are ones that can identify one of the other tracks. An example of a media extractor for a box. Another way to implement an extractor is to enable grouping of samples from samples of another trajectory, as shown in Figure 13. To provide more specific support for time-adjustable capabilities, as shown in Figure Η The 'transportable time identifier is shown. Figure 16 through Figure 22 are examples of Mvc's media extractor capable of extracting one or more potentially discontinuous NAL units from each video sample box (access unit). Various examples of extractors are 151028.doc •57- 201119346 based on offsets and byte lengths in archives or access units, while other instances can be based purely on the index of the overall NAL unit, so the subpoena The tuple range is not necessary. The mechanism of the index communication extractor of the overall NAL unit can also be extended to the SVC file format. The examples of Figures 8 through 21 can also be directly applied to the 3GPP file format as an extension to the 3GPP file format. The elements and concepts of - or more of Figures 8 through 21 may also be combined with elements of the other of Figures 8 through 22 to form other extractors, although Figures 8 through 2 are described with respect to a particular file format. Some of the figures, but in general, the examples of Figures 8 through 21 can be used with respect to any file format having similar characteristics (e.g., an extension of the IS) base media file format or the iso base media file format. In the example of 21, in order to facilitate the use of the proposed extractor in 3Gpp, the 3Gpp trajectory selection phase may be extended to include more features of each of the (extracted) alternate trajectories, such as a time identifier, The number of views to be displayed, and the number of views to be decoded. Figure 8 is a block diagram illustrating an example media extractor 3, which illustrates the format of a monthly media extractor. In the example of Figure 8, media extractor 300 includes a track reference index 3〇2 and a sample offset value of 3〇4. In accordance with the teachings of the present invention, the media extractor 3 can correspond to a definition of a data structure that can be instantiated within the media extractor trajectory. The multiplexer 30 can be configured to include an extractor that follows an instance of the media extraction benefit 300 in a media extractor trace of the video archive to identify NAL units of different trajectories of the video archive. The decimator 38 can be configured to use the extractor compliant with the media extractor 3 to fetch the identified NAL unit. 151028.doc • 58- 201119346 Trace > The test index 302 may correspond to an identifier of the track in which the identified NAl unit is present. A unique cable can be assigned to each track of the video file to distinguish the track of the video file. The Tracking Reference Index 302 can specify the factory> index to find the trajectory for extracting data. For the extraction; the samples in the trajectory can be accurately time aligned (in the media decoding timetable, the 'use time·sample table' is deflated by the offset specified by the sample offset value 3〇4), where This sample contains an extractor. In some examples, the first track of the channel has an index value of Γ1": therefore, the multi-working center (9) (four) reference index value 3 () 2 is assigned as "the value of 匕 to refer to the first track of the video file. The value of the track reference index value is "〇" for future use. The offset value 304 is the offset value from the time slot of the media extractor in the media extractor track to the identified NAL unit of the track indicated by the track reference index 3()2. That is, the sample offset value 3〇4 gives a value of the relative value of the sample offset value 3〇4 of the sample in the linked track used as the information source, and has the same value as the sample containing the extractor. The decoding time is either immediately before the sample containing the sample of the extractor. The sample is the same as the following sample-1 is the same as before, and so on. For example, when using a two-body extractor that complies with the media extractor in H.263 or MPEG_4 Part 2, the media extract is used to extract the time of the video censored by the profiling reference. Subset. The following pseudo code provides an instance definition similar to the media extraction number category of media extractor 3 (10). Class aligned(8) MediaExtractor () { 151028.doc •59- 201119346 unsigned int(8) track_ref_index; signed int(8) sample—offset; } multiplexer 30 and demultiplexer 38 can be used in the above example pseudocode The media extractor defined in the example is used to instantiate the media extractor data object. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor when extracting data from the selected trajectory to freely retrieve the identified material from another trajectory referenced by the instantiated media extractor. In the example pseudocode, 'class MediaExtractor() is aligned by the byte. That is, when the extractor is instantiated by the MediaExtractor() class, the extractor is aligned on the octet boundary. The variable rtrack_ref_index" corresponds to the track reference index value 302, and in this example pseudo code corresponds to an octet integer value without a sign. The variable r sample-〇ffset" corresponds to the sample offset value 304' and in this example corresponds to an octet integer value with a sign. Figure 9 is a block diagram showing another example of the media extractor 31. Media extractor 310 includes track reference index 314 and sample offset value 316, and additionally includes sample header 312. Track reference index 314 and sample offset value 316 may typically include data similar to track reference index 3〇2 and sample offset value 3〇4 (Fig. 8). In an example corresponding to H.264/AVC, the sample header 312 can be constructed from the naL unit header of the video sample referenced by the media extractor 310. The sample header 3 12 may contain one byte data with three syntax elements: forbidden one zero_bit, nal_ref_idc (which may contain 3 bits), I51028.doc • 60· 201119346 nal—unit—type (which may include The value of 5 bits % Λ ' ^ ; nal-unit_type " is 29 (or any other reserved number), and the other two words = the same grammatical elements in the video sample. For compliance 4 For some examples of "vision", the sample header (1) may contain a four-digit two-group code, and the code may include the start code of the "Gx1" code and

The starting value of Ox C5" (or any other reserved number) where "if" means not 〇χ is the hexadecimal value. For H 263, the sample header 312 may also include a starting code of the bit group alignment that is different from the start code of the normal video sample. The sample header 312 can be used by the demultiplexer 38 for synchronization purposes so that the extractor can be considered a normal video sample. The following pseudocode provides an example definition of a media extractor class similar to media extractor: class aligned(8) MediaExtractor () {

SampleHeader (); unsigned int (8) track_ref_index; signed int (8) sample offset; } multiplexer 30 and demultiplexer 38 may use the media extractor defined in the above example pseudocode to instantiate the media extractor data object . Therefore, the solution to the crying 38 (for example) may refer to the exemplified media extractor when extracting data from the selected trajectory, so as to freely obtain the identified data from the other exemplified media extractor reference. 10 is a block diagram illustrating an example media extractor 320 that identifies NAL units by communicating in the extractor a range of bits of the identified NAL unit • 61 - 151028.doc 201119346. Media extractor 320 includes a sample header 322 that can be similar to sample header 312, and a track reference index 324 that can be similar to track reference index 302. However, examples of media extractor 320 include data offset value 326 and data length value 328 rather than sample offset values. The data offset value 326 can describe the starting point of the material identified by the media extractor 320. That is, the data offset value 326 can include a value indicative of the offset of the first byte within the track identified by the track index value 324 to be copied. The data length value 328 can describe the number of bytes to be copied, and thus can be equivalent to the length of the referenced sample (or multiple samples when referring to multiple NAL units). The following pseudocode provides an example definition similar to the media extractor class of media extractor 320: class aligned(8) MediaExtractor () {

SampleHeader (); unsigned int (8) track_ref_index; unsigned int (32) data_offset; signed int (32) data_length; } _ multiplexer 30 and demultiplexer 38 may use the media extractor defined in the above example pseudo code To instantiate the media extractor data object. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor when extracting data from the selected trajectory to freely retrieve the identified material from another trajectory referenced by the instantiated media extractor. 11 is a block diagram illustrating an example media extractor 340 that contains reserved bits for future scalability. Media extractor 340 includes track reference index 342 and sample offset value 346 that may be similar to media extractor 3〇2 and sample offset values, respectively. In addition, media extractor 3 40 includes reserved bits 344, which may contain reserved bits for future expansion of the media extractor. The following pseudocode provides an instance class definition similar to the media extractor class of media extractor 34: class aligned(8) MediaExtractor () { unsigned int(8) track_ref_index; unsigned int(8) reserved_bits; signed int(8 Sample_offset; } The multiplexer 30 and the demultiplexer 38 may instantiate the media extractor data object using the media extractor defined in the above example pseudo code. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor when extracting data from the selected trajectory to freely retrieve the identified material from another trajectory referenced by the instantiated media extractor. Figure 12 is a block diagram illustrating an example media extractor 35 that uses track identifier values instead of track reference index values. Using the track identifier value to identify the track can refer to presenting the track reference box in the IS〇 base media file format. Examples of media extractor 35G include a trace identifier 352, a reserved bit 354, and a sample offset value 356. The reserved bit 354 is optional as indicated by the dashed line surrounding the reserved bit 354. That is, some instances may include reserved bits 354, while other instances may omit reserved bits 354. The sample offset value 356 can be similar to the sample offset value of 3〇4. J51028.doc -63- 201119346 The track identifier 352 specifies the track 1〇 for the track of the extracted data. The samples in the trajectory for extracting data may be aligned in time (in the media decoding time table, using the time-sample table, adjusted by the offset specified by the sample offset 356), where the sample contains the media extractor 35〇. An identifier value of 1 can be assigned to the first trace reference. It can be retained as a value for future use and expansion. The following pseudocode provides an example definition of a media extractor class similar to media extractor 35: class aligned(8) MediaExtractor () { unsigned int(8) trackjd; unsigned int(8) reserved_bits; signed int(8) sample_offset; The multiplexer 30 and the demultiplexer 38 can instantiate the media extractor data object using the media extractor defined in the above example pseudo code. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor' when extracting data from the selected trajectory to freely retrieve the identified material from another trajectory referenced by the instantiated media extractor. 13 is a block diagram illustrating an example media extractor sample group 36. The multiplexer 30 can include the media extractor sample group 36 in a message type box (having a type identifier "MESG"), a sample list box container. The multiplexer 3〇 can be configured to include zero or one media extractor sample group 36 〇 objects in the message box. In the example of FIG. 13, media extractor sample group 360 includes track reference index 362, group type 364, group number count 151028.doc • 64·201119346 366, reserved bit 368, and group description index 370. The trajectory reference index 362 specifies an index of trajectory references used to find trajectories from the sample group for extracting data under a certain criterion. That is, the trajectory reference index 362 identifies the trajectory for extracting the material identified by the media extractor in a manner similar to the trajectory reference index 302. The group type value 364 identifies the type of sample group that the media extractor sample group 36 〇 corresponds to. The group type value 364 typically identifies criteria for forming a sample group of sample groups and links the criteria to a sample group description table having the same group type value among the tracks identified by the track reference index 3 62. Group type value 364 can include an integer value. In this manner, the group type value of the media extractor sample group 360 can be the same as the group type of the track indicated by the tracking reference index 362. Alternatively, for the video time subset, the group type value 364 can be defined as "vtst", the media extractor sample group can be defined only for the group type, and the syntax element will not require the syntax element "grouping_type". The group number juice value 366 may describe the number of sample groups in the media extraction track including the media extractor sample group (4). A value of zero for the group Hamas count value 366 can be expressed by the group of samples referenced by the group type value (10) under the criterion to form a media extractor trajectory. The group description index (10) defines an index of the sample group items used to form the media extractor trajectory in the sample group description table. According to the technique of the present invention, the π m 7 port process can be used to place all samples in the sample, so that the samples are sorted by time, so that the sample 指 refers to the sample search after the sample B in the media extraction benefit track.曰 No sample A is after sample B in the trace of the track reference index 362 15I028.doc -65· 201119346. The following pseudocode provides an example definition of a media extractor sample group class similar to media extractor sample group 360: class aligned(8) MedEtrSampleGroup () { unsigned int(8) track_ref_index; unsigned int(32) grouping_type; unsigned int (32) group_number_count; for (i =0; i< group number count; i++ ) unsigned int (32) group-description_index; } multiplexer 30 and demultiplexer 38 can be defined in the above example pseudo code The media extractor is used to instantiate the media extractor data object. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor when extracting data from the selected trajectory to freely retrieve the identified material from another trajectory referenced by the instantiated media extractor. Figure 14 is a block diagram illustrating an example media extractor 380 that can be used in the context of a video archive in accordance with the AVC file format. Examples of media extractor 380 include a trace reference index 382, a time identifier value 3 84, a reserved bit 386, and a sample offset value 388. The trajectory reference index 382 and the sample offset value 388 can be used in a manner similar to the trajectory reference index 302 and the sample offset value 304, respectively. The reserved bit 386 can be reserved for future use and no semantic values are assigned to it at this time. The time identifier value 384 specifies the time level of the sample to be extracted by the media extractor 380. In one example, the time scale is in the range of -66· 151028.doc 201119346 of 〇 to 7 (including 0 and 7). As discussed above, the encoded picture may correspond to a time level, where the time level generally describes the coding level between frames. For example, the key frame (also known as the anchor frame) can be assigned the highest time level, and the frame that is not used as the reference frame can be assigned a relatively low time level. In this manner, media extractor 380 can identify the extracted samples from the trajectory referred to by trajectory reference index 382 by reference to the temporal level of the samples rather than explicitly identifying the samples themselves. The media extractor trajectory of the media extractor having a value above a value defined by the time identifier value 384 may correspond to an operating point having a two frame rate of the parent. The following pseudocode provides an example definition of a media extractor class similar to media extractor 38: class aligned(8) MediaExtractor () { unsigned int(8) track_ref_index; unsigned int(3) temporal_id; unsigned int(5) Reserved_bits; signed int(8) sample_offset; } The multiplexer 30 and the demultiplexer 38 can instantiate the media extractor data object using the media extractor defined in the above example pseudo code. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor when extracting data from the selected trajectory so that the exemplified media extractor can refer to the other trajectory to retrieve the identified material. 15 is a block diagram illustrating an example MVC media extractor 42 that can be used to modify the MVC to include media extraction 151028.doc • 67-201119346 trajectories. Examples of media extractor 420 include an optional NAL unit header 422, track reference index 424, sample offset 426, consecutive byte set count 428', and a value including data offset value 430 and data length value 432. Loop. The MVC media extractor 420 can be used to extract a plurality of NAL units from a subset of the view components from a particular profile. Example of MVC Media Extractor 420 The View component in the trajectory can be skipped when the data is extracted from the sample of the referenced trajectory. When present, the NAL unit header 422 can mirror the NAL unit header of the NAL unit identified by the MVC media extractor 420. That is, the syntax elements of NAL unit header 422 may be generated based on the NAL unit header syntax in the extractor or the summary tool generation process defined in the MVC file format. In some examples, the extractor may not require the NAL unit header 422, for example, when a series of extractors are to be generated to include the associated NAL unit header. The trajectory reference index value 424 specifies an index of the trajectory reference for finding a trajectory for extracting data. The samples in the trace for the extracted data may be temporally aligned in the media decode schedule, adjusted by the offset specified by the sample offset value 426, where the sample contains the Mvc media extractor 42. The first trace reference can be assigned to receive an index value of 】, and the value of the track reference index value of zero can be retained. The π sample offset value 426 defines the offset of the sample to be extracted relative to the time position of the MVC media extraction, which is located in the track towel referred to by the track reference index value 424. A value with a sample offset value of zero indicates that the sample is to be fetched = the sample system is at the same time position, _丨 indicates the previous sample, +ι indicates the next sample, and so on. 151028.doc • 68 · 201119346 Continuous byte set count 42 8 describes the number of consecutive byte sets for the sample of the trace of the extracted data. If the consecutive byte set count 428 has a value of zero, then the entire referenced sample in the trajectory will be extracted. A collection of consecutive bytes can also be referred to as a separate part of the sample. The data offset value 430 and the data length value 432 appear in a loop. In general, the number of repetitions of the loop (i.e., the number of data offset values 430 and data length values 432) is related to the number of portions of the sample to be retrieved (e.g., the number of consecutive sets of bytes). Thus, MVC media extractor 420 can be used to extract two or more portions of a sample. For each portion of the sample to be extracted, the corresponding one of the data offset values 430 indicates the beginning of the portion (eg, the first byte of the portion, relative to the first byte of the sample), and the data The corresponding one of the length values 432 indicates the length to be copied (eg, the number of bytes). In some examples, a value of zero of one of the data length values 432 may indicate that all remaining bytes in the sample will be copied, ie, the portion corresponds to the bit indicated by the corresponding one of the data offset values 430. The tuple and all other consecutive bytes up to the end of the sample. The following pseudocode provides an example definition similar to the media extractor class of the MVC media extractor 420: class aligned(8) MediaExtractorMVC () { NALUnitHeader(); // may be omitted in some instances unsigned int(8) track_ref_index; Int(8) sample-offset; unsigned int(8) continuous_byte_set_count; for (i = 0 ; i <continuous_byte_set_count; i++ ) { 151028.doc -69- 201119346 unsigned int((lengthSizeMinusOne+1 ) *8) data-offset; unsigned int((lengthSizeMinusOne+1)* 8) data_length; The multiplexer 30 and the demultiplexer 38 may instantiate the media extractor data object using the media extractor defined in the above example pseudo-drink order. Thus, the demultiplexer 38, for example, may refer to a media extractor when extracting data from the selected trajectory so as to freely exemplify the media extractor actor 1 < another trajectory to retrieve the identified material. Figure 16 is a diagram illustrating another example MVC media extractor 44, which may be used to modify the MVC to include the media extractor trajectory. In contrast to the specific bit group of the sample as described with respect to the example of Fig. 15, the real gamma of the MVC media extractor 440...

The column identifies a particular NAL unit for extraction. In the example of FIG. 16, the MVC mediator 440 includes an optional NAL unit header 442, a track reference index 4, a sample offset Yang, and a consecutive NALU (NAL unit) set count 448. And a loop of the NALU offset value 450 and the number 452 of consecutive NAL units. The NAL unit header 442, the track reference index 444, and the sample offset value 446 are typically deprecated in the same manner as the NAL unit header 422, the tracking reference index 424, and the sample offset value 426, respectively. The continuous NALU set count 4 4 8 describes the number of consecutive NAL units of samples for the trajectory of the extracted data. In some examples, if this value is set to 151028.doc -70-201119346 zero' then the entire reference in the extracted trajectory Sample. The NALU offset value 450 and the number 452 of consecutive NALUs appear in a loop. In general, as defined by the consecutive NALU set count 448, there are as many instances of the NALU offset value and the number of consecutive NALUs as there are consecutive NALU sets. Each NALU offset value describes the offset of the corresponding NAL unit at the sample for the trace of the extracted data. This extractor can be used to extract NAL units starting from this offset of the NAL unit. Each value of the number of consecutive NALUs describes the number of entire single referenced NAL units of the corresponding NAL unit set to be copied. The following pseudocode provides an example definition of the media extractor class similar to MVC media extractor 440: class aligned(8) MediaExtractorMVC () { NALUnitHeaderQ; //In some instances, unsigned int(8) track_ref_index; signed int(8) Sample_offset; unsigned int(8) continuous_NALU_set_count; for (i = 0 ; i <continuous_NALU_Set_Count; i++ ) { unsigned int((lengthSizeMinusOne+1 )* 8) NALU_offset; • unsigned int((lengthSizeMinusOne+1 ) * 8) num_continuous_NALUs } } The multiplexer 30 and the demultiplexer 38 can use the 151028.doc • 71· 201119346 media extractor defined in the above example pseudocode to instantiate the media extractor data object. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor when extracting data from the selected trajectory to freely retrieve the identified material from another trajectory referenced by the instantiated media extractor. Figure 17 is a block diagram illustrating another example MVC media extractor 460 that summarizes NAL units in the same view component when there is more than one person of a view component. The MVc media extractor 460 can then be used to extract the identified view components. In the example of FIG. 7, MVC media extractor 460 includes an optional unit header 462, track reference index 464, sample offset 466, continuous view set count 468, and view component offset value 470. And the view component counts 472 the loop. The NAL unit header 462, the trajectory reference index 466, and the sample offset value 466 are typically defined in the same manner as the NAL unit header 422, the trajectory reference index 424, and the sample offset value 426, respectively. The continuous view set count 468 defines the number of consecutive view components of the identified samples in the trace for extracting data identified by the trace reference index 464. The multiplexer 30 can set the value of the continuous view set count 468 to zero to indicate that the entire referenced sample in the trajectory will be extracted. The view component offset value 470 and the view component count 472 appear in the -loop. In general, there are as many loop repeats as the value of the continuous view set count 468, and each loop corresponds to the one in the continuous view set. Each of the view component offset values 470 indicates an offset of the first view component at the sample of the track of the corresponding continuous view set for extracting the data. The view component of the self-view component can be extracted using the MVC media extractor 460 as follows: 151028.doc • 72· 201119346. Each of the view component counts 472 describes the number of entire referenced view components to be copied in the samples of the respective successive view sets. The following pseudocode provides an example definition similar to the media extractor class of the MVC media extractor 460: class aligned(8) MediaExtractorMVC () { NALUnitHeader(); // In some instances, unsigned int(8)trak_ref_index; signed int may be omitted (8) sample_offset; unsigned int(8) continuous_view_set_count; for ( i = 0 ; i <continuous_view_set_count; i++) { unsigned int((lengthSizeMinusOne+1 )* 8) view_component_offset; unsigned int((lengthSizeMinusOne+1 )* 8) view_component_count } The multiplexer 30 and the demultiplexer 38 may instantiate the media extractor data object using the media extractor defined in the above example pseudo code. Thus, the demultiplexer 38 can, for example, refer to the exemplified media extractor' when freeing data from the selected trajectory to freely retrieve the identified material from another trajectory referenced by the media extractor as exemplified. Figure U is a block diagram illustrating another example of an MVC media extractor 480 that can be used to reference various trajectories. In the example of FIG. a, MVC media extractor 480 includes an optional NaL unit header 482, a continuous view set count 484, and a sample offset value 486, a trace reference index 15l028.doc -73 - 201119346 value 488, view The component offset value 490 and the loop of the view component count 492. NAL unit header 482 may be defined similar to NAL unit header 422 and may be omitted in some examples. The continuous view set count 484 gives the number of consecutive view components of the sample with the track reference index track_ref Jndex for the extracted data. The track_ref_index value specifies the index of the track reference to find the track for extracting data. The view components in the trajectory for extracting data may be aligned in time (in the media decoding time table, using the time sample table, adjusted by the offset specified by the corresponding one of the sample offset values 486) The sample contains MediaExtractorMVC. The first track spring has an index value of 1; the value can be retained for future use. An example of MVC media extractor 480 includes each of sample offset value 牦6, trajectory reference index value 488, view component offset value 490, and view component count in a loop. Each loop repeatedly corresponds to a particular trajectory for extracting data corresponding to samples of the MVC media extractor 480. The sample offset value 486 defines the relative index of the samples in the trajectory, as indicated by the corresponding one of the trajectory reference index values 488, which can be used as information:. ^ 零 (zero) is the sample of the track reference index value 488, which is the same as the sample with the MVC media extractor 48〇, or the sample before the sample of the MVC media extractor 480. , sample 1 is inclusive, sample-1 is the same as before, and so on. Proof Each of the refurbishment reference index values 488 specifies a refusal reference for finding the information for extracting the corresponding lap & "引引 = track reference index value, MVC media extraction 9 uses multiple T from multiple different tracks 15I028.doc • 74, 201119346 trace extraction data. View component offset value 490 describes each of the traces for extracting data An offset of the first view component at the sample, the trajectory having a trajectory reference index corresponding to the corresponding one of the trajectory reference index values 488 in the loop. The MVC media extractor 480 can be used to extract the self-view component Offset starting view component. In some examples, a media extractor having a nested loop structure similar to the media extractors of Figures 15-17 can be constructed, in the nested loop structure, The outer loop is overlaid via the trajectory for extracting the sample, and the inner loop is overlaid by the sample to be extracted from the corresponding trajectory. Each of the view component counts 492 describes the number of referenced view components in the sample of the trajectory, The trajectory has a trajectory reference index corresponding to the current one of the trajectory reference index values 488 in this loop. The following pseudocode provides a media extractor class similar to the MVC media extractor 480. Example definition: class aligned(8) MediaExtractorMVC () { NALUnitHeader(); //In some instances you can omit unsigned int(8) continuous_view_set_count; for (i = 0 ; i <continuous_view_set_count; i++) { signed int(8 ) sample-offset; unsigned int(8) track-ref_index; unsigned int((lengthSizeMinusOne+1)* 8) view_component_offset; unsigned int((lengthSizeMinusOne+1 )* 8) view—component—count 151028.doc -75- 201119346 The 1/multiplexer 3G and demultiplexer 38 can instantiate the media extractor data object using the media extractor defined in the pseudo code of the above example. Thus, the demultiplexer 38 (for example) captures from the selected trajectory The data may be referred to the exemplified media extractor so as to freely retrieve the identified data from the other exemplified media extractor reference. Figure 19 is a block diagram illustrating another example MVC media extractor 5 The duration of the MVC media extractor 500 communication extractor. When different samples in the media extractor trajectory share the same syntax elements of the extractor, the MVC media extractor 500 can provide Or a plurality of advantages. In the example of FIG. 19, the MVC media extractor 500 includes a sample count 5〇2, a continuous view set count 504, a sample offset value 506, a track reference index 5〇8, a view component offset 5 1 0 , and the view component counts 5 12. The continuous view set count 504, the sample offset value 506, the track reference index 508, the view component offset 510, and the view component count 512 can generally be based on a continuous view set count 484, a sample offset value 486, a track reference index 488, a view component bias. The corresponding one of the shift 490 and the view component count 492 is defined. The sample count 502 can define the number of consecutive samples using the same media extractor in the media extractor trajectory containing the MVC media extractor. The following pseudocode provides an example definition similar to the media extractor class of the MVC media extractor 500: class aligned(8) MediaExtractorMVC () { unsigned int(8) sample count; 151028.doc •76· 201119346 unsigned int(8) continuous_view_set_count ; ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( lengthSizeMinusOne+1) * 8) view_component_count } } The multiplexer 30 and the demultiplexer 38 can instantiate the media extractor data object using the media extractor defined in the above example pseudo code. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor when extracting data from the selected trajectory to freely retrieve the identified material from another trajectory referenced by the instantiated media extractor. 20 is a block diagram illustrating another example MVC media extractor 520 that defines a set of different extractors. For each sample in the media extractor trajectory, the sample may use one or more of the extractors in the set of extractors, or reference one of the extractors. That is, one set of media extractors similar to MVC media extractor 520 can be defined, and each sample can use one or more extractors in the set of extractors or one of the extractors to identify another A sample of the falsification. Examples of MVC media extractor 520 include extractor identifier value 522, sample offset value 524, track reference index value 526, continuous view set count 151028.doc -77 - 201119346 528, and include view component offset 530 and view component Count the loop of 532. Sample offset value 524, continuous view set count 528, view component offset 5 30, and view component count 532 may be based on successive view set count 484, sample offset value 486, view component offset 490, and view component count 492. To define. The trajectory reference index value 526 can be defined in accordance with, for example, the trajectory reference index 464. Extractor identifier value 522 defines the identifier of the extractor (i.e., MVC media extractor 520). The extractor in the same media extractor trajectory is assigned a different extractor identifier such that samples in the media extractor trajectory can refer to the extractor identifier value to use the media extractor. The reference extractor bin can also be defined to include the number of extractors and the reference extractor identifier. The value of the number of extractors provides the number of extractors used to copy the data of the samples in the extractor trajectory. When the value of the number of extractors is equal to zero, an extractor having a predetermined extractor identifier (e.g., an extractor identifier equal to zero) can be used. The reference extractor identifier provides an extractor identifier for the extractor that copies the data for the samples in the extractor trajectory. This box can be included in the sample of the media extractor track. The following pseudocode provides an example definition similar to the media extractor class of the MVC media extractor 520: class aligned(8) MediaExtractorMVC () { unsigned int((lengthSizeMinusOne+l)*8) extractor_id; signed int(8) sample_offset; unsigned Int(8) track_ref_index; for ( i = 0 ; i <continuous_view_set_count; i++) { 151028.doc -78 - 201119346 unsigned int((lengthSizeMinusOne+1 )* 8) view component offset; unsigned int((lengthSizeMinusOne+1 ) * 8) viewcomponentcount; } } The multiplexer 3 〇 and multiplexer 38 can instantiate the media extractor data object using the media extractor defined in the above example pseudo code. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor' when extracting data from the selected trajectory to freely retrieve the identified material from another exemplified media extractor reference. The following pseudocode provides an example definition of the reference extractor box class of the reference extractor box described above: class aligned(8) RefExtractorMVC () { unsigned int((lengthSizeMinusOne+1 )* 8) num—extractor; for (i = 0 ; i <num_extractor; i++) ref_extractor_id ; } } Figure 21 is a block diagram illustrating an example MVC media extractor 550, which may be formed using a group of mapped samples. An instance of Mvc Media Extractor 550 specifies a group of NAL units from a series of sample items, each of which contributes to the $151028.doc •79-201119346 NAL unit in the mapped sample group. In the example of FIG. 22, MVC media extractor 550 includes a NALU group count 552, and a loop including track index 554, group description index 556, NALU start map sample 558, and NALU view count 560. The NALU group count 552 specifies the number of NAL unit groups from the mapped sample group entries in the reference trajectory. The trajectory reference index 554 each specifies an index of the trajectory reference for finding a trajectory for extracting the data of the corresponding loop. The group description index 556 each specifies an index of the mapped sample group item used to form the NAL unit group of the corresponding loop. The NALU start mapping samples 558 each specify an offset of a NAL unit in a mapped sample group in the corresponding loop, the mapped sample group having a mapped sample item index of the corresponding one of the group description index 556. The NALU view count 560 specifies the number of consecutive NAL units in the corresponding mediator to be extracted into the media extractor in the mapped sample group, the mapped sample group having the mapped sample item index of the corresponding one of the group description index 556 . The following pseudocode provides an example definition similar to the media extractor class of the MVC media extractor 550: class aligned(8) MedEtrMapSampleGroup () { unsigned int(32) NALU group count; for ( i =0; i<NALU_group_count; i++ ) { unsigned int(8) track_ref_index; unsigned int(32) group_description_index; unsigned int(8) NALU_start_map_sample; unsigned int(8) NALU_view_count; I51028.doc • 80 · 201119346 Multiplexer 30 and multiplexer 38 can be used above The media extractor defined in the example pseudocode is used to instantiate the media extractor data object. Thus, the demultiplexer 38, for example, may refer to the exemplified media extractor when extracting data from the selected trajectory to freely retrieve the identified material from another exemplified by the instantiated media extractor. The techniques of this disclosure may include a combination process for configuring a view component of a sample into a sample group. The view components in the sample of the sample group item are sorted in a time manner such that if sample A is after sample B in the original track (with the index of the track reference index), then the view component in sample eight is

After the view component in sample B in the media extractor trajectory; if sample A has a decoding time earlier than the decoding time of sample B, then the view component in sample a is behind the view component in sample B in the media extractor trajectory The two view components in the same sample of the track follow the presentation order in the syntax table of the media extractor mapping sample group; if two view components in the same sample of the track belong to the same group of NAL units, ie, The sample group is mapped by the media extractor, and the syntax elements of the same loop in the group are extracted, then the two view components follow the original order; and if the samples of the same time stamp are extracted from different traces, the two view components are extracted. , then the two view groups = follow the order of the view order index as indicated in the view identifier box of the MVC slot format. Figure 2 2 is a block diagram illustrating an example of a modified 3GPP trajectory selection box 39 for additional attributes of the trajectory selection box. According to this document (4) (4), the most 15I028.doc 201119346 new 30?? standard specifies an entry "1^1^61^31, which includes attributes describing the following: language, bandwidth, codec, screen size, maximum packet Size, and media type. The attribute list 392 of the 3GPP trace selection box 390 includes a language value 394, a bandwidth value 396, a codec value 398, and a screen size value 400, which are communicated according to existing 3GPP standards. Moreover, the techniques of this disclosure may modify an existing 3GPP track selection box to include a frame rate value 406, a time identifier value 408, and (in some cases) a view view number value 410 and an output view list value 412. The language value 394 defines the value of the group type LANG of the "alternating group" attribute in the session level SDP as defined in clause 5.3.3.4 of the existing 3GPP standard. The bandwidth value 396 defines the value of the "b=AS" attribute in the media level SDP. The codec value 398 defines the SampleEntry value in the sample description box of the media track. The honor size value 400 defines the width and height fields of the MP4VisualSampleEntry value and the H263SampleEntry value in the media trace. The maximum packet size value 402 defines the value of the MaxPacketSize field in the RTPHintSampleEntry (eg, in the RTP schematic trace). The media type value 404 describes the HandlerType in the handler box of the media track. In general, these values correspond to existing 3GPP standards. The frame rate value 406 describes the frame rate corresponding to the video track or media extractor track of the 3GPP Strike Selection Box 390. The time identifier value 408 corresponds to the time identifier corresponding to the video track of the 3GPP track selection box 390 and may be subject to the presence of a lower time identifier value. In some examples, multiplexer 30 may indicate that the value of time identifier value 408 is not specified by setting the value to a pre-configured "unspecified" value (e.g., 8). Generally and 151028.doc -82- 201119346. Xigongyi 30 may indicate that the value of the time identifier value 408 of the non-video track is not specified. In some instances, the multiplexer may also indicate that the time slot value is not specified when the corresponding video track does not contain a media extractor and/or is not referenced by other time material subsets. Value. In the example of considering Mvc in 3, the multiplexer 3 may include additional attributes: a display view number value of 41 〇 and an output view list value 412. In such an example, multiplexer 30 may omit the time identifier value 4〇8. The number of views is displayed. The number of views of the corresponding trajectory of the 4HM field is output. For example, when the view to be displayed is encoded in a view not shown, the number of pictures to be outputted 2 does not have to be the same as the number of views to be decoded. Output View The π-single value 412 defines a list of view identifiers that identify the views to be output. Figure 23 is a flow diagram of an example method for using a media extractor in accordance with the teachings of the present invention. Initially, such as a Α/ν source device 2 (the source device of the figure is constructed according to the technique of the present invention to construct a video track conforming to a file of a file format. That is, the multiplexer 30 combines the encoded video data into the track. The video track includes a coded video sample, the video sample includes one or more NAL units (600), and the multiplexer 3A also constructs one or all of the reference video tracks or some or all of the NAL units. An extractor (6〇2) and constructing an extractor track (6〇4) including the extractor. In addition, the multiplexer 3 can include the encoded video sample in the media extractor track and include the encoded In the additional trace of the video sample and/or media extractor, the multiplexer 30 can then output the file (6〇6). The file can be output to the transmitter, transceiver, network interface, modem or other signal output component to - 151028.doc •83- 201119346 The signal 'or file can be output to the storage medium via a hardware interface such as a USB interface, magnetic media recorder, optical recorder or other hard interface. A/V destination device 4 〇 (eg The file is finally received (608) by receiving a signal or reading the storage medium. The demultiplexer 38 can select one of two (or more) tracks for decoding (61 〇). W can select one of the trajectories of the decoding capability of the video decoder 48, the rendering capability of the video output 44, or its T criterion. ◎ When an extractor is selected, the demultiplexer 38 can be executed from there. Taking the NAL unit of the extractor in the extractor trajectory, the encoded video sample identified by the extractor is stored in the trajectory. The multiplexer 38 can be discarded and not in the selected trajectory and is not An encoded video sample (or 2 other NAL unit) identified by at least one of the selected trajectories. That is, the 'demultiplexer 38 can prevent such encoded samples from being sent to the video decoding (4), such that There is no need to assign a task to the video decoder to decode unused video material. In the case of - or multiple instances, the functions described may be implemented in hardware, software, or any combination of them. These functions can be used as - or more Or the program code is stored on a computer readable medium (4) or transmitted via a computer readable medium. The computer readable medium can include a computer readable storage medium such as a data storage medium or a communication medium, the communication medium including facilitating the computer program from one place. Any medium that is transferred to another location. The data storage medium can be any available medium that can be accessed by - or multiple computers or - or multiple processors to retrieve instructions for implementing the techniques described in this disclosure, The code and / or data structure. By way of example and without limitation, 151028.doc -84 - 201119346 This computer readable storage medium can include: ram, r〇m, eepr〇m, ROM or other optical disk storage, magnetic A disk storage or other magnetic storage device, flash memory, or any other medium that can be used to store a desired code in the form of an instruction or data structure and accessible by a computer. Also, any connection is properly termed a computer-readable medium. For example, if you use coaxial wire, fiber optic cable, twisted pair cable, digital subscriber line (dsl), or wireless technology such as external lines, radio and microwave, from websites, feeders, or other remote sources. Transmission instructions 'coaxial cable' fiber optic cable, twisted pair, SL or wireless technologies such as infrared, radio and microwave are included in the definition of the media. However, it should be understood that computer readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media. As used herein, magnetic disks and optical discs include compact discs (CDs), laser discs, optical discs, digital audio and video discs (_), flexible magnetic discs and Blu-ray discs. One of the magnetic discs usually reproduces data magnetically, while the optical discs are used. The combination of each of the above by laser, and the combination of the above should also be included in the scope of computer readable media. The instructions encoded in the computer readable medium can be executed by one of, for example, one or more processors: one or more digital signal processors (DSPs), pass-through microprocessors, special application integrated circuits (ASICs) Field programmable logic (UFPGA) 'or other equivalent integrated or discrete logic circuits. Accordingly, the term "processor" as used in this specification may refer to any of the foregoing structures or any other structure suitable for the techniques described in the gamma. In this case, the functionality described herein can be provided in a dedicated and/or software module that is configured for encoding and decoding or as a combination of 151028.doc -85-201119346 In the decoder. Moreover, such techniques can be fully implemented in one or more circuits or logic elements. The techniques of this disclosure may be implemented in a variety of devices or devices, including wireless handsets, integrated circuits (1C) or a collection of 1C (eg, a chipset). Various components, modules or units are described in the present invention. To emphasize the functional aspects of devices configured to perform the disclosed techniques, but not necessarily required to be implemented by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit, or combined with a suitable software and/or set of interoperable hardware units including one or more processors as described above. Or firmware east provides. Various embodiments have been described. These and other examples are within the scope of the following patent application. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an example system in which an audio/video (A/V) source device transmits audio data and video data to an A/ν destination device. 2 is a block diagram showing an example configuration of components of a multiplexer. Figure: is a block diagram of an example-example file, the example file includes a first sample of a set of fl samples, a first track of a set, and a second track 4 with an extractor. A subset of it. Figure 4 is a block diagram illustrating an embodiment of a file including two different extractor traces. 6A through 6C are blocks 151028.doc-86-201119346 of an example of a media data box illustrating a set of tracks and another file of two media extractor tracks, the media data box including media extraction of various media extractor tracks An instance of the device. Figure 7 is a conceptual diagram illustrating an example MVC prediction pattern. Figures 8 through 21 are block diagrams showing various examples of the material structure of the media extractor and other supporting data structures that may be used in accordance with the teachings of the present invention. Figure 22 is a block diagram showing an example modified third generation partnership plan (3Gpp) track selection box for communicating additional attributes of the track selection box. 23 is a flow diagram of an example method for using a media extractor in accordance with the teachings of the present invention. [Main component symbol description] 10 System 20 Audio/Video (A/V) source device 22 Audio source 24 Video source 26 Audio encoder 28 Video encoder 30 Multiplexer 32 Output interface 34 Computer readable media 36 Input interface 38 Solution Multiplexer 40 Audio/Video (A/V) Destination 42 Audio Output 15I028.doc -87· 201119346 44 Video Output 46 Audio Decoder 48 Video Decoder 60 Stream Management Unit 62 NAL Unit Builder 66 Stream Identifier (Stream ID) Search unit 64 Track generation unit 68 Extractor generation unit 80 Video input interface 82 Audio input interface 84 Multiple stream output interface 88 Program specific information table 100 File 102 MOOV box 104 Complete subset track 106 Media extractor track 110 Media data (MD AT) box 112 I-coded sample 114 P-coded sample 116 B-coded sample 118 B-coded sample 120 Extractor 122 Extractor 124 Extractor 151028.doc • 88. 201119346 140 File 142 MOOV box 144 Complete Subsection Execution 146 Extractor Trajectory 148 Extractor Trajectory 150 Media Data (MD AT) Box 152 I-coded sample 154 P-coded sample 156 B-coded sample 158 B-coded sample 160 Extractor 162 Extractor 164 Extractor 166 Extractor 168 Extractor 180 File 182 MOOV Box 184 Media Extractor Track 186 Media Extractor Trajectory 188 Subset Trace 190 Media Data (MD AT) Box 192 Level I Coded Sample 194 P Coded Sample 198 Extractor 151028.doc -89- 201119346 200 Extractor 202 B Coded Sample 204 Extractor 206 Extractor 208 B-coded sample 210 Extractor 220 Media Data (MD AT) box 222 Missed sample 223 Non-Cat Sample 224A View 0 Sample 224B View 0 Sample 226A View 2 Sample 226B View 2 Sample 228A View 1 Sample 228B View 1 Sample 230A View 4 Sample 230B View 4 Sample 232A View 3 Sample 232B View 3 Sample 240 Extractor Set 242A Extractor 242N Extractor 244 Extractor Set 246A Extractor 151028.doc -90- 201119346 246N Extractor 250 Extractor 252A Extractor Sample 252N Extractor Sample 254A Extractor 254B Extractor 256A 256B extractor 300 media extractor 302 trajectory reference index 304 sample offset value 310 media extractor 312 sample header 314 trajectory reference index 316 sample offset value 320 media extractor 322 sample header 324 trajectory reference index 326 data bias Shift value 328 data length value 340 media extractor 342 track reference index 344 reserved bit 346 sample offset value 151028.doc -91 201119346 350 media extractor 352 track identifier 354 reserved bit 356 sample offset value 360 media extractor Sample Group 362 Tracking Reference Index 364 Group Type 366 Group Number Count 368 Reserved Bit 370 Group Description Index 380 Media Extractor 382 Track Reference Index 384 Time Identifier Value 386 Reserved Bit 388 Sample Offset Value 390 3GPP Track selection box 392 Attribute list 394 Language value 396 Band value 398 Codec value 400 Screen size value 402 Maximum packet size value 404 Media type value 406 Frame rate value 151028.doc -92- 201119346 408 Time identifier value 410 display View number value 412 Output view list value 4 20 Media Extractor 422 NAL Unit Header 424 Track Reference Index 426 Sample Offset 428 Continuous Byte Set Count 430 Data Offset Value 432 Data Length Value 440 MVC Media Extractor 442 NAL Unit Header 444 Tracking Reference Index 446 Sample Offset value 448 Continuous NALU (NAL unit) set count 450 NALU offset value 452 Number of consecutive NAL units 460 MVC media extractor 462 NAL unit header 464 Trajectory reference index 466 Sample offset 468 Continuous view set count 470 View component Offset value 472 View component count 151028.doc -93- 201119346 480 MVC media extractor 482 NAL unit header 484 continuous view set count 486 sample offset value 488 track reference index value 490 view component offset value 492 view_component count 500 MVC Media Extractor 502 Sample Count 504 Continuous View Set Count 506 Sample Offset Value 508 Excluded Reference Index 510 View Component Offset 512 View Component Count 520 MVC Media Extractor 522 Extractor Identifier Value 524 Sample Offset Value 526 Track Trace Reference Index Value 528 Continuous View Set Count 53 0 View Component Offset 532 View Component Count 550 MVC Media Extractor 552 NALU Group Count 554 Track Index 151028.doc -94- 201119346 556 Group Description Index 558 NALU Start Map Sample 560 NALU View Count SO View SI View S2 View S3 view S4 view S5 view S6 view S7 view 151028.doc -95-

Claims (1)

201119346 VII. Patent application scope: 1' A method for encoding video data, the method comprising: constructing a trajectory based on the encoded video data by a source video device, wherein the first trajectory comprises a plurality of a video sample of a network access layer (NAL) unit, wherein the video sample is included in an access unit; and the source video device constructs a second track including an extractor, the extracting theft identifying the first At least one of the plurality of NAL units in the video sample of the trajectory, the at least one of the plurality of nal units includes a first identified NAL unit, and wherein the extractor identifies the access unit a second NAL unit, wherein the first identified NAL unit and the second identified NAL unit are discontinuous; the "Hai trajectory" and the far second trajectory are included in an at least partial compliance with international standards ( ISO) in the video file of the basic media file format; and rotate the video file. 2. The method of claim 1, wherein the video file conforms to the iso base media file format. 3. The method of claiming, wherein the video file complies with at least one of: an expandable video code (SVC) file format, an advanced video code (AVC) file format, and a third generation cooperation. Partner Program (3Gpp) file format, and Multiview Video Recording (MVC) file format. 4. The method of claim 1 wherein the constructing the second manifest further comprises including one or more additional NAL units in the first or a plurality of additional NAL units based on the encoded data, not included in the first An I51028.doc 201119346 trajectory of the plurality of NAL units. 5. The method of claim 4, further comprising constructing a second trajectory comprising - identifying a first extractor of one or more of the plurality of NAL units of the first trajectory - and - A second extractor that identifies at least one of the one or more NAL units of the second trajectory. 6. The method of claim 5, which is not included in the first' wherein constructing the third track further comprises including one or more NAL units in the track and the second track in the third track. A method of claim 1 wherein constructing the second track comprises constructing an extractor for identifying each of the plurality of NAL units of the video sample of the first track and wherein the extractor-destination device is overall Each of the plurality of NAL units of the video sample is extracted. 8. The method of claimant, wherein constructing the second track comprises constructing an extractor of the plurality of nal units identifying the video sample by the following operation: designating the first of the video files A one-tuple range of the one or more of the plurality of NAL units of the video sample in the track. The method of claim 1, wherein the NAL unit of the video sample in the first track comprises at least one of: a slice of a common picture, a non-video code layer ( VCL) NAL unit, Supplemental Enhancement Information (SEI) message Nal unit, video layer of the access unit, different view components of the access unit, and NUL unit 0 aggregated from multiple NAL units 10. If requested The method, wherein the plurality of nal units includes a first plurality of NAL units 151028.doc 201119346, the method further comprising constructing a third track comprising the second plurality of NAL units based on the encoded video data, wherein the second A plurality of NAL units form part of the access unit, and wherein the second plurality of NAL units includes the second identified NAL unit identified by the extractor. The method of claim 1, wherein the video sample comprises a first video sample, wherein the plurality of NAL units comprise a first plurality of Nal units, wherein the first track of the first path further comprises a second plurality of nal units a second sample, wherein the access unit includes the second sample, and wherein the second plurality of NAL units includes the second NAL unit identified by the extractor. The method of claim 1, wherein the second NAL unit includes a second element of the plurality of NAL units of the video sample of the first track and the first NAL unit of the video sample The identified NAL unit separates at least one tuple of information. Wherein the first track and the second track form 1 3. If the method of requesting item 1 is to switch the group, the second track is recruited; 151028.doc 201119346 Message 1 represents the value of the number of views to be displayed after decoding the first - track; the message is used for the last track of the second track. _. One or more views of the view to be displayed The identifier value; and the message 1 indicates the value of the number of views to be decoded for the (four) two tracks. 15. A device for encoding video data, the device comprising: an encoder H configured to encode video data; 彳工益', configured to: construct a first based on the encoded video data - The obstruction 'the first-existing includes--including a plurality of network access layers (a video sample of NAL#, the video sample is included in an access 7L early, constructed - including - the second track of the extractor, The extractor identifies at least one of the plurality of cells in the video sample of the meta-track, the at least one of the plurality of NAL units including a first identified NAL unit, and wherein the extractor Identifying a second NAL unit of the access unit, wherein the first identified NAL unit and the second identified muscle unit are discontinuous; the first track and the second track are included in at least one Partially in accordance with the International Standards Organization (IS0) video file format of the basic media file format; and an output interface configured to output the video file. 16. The device of claim 15 wherein the video file complies with At least one : ISO base media file format, expandable video code (SVC) file format, advanced video code (AVC) file format, 3rd Generation Partnership Project (3GPP) file format, and multi-view video code 151028.doc -4- 201119346 17. 18. 19. 20. 21. (MVC) file format. 3 Provider 15 device 'where the multiplexer is configured to be based on the warp = the current data will be - or more The unit is included in the second track. The one or more NAL units are not included in the first track. The device of the member 17, wherein the multiplexer is configured to construct a third track. a first extractor for identifying one or more of the plurality of early detection elements of the first track, and a second extractor for identifying the plurality of NAL units of the second track or more Ρ求^15的设置' wherein the extractor includes a -th extractor, 八中忒工工 is configured to construct a third extractor based on the encoded video data, including a plurality of NAL units a trajectory, and wherein the multiplexed U state constructs the second trajectory to include an identification of the third trajectory The device of claim 16, wherein the device comprises one of: an integrated circuit; a microprocessor; a wireless communication device, comprising the video encoder and the multiplexer. The device is configured to: encode a video device, and the farm includes: configured to: construct a first track in the encoded video data The component-track includes a video sample including a plurality of network access layers (NAL), wherein the video sample is included in an access unit; and is configured to construct a component including a second track of an extractor The extractor identifies at least one of the plurality of NAL units I51028.doc 201119346 in the video sample of the first track ‘the at least one of the plurality of NAL units includes a first identified NAL unit And wherein the extractor identifies a second NAL unit of the access unit, wherein the first identified NAL unit and the second NAL unit are discontinuous; for the first track and the second The trajectory is included in an at least partial compliance with international standards Organization (ISO) base media file format of the video file in the member; and said means for outputting the video file. 22. The device of claim 21, wherein the video file complies with at least one of: an ISO base media file format, an expandable video write stone lane (svc) file format, and an advanced video code (AVC) file. Format, Second Partner Program (3GPP) building format, and Multiview Video (MVC) file format. 23. The apparatus of claim 21, further comprising: the one or more NAL units included in the second trajectory based on the data = not included in the first trajectory . 24. The device of claim 23, further comprising means for constructing a second track 33⁄4, the third track comprising - a first extractor identifying one or more of the complex NAL units of the first track, And identifying at least one of the one or more NAL units of the second track. The second device of claim 2, wherein the device of claim 21, wherein the extractor comprises a first extractor, the device further comprising: constructing, based on the encoded data, a plurality of NAL units The third extractor track <photograph, and wherein 151028.doc -6 - 201119346 is used to construct the second track to include - the recognition 3 uses (4) to construct the second track. ° ^ the first track A component of a second extractor of one or more of the plurality of NAL units. 26. A computer readable computer comprising instructions for causing a processor to perform a storage of media based on the encoded Video data to construct a - trajectory, the squad, suffix & a plurality of video access layer (nal) unit video images, wherein the video sample is included in an access unit; a second trajectory of the extractor, the extractor identifying at least one of the plurality of nal units in the meta-synchronization sample of the first trajectory, the _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Single το, and where the extraction Identifying the first NAL unit of the access unit, wherein the first identified nal unit and the second identified NAL unit are discontinuous; the first trajectory and the second trajectory are included in at least one Partially complying with the International Standards Organization (ISO) basic media file format in the video building case; and outputting the video standard. 27. 28. The computer readable storage medium of claim 26, wherein the video file complies with At least one: iso basic media file format, expandable afl code (SVC) file format, advanced video code (AVC) file format, 3rd Generation Partnership Project (3GPP) file format, and multi-view video A code-readable (MVC) file format, such as the computer readable storage medium of claim 26, further comprising causing the processor to include one or more NAL units based on the encoded material in the 151028.doc 201119346 second track The one or more nal units are not included in the first trajectory. 29. The computer readable storage medium of claim 28, further comprising causing the processor to construct a third trajectory The third track includes a first extractor that identifies one or more of the plurality of NAL units of the first track, and a one or more NAL units that identify the second track A second extractor of at least one of the foregoing. 30. The computer readable storage medium of claim 26, wherein the extractor comprises a first extractor, the computer readable storage medium further comprising the processor based on the The encoded video material constructs an instruction including a second extractor track of the plurality of NAL units, and wherein the instructions for causing the processor to construct the second trace include causing the processor to construct the second track to cause An instruction to identify a second extractor of one or more of the plurality of NAL units of the third track. 3 1 - A method for decoding video data, the method comprising: receiving, by a multiplexer of a destination device, a video file at least partially conforming to an International Standards Organization (IS0) basic media standard format, The video file includes a first track and a second track. The first track includes a video sample including a plurality of network access layer (NAL) units corresponding to the encoded video data, wherein the video sample is included in the video file. And the second track includes an extractor identifying at least one of the plurality of NAL units of the first track, the at least one of the plurality of nal units including a first identified a NAL unit, and wherein the extractor identifies a second NAL unit of the access unit, wherein the identified NAL unit and the second identified NAL unit are discontinuous; the second is selected Destroy for decoding; and will be. The encoded video data of the first NAL unit and the second NAL unit identified by the ridge of the ridge is transmitted to one of the video decoders of the destination device. 32. The method of claim 31, further comprising discarding the first trajectory. Each of the plurality of NAL units identified by the extractor of the trajectory. The method of claim 3, wherein the second trajectory further comprises one or more NAL units not included in the first trajectory, the method further comprising the one or more of the second trajectory The encoded video material of the NAL unit is sent to the video decoder. The method of claim 3, wherein the video file further comprises a third track comprising a plurality of Nal units corresponding to the encoded video data, the method further comprising encoding the plurality of NAL units of the third track Video data is sent to the video decoder. An apparatus for decoding video data, the apparatus comprising: a video decoder 'configured to decode video data; and an instant processor configured to: receive an at least partial compliance with an International Standards Organization (IS0) a video file of a basic media file format, the video file including a first track and a second track, the first track including a video sample including a plurality of network access layer (NAL) units corresponding to the encoded video data The video sample is included in an access unit, 151028.doc 201119346 and the second track includes an extractor identifying at least one of the plurality of NAL units of the first track, the plurality of NALs The at least one of the cells includes a first identified NAL unit, and wherein the extractor identifies the second NAL unit of the access unit, wherein the first identified NAL single το and the second identified The ^^ octet unit is discontinuous; the second trajectory is selected for decoding; and the first NAL unit and the second NAL unit are encoded by the extractor of the second trajectory The data is sent to the video decoder. 36. 38. 38. 39. The apparatus of claim 3, wherein the demultiplexer is configured to discard the plurality of NALs of the first trajectory that are not recognized by the extractor of the second trajectory Each of the units. The apparatus of claim 35, wherein the second trajectory further comprises one or more NAL units not included in the first trajectory, and wherein the solution is configured to use the second trajectory The encoded video data of the one or more NAL units is sent to the video decoder. The device of claim 35, wherein the video file further comprises a third track comprising a plurality of NAL units corresponding to the encoded video material, and wherein the demultiplexer is configured to the plurality of tracks of the third track The encoded video data of the NAL units is sent to the video decoder. An apparatus for decoding video data, the apparatus comprising: means for receiving a video file at least partially in accordance with an International Standards Organization (IS) basic media standard format, the video file including a first track and a second a trajectory, the first trajectory comprising a video sample comprising a plurality of network access layer (NAL) units corresponding to the encoded video data, wherein the video sample is included in an access unit in 151028.doc -10- 201119346 And the second track includes a & picker that identifies at least one of the plurality of NAL units of the first track. The at least one of the plurality of NAL units includes - the first identified NAL unit, And wherein the extractor identifies one of the first NAL units of the access unit, wherein the first identified NAl unit and the second identified NAL unit are discontinuous; for selecting the second track for performing a decoding component; and means for transmitting the encoded video data of the first NAL unit and the first NAL unit identified by the extractor of the second representation to a device sfl decoder Pieces. 40. The nightmare of claim 39, wherein the device further comprises a plurality of hunger units for discarding the trajectory that are not identified by the extractor of the second trajectory The component of one. Six Dingle 41. If the device of μ is 39 in the first track—or a plurality of NAL units, the device further LIS encodes the bee of the second track or the plurality of muscle units The component sent to the video decoder. 42 I;::!: The setting of 9, wherein the video slot entry-step includes - the third slogan of the NAL^ including the «set=code video# material, the second edit it contains for the first The plurality of NALs 43 of the three tracks are sent to the components of the video decoder.匕 “Japanese-language computer-readable storage media, (iv) enable-process H to operate with τ: in the capture of the receiver to receive 丨 | — at least in part in accordance with the International Standards Organization (IS0) basic media 151028.doc 201119346 file format After the video file, the video file includes a first track and a second track, the first track including a video sample including a plurality of network access layer (NAL) units corresponding to the encoded video data, wherein The video sample is included in an access unit, and the second track includes an extractor that identifies at least one of the plurality of NAL units of the first track, the at least one of the plurality of NAL units including a first identified NAL unit, and wherein the extractor identifies a second NAL unit of the access unit, wherein the first identified 2ΝΛχ unit and the second identified NAL unit are discontinuous, selecting the a second track for decoding; and transmitting the encoded video data of the first Nal unit and the second NAL unit identified by the extractor of the second track to a video decoder. 4. The computer readable storage medium of claim 43, further comprising discarding each of the plurality of NAL units of the first trajectory that are not identified by the extractor of the second trajectory. The computer readable storage medium of item 43, wherein the second trajectory further comprises one or more NAL units not included in the first trajectory, the method further comprising the one or more NAL units of the second trajectory The computer-readable storage medium of claim 43, wherein the video file further comprises a second track comprising a plurality of Nal units corresponding to the encoded video material, The method further includes transmitting the encoded video material of the plurality of NAL units of the third track to the video decoder. 151028.doc • 12·