WO2006030401A2 - Melange multicouche de video et graphique avec identification de regions non transparentes mises en composition dans la superposition graphique multipliee par alpha - Google Patents

Melange multicouche de video et graphique avec identification de regions non transparentes mises en composition dans la superposition graphique multipliee par alpha Download PDF

Info

Publication number
WO2006030401A2
WO2006030401A2 PCT/IB2005/053054 IB2005053054W WO2006030401A2 WO 2006030401 A2 WO2006030401 A2 WO 2006030401A2 IB 2005053054 W IB2005053054 W IB 2005053054W WO 2006030401 A2 WO2006030401 A2 WO 2006030401A2
Authority
WO
WIPO (PCT)
Prior art keywords
regions
region
transparent
data
determination
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2005/053054
Other languages
English (en)
Other versions
WO2006030401A3 (fr
Inventor
Koul Manoj
Torsten Fink
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips Electronics NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV filed Critical Koninklijke Philips Electronics NV
Publication of WO2006030401A2 publication Critical patent/WO2006030401A2/fr
Publication of WO2006030401A3 publication Critical patent/WO2006030401A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14—Display of multiple viewports
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2622—Signal amplitude transition in the zone between image portions, e.g. soft edges
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/10—Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
    • G09G2340/125—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels wherein one of the images is motion video

Definitions

  • the present invention relates generally to video processing, and more particularly relates to reducing computational requirements for multi- layer video/graphics blending.
  • the pre-recorded data available on DVDs often includes, in addition to video program content, such items as menu elements, sub-titles, and related graphic overlay features. These menu items, sub-titles, and related features are often required to be displayed by the DVD playback systems. Further, such items may be required to blended with video data from the main video plane to create the desired display.
  • each display field, or frame i.e., combining a field or frame of main video data, with a corresponding field, or frame, of overlay material, which is first color converted and then alpha blended with the main video data, consumes substantial computational resources.
  • these computational resources include, but are not limited to, CPU cycles, memory bandwidth, and data cache resources.
  • methods in accordance with the present invention identify composited non- transparent regions in a video/graphics overlay during color conversion and alpha blending. Individual non-transparent regions are merged into larger such regions if not separated by at least a predetermined distance. Regions that are so determined are blended with target regions in the main video plane. Transparent regions are not blended with the main video data, thereby reducing the computational and memory bandwidth requirements.
  • the distance is predetermined, it is dynamically scaled to optimize CPU resources.
  • Fig. 1 is a memory map logically representing the layout of a display screen, and showing the display coordinates of two non-overlapping, non-transparent regions within the memory.
  • Fig. 2 is a flow diagram illustrating a method in accordance with the present invention.
  • Fig. 3 is a flow diagram illustrating a method in accordance with the present invention.
  • Various embodiments of the present invention provide means for reducing the processing power and bandwidth requirements for blending overlays, or generating and blending sub-pictures, menus, JPEGs, GIFs, and so on, with a main video plane for a DVD playback and picture display. It is noted that embodiments of the present invention are not limited to DVD playback, but are suitable for use in any type of picture display where composition and mixing from different planes and/or sources is required, such as, but not limited to, video playback on advanced television monitors, displays from portable video jukeboxes, media streaming over the Internet, and so on.
  • Methods in accordance with the present invention reduce, as compared to conventional methods, the computational requirements. Such methods also reduce the impact on a data cache associated with the computational resources, which in turn reduces the number of CPU stall cycles that would otherwise be encountered during processing. This reduction in CPU stall cycles increases the throughput of the computational resources.
  • video data is typically in stored in an addressable memory such as, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM).
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • a typical logical arrangement for video data is one in which such an arrangement is representative of the layout of a display screen on which the video data will be presented.
  • video data may be logically represented as consisting of a video field, with each field containing a certain number of lines, and each line containing a certain number of pixels.
  • processing operations on that video data can be expressed, or described, in terms reflective of the layout of a display screen, such as "left" and "right", and "top” and “bottom”.
  • One way to organize the addressing of these fields of video data is as memory arrays.
  • the video data may thought of as occupying a two-dimensional array that corresponds to a display screen.
  • the address of the memory location where the pixel data corresponding to the leftmost pixel of the top line of the display may be mapped to a screen address, for example (0,0), if screen addressing begins in the upper left-hand corner of the display screen.
  • Logical operations on the video data may then be described in terms of display screen locations rather than in terms of actual physical memory addresses. It is also noted that mapping between physical and logical addresses is well-known in this field, and therefore the details thereof will not be further described herein.
  • each pixel may be represented by an arbitrary number of bits or bytes, although some common arrangements include representing a pixel with eight bits of data (one byte) or with 32 bits of data (four bytes).
  • Various embodiments of the present invention provide a method of determining composited non-transparent regions in a video/graphics overlay during color conversion and alpha blending, and then only blending the regions thus determined with the target regions in the main video plane.
  • An advantage of such methods is that fewer CPU cycles are consumed, i.e., fewer instruction need to be executed, in performing the conversion and blending.
  • Another advantage is that memory bandwidth requirements are reduced.
  • a further advantage of some embodiments is that fewer data cache stalls are experienced in the embodiments that include data caches.
  • the overlay data is converted to the same video format as that of the main picture data, and multiplied with the alpha ( ) value required for blending. In some embodiments, the conversion and multiplication occur simultaneously.
  • the conversion and multiplication occur as separate steps.
  • the overlay can be decomposed into a set of composited rectangular non-transparent, and transparent non-overlapping regions.
  • the regions may not be completely non-transparent, and may contain pixel data after alpha multiplication that are transparent. Hence the regions are composited from non-transparent multiplied pixel data which may contain transparent multiplied pixel data.
  • the regions are identified by methods in accordance with the present invention that merge smaller non- transparent regions into larger regions recursively while scanning the data in a single pass. Such methods use a mean distance calculation between different areas in the picture to generate optimal size areas depending upon the impact on the data cache and the efficiency in processing the regions.
  • a datacahe is arranged in terms of cache sets.
  • the data cache is 8-way set associative, there are a total of 32 sets, and each set has 8 blocks, with each block being 64-bytes.
  • On a cache miss one cache line is fetched, with the word at which the miss occured being the first one fetched.
  • this embodiment makes a decision, based on the entropy in the data, as to whether the regions should be some integer multiple of cachelines in width, and whether the seperation between the regions is also some integer mutiple of cachlines. This helps to produce more efficient cache/memory usage.
  • entropy is the sum of energy of the area of the picture being loaded and being described as regions. If the entropy is greater than a thresold limit the value of which is typically determined by experimentaiton, then the illustrative embodiment aligns the regions to cacheline boundaries.
  • Various methods in accordance with the present invention use search and recursive region overlap identification processes on the picture data to identify these regions while performing a single pass over the data. For example, in typical sub-pictures for DVD, 70% of the picture is transparent, therefore efficient search and blend processes can reduce the computational load and memory bandwidth required. Further in accordance with the present invention, only the non-transparent regions are blended with the target regions in the main video plane, simultaneously multiplying the (1- ) values to the main picture data only for those target regions.
  • These processes can be used, for example, in blending images defined in various formats, such as but not limited to, JPEG, GIF, and BMP, with the main video to produce a final output result. Similarly, these processes can be used in blending images or video of two or more planes/sources whose origin is JPEG, GIF, or BMP, with the main video to produce a final output result.
  • the sub-picture data represented in a DVD is in the format of two bits/pixel.
  • the sub-pictures may be for creating the menus, or the sub-titles of the DVD.
  • These sub-pictures are blended with the main video picture data in order to produce the desired display results.
  • the sub-picture data Prior to blending such sub-picture data with the main video picture, the sub-picture data needs to be converted to target YUV data. In the case of normal overlay, data conversion is done to the target format, the value is used for the overlay, and it is used in a multiplication operation at this time.
  • this method can be described as including a number of operations, such as, creating an alpha-multiplied, target-format sub-picture, starting with 2- bit/pixel data, and using color palette tables, and contrast and color lookup tables.
  • the data is multiplied with the alpha value and converted to the main target video format.
  • the method also includes determining the first composited non-transparent blocks in the first non-transparent line in the alpha multiplied sub-picture/overlay data.
  • the method further includes determining and recording the regions in the first line of a field of pixel data to be blended, based on one or more parameters selected for the acceptable mean distance in the X-axis between the two adjacent regions.
  • This acceptable width is the mean distance that is suitable in terms of data cache performance and blending efficiency. Except for the first line, at the end of every line, the method updates the region list and gets a fix in the Y-axis (i.e., the vertical direction) from the recorded line lists, based on one or more parameters selected for acceptable distance in the Y-axis between two adjacent regions. This acceptable distance is the mean distance that is suitable in terms of data cache performance and blending efficiency. The method further includes recursively sweeping the adjacent regions recorded in the region list for overlaps and generating updated inclusive regions and removing overlaps.
  • the operations of determining and recording the regions in the first line, updating the region list, and recursively sweeping and generating inclusive regions, are repeated until the end of the sub- picture/overlay is reached.
  • the alpha multiplied pixels of the recorded regions are blended with the main video pixels by simultaneously multiplying (1- ) with the target video data.
  • the final output picture pixels can be represented as:
  • Pout *Poverlay + (1- ) * Pmain where ranges from 0 to 1.
  • the next step is to find the non-overlapping, non-transparent regions in the overlay layer, and the target regions in the main layer. Two regions are marked distinct only if: (a) the distance between them in the X-direction is above a first threshold, referred to as the X_Threshold; and (b) the distance between them in the Y-direction is above a second threshold referred to as the Y_Threshold.
  • X_Threshold a first threshold
  • Y_Threshold the distance between them in the Y-direction is above a second threshold referred to as the Y_Threshold.
  • the values for X_Threshold and Y Threshold are chosen as 32 pixels (i.e., 8 DWORDS), and four lines respectively.
  • Rgn(i) and Rgn(i+1) are distinct only if all the coordinates of the two regions satisfy the following conditions:
  • Fig. 1 illustrates two distinct regions, Rgn(i) 102 and Rgn(i+1) 104, which are determined in accordance with Equations 2.1 - 2.4. It will be appreciated that Fig. 1 is a memory mapping of an actual memory storage arrangement into a "space" representative of a display screen. For convenience of this description, such a memory mapping may be referred to as a display space.
  • the buffers are aligned to double word (DWORD) boundaries so as to enable 32-bit load operations, which are more efficient than performing four 8-bit loads, for four pixels at a time.
  • the region search is started by analyzing the overlay data per four pixels, one DWORD at a time per line.
  • the first non-zero value (i.e., a set of four pixels with one or more being non- transparent) is recorded as the initial coordinate of the first region.
  • the line is progressively scanned and the next zero value (i.e., a set of four pixels with none being non-transparent) marks the end of the region. However, if the next non-zero value starts within the X_Threshold, it is recorded as a single region. If the region does not end until the end of the line, then the regions is terminated at the end of the line. Lines are progressively scanned following the procedure described above except that the coordinates are now also compared in the vertical direction for the previously recorded regions, and their exact coordinates are then fixed.
  • Various embodiments of the present invention may be computer-implemented. Such computer based implementations, may include the use of general purpose or special purpose processors.
  • general purpose processors it is noted the present invention is not limited to any particular architecture or instruction set, and may be implemented with a microprocessor or a processor implements from a plurality of integrated circuits.
  • special purpose processors such as for example, digital signal processors
  • the present invention is not limited to any particular digital signal processor design or architecture, and may be implemented with single chip or multiple chip integrated circuit solutions that provide the hardware resources for a complete hardware implementation, or a hardware/software combination.
  • Sub-picture data and overlay pixel data are provided 202, 204.
  • the sub-picture data and overlay pixel data are color converted to a target format and alpha multiplied 206.
  • Composited non-transparent active regions are found and identified 208.
  • a list of regions determined at 208 is established and maintained 210.
  • a region is generated 304 from the first non-transparent line .
  • any 4-pixel set with a value of 0 is regarded as transparent.
  • a determination 306 is made as to whether there is another line (i.e., a next line) to process. If the determination of 306 is negative, then method 300 has completed its currently assigned processing, and returns control 308 to another process. If the determination of 306 is affirmative, then the pixels of the next line are processed 310. Still referring to Fig. 3, a determination 312 is made as whether a region has been identified.
  • a determination 314 is made as to whether the end of the current line has been reached. If the determination of 314 is negative, then method 300 returns to step 310 as shown in Fig. 3. However, if the determination of 314 is affirmative, then a determination 316 is made as to whether a new region has been found. If the determination of 316 is negative, then method 300 returns to step 306 to determine whether there is another line to process. However, if the determination of 316 is affirmative, then the previously recorded regions are re-sized, and any redundant regions are eliminated 318. Subsequent to the operations of re-sizing and eliminating redundant regions, method 300 returns to step 306 to determine whether there is a next line to process.
  • a determination 320 is made as to whether there is an overlap between regions. If the determination of 320 is negative (i.e., a region has been found and the region does not overlap another region), then this region is recorded 322. Subsequent to the recording of 322, method 300 returns to determination 314 to see if the end of the line has been reached. However, if the determination of 320 is affirmative (i.e., a regions has been found but it overlaps with another region), then region boundaries are re-calculated 324 to remove the overlaps. This can be thought of as merging the overlapping regions.
  • method 300 returns to step 310 as shown in Fig. 3.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

La présente invention concerne des procédés (200, 300) permettant d'identifier les régions non transparentes mises en composition dans une superposition vidéo/graphique pendant la conversion des couleurs et le mélange alpha (206). Les différentes régions non transparentes sont fusionnées en des régions plus grandes si elles ne sont pas séparées d'au moins une distance déterminée. Les régions ainsi déterminées sont mélangées (214) aux régions cibles dans le plan vidéo principal. Les régions transparentes ne sont pas mélangées aux données vidéo principales, ce qui réduit les besoins en calcul et en largeur de bande mémoire. Dans un autre aspect de l'invention, bien que la distance soit déterminée, elle peut faire l'objet d'un redimensionnement dynamique permettant d'optimiser les ressources de l'unité centrale.
PCT/IB2005/053054 2004-09-16 2005-09-16 Melange multicouche de video et graphique avec identification de regions non transparentes mises en composition dans la superposition graphique multipliee par alpha Ceased WO2006030401A2 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US61075104P 2004-09-16 2004-09-16
US60/610,751 2004-09-16
US64059604P 2004-12-31 2004-12-31
US60/640,596 2004-12-31

Publications (2)

Publication Number Publication Date
WO2006030401A2 true WO2006030401A2 (fr) 2006-03-23
WO2006030401A3 WO2006030401A3 (fr) 2006-05-11

Family

ID=35285594

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2005/053054 Ceased WO2006030401A2 (fr) 2004-09-16 2005-09-16 Melange multicouche de video et graphique avec identification de regions non transparentes mises en composition dans la superposition graphique multipliee par alpha

Country Status (1)

Country Link
WO (1) WO2006030401A2 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008082940A1 (fr) 2006-12-29 2008-07-10 Intel Corporation Utilisation d'informations supplémentaires de zones de délimitation dans une composition vidéo multicouche
WO2012047229A1 (fr) * 2010-10-08 2012-04-12 Hewlett-Packard Development Company, L.P. Combinaison d'une pluralité de flux vidéo
EP2299691A3 (fr) * 2009-09-08 2014-03-26 Samsung Electronics Co., Ltd. Appareil et procédé de traitement d'images

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5351067A (en) * 1991-07-22 1994-09-27 International Business Machines Corporation Multi-source image real time mixing and anti-aliasing
US6028583A (en) * 1998-01-16 2000-02-22 Adobe Systems, Inc. Compound layers for composited image manipulation
JP2003283925A (ja) * 2002-03-20 2003-10-03 Sony Corp 画像合成装置及び方法
GB2390950A (en) * 2002-07-17 2004-01-21 Sony Uk Ltd Video wipe generation based on the distance of a display position between a wipe origin and a wipe destination

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008082940A1 (fr) 2006-12-29 2008-07-10 Intel Corporation Utilisation d'informations supplémentaires de zones de délimitation dans une composition vidéo multicouche
EP2100269A4 (fr) * 2006-12-29 2014-01-08 Intel Corp Utilisation d'informations supplémentaires de zones de délimitation dans une composition vidéo multicouche
EP2299691A3 (fr) * 2009-09-08 2014-03-26 Samsung Electronics Co., Ltd. Appareil et procédé de traitement d'images
US8787701B2 (en) 2009-09-08 2014-07-22 Samsung Electronics Co., Ltd. Image processing apparatus and image processing method
WO2012047229A1 (fr) * 2010-10-08 2012-04-12 Hewlett-Packard Development Company, L.P. Combinaison d'une pluralité de flux vidéo

Also Published As

Publication number Publication date
WO2006030401A3 (fr) 2006-05-11

Similar Documents

Publication Publication Date Title
US6819328B1 (en) Graphic accelerator with interpolate function
CN100378867C (zh) 半导体器件、图像数据处理装置和方法
US7190839B1 (en) Methods and apparatus for generating multi-level graphics data
JP3081774B2 (ja) テクスチャーパターンメモリ回路
US7880745B2 (en) Systems and methods for border color handling in a graphics processing unit
US8130234B2 (en) Computer graphics rendering apparatus and method
JP2001507836A (ja) タイルリニアホストテクスチャストレージ
JP2010102729A (ja) テクスチャ情報をコード化するための方法および装置
JP2000089748A (ja) 画像処理装置及び画像処理方法
JPH11296154A (ja) 図形処理装置及び図形処理方法
US8264612B2 (en) Method of raster-scan search for multi-region on-screen display and system using the same
JP2008514160A (ja) 画像処理におけるエッジハンドリングの装置及び方法
US6693644B1 (en) Graphic accelerator reducing and processing graphics data
WO2006030401A2 (fr) Melange multicouche de video et graphique avec identification de regions non transparentes mises en composition dans la superposition graphique multipliee par alpha
US20050195200A1 (en) Embedded system with 3D graphics core and local pixel buffer
US20030231176A1 (en) Memory access device, semiconductor device, memory access method, computer program and recording medium
JPH11219429A (ja) 記憶装置、並びに書き込み方法および読み出し方法
US6940519B2 (en) Graphics processor, graphics card and graphics processing system
EP0593012B1 (fr) Dispositif d'affichage d'images vidéo et méthode de contrôle d'un affichage d'images vidéo
JP3132771B2 (ja) 画像蓄積装置及びこれを具える画像処理装置
KR900002631B1 (ko) 화상데이터의 처리방법 및 장치
JPH11154237A (ja) テクスチャマッピング用メモリ装置
US6903744B2 (en) Graphics processing system
JPH09270024A (ja) 情報処理装置
US8115874B2 (en) Memory optimization for video processing

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV LY MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase