WO2004015985A1 - Traitement irregulier de donnees mediatiques a sortie continue - Google Patents

Traitement irregulier de donnees mediatiques a sortie continue Download PDF

Info

Publication number
WO2004015985A1
WO2004015985A1 PCT/IB2003/002820 IB0302820W WO2004015985A1 WO 2004015985 A1 WO2004015985 A1 WO 2004015985A1 IB 0302820 W IB0302820 W IB 0302820W WO 2004015985 A1 WO2004015985 A1 WO 2004015985A1
Authority
WO
WIPO (PCT)
Prior art keywords
global
processing
global parameter
frame
data frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2003/002820
Other languages
English (en)
Inventor
Christian Hentschel
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips Electronics NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV filed Critical Koninklijke Philips Electronics NV
Priority to AU2003242920A priority Critical patent/AU2003242920A1/en
Publication of WO2004015985A1 publication Critical patent/WO2004015985A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115—Selection of the code volume for a coding unit prior to coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/527—Global motion vector estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region

Definitions

  • the invention relates to media processing, and more particularly to a method and apparatus for monitoring the amount of available resources in a media processing system and suspending calculation of global parameters when the available resources falls below a predetermined level.
  • a typical video processing chain in a consumer device like a television or a set top box includes processing units for video decoding, video enhancement, and display processing. In each unit, images have to be processed and output in a specific time period or frame.
  • the processing units may comprise a set of algorithms for producing the output image, and currently all algorithms have the same regular processing deadlines to produce a continuous stream of images. As long as these algorithms are implemented in dedicated hardware like ASICS, they are designed for worst case processing, and all of the functionality is guaranteed to be finished in a given time period or frame.
  • Image analysis as a part of image processing, can consume quite a lot of resources.
  • each frame of the incoming media data can be analyzed and global parameters such as, for example, noise level, average image spectrum characteristics, bit rate of digital streams, global motion vectors (zoom, tilt) are determined and provided to the image processing units for use during processing of each frame.
  • the overall system control decides on a shortage of resources, e.g. when the available resources falls below the predetermined level, or is expected to fall below the predetermined level, the media analysis for global parameters of each frame is suspended and the global parameters from the last fully processed frame are sent to the media processing units.
  • the media processing on individual images/data is still performed with little or no influence on the output image quality.
  • a method and apparatus for processing data frames is disclosed.
  • the resources needed to calculate at least one global parameter for the data frame is estimated.
  • the resources needed to calculate the at least one global parameter can also be pre-knowledge and stored in e.g. a look-up-table. It is then determined if the available resources are greater than a predetermined threshold. If the available resources are below the predetermined threshold, the calculation of the at least one global parameter is skipped.
  • the at least one previously calculated global parameter is then sent to a processing unit which processes the data frame using the sent parameter.
  • a method and apparatus for processing data frames is disclosed.
  • a data frame is received, at least one global parameter is calculated for the data frame.
  • the at least one global parameter is then sent to a processing unit which processes the data frame using the calculated global parameters.
  • the calculation of at least one global parameter for the next received frame is skipped when the available resources are less than the predetermined threshold, wherein previously calculated global parameters are sent to the processing unit for processing of the next received frame.
  • Figure 1 illustrates a known image processing system
  • Figure 2 illustrates a detailed image processing system according to one embodiment of the invention
  • FIG. 3 is a flowchart illustrating an image processing method according to one embodiment of the invention.
  • Figure 4 is a flowchart illustrating an image processing method according to one embodiment of the invention.
  • Adaptive algorithms using image analysis are well-known in image processing.
  • the adaptation may be locally image content dependent or globally dependent on the image input parameters like noise level, signal spectrum characteristics, global motion vectors, etc.
  • the image analysis can take quite a lot of resources and in the case of global adaptation, the result may change only by a small amount in adjacent fields or frames.
  • the image analysis is skipped in situations where the amount of available resources becomes small or falls below a predetermined threshold. In this situation, the global parameters from the previous frame are used in the image processing of the next frame.
  • FIG. 1 illustrates a media processing system 100.
  • the media processing system 100 has a media analysis unit 102 and a media processing unit 104.
  • the media analysis unit 102 receives each frame of the incoming data and analyzes each frame to determine at least one global parameter.
  • the global parameters are then sent to the media processing unit 104.
  • the media processing unit 104 processes each incoming data frame using the global parameters from the media analysis unit 102.
  • the media processing unit 104 then outputs a processed frame for display.
  • FIG. 2 illustrates an image processing system 200 which is a more detailed example of the media processing system 100.
  • the media analysis unit 102 comprises a noise level detector 204, a detector for global motion vectors 206, and a signal spectrum analyser208, all connected to the input signal.
  • the outputs for the global parameters are the input parameters for the media processing units 214, 216, and 218.
  • a controller 202 is connected to the media analysis units 204, 206, and 208.
  • the media " analysis unit is not limited to the components described above, but rather can contain any means necessary for calculating global parameters for the media processing unit 104.
  • the media processing unit 104 can be comprised of a video decoding unit 214, a video enhancement unit 216 and a video display processing unit 218, but the media processing unit 104 is not limited thereto.
  • the controller 202 monitors the available resources of the media processing system 200. It will be understood that a different controller in the media processing system 200 could alternatively be used to monitor the available resources of the system 200. When the available resources drops below a predetermined threshold, the controller 202 halts at least part of the image analysis on the received frame. For example, the controller 202 may halt processing in one or more of the noise level detection unit 204, the detector for global motion vectors unit 206, and the signal spectrum analyzer unit 208. For example, noise level is a global parameter and will change slightly from frame to frame.
  • the previously calculated noise level value can be sent to the media processing unit 104 and the freed resources can be used for other more important purposes. Since the last noise level value is most likely very close to the value that would have been calculated, the output quality from the media processing unit 104 is not greatly affected.
  • FIG. 3 is a flow chart illustrating an image processing method according to one embodiment of the invention.
  • the controller 202 estimates the resources needed to calculate the global parameters for the frame in step 304. If the resources needed to calculate the global parameters for the frame are pre-knowledge, step 304 can be skipped. If it is determined in step 306 that the available resources are greater than a predetermined threshold, the media analysis unit calculates the global parameters in step 308 and analyzes the frame in the normal manner and outputs the calculated global parameters to the media processing unit 104 in step 310.
  • the media analysis unit 102 skips the calculation of at least one of the global parameters in step 312.
  • the media analysis unit 102 then sends at least one previously calculated global parameter to the media processing unit 104 in step 314.
  • the controller 202 may prevent one or more of the units 204, 206, 208 from calculating a new global parameter.
  • the unit 204, 206, 208 which is prevented from calculating a new global parameter sends the previously calculated global parameter to the media processing unit 104.
  • the media processing unit 104 then processes the received frame in the normal manner using the previously calculated global parameters in step 316.
  • Figure 4 is a flow chart illustrating an image processing method according to another embodiment of the invention.
  • the media analysis unit 102 calculates at least one global parameter for the frame in step 404.
  • the media analysis unit 102 then sends the at least one calculated global parameter to the media processing unit 104 in step 406.
  • the media processing unit 104 then processes the frame using the at least one global parameter in step 408. While the media processing unit 104 is processing the frame, the controller 202 estimates the resources needed to calculate the global parameters for the next frame in step 410. If it is determined in step 412 that the available resources are greater than a predetermined threshold, the media analysis unit 102 will analyze the next frame in the normal manner.
  • the media analysis unit 102 will skip the calculation of at least one of the global parameters for the next frame in step 414.
  • the media analysis unit 102 will then send at least one previously calculated global parameter to the media processing unit 104 at the appropriate time for the next frame in step 416.
  • the controller 202 may prevent one or more of the units 204, 206, 208 from calculating a new global parameter for the next frame. In this situation, the unit 204, 206, 208 which is prevented from calculating a new global parameter sends the previously calculated global parameter to the media processing unit 104 for processing of the next frame.
  • the controller 202 may continuously monitor the amount of available resources and suspend calculation of one or more of the global parameters when the amount of available resources falls below a predetermined level. In this situation, the units which have suspended calculation of the new global parameter will send the last previously calculated global parameter to the media processing unit 104.
  • the above-described invention has many advantages. First, media processing is almost unaffected, while programmable processing resources can be used in a very flexible manner. The former "hard ceiling" of the maximal available programmable resources becomes a "soft ceiling”.
  • the invention is applicable for scalable and non-scalable media processing. The invention can be implement in an inexpensive manner using control software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

La présente invention concerne un procédé et un appareil pour traiter efficacement des données médiatiques lorsque la quantité de ressources disponibles est inférieure à un niveau prédéfini. Lorsque la commande du système global décide qu'il y a un déficit de ressources, c'est-à-dire lorsque les ressources disponibles sont ou vont être inférieures à un niveau prédéfini, l'analyse médiatique de paramètres globaux de chaque trame est interrompue et les paramètres globaux de la dernière trame complètement traitée sont envoyés aux unités de traitement médiatique.
PCT/IB2003/002820 2002-07-29 2003-06-26 Traitement irregulier de donnees mediatiques a sortie continue Ceased WO2004015985A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2003242920A AU2003242920A1 (en) 2002-07-29 2003-06-26 Irregular processing of media data with continuous output

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP02078098.7 2002-07-29
EP02078098 2002-07-29

Publications (1)

Publication Number Publication Date
WO2004015985A1 true WO2004015985A1 (fr) 2004-02-19

Family

ID=31502765

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2003/002820 Ceased WO2004015985A1 (fr) 2002-07-29 2003-06-26 Traitement irregulier de donnees mediatiques a sortie continue

Country Status (2)

Country Link
AU (1) AU2003242920A1 (fr)
WO (1) WO2004015985A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0910219A2 (fr) * 1997-10-16 1999-04-21 Matsushita Electric Industrial Co., Ltd. Méthode de réduction des exigences de puissance de traitement d'un décodeur vidéo
JPH11146398A (ja) * 1997-11-04 1999-05-28 Matsushita Electric Ind Co Ltd マルチメディア情報処理装置
US20020009149A1 (en) * 1999-12-14 2002-01-24 Rodriguez Arturo A. System and method for adaptive video processing with coordinated resource allocation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0910219A2 (fr) * 1997-10-16 1999-04-21 Matsushita Electric Industrial Co., Ltd. Méthode de réduction des exigences de puissance de traitement d'un décodeur vidéo
JPH11146398A (ja) * 1997-11-04 1999-05-28 Matsushita Electric Ind Co Ltd マルチメディア情報処理装置
US20020009149A1 (en) * 1999-12-14 2002-01-24 Rodriguez Arturo A. System and method for adaptive video processing with coordinated resource allocation

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 1999, no. 10 31 August 1999 (1999-08-31) *

Also Published As

Publication number Publication date
AU2003242920A1 (en) 2004-02-25

Similar Documents

Publication Publication Date Title
US6081299A (en) Methods and systems for encoding real time multimedia data
EP2727344B1 (fr) Sélection de trames à être codées basée sur la similarité, la qualité et la signifiance des trames
US20110299588A1 (en) Rate control in video communication via virtual transmission buffer
US20070046821A1 (en) Video image processing with remote diagnosis and programmable scripting
US6115499A (en) Repeat field detection using checkerboard pattern
US10362264B2 (en) Profile for frame rate conversion
CN111182352B (zh) 一种视频播放的自适应码流控制装置及方法
US7653265B2 (en) Video image processing with utility processing stage
US5572654A (en) Method and apparatus for graceful degradation of image playback frames rates
US20070217505A1 (en) Adaptive Decoding Of Video Data
US7106800B2 (en) Image signal decoder selectively using frame/field processing
WO2007011732A1 (fr) Traitement d'images video avec affectation de temps de traitement
US6412013B1 (en) System for controlling data output to a network
US6728312B1 (en) Adaptive video decoding and rendering with respect to processor congestion
US20210103735A1 (en) System for computationally efficient analysis of traffic details in traffic video stream and a method thereof
WO2007024362A2 (fr) Traitement d'images video au moyen d'un script programmable
US6751404B1 (en) Method and apparatus for detecting processor congestion during audio and video decode
US7274739B2 (en) Methods and apparatus for improving video quality in statistical multiplexing
US20060274830A1 (en) Method and apparatus for compressing image data
WO2004015985A1 (fr) Traitement irregulier de donnees mediatiques a sortie continue
US5530477A (en) Method and apparatus for selectively omitting video frames during playback
JP2004153553A (ja) 動画像復号化装置および動画像復号化方法
CN115002520B (zh) 一种视频流数据处理方法、装置、设备及存储介质
KR101732995B1 (ko) 스트리밍 레이턴시 최소화 시스템 및 이를 사용하는 방법
JPH0547031B2 (fr)

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP