WO2022100522A1 - 一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品 - Google Patents

一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品 Download PDF

Info

Publication number
WO2022100522A1
WO2022100522A1 PCT/CN2021/128969 CN2021128969W WO2022100522A1 WO 2022100522 A1 WO2022100522 A1 WO 2022100522A1 CN 2021128969 W CN2021128969 W CN 2021128969W WO 2022100522 A1 WO2022100522 A1 WO 2022100522A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
playback
information
decoding
strategy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/128969
Other languages
English (en)
French (fr)
Inventor
李志成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to EP21891048.7A priority Critical patent/EP4192015A4/en
Publication of WO2022100522A1 publication Critical patent/WO2022100522A1/zh
Priority to US17/963,829 priority patent/US11909984B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/352—Details of game servers involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A63F13/358—Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/131—Protocols for games, networked simulations or virtual reality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/4424—Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/4508—Management of client data or end-user data
    • H04N21/4516—Management of client data or end-user data involving client characteristics, e.g. Set-Top-Box type, software version or amount of memory available

Definitions

  • the embodiments of the present application relate to video encoding technologies, and in particular, to a video encoding method, a video decoding method, an apparatus, an electronic device, a storage medium, and a computer program product.
  • the embodiments of the present application provide a video encoding method, a video decoding method, an apparatus, an electronic device, a storage medium, and a computer program product, which can utilize network bandwidth information, decoding computing power information, and capacity configuration information of a playback buffer pool, Dynamically adjust the video playback coding strategy to determine the video coding strategy that matches the target video playback environment, reduce video playback freezes caused by changes in the network environment and insufficient terminal decoding capabilities, and improve user experience.
  • An embodiment of the present application provides a video encoding method, which is executed by an electronic device, including:
  • the target video is encoded by the video encoding strategy.
  • the embodiment of the present application provides a video decoding method, which is executed by an electronic device and applied to a target video encoded by the preceding video encoding method;
  • the method includes:
  • the encoded target video is decoded through a video decoding strategy corresponding to the video encoding strategy.
  • the embodiment of the present application also provides a video encoding device, including:
  • an information processing module configured to determine network bandwidth information based on the network bandwidth environment of the video playback terminal
  • the information processing module is further configured to determine the decoding computing power information of the video playback terminal;
  • the information processing module is further configured to obtain the capacity configuration information of the playback buffer pool in the video playback environment;
  • the information processing module is further configured to dynamically adjust the encoding strategy for video playback based on at least one of the network bandwidth information, the decoding computing power information, and the capacity configuration information of the playback buffer pool, to obtain a A video coding strategy that matches the playback environment of the target video;
  • An information encoding module configured to encode the target video through the video encoding strategy.
  • the embodiment of the present application also provides a video decoding apparatus, which is applied to the target video encoded by the pre-order video encoding method;
  • the device includes:
  • an acquisition module configured to acquire the encoded target video
  • the information decoding module is configured to decode the encoded target video through a video decoding strategy corresponding to the video encoding strategy.
  • the embodiment of the present application also provides an electronic device, the electronic device includes:
  • the processor is configured to implement a pre-order video encoding method or a pre-order video decoding method when executing the executable instructions stored in the memory.
  • Embodiments of the present application further provide a computer-readable storage medium storing executable instructions, characterized in that, when the executable instructions are executed by a processor, a pre-order video encoding method or a pre-order video decoding method is implemented .
  • Embodiments of the present application provide a computer program product, including computer programs or instructions, which implement a pre-order video encoding method or a pre-order video decoding method when the computer program or instructions are executed by a processor.
  • Video playback freezes caused by insufficient decoding capability, which improves the smoothness of video playback, thereby improving user experience.
  • FIG. 1 is a schematic diagram of a usage environment of a video encoding method and a video decoding method provided by an embodiment of the present application;
  • FIG. 2 is a schematic diagram of a composition structure of a video encoding apparatus provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a processing method for video coding and display in an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a video encoding method provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a usage environment of the video encoding method provided by the embodiment of the present application.
  • FIG. 6 is a schematic diagram of a usage environment of the video encoding method provided by the embodiment of the present application.
  • FIG. 7A is a schematic diagram of a usage environment of the video encoding method provided by the embodiment of the present application.
  • FIG. 7B is a schematic diagram of a usage environment of the video encoding method provided by the embodiment of the present application.
  • FIG. 8 is a schematic diagram of a usage environment of the video encoding method provided by the embodiment of the present application.
  • FIG. 9 is a schematic diagram of a front-end display of a video encoding method according to an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of a video encoding method provided by an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a front-end display of a video encoding method according to an embodiment of the present application.
  • Video Transcoding refers to converting a compressed and encoded video stream into another video stream to adapt to different network bandwidths, different terminal processing capabilities and different user needs.
  • Client a carrier that implements a specific function in a terminal, for example, a mobile client (APP) is a carrier of a specific function in a mobile terminal, such as the function of performing online live broadcast (video streaming) or the function of playing online video.
  • APP mobile client
  • one or more of the executed operations may be real-time, or may have a set delay; Unless otherwise specified, there is no restriction on the order of execution of multiple operations to be executed.
  • Virtual environment is the virtual environment displayed (or provided) when the application is running on the terminal.
  • the virtual environment may be a simulated environment of the real world, a semi-simulated and semi-fictional three-dimensional environment, or a purely fictional three-dimensional environment.
  • the virtual environment may be any one of a two-dimensional virtual environment, a 2.5-dimensional virtual environment, and a three-dimensional virtual environment.
  • the following embodiments illustrate that the virtual environment is a three-dimensional virtual environment, but this is not limited.
  • the virtual environment is also used for virtual environment battles between at least two virtual objects.
  • the virtual environment is also used for fighting between at least two virtual objects using virtual firearms.
  • the virtual environment is also used for combat using virtual firearms between at least two virtual objects within a target area range, and the target area range is continuously reduced with the passage of time in the virtual environment.
  • Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize the calculation, storage, processing and sharing of data.
  • the general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model application can form a resource pool, which can be used on demand and is flexible and convenient.
  • Cloud computing technology will become an important support. Background services of technical network systems require a lot of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the back-end system for logical processing. Data of different levels will be processed separately, and all kinds of industry data need to be strong. The system backing support can only be achieved through cloud computing.
  • Cloud game refers to the game itself running on the cloud server device, after encoding the game screen rendered by the cloud device, and then transmitting it to the user terminal through the network, and the user terminal decodes the encoded file and renders it to the display screen for display. Therefore, the user does not need to install the game locally, but only needs to establish a communication network connection with the cloud to complete the game interaction process.
  • FPS Frames Per Second
  • image field refers to the number of frames per second transmitted by the picture, that is, the number of pictures of animation or video.
  • FPS is a measure of the amount of information used to save and display dynamic video. The more frames per second, the smoother the motion displayed. Among them, 30FPS is the minimum requirement to avoid unsmooth action.
  • FIG. 1 is a schematic diagram of a usage scenario of the video encoding method and the video decoding method provided by the embodiment of the present application.
  • the following describes the usage environment of the video encoding method and the video decoding method provided by the embodiment of the present application:
  • the terminal (including the terminal 10-1 and the terminal 10-2) is provided with a running video playback client (for example, a client of a cloud game), and the user can obtain the game content stored by the cloud server through the set cloud game software client;
  • the terminal connects to the server 200 (ie, the cloud server) through the network 300.
  • the network 300 may be a wide area network or a local area network, or a combination of the two, and uses a wireless link to realize data transmission.
  • the embodiment of this application does not make specific restrictions, wherein the game itself runs in the cloud server, and the cloud server renders the After the game screen is encoded, the encoded file is obtained and transmitted to the terminal through the network 300, and the encoded file is decoded and rendered by the terminal to display the game on the display interface. Therefore, the user does not need to install the game locally, but only needs to establish a connection with the cloud.
  • the communication network connection can complete the game interaction process.
  • the videos in the embodiments of the present application are not limited to game videos, and may also be short videos, live videos, and the like.
  • the above-mentioned terminal may include various types of applications, for example, instant messaging applications, video playback applications, short video applications, live broadcast applications, and the like.
  • the above-mentioned video encoding method and video decoding method may also be not limited to be applied to scenarios such as cloud games, video calls, and live video broadcasts.
  • the organizational structure of the cloud game includes a terminal and a cloud server, wherein the terminal is used to receive the user's control operation on the game process, and send the control instructions corresponding to the control operation to the cloud server, and the cloud server is used to control the game.
  • the process controls and sends the video stream during the game to the terminal for playback.
  • the terminal is mainly used to receive the game video and game audio sent by the cloud server during the game process, and perform rendering and playback, and the operation of the game user on the terminal side (including but not limited to the game user.
  • the processing content of the cloud server includes: 1) The calculation and game rendering in the running of the cloud game, the screen capture and encoding, the sound card sound capture and encoding, and will be processed by the cloud game.
  • the encoded video information and audio information are sent to the terminal in the form of streaming media; 2) Receive the operation instructions of the terminal, and send the operation instructions to the corresponding mouse driver process, sound card driver process and keyboard driver process, so as to realize cloud Game controls.
  • FIG. 2 is a schematic diagram of the composition and structure of a video encoding apparatus provided by an embodiment of the present application. It can be understood that FIG. 2 only shows an exemplary structure of the video encoding apparatus, but not the entire structure. Part of the structure shown in FIG. 2 or full structure.
  • the video encoding apparatus includes: at least one processor 201 , a memory 202 , a user interface 203 , and at least one network interface 204 .
  • the various components in the video encoding apparatus are coupled together by a bus system 205 .
  • the bus system 205 is used to implement the connection communication between these components.
  • the bus system 205 also includes a power bus, a control bus and a status signal bus.
  • the various buses are labeled as bus system 205 in FIG. 2 .
  • the user interface 203 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen, and the like.
  • the memory 202 may be either volatile memory or non-volatile memory, and may include both volatile and non-volatile memory.
  • the memory 202 in the embodiment of the present application can store data to support the operation of the terminal (eg, the terminal 10-1). Examples of such data include: any computer programs, such as operating systems and applications, used to operate on a terminal such as terminal 10-1.
  • the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks.
  • Applications can contain various applications.
  • the video encoding apparatus provided by the embodiments of the present application may be implemented by a combination of software and hardware.
  • the video encoding apparatus provided by the embodiments of the present application may be a processor in the form of a hardware decoding processor. It is programmed to execute the video encoding method provided by the embodiments of the present application.
  • a processor in the form of a hardware decoding processor may employ one or more application specific integrated circuits (ASIC, Application Specific Integrated Circuit), DSP, Programmable Logic Device (PLD, Programmable Logic Device), complex programmable logic engine Device (CPLD, Complex Programmable Logic Device), Field Programmable Gate Array (FPGA, Field-Programmable Gate Array) or other electronic components.
  • ASIC Application Specific Integrated Circuit
  • DSP Programmable Logic Device
  • PLD Programmable Logic Device
  • CPLD Complex Programmable Logic Device
  • FPGA Field-Programmable Gate Array
  • the video encoding apparatus provided by the embodiment of the present application may be directly embodied as a combination of software modules executed by the processor 201, and the software modules may be located in a computer-readable storage Among the media, the computer-readable storage medium is located in the memory 202, and the processor 201 reads the executable instructions included in the software modules in the memory 202, and combines necessary hardware (for example, including the processor 201 and other components connected to the bus 205) to complete the present invention.
  • the processor 201 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic engine devices, discrete gate or transistor logic Engine devices, discrete hardware components, etc., where a general-purpose processor may be a microprocessor or any conventional processor, or the like.
  • DSP Digital Signal Processor
  • the apparatus provided in this embodiment of the present application may be directly executed by a processor 201 in the form of a hardware decoding processor, for example, dedicated to one or more applications.
  • Integrated Circuit ASIC, Application Specific Integrated Circuit
  • DSP Programmable Logic Engine Device
  • PLD Programmable Logic Device
  • CPLD Complex Programmable Logic Engine Device
  • FPGA Field -Programmable Gate Array
  • the memory 202 in the embodiment of the present application is used to store various types of data to support the operation of the video encoding apparatus. Examples of these data include: any executable instructions for operating on a video encoding device, such as executable instructions, and a program implementing the slave video encoding method of the embodiments of the present application may be included in the executable instructions.
  • the video encoding apparatus may be implemented in software.
  • FIG. 2 shows the video encoding apparatus stored in the memory 202, which may be software in the form of programs and plug-ins, and includes a
  • the series of modules, as an example of the program stored in the memory 202 may include a video encoding device, and the video encoding device includes the following software modules, an information processing module 2081 and an information encoding module 2082.
  • the software module in the video encoding device is read into the RAM by the processor 201 and executed, the video encoding method provided by the embodiment of the present application will be realized, wherein the functions of each software module in the video encoding device include:
  • the information processing module 2081 is configured to determine network bandwidth information based on the network bandwidth environment of the video playback terminal; the information processing module 2081 is further configured to determine the decoding computing power information of the video playback terminal; the information processing module 2081, is also configured to obtain the capacity configuration information of the playback buffer pool in the video playback environment; the information processing module 2081 is further configured to be based on the network bandwidth information, the decoding computing power information and the capacity configuration information of the playback buffer pool At least one of the encoding strategies for video playback is dynamically adjusted to obtain a video encoding strategy that matches the playback environment of the target video; the information encoding module 2082 is configured to encode the target video through the video encoding strategy .
  • the structure of the video decoding apparatus will be briefly described below.
  • the video decoding apparatus may be implemented in various forms, such as a dedicated terminal with the processing function of the video decoding apparatus, or may be an electronic device provided with the processing function of the video decoding apparatus. (for example, a mobile phone or a tablet computer), for example, the terminal 10-1 or the terminal 10-2 in the preceding Figure 1 .
  • the functions of each software module in the video decoding device include:
  • an acquisition module configured to acquire the encoded target video
  • the information decoding module is configured to decode the encoded target video through a video decoding strategy corresponding to the video encoding strategy.
  • an embodiment of the present application further provides a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the different implementations provided in the various optional implementations of the above-mentioned video encoding method and video decoding method Examples and combinations of examples.
  • FIG. 3 is a schematic diagram of a processing method for video encoding and display provided by an embodiment of the present application.
  • the desktop capture and encoding scheme in the cloud game can capture the game screen and then encode and transmit it to the terminal, and the desktop captures the frame rate.
  • the graphics processor GPU, Graphics Processing Unit
  • Vertex processing The GPU reads the vertex data describing the appearance of the 3D graphics, and determines the shape and positional relationship of the 3D graphics according to the vertex data, and establishes the skeleton of the 3D graphics.
  • it can be applied to different game environments including but not limited to virtual reality applications, three-dimensional map programs, military simulation programs, first-person shooting games (FPS, First-Person Shooting Game), multiplayer online tactical competitive games (MOBA, Multiplayer Online Battle Arena Games) as an example, the skeleton of the 3D graphics in the 3D virtual environment is finally presented on the user interface (UI, User Interface), wherein the vertex data describing the appearance of the 3D graphics can be obtained through the texture coordinates of the vertices in the set of textures.
  • UI User Interface
  • a corresponding texture coordinate line segment can be formed between the texture coordinates of the vertices in the texture coordinate set. If the shortest path between all texture coordinate vertices passes through a certain node, then this node is considered to be The root node of the skeleton (Betweenness Centrality), from which the skeleton of the 3D graphics can be established.
  • Betweenness Centrality The root node of the skeleton
  • Rasterization calculation convert vertex data into fragments, where the image actually displayed on the display includes pixel groups, convert the vertex data formed in the vertex processing stage into corresponding pixel points through a conversion algorithm, and convert a vector graphic into The process of a series of pixels is called rasterization. For example, the vertex data based on the skeleton of the 3D graphic represented by the oblique line segment is converted into step-shaped continuous pixel points through rasterization calculation.
  • Texture map The vertex data generation stage of the skeleton of 3D graphics only generates polygons that only constitute the outline of the 3D object, and texture mapping completes the map of the multi-deformable surface, that is, the surface of the polygon. Paste the corresponding image to generate a "real" graphic.
  • Pixel processing In the stage of performing rasterization calculation, in the process of converting each pixel, the GPU completes the calculation and processing of the pixel, thereby determining the final attribute of each pixel.
  • the encoding and transmission process shown in Figure 3 is not dynamically adjusted according to the user's network and terminal decoding capabilities, which leads to unstable user networks or limited decoding computing power.
  • the related technology is only to read the current latest frame data from the frame buffer (buffer) at a fixed frame rate, and then send it to the encoder for encoding, and then transmit the encoded and compressed frame data to the cloud.
  • the game terminal is displayed on the display interface of the cloud game terminal.
  • the terminal load is relatively high, and the decoding ability cannot keep up with the cloud frame rate screen. , resulting in frequent freezes and slow response.
  • the terminal hardware load is relatively high due to decoding and high frame rate rendering, resulting in terminal battery overheating and frequent Causes forced frequency reduction and affects the user experience.
  • FIG. 4 is a schematic flowchart of the video encoding method provided by the embodiment of the present application. It can be understood that FIG. 4 The steps shown can be performed by various electronic devices running the video encoding apparatus, such as a dedicated terminal with video encoding function, a server or a server cluster. The steps shown in FIG. 4 will be described below.
  • Step 401 The video encoding apparatus determines network bandwidth information based on the network bandwidth environment of the video playback terminal.
  • Step 402 The video encoding apparatus determines the decoding computing power information of the video playback terminal.
  • the decoding computing power information is used to represent the number of frames that the video playback terminal (such as terminal 1 and terminal 2 in Figure 4) can perform video decoding processing in a unit time.
  • determining the decoding computing power information of the video playback terminal may be implemented in the following ways: acquiring hardware type information of the video playback terminal; determining the configuration information of the video playback terminal based on the hardware type information of the video playback terminal; The configuration information of the playback terminal determines the decoding computing power information of the video playback terminal.
  • the decoding computing power information of the video playback terminal determines the use environment with low decoding computing power and unstable computing power of the terminal.
  • the configuration information of the terminal determines the decoding computing power of the video playback terminal.
  • the decoding delay for decoding a video frame is greater than or equal to the parameter A, it indicates that the decoding computing power of the video playback terminal is low; wherein, the decoding delay is used to represent the video playback.
  • the time consumed by the terminal to decode one frame of video frame, for example, parameter A is defined as 1000/frame rate * 0.8, which can trigger the adjustment of the video coding strategy, reduce the freezing of cloud game videos, and improve the user experience.
  • the configuration of the video playback terminal can be determined through the corresponding hardware type information.
  • Information for example, for the XX model mobile phone of the XX brand, the configuration information of the video playback terminal can be determined by the hardware type information (or the network access code of the XX model mobile phone) as the central processing unit (CPU, Central Processing Unit) of the A model, and the B model.
  • the information of the GPU and the motherboard of the C model, the decoding capability information corresponding to the video playback terminal can be determined through the determined configuration information of the video playback terminal, and the high-specification CPU or GPU in the configuration information of the video playback terminal can improve the video playback.
  • the decoding capability corresponding to the terminal because cloud game users can flexibly switch between different video playback terminals according to different operating environments, therefore, the decoding computing power information of the video playback terminal can be determined through the configuration information of the video playback terminal, which can be timely and accurate.
  • the terminal situation of game users can be obtained in a timely manner, so that the video coding strategy can be adjusted in time to adapt to different use environments.
  • determining the decoding computing power information of the video playback terminal can be implemented by the following methods: acquiring real-time load information of the video playback terminal; and determining the decoding computing power information of the video playback terminal based on the real-time load information of the video playback terminal.
  • the decoding capability of the same video playback terminal is also limited by the number of decoding tasks it carries, changes in the real-time load information of the video playback terminal will cause changes in the real-time decoding capability of the video playback terminal.
  • the real-time decoding capability is 10 frames/second.
  • the configuration information and network status remain unchanged
  • the video playback terminal carries both local game and video playback decoding tasks
  • the number of decoding service processes it carries is 3
  • the real-time decoding capability is 3 frames per second, which is significantly reduced. Therefore, it is necessary to adjust the video coding strategy in time to adapt to the video Changes in the coding environment provide users with a better experience.
  • Step 403 The video encoding apparatus obtains the capacity configuration information of the playback buffer pool in the video playback environment.
  • FIG. 5 is a schematic diagram of the use environment of the video encoding method provided by the embodiment of the present application, wherein, acquiring the capacity configuration information of the playback buffer pool in the video playback environment can be realized by the following methods: acquiring the real-time data of the playback buffer pool Frame queue length; based on the comparison result of the real-time data frame queue length of the playback buffer pool and the length threshold, determine the capacity configuration information of the playback buffer pool in the video playback environment.
  • the video playback terminal when playing the video of the cloud game, receives the video frame encoded by the server and decodes and plays it.
  • the video playback terminal receives the video frame of the game encoded by the server After the video frame or the data frame carrying the video information, it is necessary to save the received video frame of the game video or the data frame carrying the video information in the playback buffer pool of the video playback terminal.
  • the video frame of the game video or the data frame carrying the video information in the real-time data frame queue is decoded, and the video frame 501 of No. 1, the video frame 502 of No. 2 and the video frame 503 of No.
  • 3 in the game video should be displayed by decoding, This enables game users to obtain three complete video frames of the game video.
  • the stronger the decoding computing power of the video playback terminal the fewer the frames in the real-time data frame queue of the playback buffer pool.
  • the number of frames in the real-time data frame queue of the playback buffer pool can be kept at 0, thereby ensuring that the game video without stuttering can be obtained during the running of the cloud game.
  • FIG. 6 is a schematic diagram of the use environment of the video encoding method provided by the embodiment of the present application.
  • the frame (Frame) queue buffered in the buffer pool (buffer) of the player is longer than 1 frame for a long time, it is explained that the real-time decoding computing power of the video playback terminal is It is relatively low and cannot adapt to the current cloud game video load, or the decoding computing power of the video playback terminal is unstable and cannot adapt to the current game video playback in cloud games.
  • the real-time decoding computing power of the video playback terminal is low or unstable.
  • the game user of the cloud game can only obtain the video frame 501 of the number 1 and the video frame 502 of the game video number 2, which causes the game video to freeze. You can trigger dynamic adjustment of the frame rate when you want to avoid game video lag affecting the playback of cloud game videos.
  • Step 404 The video encoding device dynamically adjusts the encoding strategy for video playback based on at least one of the network bandwidth information, the decoding computing power information, and the capacity configuration information of the playback buffer pool to obtain a video encoding that matches the playback environment of the target video. Strategy.
  • the coding strategy for video playback is dynamically adjusted to obtain a video coding matching the playback environment of the target video.
  • the strategy can be implemented in the following manner: when the resolution of the video playback terminal is lower than the first resolution, and the network bandwidth information indicates that the network bandwidth environment of the video playback terminal is unstable, determining a video encoding that matches the playback environment of the target video.
  • the strategy is to reduce the frame rate encoding; when the resolution of the video playback terminal is higher than or equal to the first resolution, and the network bandwidth information indicates that the network bandwidth environment of the video playback terminal is unstable, determine the video that matches the playback environment of the target video
  • the encoding strategy is to reduce the video bit rate encoding.
  • FIG. 7A is a schematic diagram of the use environment of the video encoding method provided by the embodiment of the present application.
  • the encoding strategy is dynamically adjusted to reduce the frame rate encoding (for example, the frame reduction processing strategy can be used to select an appropriate frame rate reduction strategy in the number of transmission frames per second greater than or equal to 30).
  • Game users of cloud games can obtain the video frame 501 of No. 1 and the video frame 502 of the game video No. 2, to avoid that when the resolution is less than 720P due to the encoding strategy of high transmission frames per second, only the video frame No. 1 can be displayed. Defects in game video frames reduce the user experience degradation caused by dropped frames and freezes.
  • FIG. 7B is a schematic diagram of a usage environment of the video encoding method provided by the embodiment of the present application.
  • the first resolution as 720P as an example
  • the resolution selected by the user in the cloud game process is greater than or equal to 720P
  • the unit time of data transmission can be reduced.
  • the number of transmitted data bits ie bit rate
  • the first resolution can be increased from 720P to 720P.
  • the second resolution is 1080P, while maintaining a low number of data bits transmitted per unit time during data transmission, so that the video playback terminal can run cloud games normally, and at the same time, users can obtain a better visual experience.
  • the bit rate adjustment is relatively low and still appears to be stuck, it can be dynamically adjusted to reduce the frame rate. For example, when it is determined that the video coding strategy that matches the playback environment of the target video is coding with reduced video bit rate, the playback fluency of the target video is detected; when it is detected that the playback fluency of the target video is less than the fluency threshold (that is, playback When a freeze occurs), it is determined that the video encoding strategy that matches the playback environment of the target video is to reduce the frame rate encoding and the video bit rate encoding at the same time.
  • the fluency threshold that is, playback When a freeze occurs
  • the first image frame and the second image frame in the target video may be acquired, wherein the first image frame and the second image frame are the playback of the target video.
  • the image frame corresponding to the screen is displayed; the difference image corresponding to the first image frame and the second image frame is obtained; the difference image is converted into a matching grayscale image, and a plurality of pixels included in the grayscale image are determined; The grayscale values of multiple pixels in the grayscale image are used to detect the playback fluency of the target video.
  • the difference between the first image frame and the second image frame is positively correlated with the grayscale value in the grayscale image, that is, the larger the grayscale value in the grayscale image, the higher the grayscale value of the first image.
  • the greater the difference between the frame and the second image frame it can also be determined that there is no jamming situation between the first image frame and the second image frame; similarly, in response to the smaller the grayscale value in the grayscale image , it means that the difference between the first image frame and the second image frame is smaller, and when the maximum grayscale value in the grayscale image is less than the preset threshold, it is determined that a card is generated between the first image frame and the second image frame. Dayton situation.
  • the video encoding method provided in the embodiment of the present application is used to process the game video in the cloud game
  • the cloud game is operated by the cloud server during operation, and the video stream obtained after the operation is transmitted.
  • Play to the video playback terminal that is, the video playback terminal is used for video playback during the running process of the cloud game.
  • the running process of the cloud game does not have very high requirements on the computing power of the video playback terminal, and it is not easy to play the game video. Stalling occurs during the playback process, and the cloud server requires a large amount of computation during the game running process, which is prone to stalling.
  • the video stream generated by the cloud server itself is the video stream with stalling, and the video playback terminal In the process of playing the video stream, there is no stutter caused by the video playback terminal, that is, the FPS cannot accurately detect the stutter during the running of the cloud game.
  • the FPS cannot accurately detect the stutter during the running of the cloud game.
  • the accuracy of the stuck detection is improved. Accuracy.
  • the coding strategy for video playback is dynamically adjusted to obtain a video coding matching the playback environment of the target video.
  • the strategy can be implemented in the following ways: based on the decoding computing power information, determine the decoding computing power decoding time-consuming delay parameter of the video playback terminal; target frame rate; based on the target frame rate, determine the video coding strategy that matches the playback environment of the target video.
  • the dynamic adjustment frame rate selects the largest value among the constant value of 30 and the dynamic value of 1000/decode delay (DecodeDelay), and ensures the minimum interactive experience of the game through the lowest 30FPS, and presents notification information to game users. It is recommended that game users Replace the terminal for a better user experience.
  • the bit rate of cloud game audio accounts for a relatively small proportion of the total, and the decoding computing power required for audio decoding is low.
  • the audio and video can be out of sync when the game audio and video are rendered, and they are rendered separately to prevent the audio and video synchronization problem caused by video frame rate adaptation.
  • Step 405 The video encoding apparatus encodes the target video according to the determined video encoding strategy.
  • the encoded target video is sent to the video playback terminal, and the video playback terminal performs video decoding by using the video decoding method provided in the embodiment of the present application, that is, by using the video corresponding to the video encoding strategy.
  • the decoding strategy decodes the encoded target video to play the target video on the video playback terminal.
  • the video decoding strategy is to reduce frame rate decoding
  • the video encoding strategy is to reduce video bit rate encoding
  • the video decoding strategy is to reduce video bit rate decoding
  • the video encoding strategy is to reduce frame rate encoding and video bit rate encoding
  • the video decoding strategy is to reduce frame rate decoding and video bit rate decoding.
  • Cloud technology refers to unifying a series of resources such as hardware, software, and network in a wide area network or a local area network to realize data computing, storage, and storage.
  • a hosting technology for processing and sharing can also be understood as a general term for network technology, information technology, integration technology, management platform technology and application technology based on cloud computing business model applications. Background services of technical network systems require a lot of computing and storage resources, such as video websites, picture websites and more portal websites, so cloud technology needs to be supported by cloud computing.
  • cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space and information services as needed.
  • the network that provides the resources is called the “cloud”.
  • the resources in the "cloud” are infinitely scalable from the user's point of view, and can be obtained at any time, used on demand, expanded at any time, and paid for according to usage.
  • a cloud platform As a basic capability provider of cloud computing, it will establish a cloud computing resource pool platform, referred to as a cloud platform, generally referred to as Infrastructure as a Service (IaaS, Infrastructure as a Service), deploying various types of virtual resources in the resource pool for External customers choose to use.
  • the cloud computing resource pool mainly includes: computing devices (which can be virtualized machines, including operating systems), storage devices, and network devices.
  • the identification information of the target user can be determined; based on the cloud server network, the data source cluster matching the identification information is determined; according to the data source cluster, the corresponding cloud
  • the game server stores the historical information of users that match the target users, so that the game data corresponding to the game users can be updated in time.
  • the cloud server can adjust the video encoding strategy in time through the decoding computing power information and video encoding strategy of the video playback terminal saved in the blockchain network, and play the video currently used by the game user.
  • the terminal adapts the corresponding video coding strategy, reduces the manual adjustment operation of the user, and improves the user experience.
  • FIG. 8 is a schematic diagram of the usage environment of the video encoding method provided by the embodiment of the present application, wherein the game application is in the During the process of running in the cloud server, the operation of the game application program is acquired and encoded, wherein the acquisition process includes acquiring the desktop image frame (ie the game image frame) from the buffer area of a Digital Differential Analyzer (DDA), and The structure information of the desktop image frame.
  • the structure information includes the rendering time of the desktop image frame, that is, the refresh time of the desktop image frame.
  • the refresh time is used as the display time stamp of the desktop image frame.
  • first-person shooting game FPS, First Person Shooting Game
  • MOBA multiplayer online tactical competitive game
  • multiplayer gun battle survival games battle royale type shooting games, etc.
  • FIG. 9 is a schematic diagram of a front-end display of a video encoding method according to an embodiment of the present application, wherein the running logic of the cloud application program is pre-stored in the server, and after the server receives the control instruction sent by the terminal, the operation logic according to the control instruction and the pre-stored operation logic , control the cloud application to run in the server, and generate the running screen (ie image frame).
  • the running screen is rendered by a graphics processor (GPU, Graphics Processing Unit), and the running screen is displayed on the desktop.
  • the server can display the running picture through the display, or the server can only generate and encode the running picture to obtain the encoded file, so as to send the encoded file to the terminal to display the running picture. Since the terminal with limited decoding computing power runs the cloud game due to Insufficient hardware decoding capability leads to high terminal load, and the decoding capability cannot match the frame rate of the cloud server, resulting in the freeze as shown in Figure 9.
  • FIG. 10 is a schematic flowchart of a video encoding method provided by an embodiment of the present application.
  • the processing flow of the video encoding method provided by an embodiment of the present application includes the following steps:
  • Step 1001 Estimate the network environment of the cloud game to obtain environment estimation information.
  • Step 1002 Estimate the decoding computing power of the user terminal to obtain decoding computing power estimation information.
  • Step 1003 Acquire the capacity configuration information of the playback buffer pool in the cloud game operating environment.
  • Step 1004 Dynamically adjust the encoding strategy for video playback based on the capacity configuration information, environment estimation information, and decoding computing power estimation information of the playback buffer pool. For example, when the resolution in the cloud game process is less than 720P, the frame is reduced to 30FPS, and when the resolution in the cloud game process is greater than or equal to 720P, the video bit rate is reduced.
  • FIG. 11 is a schematic diagram of a front-end display of a video encoding method provided by an embodiment of the present application, wherein, through the video encoding process shown in FIG. 10 , a minimum interactive experience of the game is guaranteed at a minimum of 30 FPS, clear game video image frames are presented to the user, and clear game video image frames are presented to the user. Cloud game users are presented with notification information, and users are advised to replace the terminal for a better experience.
  • the software modules stored in the video decoding apparatus in the memory 202 may include: information Processing module 2081, information encoding module 2082.
  • the information processing module 2081 is further configured to obtain hardware type information of the video playback terminal; determine the configuration information of the video playback terminal based on the hardware type information of the video playback terminal; determine the video playback terminal based on the configuration information of the video playback terminal Play the decoding computing power information of the terminal.
  • the information processing module 2081 is further configured to detect real-time load information of the video playback terminal; and determine the decoding computing power information of the video playback terminal based on the real-time load information of the video playback terminal.
  • the information processing module 2081 is further configured to obtain the real-time data frame queue length of the playback buffer pool in the video playback environment; based on the comparison between the real-time data frame queue length of the playback buffer pool and the length threshold As a result, the capacity configuration information of the playback buffer pool in the video playback environment is determined.
  • the information processing module 2081 is further configured to, when the resolution of the video playback terminal is lower than the first resolution, and the network bandwidth information indicates that the network bandwidth environment of the video playback terminal is unstable, It is determined that the video coding strategy that matches the playback environment of the target video is coding with reduced frame rate; when the resolution of the video playback terminal is higher than or equal to the first resolution, and the network bandwidth information represents the When the network bandwidth environment of the video playback terminal is unstable, it is determined that the video coding strategy that matches the playback environment of the target video is coding by reducing the video bit rate.
  • the information processing module 2081 is further configured to detect the playback fluency of the target video when it is determined that the video encoding strategy that matches the playback environment of the target video is encoding at a reduced video bit rate; When it is detected that the playback fluency of the target video is less than the fluency threshold, it is determined that the video coding strategy that matches the playback environment of the target video is frame rate reduction coding and video bit rate coding.
  • the information processing module 2081 is further configured to obtain the first image frame and the second image frame, wherein the first image frame and the second image frame are image frames corresponding to the display screen during the playback of the target video; acquiring the first image frame corresponds to the second image frame the difference image; convert the difference image into a matching grayscale image, and determine a plurality of pixels included in the grayscale image; based on the grayscale values of the plurality of pixels in the grayscale image, determine The playback fluency of the target video is detected.
  • the information processing module 2081 is further configured to, based on the decoding computing power information, determine a decoding computing power decoding time-consuming delay parameter of the video playback terminal; decoding time-consuming delay based on the decoding computing power parameters and the threshold of the number of transmitted frames per second, determine a matching target frame rate; based on the target frame rate, determine a video encoding strategy that matches the playback environment of the target video.
  • the software modules stored in the service data processing apparatus of the memory 202 may include: a first processing module configured to determine the identification information of the target object when the target video is a cloud game video; based on the cloud server The network determines the data source cluster matching the identification information; based on the data source cluster, the cloud game history information matching the target object is stored to the cloud game server
  • the software module stored in the service data processing apparatus of the memory 202 may include: a second processing module configured to convert the identification information of the video playback terminal, the decoding computing power information of the video playback terminal, and the The video coding strategy is stored in a content storage network, where the content storage network is a blockchain network or a network composed of at least one cloud server.
  • the software modules stored in the video decoding apparatus in the memory may include: an acquisition module and an information decoding module.
  • an acquisition module configured to acquire the encoded target video
  • the information decoding module is configured to decode the encoded target video through a video decoding strategy corresponding to the video encoding strategy.
  • the network bandwidth information is determined based on the network bandwidth environment of the video playback terminal; the decoding computing power information of the video playback terminal is determined; the capacity configuration information of the playback buffer pool in the video playback environment is obtained; based on the network bandwidth information, the decoding computing power information, and the capacity configuration information of the playback buffer pool, dynamically adjust the encoding strategy for video playback, and determine a video encoding strategy that matches the playback environment of the target video; through the determined video encoding strategy
  • the target video is encoded, thereby, the network bandwidth information, the decoding computing power information and the capacity configuration information of the playback buffer pool can be used to dynamically adjust the encoding strategy for video playback, and determine the playback of the target video.
  • the video coding strategy that matches the environment can reduce video playback freezes caused by changes in the network environment and insufficient terminal decoding capabilities, and improve user experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品，方法包括：基于视频播放终端的网络带宽环境，确定网络带宽信息；确定视频播放终端的解码算力信息；获取视频播放环境中的播放缓存池的容量配置信息；基于网络带宽信息、解码算力信息以及播放缓存池的容量配置信息中的至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略；通过视频编码策略对目标视频进行编码。

Description

一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品
相关申请的交叉引用
本申请实施例基于申请号为202011269523.8、申请日为2020年11月13日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请实施例作为参考。
技术领域
本申请实施例涉及视频编码技术，尤其涉及一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品。
背景技术
随着相关技术中云技术的发展，例如部署在云服务器中云游戏数量越来越多，但是云游戏的编码和传输并没有根据用户网络和终端解码能力自适应地动态调整，这样导致不稳定的用户网络或解码算力有限的用户网络，在运行云游戏时就会出现频繁卡顿、响应慢。
发明内容
有鉴于此，本申请实施例提供一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品，能够利用网络带宽信息、解码算力信息以及播放缓存池的容量配置信息，对视频播放的编码策略进行动态调整，确定与目标视频的播放环境相匹配的视频编码策略，减少由于网络环境的变化和终端解码能力的不足所造成的视频播放卡顿，提升用户的使用体验。
本申请实施例的技术方案是这样实现的：
本申请实施例提供了一种视频编码方法，由电子设备执行，包括：
基于视频播放终端的网络带宽环境，确定网络带宽信息；
确定所述视频播放终端的解码算力信息；
获取视频播放环境中的播放缓存池的容量配置信息；
基于所述网络带宽信息、所述解码算力信息以及所述播放缓存池的容量配置信息中的至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略；
通过所述视频编码策略对所述目标视频进行编码。
本申请实施例提供了一种视频解码方法，由电子设备执行，应用于前序的视频编码方法所编码后的目标视频；
所述方法包括：
获取所述编码后的目标视频；
通过与视频编码策略对应的视频解码策略对所述编码后的目标视频进行解码。
本申请实施例还提供了一种视频编码装置，包括：
信息处理模块，配置为基于视频播放终端的网络带宽环境，确定网络带宽信息；
所述信息处理模块，还配置为确定所述视频播放终端的解码算力信息；
所述信息处理模块，还配置为获取视频播放环境中的播放缓存池的容量配置信息；
所述信息处理模块，还配置为基于所述网络带宽信息、所述解码算力信息以及所述播放缓存池的容量配置信息中的至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略；
信息编码模块，配置为通过所述视频编码策略对所述目标视频进行编码。
本申请实施例还提供了一种视频解码装置，应用于前序的视频编码方法所编码后的目标视频；
所述装置包括：
获取模块，配置为获取所述编码后的目标视频；
信息解码模块，配置为通过与视频编码策略对应的视频解码策略对所述编码后的目标视频进行解码。
本申请实施例还提供了一种电子设备，所述电子设备包括：
存储器，用于存储可执行指令；
处理器，用于运行所述存储器存储的可执行指令时，实现前序的视频编码方法，或前序的视频解码方法。
本申请实施例还提供了一种计算机可读存储介质，存储有可执行指令，其特征在于，所述可执行指令被处理器执行时实现前序的视频编码方法，或前序的视频解码方法。
本申请实施例提供一种计算机程序产品，包括计算机程序或指令，所述计算机程序或指令被处理器执行时实现前序的视频编码方法，或前序的视频解码方法。
本申请实施例具有以下有益效果：
利用网络带宽信息、解码算力信息或播放缓存池的容量配置信息，对视频播放的编码策略进行动态调整，确定与目标视频的播放环境相匹配的视频编码策略，减少由于网络环境的变化和终端解码能力的不足所造成的视频播放卡顿，提高视频播放的流畅性，从而提升用户的使用体验。
附图说明
图1是本申请实施例提供的一种视频编码方法以及视频解码方法的使用环境示意图；
图2为本申请实施例提供的视频编码装置的组成结构示意图；
图3为本申请实施例中视频编码显示的处理方法的示意图；
图4为本申请实施例提供的视频编码方法的流程示意图；
图5为本申请实施例提供的视频编码方法的使用环境示意图；
图6为本申请实施例提供的视频编码方法的使用环境示意图；
图7A为本申请实施例提供的视频编码方法的使用环境示意图；
图7B为本申请实施例提供的视频编码方法的使用环境示意图；
图8为本申请实施例提供的视频编码方法的使用环境示意图；
图9为本申请实施例视频编码方法的前端显示示意图；
图10为本申请实施例提供的视频编码方法的流程示意图；
图11为本申请实施例视频编码方法的前端显示示意图。
具体实施方式
为了使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请作进一步地详细描述，所描述的实施例不应视为对本申请的限制，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。
在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。
对本申请实施例进行进一步详细说明之前，对本申请实施例中涉及的名词和术语进行说明，本申请实施例中涉及的名词和术语适用于如下的解释。
1)视频编码(Video Transcoding)：指将已经压缩编码的视频码流转换成另一个视频码流，以适应不同的网络带宽、不同的终端处理能力和不同的用户需求。
2)客户端：终端中实现特定功能的载体，例如移动客户端(APP)是移动终端中特定功能的载体，例如执行线上直播(视频推流)的功能或者是在线视频的播放功能。
3)响应于：用于表示所执行的操作所依赖的条件或者状态，当满足所依赖的条件或状态时，所执行的一个或多个操作可以是实时的，也可以具有设定的延迟；在没有特别说明的情况下，所执行的多个操作不存在执行先后顺序的限制。
4)虚拟环境：是应用程序在终端上运行时显示(或提供)的虚拟环境。 该虚拟环境可以是对真实世界的仿真环境，也可以是半仿真半虚构的三维环境，还可以是纯虚构的三维环境。虚拟环境可以是二维虚拟环境、2.5维虚拟环境和三维虚拟环境中的任意一种，下述实施例以虚拟环境是三维虚拟环境来举例说明，但对此不加以限定。例如，该虚拟环境还用于至少两个虚拟对象之间的虚拟环境对战。该虚拟环境还用于至少两个虚拟对象之间使用虚拟枪械进行对战。该虚拟环境还用于在目标区域范围内，至少两个虚拟对象之间使用虚拟枪械进行对战，该目标区域范围会随虚拟环境中的时间推移而不断变小。
5)云技术(Cloud technology)：指在广域网或局域网内将硬件、软件、网络等系列资源统一起来，实现数据的计算、储存、处理和共享的一种托管技术。基于云计算商业模式应用的网络技术、信息技术、整合技术、管理平台技术、应用技术等的总称，可以组成资源池，按需所用，灵活便利。云计算技术将变成重要支撑。技术网络系统的后台服务需要大量的计算、存储资源，如视频网站、图片类网站和更多的门户网站。伴随着互联网行业的高度发展和应用，将来每个物品都有可能存在自己的识别标志，都需要传输到后台系统进行逻辑处理，不同程度级别的数据将会分开处理，各类行业数据皆需要强大的系统后盾支撑，只能通过云计算来实现。
6)云游戏：是指游戏本身运行在云端服务器设备中，将云端设备渲染后的游戏画面编码后，通过网络传输至用户终端，由用户终端对编码文件进行解码后渲染至显示屏进行显示的游戏，从而，用户无需在本地安装游戏，而仅需要建立与云端的通信网络连接便能完成游戏交互过程。
7)每秒传输帧数(FPS，Frames Per Second)：FPS是图像领域中的定义，是指画面每秒传输帧数，即动画或视频的画面数。FPS是测量用于保存、显示动态视频的信息数量。每秒钟帧数越多，所显示的动作就会越流畅。其中，30FPS是避免动作不流畅的最低要求。
参考图1，图1为本申请实施例提供的视频编码方法以及视频解码方法的使用场景示意图，下面对本申请实施例所提供的视频编码方法以及视频解码方法的使用环境进行说明：
终端(包括终端10-1和终端10-2)上设置有运行视频播放客户端(例如云游戏的客户端)，用户通过所设置的云游戏软件客户端可以获得云服务器所存储的游戏内容；终端通过网络300连接服务器200(即云服务器)，网络300可以是广域网或者局域网，又或者是二者的组合，使用无线链路实现数据传输，其中，本申请实施例所提供的视频编码方法可以作为云服务的形式服务于终端(封装于专业游戏终端或者封装于不同的移动电子设备中)，本申请实施例不做具体限制，其中，游戏本身运行在云服务器中，将云服务器渲染后的游戏画面编码后，得到编码文件，并通过网络300传输至终端，由终端对编码文件进行解码后渲染，以在显示界面显示游戏，从而，用户无需在本地安装游戏，而仅需要建立与云端的通信网络连接便 能完成游戏交互过程。
需要说明的是，本申请实施例中的视频并不局限于游戏视频，还可以是短视频、直播视频等。其中，上述终端可以包括各种类型的应用，例如，即时通讯应用、视频播放应用、短视频应用、直播应用等。
在一些实施例中，上述视频编码方法以及视频解码方法还可以是不限于应用于云游戏、视频通话、视频直播等场景中。
需要说明的是，云游戏的组织架构中包括终端和云服务器，其中，终端用于接收用户对游戏过程的控制操作，并将控制操作对应的控制指令发送至云服务器，云服务器用于对游戏进程进行控制，并将游戏过程中的视频流发送至终端进行播放。也即在云游戏的组织架构中，终端主要用于在游戏过程中接收云服务器发送的游戏视频和游戏音频，并进行渲染与播放，将游戏用户在终端侧的操作(包括但不限于游戏用户通过鼠标、键盘、操作手柄、语音指令、触摸指令等方式实现的操作)，通过和云服务器连接的信令通道发送给云服务器，云服务器把这些操作信息发送给相匹配的鼠标驱动进程、声卡驱动进程以及键盘驱动进程，从而实现对云端游戏的控制，具体来说，云服务器的处理内容包括：1)云游戏运行中的运算以及游戏渲染、画面采集编码、声卡声音采集编码，并将经过编码处理的视频信息和音频信息通过流媒体的形式发送给终端；2)接收终端的操作指令，并将操作指令发送至相对应的鼠标驱动进程、声卡驱动进程以及键盘驱动进程，从而实现对云游戏的控制。
下面对本申请实施例的视频编码装置的结构做详细说明，视频编码装置可以各种形式来实施，如带有视频编码装置处理功能的专用终端，也可以为设置有视频编码装置处理功能的电子设备(例如手机、或平板电脑)，例如前序图1中的终端10-1或者终端10-2。图2为本申请实施例提供的视频编码装置的组成结构示意图，可以理解，图2仅仅示出了视频编码装置的示例性结构而非全部结构，根据需要可以实施图2示出的部分结构或全部结构。
本申请实施例提供的视频编码装置包括：至少一个处理器201、存储器202、用户接口203和至少一个网络接口204。视频编码装置中的各个组件通过总线系统205耦合在一起。可以理解，总线系统205用于实现这些组件之间的连接通信。总线系统205除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图2中将各种总线都标为总线系统205。
其中，用户接口203可以包括显示器、键盘、鼠标、轨迹球、点击轮、按键、按钮、触感板或者触摸屏等。
可以理解，存储器202可以是易失性存储器或非易失性存储器，也可包括易失性和非易失性存储器两者。本申请实施例中的存储器202能够存储数据以支持终端(如终端10-1)的操作。这些数据的示例包括：用于在 终端(如终端10-1)上操作的任何计算机程序，如操作系统和应用程序。其中，操作系统包含各种系统程序，例如框架层、核心库层、驱动层等，用于实现各种基础业务以及处理基于硬件的任务。应用程序可以包含各种应用程序。
在一些实施例中，本申请实施例提供的视频编码装置可以采用软硬件结合的方式实现，作为示例，本申请实施例提供的视频编码装置可以是采用硬件译码处理器形式的处理器，其被编程以执行本申请实施例提供的视频编码方法。例如，硬件译码处理器形式的处理器可以采用一个或多个应用专用集成电路(ASIC，Application Specific Integrated Circuit)、DSP、可编程逻辑引擎器件(PLD，Programmable Logic Device)、复杂可编程逻辑引擎器件(CPLD，Complex Programmable Logic Device)、现场可编程门阵列(FPGA，Field-Programmable Gate Array)或其他电子元件。
作为本申请实施例提供的视频编码装置采用软硬件结合实施的示例，本申请实施例所提供的视频编码装置可以直接体现为由处理器201执行的软件模块组合，软件模块可以位于计算机可读存储介质中，计算机可读存储介质位于存储器202，处理器201读取存储器202中软件模块包括的可执行指令，结合必要的硬件(例如，包括处理器201以及连接到总线205的其他组件)完成本申请实施例提供的视频编码方法。
作为示例，处理器201可以是一种集成电路芯片，具有信号的处理能力，例如通用处理器、数字信号处理器(DSP，Digital Signal Processor)，或者其他可编程逻辑引擎器件、分立门或者晶体管逻辑引擎器件、分立硬件组件等，其中，通用处理器可以是微处理器或者任何常规的处理器等。
作为本申请实施例提供的视频编码装置采用硬件实施的示例，本申请实施例所提供的装置可以直接采用硬件译码处理器形式的处理器201来执行完成，例如，被一个或多个应用专用集成电路(ASIC，Application Specific Integrated Circuit)、DSP、可编程逻辑引擎器件(PLD，Programmable Logic Device)、复杂可编程逻辑引擎器件(CPLD，Complex Programmable Logic Device)、现场可编程门阵列(FPGA，Field-Programmable Gate Array)或其他电子元件执行实现本申请实施例提供的视频编码方法。
本申请实施例中的存储器202用于存储各种类型的数据以支持视频编码装置的操作。这些数据的示例包括：用于在视频编码装置上操作的任何可执行指令，如可执行指令，实现本申请实施例的从视频编码方法的程序可以包含在可执行指令中。
在一些实施例中，本申请实施例提供的视频编码装置可以采用软件方式实现，图2示出了存储在存储器202中的视频编码装置，其可以是程序和插件等形式的软件，并包括一系列的模块，作为存储器202中存储的程序的示例，可以包括视频编码装置，视频编码装置中包括以下的软件模块信息处理模块2081和信息编码模块2082。当视频编码装置中的软件模块被 处理器201读取到RAM中并执行时，将实现本申请实施例提供的视频编码方法，其中，视频编码装置中各个软件模块的功能，包括：
信息处理模块2081，配置为基于视频播放终端的网络带宽环境，确定网络带宽信息；所述信息处理模块2081，还配置为确定所述视频播放终端的解码算力信息；所述信息处理模块2081，还配置为获取视频播放环境中的播放缓存池的容量配置信息；所述信息处理模块2081，还配置为基于所述网络带宽信息、所述解码算力信息以及所述播放缓存池的容量配置信息中的至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略；信息编码模块2082，配置为通过所述视频编码策略对所述目标视频进行编码。
下面对本申请实施例的视频解码装置的结构做简要说明，视频解码装置可以各种形式来实施，如带有视频解码装置处理功能的专用终端，也可以为设置有视频解码装置处理功能的电子设备(例如手机、或平板电脑)，例如前序图1中的终端10-1或者终端10-2。其中，视频解码装置中各个软件模块的功能，包括：
获取模块，配置为获取所述编码后的目标视频；
信息解码模块，配置为通过与视频编码策略对应的视频解码策略对所述编码后的目标视频进行解码。
根据图2所示的电子设备，本申请实施例还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述视频编码方法以及视频解码方法的各种可选实现方式中所提供的不同实施例及实施例的组合。
参考图3，图3为本申请实施例提供的视频编码显示的处理方法的示意图，视频编码过程中，云游戏中的桌面采集编码方案可以采集游戏画面然后编码、传输给终端，桌面采集帧率和游戏帧率同步，以图形处理器(GPU，Graphics Processing Unit)渲染3D图像为例，包括以下几个方面：
1)顶点处理：GPU读取描述3D图形外观的顶点数据，并根据顶点数据确定3D图形的形状及位置关系，建立起3D图形的骨架。其中，以应用于不同的游戏环境包括但不限虚拟现实应用程序、三维地图程序、军事仿真程序、第一人称射击游戏(FPS，First-Person Shooting Game)、多人在线战术竞技游戏(MOBA，Multiplayer Online Battle Arena Games)为例，最终在用户界面(UI，User Interface)上呈现出三维虚拟环境中的3D图形的骨架，其中，描述3D图形外观的顶点数据可以通过纹理坐标集合中的顶点的纹理坐标实现，具体来说，纹理坐标集合中的顶点的纹理坐标之间可以形成相应的纹理坐标线段，如果所有的纹理坐标顶点之间的最短路径都经过了某个节点，那么就认为这个节点为骨架的根节点(Betweenness  Centrality)，由此，可以建立起3D图形的骨架。
2)光栅化计算：将顶点数据转换为片元，其中显示器实际显示的图像包括像素组，将顶点处理阶段所形成的顶点数据通过转换算法转换为相应的像素点，实现把一个矢量图形转换为一系列像素点的过程就称为光栅化。例如，通过光栅化计算将基于斜线段所表示的3D图形的骨架的顶点数据转化成阶梯状的连续像素点。
3)纹理帖图：3D图形的骨架的顶点数据生成阶段仅仅生成了只构成了3D物体的轮廓的多边形，而纹理映射(texture mapping)完成对多变形表面的帖图，也就是将多边形的表面贴上相应的图片，从而生成“真实”的图形。
4)像素处理：在执行光栅化计算阶段，在对每个像素进行转化的过程中，GPU完成对像素的计算和处理，从而确定每个像素的最终属性。
5)最终输出：由光栅化引擎完成像素的输出，1帧图像渲染完毕后，将渲染后的图像发送至显存帧缓冲区。
但是图3所示的编码和传输过程并没有根据用户网络和终端解码能力自适地应动态调整，这样导致一些用户网络不稳定或解码算力有限的用户网络在玩云游戏时就会频繁卡顿、响应慢等问题，具体来说相关技术仅是定时帧率从帧缓冲区(buffer)里读取当前最新帧数据，然后发送给编码器进行编码，编码压缩后的帧数据再传输给云游戏的终端，并在云游戏的终端的显示界面显示，但是，解码算力有限的用户在玩云游戏时由于硬件解码能力不足，而导致终端负载比较高，解码能力跟不上云端帧率画面，从而导致频繁卡顿、响应慢等问题，对于帧率、分辨率、码率比较高的云游戏，由于解码以及高帧率渲染画面时会导致终端硬件负载比较高，造成终端电池过热，频繁引起强制降频，影响用户的使用体验。
为了克服上述问题，结合图2示出的电子设备说明本申请实施例提供的视频编码方法，参见图4，图4为本申请实施例提供的视频编码方法的流程示意图，可以理解地，图4所示的步骤可以由运行视频编码装置的各种电子设备执行，例如可以是如带有视频编码功能的专用终端、服务器或者服务器集群。下面针对图4示出的步骤进行说明。
步骤401：视频编码装置基于视频播放终端的网络带宽环境，确定网络带宽信息。
步骤402：视频编码装置确定视频播放终端的解码算力信息。
其中，解码算力信息用于表征视频播放终端(如图4中的终端1、终端2)在单位时间内能够进行视频解码处理的帧数，解码算力信息的数值越大，表明视频播放终端在单位时间内能够进行视频解码处理的帧数越多。
在一些实施例中，确定视频播放终端的解码算力信息，可以通过以下方式实现：获取视频播放终端的硬件类型信息；基于视频播放终端的硬件类型信息，确定视频播放终端的配置信息；基于视频播放终端的配置信息， 确定视频播放终端的解码算力信息。
例如，在运行云游戏进程，即播放云游戏的视频时，首先确定视频播放终端的解码算力信息，即确定终端解码算力较低、算力不稳定的使用环境，例如，可以根据视频播放终端的配置信息确定视频播放终端的解码算力，当解码一帧视频帧的解码延时大于或者等于参数A时，说明视频播放终端解码算力较低；其中，解码延时用于表征视频播放终端解码一帧视频帧所消耗的时间，例如参数A定义为1000/画面帧率*0.8，由此，可以触发对视频编码策略进行调整，减少云游戏视频的卡顿，提升用户的使用体验。
例如，由于不同的视频播放终端的解码能力不同，因此需要确定视频播放终端的解码算力信息，对于已经获知硬件类型信息的视频播放终端，可以通过相应的硬件类型信息，确定视频播放终端的配置信息，例如对于XX品牌的XX型号的手机，可以通过硬件类型信息(或者XX型号手机的入网代码)确定视频播放终端的配置信息为A型号的中央处理器(CPU，Central Processing Unit)、B型号的GPU以及C型号的主板信息，通过所确定的视频播放终端的配置信息即可确定视频播放终端所对应的解码能力信息，视频播放终端的配置信息型号中高规格的CPU或者GPU均能够提升视频播放终端所对应的解码能力，由于云游戏的用户可以根据不同的操作环境灵活地切换不同的视频播放终端，因此，通过视频播放终端的配置信息，确定视频播放终端的解码算力信息，可以及时准确地获取游戏用户的终端情况，便于及时调整视频编码策略以适应不同的使用环境。
在一些实施例中，确定视频播放终端的解码算力信息，可以通过以下方式实现：获取视频播放终端的实时负载信息；基于视频播放终端的实时负载信息，确定视频播放终端的解码算力信息。
例如，由于同一个视频播放终端的解码能力还受到所承载的解码任务的数量限制，因此，由于频播放终端的实时负载信息的变化会造成视频播放终端的实时解码能力的变化，例如对于配置信息为A型号的CPU、B型号的GPU以及C型号的主板的手机，在承载的解码业务进程数量为1(负载信息)时，实时解码能力为10帧/秒，当配置信息和网络状态不变时，视频播放终端同时承载本地游戏和视频播放的解码任务，其承载的解码业务进程数量为3，实时解码能力3帧/秒，出现明显的下降，因此，需要及时调整视频编码策略以适应视频编码使用环境的变化，为用户提供更好的使用体验。
步骤403：视频编码装置获取视频播放环境中的播放缓存池的容量配置信息。
参考图5，图5为本申请实施例提供的视频编码方法的使用环境示意图，其中，获取视频播放环境中的播放缓存池的容量配置信息，可以通过以下方式实现：获取播放缓存池的实时数据帧队列长度；基于播放缓存池的实时数据帧队列长度与长度阈值的比对结果，确定视频播放环境中的播放缓 存池的容量配置信息。
其中，以云游戏为例，如图5所示，当播放云游戏的视频时，视频播放终端接收服务器编码后的视频帧并进行解码播放，当视频播放终端接收到经过服务器编码的游戏视频的视频帧或者携带视频信息的数据帧后，需要将所接收的游戏视频的视频帧或者携带视频信息的数据帧保存在视频播放终端的播放缓存池中，视频播放终端的解码装置对播放缓存池的实时数据帧队列中的游戏视频的视频帧或者携带视频信息的数据帧进行解码，通过解码应该展示游戏视频中的编号1的视频帧501、编号2的视频帧502以及编号3的视频帧503，使得游戏用户获得三个完整的游戏视频的视频帧，这一过程中，视频播放终端的解码算力越强，播放缓存池的实时数据帧队列中的帧数越少，在云游戏使用场景中，播放缓存池的实时数据帧队列中的帧数可以保持为0，由此，可以保证在云游戏的运行过程中获得无卡顿的游戏视频。
参考图6，图6为本申请实施例提供的视频编码方法的使用环境示意图，当播放器缓冲池(buffer)缓存的帧(Frame)队列长期大于1帧，说明视频播放终端的实时解码算力较低，不能够适应当前的云游戏视频负载，或者视频播放终端的解码算力不稳定，无法适应当前云游戏中的游戏视频播放，此时，如图6所示，由于视频播放终端的实时解码算力较低或者不稳定，与前序图5相比，云游戏的游戏用户仅能够获得编号1的视频帧501和编号2的游戏视频的视频帧502，造成游戏视频的卡顿，此时可以触发动态调整帧率，避免游戏视频的卡顿影响云游戏视频的播放。
步骤404：视频编码装置基于网络带宽信息、解码算力信息以及播放缓存池的容量配置信息中至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略。
在一些实施例中，基于网络带宽信息、解码算力信息以及播放缓存池的容量配置信息中至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略，可以通过以下方式实现：当视频播放终端的分辨率低于第一分辨率、并且网络带宽信息表征视频播放终端的网络带宽环境不稳定时，确定与目标视频的播放环境相匹配的视频编码策略为降低帧率编码；当视频播放终端的分辨率高于或者等于第一分辨率、并且网络带宽信息表征视频播放终端的网络带宽环境不稳定时，确定与目标视频的播放环境相匹配的视频编码策略为降低视频码率编码。
例如，继续以云游戏为例进行说明，参考图7A，图7A为本申请实施例提供的视频编码方法的使用环境示意图，以第一分辨率为720P(行像素)为例，当用户在云游戏进程中选择的分辨率小于720P时，用户终端的网络由宽带接入云游戏服务器切换至移动网络接入云游戏服务器，会出现信息传输速率的波动，造成带宽不稳定，通过对视频播放的编码策略进行动态调整，降低帧率编码(例如可以通过降帧处理策略，在大于等于30每秒传 输帧数中选择相适应的降低帧率策略)，由此，可以如图7A所示，使得云游戏的游戏用户能够获得编号1的视频帧501和编号2的游戏视频的视频帧502，避免在分辨率是小于720P时由于高每秒传输帧数编码策略所造成的仅能够显示编号1的游戏视频帧的缺陷，减少用户由于掉帧和卡顿所造成的使用体验下降。
参考图7B，图7B为本申请实施例提供的视频编码方法的使用环境示意图，以第一分辨率为720P为例，当用户在云游戏进程中选择的分辨率大于等于720P时，用户终端的网络由宽带接入云游戏服务器切换至移动网络接入云游戏服务器时，会出现信息传输速率的波动，造成带宽不稳定，通过对视频播放的编码策略进行动态调整，可以降低数据传输时单位时间传送的数据位数(即码率)，以保证用户获得较高分辨率的游戏视频进行观看，由于视频播放终端的配置信息不同，对于硬件配置较高的终端，可以提升第一分辨率720P至第二分辨率1080P，同时保持低数据传输时单位时间传送的数据位数，可以使得视频播放终端正常运行云游戏的同时，使得用户的获得更好的视觉体验。
例如，如果码率调整比较低以后仍然出现卡顿的情况可以动态调整，以降低帧率。例如，当确定与目标视频的播放环境相匹配的视频编码策略为降低视频码率编码时，对目标视频的播放流畅度进行检测；当检测到目标视频的播放流畅度小于流畅度阈值(即播放出现卡顿)时，确定与目标视频的播放环境相匹配的视频编码策略为同时降低帧率编码与降低视频码率编码。
在一些实施例中，当需要对目标视频的播放流畅度进行检测时，可以获取目标视频中的第一图像帧和第二图像帧，其中，第一图像帧和第二图像帧为目标视频播放过程中显示画面对应的图像帧；获取第一图像帧与第二图像帧对应的差异图像；将差异图像转换为相匹配的灰度图像，并确定灰度图像所包括的多个像素点；根据灰度图像中多个像素点的灰度值，对目标视频的播放流畅度进行检测。
需要说明的是，第一图像帧和第二图像帧之间的差异，与灰度图像中的灰度值呈正相关关系，也即灰度图像中的灰度值越大，则表示第一图像帧和第二图像帧之间的差异越大，从而也能够确定，第一图像帧和第二图像帧之间未产生卡顿情况；同理，响应于灰度图像中的灰度值越小，则表示第一图像帧和第二图像帧之间的差异越小，当灰度图像中的最大灰度值小于预设阈值时，则确定第一图像帧和第二图像帧之间产生卡顿情况。
需要说明的是，在通过本申请实施例提供的视频编码方法对云游戏中的游戏视频进行处理时，由于云游戏在运行过程中是由云服务器进行运算，并将运算后得到的视频流传输至视频播放终端进行播放，也即视频播放终端在云游戏的运行过程中用于进行视频播放，云游戏的运行过程对视频播放终端的计算能力并没有非常高的要求，也不易在游戏视频的播放过程中 产生卡顿情况，而云服务器在游戏运行过程中的运算量较大，易产生卡顿情况，从而云服务器生成的视频流本身即为存在卡顿情况的视频流，而视频播放终端播放视频流的过程中并不存在视频播放终端导致的卡顿，也即通过上述FPS并不能对云游戏运行过程中的卡顿情况进行准确的检测。而本申请实施例中，通过对云游戏运行过程中的视频进行图像帧的采集，并根据图像帧之间的差异度对云游戏的运行过程是否产生卡顿进行检测，提高了卡顿检测的准确率。
在一些实施例中，基于网络带宽信息、解码算力信息以及播放缓存池的容量配置信息中至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略，可以通过以下方式实现：基于解码算力信息，确定视频播放终端的解码算力解码耗时延时参数；基于解码算力解码耗时延时参数和每秒传输帧数阈值，确定相匹配的目标帧率；基于目标帧率，确定与目标视频的播放环境相匹配的视频编码策略。
其中，动态调整帧率在常数数值30以及动态数值1000/解码延时(DecodeDelay)中选择最大的数值使用，通过最低的30FPS保证游戏的最低交互体验，并向游戏用户呈现通知信息，建议游戏用户更换终端以获取更好的使用体验。
例如，由于视频帧率动态调整，云游戏音频由于码率占总比比较少，而且音频解码所需要得到解码算力较低，云游戏中音频仅包括背景音乐和用户的语音交流声音，所以云游戏的运行在游戏音视频渲染时音画可以不同步，分开渲染，防止视频帧率自适应带来的音画同步处理卡顿的问题。
步骤405：视频编码装置通过所确定的视频编码策略对目标视频进行编码。
需要说明的是，在对目标视频编码后，将编码后的目标视频发送至视频播放终端，视频播放终端通过本申请实施例提供的视频解码方法进行视频解码，即通过与视频编码策略对应的视频解码策略对编码后的目标视频进行解码，以在视频播放终端播放目标视频。
例如，视频编码策略为降低帧率编码，则视频解码策略为降低帧率解码；视频编码策略为降低视频码率编码，则视频解码策略为降低视频码率解码；视频编码策略为降低帧率编码以及视频码率编码，则视频解码策略为降低帧率解码以及视频码率解码。
其中，本申请实施例可结合云技术或区块链网络技术实现，云技术(Cloud technology)是指在广域网或局域网内将硬件、软件及网络等系列资源统一起来，实现数据的计算、储存、处理和共享的一种托管技术，也可理解为基于云计算商业模式应用的网络技术、信息技术、整合技术、管理平台技术及应用技术等的总称。技术网络系统的后台服务需要大量的计算、存储资源，如视频网站、图片类网站和更多的门户网站，因此云技术需要以云计算作为支撑。
需要说明的是，云计算是一种计算模式，它将计算任务分布在大量计算机构成的资源池上，使各种应用系统能够根据需要获取计算力、存储空间和信息服务。提供资源的网络被称为“云”。“云”中的资源在使用者看来是可以无限扩展的，并且可以随时获取，按需使用，随时扩展，按使用付费。作为云计算的基础能力提供商，会建立云计算资源池平台，简称云平台，一般称为基础设施即服务(IaaS，Infrastructure as a Service)，在资源池中部署多种类型的虚拟资源，供外部客户选择使用。云计算资源池中主要包括：计算设备(可为虚拟化机器，包含操作系统)、存储设备和网络设备。
在一些实施例中，当通过云服务器群组运行云游戏时，可以确定目标用户的标识信息；基于云服务器网络，确定与标识信息相匹配的数据源集群；根据数据源集群，向相应的云游戏服务器中存储与目标用户相匹配的用户的历史信息，由此，能够及时更新游戏用户所对应的游戏数据，同时，当同一用户通过不同配置的视频播放终端播放云服务器中的视频信息，或者登陆同一用户的云游戏账号时，云服务器可以通过区块链网络中所保存的视频播放终端的解码算力信息以及视频编码策略及时地调整视频编码策略，并为游戏用户当前所使用的视频播放终端，适配相应的视频编码策略，减少用户的手动调整操作，提升用户的使用体验。
下面以云游戏运行中的游戏视频编码处理为例，对本申请实施例的视频编码方法进行说明，其中，图8为本申请实施例提供的视频编码方法的使用环境示意图，其中，游戏应用程序在云服务器中运行的过程中，对游戏应用程序的运行进行采集编码，其中，采集过程包括从数字差分分析仪(DDA，Digital Differential Analyzer)缓存区中获取桌面图像帧(即游戏图像帧)，以及桌面图像帧的结构体信息，该结构体信息中包括桌面图像帧的渲染时刻，也即桌面图像帧的刷新时刻，将该刷新时刻作为桌面图像帧的显示时间戳，对桌面图像帧进行编码后，生成编码文件，并将编码文件发送至终端进行解码播放；其中，上述游戏可以实现为如下：第一人称射击游戏(FPS，First Person Shooting Game)、多人在线战术竞技游戏(MOBA，Multiplayer Online Battle Arena Games)、多人枪战类生存游戏、大逃杀类型的射击游戏等。
其中，图9为本申请实施例视频编码方法的前端显示示意图，其中，服务器中预先存储有该云应用程序的运行逻辑，服务器接收终端发送的控制指令后，根据控制指令以及预先存储的运行逻辑，控制云应用程序在服务器中运行，并产生运行画面(即图像帧)。例如，由图形处理器(GPU，Graphics Processing Unit)对运行画面进行渲染，并在桌面中显示运行画面。例如，服务器可以通过显示器显示运行画面，或，服务器仅生成并编码运行画面，得到编码文件，从而将编码文件发送至终端进行运行画面的显示，由于解码算力有限的终端在运行云游戏时由于硬件解码能力不足导致终端 负载比较高，解码能力无法匹配云服务器的帧率画面从而导致如图9所示的卡顿。
为了克服上述问题，参考图10，图10为本申请实施例提供的视频编码方法的流程示意图，本申请实施例所提供的视频编码方法的处理流程包括以下步骤：
步骤1001：对云游戏的网络环境进行预估，获得环境预估信息。
步骤1002：对用户终端的解码算力进行预估，获得解码算力预估信息。
步骤1003：获取云游戏运行环境中的播放缓存池的容量配置信息。
步骤1004：基于播放缓存池的容量配置信息、环境预估信息以及解码算力预估信息，对视频播放的编码策略进行动态调整。例如当云游戏进程中分辨率为小于720P时，降帧至30FPS，当云游戏进程中分辨率为大于等于720P时，降低视频码率。
图11为本申请实施例提供的视频编码方法的前端显示示意图，其中，通过图10所示的视频编码处理，最低30FPS保证游戏的最低交互体验，向用户呈现清晰的游戏视频图像帧，并向云游戏用户呈现通知信息，建议用户更换终端以获取更好的使用体验。
下面继续说明本申请实施例提供的视频编码装置的实施为软件模块的示例性结构，在一些实施例中，如图2所示，存储在存储器202的视频解码装置中的软件模块可以包括：信息处理模块2081、信息编码模块2082。
在一些实施例中，信息处理模块2081，还配置为获取视频播放终端的硬件类型信息；基于视频播放终端的硬件类型信息，确定视频播放终端的配置信息；基于视频播放终端的配置信息，确定视频播放终端的解码算力信息。
在一些实施例中，信息处理模块2081，还配置为侦测视频播放终端的实时负载信息；基于视频播放终端的实时负载信息，确定视频播放终端的解码算力信息。
在一些实施例中，信息处理模块2081，还配置为获取所述视频播放环境中的播放缓存池的实时数据帧队列长度；基于所述播放缓存池的实时数据帧队列长度与长度阈值的比对结果，确定所述视频播放环境中的播放缓存池的容量配置信息。
在一些实施例中，信息处理模块2081，还配置为当所述视频播放终端的分辨率低于第一分辨率、并且所述网络带宽信息表征所述视频播放终端的网络带宽环境不稳定时，确定与所述目标视频的播放环境相匹配的视频编码策略为降低帧率编码；当所述视频播放终端的分辨率高于或者等于所述第一分辨率、并且所述网络带宽信息表征所述视频播放终端的网络带宽环境不稳定时，确定与所述目标视频的播放环境相匹配的视频编码策略为降低视频码率编码。
在一些实施例中，信息处理模块2081，还配置为当确定与所述目标视 频的播放环境相匹配的视频编码策略为降低视频码率编码时，对所述目标视频的播放流畅度进行检测；当检测到所述目标视频的播放流畅度小于流畅度阈值时，确定与所述目标视频的播放环境相匹配的视频编码策略为降低帧率编码以及视频码率编码。
在一些实施例中，信息处理模块2081，还配置为当确定与所述目标视频的播放环境相匹配的视频编码策略为降低视频码率编码时，获取所述目标视频中的第一图像帧和第二图像帧，其中，所述第一图像帧和所述第二图像帧为所述目标视频播放过程中显示画面对应的图像帧；获取所述第一图像帧与所述第二图像帧对应的差异图像；将所述差异图像转换为相匹配的灰度图像，并确定所述灰度图像所包括的多个像素点；基于所述灰度图像中多个像素点的灰度值，对所述目标视频的播放流畅度进行检测。
在一些实施例中，信息处理模块2081，还配置为基于所述解码算力信息，确定所述视频播放终端的解码算力解码耗时延时参数；基于所述解码算力解码耗时延时参数以及每秒传输帧数阈值，确定相匹配的目标帧率；基于所述目标帧率，确定与所述目标视频的播放环境相匹配的视频编码策略。
在一些实施例中，存储在存储器202的业务数据处理装置中的软件模块可以包括：第一处理模块，配置为当所述目标视频为云游戏视频时，确定目标对象的标识信息；基于云服务器网络，确定与所述标识信息相匹配的数据源集群；基于所述数据源集群，将与所述目标对象相匹配的云游戏历史信息存储至云游戏服务器
在一些实施例中，存储在存储器202的业务数据处理装置中的软件模块可以包括：第二处理模块，配置为将所述视频播放终端的标识信息、所述视频播放终端的解码算力信息以及所述视频编码策略存储至内容存储网络中，其中，所述内容存储网络为区块链网络，或者由至少一个云服务器构成的网络。
下面继续说明本申请实施例提供的视频解码装置的实施为软件模块的示例性结构，存储在存储器的视频解码装置中的软件模块可以包括：获取模块、信息解码模块。
获取模块，配置为获取所述编码后的目标视频；
信息解码模块，配置为通过与视频编码策略对应的视频解码策略对所述编码后的目标视频进行解码。
本申请实施例通过基于视频播放终端的网络带宽环境，确定网络带宽信息；确定所述视频播放终端的解码算力信息；获取视频播放环境中的播放缓存池的容量配置信息；基于所述网络带宽信息、所述解码算力信息以及所述播放缓存池的容量配置信息，对视频播放的编码策略进行动态调整，确定与目标视频的播放环境相匹配的视频编码策略；通过所确定的视频编码策略对所述目标视频进行编码，由此，能够利用网络带宽信息、所述解 码算力信息以及所述播放缓存池的容量配置信息，对视频播放的编码策略进行动态调整，确定与目标视频的播放环境相匹配的视频编码策略，减少由于网络环境的变化和终端解码能力的不足所造成的视频播放卡顿，提升用户的使用体验。
以上所述，仅为本申请的实施例而已，并非用于限定本申请的保护范围，凡在本申请的精神和原则。

Claims (17)

  1. 一种视频编码方法，由电子设备执行，所述方法包括：
    基于视频播放终端的网络带宽环境，确定网络带宽信息；
    确定所述视频播放终端的解码算力信息；
    获取视频播放环境中的播放缓存池的容量配置信息；
    基于所述网络带宽信息、所述解码算力信息以及所述播放缓存池的容量配置信息中的至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略；
    通过所述视频编码策略对所述目标视频进行编码。
  2. 根据权利要求1所述的方法，其中，所述确定所述视频播放终端的解码算力信息，包括：
    获取所述视频播放终端的硬件类型信息；
    基于所述视频播放终端的硬件类型信息，确定所述视频播放终端的配置信息；
    基于所述视频播放终端的配置信息，确定所述视频播放终端的解码算力信息。
  3. 根据权利要求1所述的方法，其中，所述确定所述视频播放终端的解码算力信息，包括：
    获取所述视频播放终端的实时负载信息；
    基于所述视频播放终端的实时负载信息，确定所述视频播放终端的解码算力信息。
  4. 根据权利要求1所述的方法，其中，所述获取视频播放环境中的播放缓存池的容量配置信息，包括：
    获取所述视频播放环境中的播放缓存池的实时数据帧队列长度；
    基于所述播放缓存池的实时数据帧队列长度与长度阈值的比对结果，确定所述视频播放环境中的播放缓存池的容量配置信息。
  5. 根据权利要求1所述的方法，其中，所述基于所述网络带宽信息、所述解码算力信息以及所述播放缓存池的容量配置信息中的至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略，包括：
    当所述视频播放终端的分辨率低于第一分辨率、并且所述网络带宽信息表征所述视频播放终端的网络带宽环境不稳定时，确定与所述目标视频的播放环境相匹配的视频编码策略为降低帧率编码；
    当所述视频播放终端的分辨率高于或者等于所述第一分辨率、并且所述网络带宽信息表征所述视频播放终端的网络带宽环境不稳定时，确定与所述目标视频的播放环境相匹配的视频编码策略为降低视频码率编码。
  6. 根据权利要求5所述的方法，其中，所述方法还包括：
    当确定与所述目标视频的播放环境相匹配的视频编码策略为降低视频码率编码时，对所述目标视频的播放流畅度进行检测；
    当检测到所述目标视频的播放流畅度小于流畅度阈值时，确定与所述目标视频的播放环境相匹配的视频编码策略为降低帧率编码以及视频码率编码。
  7. 根据权利要求6所述的方法，其中，所述当确定与所述目标视频的播放环境相匹配的视频编码策略为降低视频码率编码时，对目标视频的播放流畅度进行检测，包括：
    当确定与所述目标视频的播放环境相匹配的视频编码策略为降低视频码率编码时，获取所述目标视频中的第一图像帧和第二图像帧，其中，所述第一图像帧和所述第二图像帧为所述目标视频播放过程中显示画面对应的图像帧；
    获取所述第一图像帧与所述第二图像帧对应的差异图像；
    将所述差异图像转换为相匹配的灰度图像，并确定所述灰度图像所包括的多个像素点；
    基于所述灰度图像中多个像素点的灰度值，对所述目标视频的播放流畅度进行检测。
  8. 根据权利要求1所述的方法，其中，所述基于所述网络带宽信息、所述解码算力信息以及所述播放缓存池的容量配置信息中的至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略，包括：
    基于所述解码算力信息，确定所述视频播放终端的解码算力解码耗时延时参数；
    基于所述解码算力解码耗时延时参数以及每秒传输帧数阈值，确定相匹配的目标帧率；
    基于所述目标帧率，确定与所述目标视频的播放环境相匹配的视频编码策略。
  9. 根据权利要求1所述的方法，其中，所述方法还包括：
    当所述目标视频为云游戏视频时，确定目标对象的标识信息；
    基于云服务器网络，确定与所述标识信息相匹配的数据源集群；
    基于所述数据源集群，将与所述目标对象相匹配的云游戏历史信息存储至云游戏服务器。
  10. 根据权利要求1-9任一项所述的方法，其中，所述方法还包括：
    将所述视频播放终端的标识信息、所述视频播放终端的解码算力信息以及所述视频编码策略存储至内容存储网络中，其中，所述内容存储网络为区块链网络，或者由至少一个云服务器构成的网络。
  11. 一种视频解码方法，由电子设备执行，应用于如权利要求1-10所述视频编码方法所编码后的目标视频；
    所述方法包括：
    获取所述编码后的目标视频；
    通过与视频编码策略对应的视频解码策略对所述编码后的目标视频进行解码。
  12. 一种视频编码装置，所述装置包括：
    信息处理模块，配置为基于视频播放终端的网络带宽环境，确定网络带宽信息；
    所述信息处理模块，还配置为确定所述视频播放终端的解码算力信息；
    所述信息处理模块，还配置为获取视频播放环境中的播放缓存池的容量配置信息；
    所述信息处理模块，还配置为基于所述网络带宽信息、所述解码算力信息以及所述播放缓存池的容量配置信息中的至少之一，对视频播放的编码策略进行动态调整，得到与目标视频的播放环境相匹配的视频编码策略；
    信息编码模块，配置为通过所述视频编码策略对所述目标视频进行编码。
  13. 根据权利要求12所述的装置，其中，
    所述信息处理模块，还配置为当确定与所述目标视频的播放环境相匹配的视频编码策略为降低视频码率编码时，对所述目标视频的播放流畅度进行检测；
    所述信息处理模块，还配置为当检测到所述目标视频的播放流畅度小于流畅度阈值时，确定与所述目标视频的播放环境相匹配的视频编码策略为降低帧率编码以及视频码率编码。
  14. 一种视频解码装置，应用于如权利要求1-10所述视频编码方法所编码后的目标视频；
    所述装置包括：
    获取模块，配置为获取所述编码后的目标视频；
    信息解码模块，配置为通过与视频编码策略对应的视频解码策略对所述编码后的目标视频进行解码。
  15. 一种电子设备，所述电子设备包括：
    存储器，用于存储可执行指令；
    处理器，用于运行所述存储器存储的可执行指令时，实现权利要求1至10任一项所述的视频编码方法，或者权利要求11所述的视频解码方法。
  16. 一种计算机可读存储介质，存储有可执行指令，所述可执行指令被处理器执行时实现权利要求1至10任一项所述的视频编码方法，或者权利要求11所述的视频解码方法。
  17. 一种计算机程序产品，包括计算机程序或指令，所述计算机程序或指令被处理器执行时实现权利要求1至10任一项所述的视频编码方法，或者权利要求11所述的视频解码方法。
PCT/CN2021/128969 2020-11-13 2021-11-05 一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品 Ceased WO2022100522A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21891048.7A EP4192015A4 (en) 2020-11-13 2021-11-05 VIDEO ENCODING METHOD, VIDEO DECODING METHOD, APPARATUS, ELECTRONIC DEVICE, STORAGE MEDIUM, AND COMPUTER PROGRAM PRODUCT
US17/963,829 US11909984B2 (en) 2020-11-13 2022-10-11 Video encoding and decoding for cloud gaming

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011269523.8 2020-11-13
CN202011269523.8A CN112104879B (zh) 2020-11-13 2020-11-13 一种视频编码方法、装置、电子设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/963,829 Continuation US11909984B2 (en) 2020-11-13 2022-10-11 Video encoding and decoding for cloud gaming

Publications (1)

Publication Number Publication Date
WO2022100522A1 true WO2022100522A1 (zh) 2022-05-19

Family

ID=73785627

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/128969 Ceased WO2022100522A1 (zh) 2020-11-13 2021-11-05 一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品

Country Status (4)

Country Link
US (1) US11909984B2 (zh)
EP (1) EP4192015A4 (zh)
CN (1) CN112104879B (zh)
WO (1) WO2022100522A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116456122A (zh) * 2023-04-25 2023-07-18 苏州创意云网络科技有限公司 直播视频数据处理方法、装置和系统

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112104879B (zh) * 2020-11-13 2021-03-23 腾讯科技（深圳）有限公司 一种视频编码方法、装置、电子设备及存储介质
CN112631447B (zh) * 2020-12-24 2024-08-06 广州虎牙科技有限公司 鼠标同步处理方法、装置、电子设备及存储介质
CN112788235B (zh) * 2020-12-31 2022-01-28 深圳追一科技有限公司 图像处理方法、装置、终端设备及计算机可读取存储介质
CN112929700A (zh) * 2021-02-01 2021-06-08 福建多多云科技有限公司 一种基于云手机的视频流动态切换方法
CN114971663A (zh) * 2021-02-26 2022-08-30 腾讯科技（深圳）有限公司 基于多媒体信息处理方法、装置、电子设备及存储介质
CN113347466B (zh) * 2021-05-18 2022-12-20 深圳市腾讯网络信息技术有限公司 一种数据处理方法、装置和存储介质
CN113726759A (zh) * 2021-08-25 2021-11-30 烽火通信科技股份有限公司 一种码率自适应方法、装置、设备及可读存储介质
CN113747203B (zh) * 2021-09-01 2022-08-30 腾讯科技（深圳）有限公司 一种视频信息传输方法、装置、电子设备及存储介质
CN113856204B (zh) * 2021-09-28 2025-10-28 北京奇艺世纪科技有限公司 一种云游戏数据解码方法、装置、电子设备及存储介质
CN113923485A (zh) * 2021-09-30 2022-01-11 杭州雾联科技有限公司 一种调整码率的方法、装置、及计算机可读存储介质
CN113891083B (zh) * 2021-10-29 2025-04-29 咪咕视讯科技有限公司 边缘智能环境下的视频处理方法、装置及计算设备
CN114051143A (zh) * 2021-11-03 2022-02-15 麒麟软件有限公司 一种视频流编解码任务调度方法
CN114143407B (zh) * 2021-11-25 2024-07-16 厦门汉印电子技术有限公司 一种远程实时预览实现设备及方法、装置、可读存储介质
CN114157710B (zh) * 2021-12-02 2024-11-12 中国电信股份有限公司 通信策略配置方法、装置、存储介质及设备
CN114222194A (zh) * 2021-12-17 2022-03-22 浙江大华技术股份有限公司 一种视频码流调整方法、装置及系统
CN114416236B (zh) * 2021-12-28 2025-05-02 深信服科技股份有限公司 一种虚拟桌面数据的处理方法、装置、设备和介质
CN114338550B (zh) * 2022-03-14 2022-06-21 杭州子默网络科技有限公司 一种减低带宽的云电脑实时编码方法及系统
WO2023218614A1 (ja) * 2022-05-12 2023-11-16 日本電信電話株式会社 配信装置、配信方法及びプログラム
CN117475052A (zh) * 2022-07-22 2024-01-30 华为云计算技术有限公司 一种图像渲染的处理方法及其相关设备
CN115348451B (zh) * 2022-08-24 2025-05-27 腾讯科技（深圳）有限公司 一种编码处理方法、装置、服务器和介质
CN115550690B (zh) * 2022-12-02 2023-04-14 腾讯科技（深圳）有限公司 帧率调整方法、装置、设备及存储介质
CN116585718A (zh) * 2023-05-19 2023-08-15 成都奇侠互娱科技有限公司 一种新型网络游戏的实时互动系统及方法
CN116634177B (zh) * 2023-06-16 2024-02-20 北京行者无疆科技有限公司 一种基于hdmi通信设备的视频通信解码处理方法
CN117082276B (zh) * 2023-08-07 2026-01-13 中国电信股份有限公司技术创新中心 数据传输方法、装置、设备、介质和程序产品
CN117173781B (zh) * 2023-08-17 2026-01-06 上海鸣啸信息科技股份有限公司 一种基于骨架提取和图卷积网络的隔栏递物行为识别方法
CN118118628B (zh) * 2024-03-29 2025-03-11 北京积加科技有限公司 监控视频调整方法、装置、电子设备和计算机可读介质
CN119545109B (zh) * 2024-11-27 2026-03-10 杭州海康威视系统技术有限公司 视频播放方法、装置及设备
CN120812200B (zh) * 2025-09-12 2025-11-21 上海创米数联智能科技发展股份有限公司 存储系统、缓存调度方法及装置、视频调参方法及装置

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101707551A (zh) * 2009-10-30 2010-05-12 杭州华三通信技术有限公司 一种数据的传输方法及装置
CN101867802A (zh) * 2010-05-17 2010-10-20 上海海维工业控制有限公司 一种基于编码器和缓存的两级码率控制方法
CN102521279A (zh) * 2011-11-25 2012-06-27 Tcl集团股份有限公司 一种流媒体文件播放方法、系统及播放器
US20140056162A1 (en) * 2012-08-27 2014-02-27 Qualcomm Incorporated Device and method for adaptive rate multimedia communications on a wireless network
US20140082144A1 (en) * 2012-09-20 2014-03-20 General Instrument Corporation Use of a receive-window size advertised by a client to a content server to change a video stream bitrate streamed by the content server
CN104427402A (zh) * 2013-09-03 2015-03-18 中国科学院声学研究所 一种无线网络流媒体质量获取方法及系统
CN105376595A (zh) * 2015-12-17 2016-03-02 杭州雄迈信息技术有限公司 一种视频混合编码系统和方法
CN112104879A (zh) * 2020-11-13 2020-12-18 腾讯科技（深圳）有限公司 一种视频编码方法、装置、电子设备及存储介质

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4988346B2 (ja) * 2003-09-15 2012-08-01 ザ・ディレクティービー・グループ・インコーポレイテッド ビデオネットワークにおける適応トランスコーディング及び速度変換のための方法及びシステム
US8990351B2 (en) * 2011-04-20 2015-03-24 Mobitv, Inc. Real-time processing capability based quality adaptation
CN102387409A (zh) * 2011-12-07 2012-03-21 江苏远为科技有限公司 基于流媒体系统的视频播放方法
US9369513B2 (en) * 2013-04-12 2016-06-14 Futurewei Technologies, Inc. Utility-maximization framework for dynamic adaptive video streaming over hypertext transfer protocol in multiuser-multiple input multiple output long-term evolution networks
ITBA20130077A1 (it) * 2013-11-25 2015-05-26 Cicco Luca De Meccanismo per il controllo del bitrate di codifica in un sistema di video streaming adattivo basato su buffer di playout e sulla stima di banda.
CN105263066A (zh) * 2014-06-13 2016-01-20 珠海全志科技股份有限公司 移动设备视频流传输控制方法及系统
US10812559B2 (en) * 2017-01-18 2020-10-20 Amazon Technologies, Inc. Just-in-time variable adaptive encoding and delivery of media content
CN109462773A (zh) * 2018-08-31 2019-03-12 北京潘达互娱科技有限公司 一种推流方法、装置、电子设备及存储介质
US10694239B1 (en) * 2018-12-12 2020-06-23 Sling Media Pvt Ltd Systems, methods, and devices for optimizing streaming bitrate based on multiclient display profiles
CN110087081B (zh) * 2019-05-05 2021-08-06 腾讯科技（深圳）有限公司 视频编码方法、装置、服务器及存储介质
CN110418170B (zh) * 2019-07-03 2021-11-19 腾讯科技（深圳）有限公司 检测方法和装置、存储介质及电子装置
US11524230B2 (en) * 2019-10-01 2022-12-13 Sony Interactive Entertainment Inc. Encoder tuning to improve tradeoffs between latency and video quality in cloud gaming applications
US11731043B2 (en) * 2019-11-19 2023-08-22 Sony Interactive Entertainment Inc. Adaptive graphics for cloud gaming
CN111882626B (zh) * 2020-08-06 2023-07-14 腾讯科技（深圳）有限公司 图像处理方法、装置、服务器及介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101707551A (zh) * 2009-10-30 2010-05-12 杭州华三通信技术有限公司 一种数据的传输方法及装置
CN101867802A (zh) * 2010-05-17 2010-10-20 上海海维工业控制有限公司 一种基于编码器和缓存的两级码率控制方法
CN102521279A (zh) * 2011-11-25 2012-06-27 Tcl集团股份有限公司 一种流媒体文件播放方法、系统及播放器
US20140056162A1 (en) * 2012-08-27 2014-02-27 Qualcomm Incorporated Device and method for adaptive rate multimedia communications on a wireless network
US20140082144A1 (en) * 2012-09-20 2014-03-20 General Instrument Corporation Use of a receive-window size advertised by a client to a content server to change a video stream bitrate streamed by the content server
CN104427402A (zh) * 2013-09-03 2015-03-18 中国科学院声学研究所 一种无线网络流媒体质量获取方法及系统
CN105376595A (zh) * 2015-12-17 2016-03-02 杭州雄迈信息技术有限公司 一种视频混合编码系统和方法
CN112104879A (zh) * 2020-11-13 2020-12-18 腾讯科技（深圳）有限公司 一种视频编码方法、装置、电子设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4192015A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116456122A (zh) * 2023-04-25 2023-07-18 苏州创意云网络科技有限公司 直播视频数据处理方法、装置和系统

Also Published As

Publication number Publication date
CN112104879B (zh) 2021-03-23
EP4192015A4 (en) 2024-05-22
US11909984B2 (en) 2024-02-20
CN112104879A (zh) 2020-12-18
US20230036405A1 (en) 2023-02-02
EP4192015A1 (en) 2023-06-07

Similar Documents

Publication Publication Date Title
WO2022100522A1 (zh) 一种视频编码方法、视频解码方法、装置、电子设备、存储介质及计算机程序产品
US11711563B2 (en) Methods and systems for graphics rendering assistance by a multi-access server
US8988501B2 (en) Moving image distribution server, moving image playback apparatus, control method, and recording medium
US9052959B2 (en) Load balancing between general purpose processors and graphics processors
CN111882626A (zh) 图像处理方法、装置、服务器及介质
KR20220124286A (ko) 애플리케이션 계산들의 분배를 위한 방법들 및 장치들
US9868060B2 (en) Moving image distribution server, moving image reproduction apparatus, control method, and recording medium
US9375635B2 (en) System and method for improving the graphics performance of hosted applications
KR20220125813A (ko) 하이브리드 스트리밍
US12155835B2 (en) Video processing method, related apparatus, storage medium, and program product
CN117616762A (zh) 利用3d图形增强视频或外部环境
CN116916028A (zh) 使用虚拟分辨率调整的视频流缩放
US10237563B2 (en) System and method for controlling video encoding using content information
US20170221174A1 (en) Gpu data sniffing and 3d streaming system and method
US20250337926A1 (en) High resolution and low latency video streaming using partial frame sampling
HK40035766B (zh) 一种视频编码方法、装置、电子设备及存储介质
HK40035766A (zh) 一种视频编码方法、装置、电子设备及存储介质
WO2024010588A1 (en) Cloud-based gaming system for supporting legacy gaming applications with high frame rate streams
US20260091313A1 (en) Systems and methods for distributing processing tasks for providing a media asset
CN120849135B (zh) 一种unity应用云端渲染方法、系统及电子设备
Athrij et al. Dynamic Load Distribution in web-based AR
Liu et al. Multistream a cross-platform display sharing system using multiple video streams
TWI532005B (zh) An animation distribution server, an animation reproduction apparatus, a control method, a program, and a recording medium
CN115883811A (zh) 一种姿态矫正方法、装置和存储介质
HK40030902A (zh) 图像处理方法、装置、服务器及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21891048

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2021891048

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2021891048

Country of ref document: EP

Effective date: 20230227

NENP Non-entry into the national phase

Ref country code: DE