WO2024174655A1 - 渲染方法和电子设备 - Google Patents

渲染方法和电子设备 Download PDF

Info

Publication number
WO2024174655A1
WO2024174655A1 PCT/CN2023/135910 CN2023135910W WO2024174655A1 WO 2024174655 A1 WO2024174655 A1 WO 2024174655A1 CN 2023135910 W CN2023135910 W CN 2023135910W WO 2024174655 A1 WO2024174655 A1 WO 2024174655A1
Authority
WO
WIPO (PCT)
Prior art keywords
patch
absolute coordinates
frame
rendering
facet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/135910
Other languages
English (en)
French (fr)
Inventor
夏文超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honor Device Co Ltd
Original Assignee
Honor Device Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honor Device Co Ltd filed Critical Honor Device Co Ltd
Priority to EP23923818.1A priority Critical patent/EP4528656A4/en
Priority to CN202380079933.1A priority patent/CN120226050A/zh
Publication of WO2024174655A1 publication Critical patent/WO2024174655A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/80—Two-dimensional [2D] animation, e.g. using sprites
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/40—Filling planar surfaces by adding surface attributes, e.g. adding colours or textures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/04—Texture mapping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video

Definitions

  • the present application relates to the field of image processing, and in particular to a rendering method and an electronic device.
  • the current mainstream method is to insert a predicted frame between or after two adjacent real frames after rendering to reduce the rendering overhead.
  • the predicted frame is obtained by interpolating or moving two real frames according to pixels.
  • the embodiments of the present application provide a rendering method and an electronic device for reducing the jitter of two-dimensional surface display content in a predicted frame.
  • a rendering method comprising: obtaining rendering parameter information of a first facet in a first real frame, the absolute coordinates of the first facet, rendering parameter information of a second facet in a second real frame, and the absolute coordinates of the second facet, wherein the first real frame and the second real frame are adjacent real frames; the first facet and the second facet refer to two-dimensional faces whose shapes remain unchanged and whose relative positions of vertices remain unchanged; if the rendering parameter information of the first facet and the rendering parameter information of the second facet are the same, and the absolute coordinates of the first facet and the absolute coordinates of the second facet are within a certain range, then determining the absolute coordinates of a target facet according to the absolute coordinates of the first facet and the absolute coordinates of the second facet, the target facet being located in a predicted frame obtained based on the first real frame and the second real frame; and rendering the target facet according to the absolute coordinates of the target facet and the rendering parameter information.
  • the rendering method provided in the embodiment of the present application for two-dimensional patches whose shapes and relative positions of vertices remain unchanged (in essence, the displayed content remains unchanged), in the process of obtaining a predicted frame based on two real frames, only the moving position of such patches is predicted in the predicted frame, and the display content of such patches is not interpolated or moved according to pixels.
  • the display content of such patches in the predicted frame still uses the display content in the real frame, which can ensure that such patches are also very clear in the predicted frame, reducing the jitter of the display content of the two-dimensional patches in the predicted frame.
  • the rendering parameter information of the first patch or the rendering parameter information of the second patch includes at least one of the following information: texture coordinates mapped to each vertex of the patch, an identifier of the program to which the patch belongs, and the number of vertices of the patch.
  • the two patches are the same, which means that these parameters of the two patches are the same.
  • the vertex shader source code includes the attribute information of the face.
  • One way is to determine whether the patch is a two-dimensional patch with a constant shape and constant relative positions of vertices by using the attribute information of the patch.
  • determining the absolute coordinates of the target facet according to the absolute coordinates of the first facet and the absolute coordinates of the second facet includes: smoothing the absolute coordinates of the first facet and the absolute coordinates of the second facet to obtain the absolute coordinates of the target facet, so that the movement of the two-dimensional facet whose shape remains unchanged and the relative positions of the vertices remain unchanged in the predicted frame and the real frame is relatively smooth.
  • P (P1*W1+P2*W2)/(W1+W2), where P1 is the absolute coordinates of the first patch, P2 is the absolute coordinates of the second patch, and W1 and W2 are weight coefficients.
  • w1 and w2 are equal, and in this case the target patch is located between the first patch and the second patch.
  • P1 is the absolute coordinates of the first patch
  • P2 is the absolute coordinates of the second patch
  • W is the weight coefficient.
  • an electronic device comprising a processor and a memory, wherein instructions are stored in the memory, and when the processor executes the instructions, the method described in the first aspect and any embodiment thereof is executed.
  • a computer-readable storage medium comprising instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to the first aspect and any embodiment thereof.
  • a computer program product comprising instructions is provided.
  • the electronic device executes the method as described in the first aspect and any embodiment thereof.
  • a chip system including a processor for supporting an electronic device to implement the functions involved in the first aspect.
  • the device also includes an interface circuit, which can be used to receive signals from other devices (such as a memory) or send signals to other devices (such as a communication interface).
  • the chip system may include a chip and may also include other discrete devices.
  • FIG1 is a schematic diagram of a two-dimensional facet with unchanged shape and unchanged relative coordinates of vertices provided by an embodiment of the present application;
  • FIG2 is a schematic diagram of three-dimensional modeling by splicing multiple facets provided in an embodiment of the present application
  • FIG3 is a schematic diagram of a texture mapping provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of an interpolated frame and an extrapolated frame provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
  • FIG6 is a schematic diagram of a software architecture for running an electronic device provided in an embodiment of the present application.
  • FIG7 is a schematic diagram of a flow chart of a rendering method provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of a facet provided in an embodiment of the present application in which the shape, relative positions of vertices, and viewing angle do not change;
  • FIG. 9 is a diagram of a target face, a first face, and a second face when an interpolation frame is provided in an embodiment of the present application. Schematic diagram of absolute coordinate relationship
  • FIG10 is a schematic diagram of the absolute coordinate relationship between a target face, a first face, and a second face when an extrapolation frame is provided in an embodiment of the present application;
  • FIG11 is a schematic diagram of a flow chart of another rendering method provided in an embodiment of the present application.
  • FIG12 is a schematic diagram of a flow chart of another rendering method provided in an embodiment of the present application.
  • FIG13 is a schematic diagram of the structure of a chip system provided in an embodiment of the present application.
  • Coupled and “connection” involved in the embodiments of the present application should be understood in a broad sense. For example, they may refer to a direct physical connection, or an indirect connection achieved through electronic devices, such as a connection achieved through resistors, inductors, capacitors or other electronic devices.
  • Graphics rendering engine is used to model the objects in the image, then render the model, such as adding color, lighting, shadows, etc., and finally display it on the screen.
  • Graphics rendering engines include open graphics library (OpenGL), Metal, Vulkan, DirectX, etc. This application takes OpenGL as an example for explanation, but is not intended to be limited to this.
  • Open Graphics Library is a cross-language, cross-platform application programming interface (API) for rendering two-dimensional (2D) or three-dimensional (3D) vector graphics.
  • OpenGL is commonly used in computer-aided design (CAD), virtual reality (VR), augmented reality (AR), scientific visualization programs, and electronic game development.
  • CAD computer-aided design
  • VR virtual reality
  • AR augmented reality
  • scientific visualization programs and electronic game development.
  • the embodiments of this application take mobile games as an example, but are not intended to be limited to this.
  • a shader is an editable program used to implement image rendering. Unless otherwise specified, the shaders involved in this application are all described using OpenGL shaders as an example. OpenGL shaders include vertex shaders and fragment shaders. Vertex shaders are mainly responsible for the geometric relationship calculation of vertices, while fragment shaders are mainly responsible for the color calculation of patches (for patches, see the following description). Since shaders are editable, a variety of image effects can be achieved without being restricted by the fixed rendering pipeline of the graphics card.
  • the vertex shader is essentially a program code that is called at each vertex to process the data of each vertex (such as vertex coordinates, normals, colors, and texture coordinates), such as vertex coordinate transformation, texture coordinate generation and transformation, lighting, color material application, etc. In other words, each vertex must execute the program in the main function of the vertex shader.
  • Absolute coordinates of vertices In OpenGL, all objects are located in 3D space, but the screen is 2D pixels, so OpenGL converts the 3D coordinates of the vertices into 2D coordinates that adapt to the screen through various transformations. The vertex is projected onto the screen, and the vertex coordinates in screen space are the absolute coordinates of the vertex.
  • the OpenGL shader language provides the following predefined variables for the vertex shader: Graphics library position variable (gl_Position), which is used to represent the absolute coordinates of the vertex in screen space.
  • Absolute coordinates of a patch Since a patch includes multiple vertices, the coordinates of each vertex in the patch are maintained in the form of an array. Therefore, for the convenience of description, this application exemplarily uses the coordinates of the first vertex in the array in the patch as the absolute coordinates of the patch.
  • the surface of an object can be modeled in three dimensions by stitching together multiple basic shapes (such as triangles). Each basic shape is called a patch. The more patches there are, the more detailed the modeling is.
  • the fragment shader is the code that runs at each pixel on each patch. It is used to calculate and fill the color of a single pixel. It can also perform operations such as interpolation, texture mapping, texture application, and color summation on pixels.
  • Texture, texture coordinates and texture mapping To make the rendered object more realistic, on the one hand, more patches can be used to model the surface of the object. On the other hand, as shown in Figure 3, the coordinates of the vertices of the patches on the surface of the object can be mapped to the coordinates of the modeled surface of the object (texture coordinates), so that the corresponding texture is displayed at the patch. This process is texture mapping, and the texture coordinates are represented by the UV coordinate system. This not only makes the rendered model surface more detailed, but also is more convenient and efficient.
  • OpenGL shaders are written in OpenGL shading language (GLSL). Since shaders are independent programs, the shader source code can be stored as a separate script for application calls, or it can be called directly in memory as a string. The shader source code is compiled to obtain a shader object, and then multiple shader objects are linked together to form a shader program. The sender's shader declares an output type variable A, and the receiver's shader declares an input type variable A. When compiling and linking, the two shaders with the same name A will be linked together.
  • GLSL OpenGL shading language
  • a uniform variable is a special variable in a shader program that can be accessed and remains consistent throughout the execution of the shader program.
  • the viewport is a rectangular area in the window used to display graphics.
  • a predicted frame is inserted between the two real frames (the interpolated frame shown in FIG4 A) or after the two real frames (the interpolated frame shown in FIG4 B) to reduce rendering overhead.
  • the predicted frame is obtained by interpolating or moving the two real frames according to the pixels.
  • the type of facets is not distinguished. For two-dimensional facets whose shapes and relative positions of vertices in the image remain unchanged, if they are also interpolated or moved according to the pixels, these facets will be blurred in the predicted frame but clear in the real frame (i.e., jitter). Users can clearly perceive the difference between the predicted frame and the real frame, which greatly reduces the gaming experience.
  • the rendering method and electronic device provided in the embodiment of the present application, for two-dimensional facets whose shapes remain unchanged and whose relative positions of vertices remain unchanged (in essence, the displayed content remains unchanged), only predicts the position of the movement of such facets in the predicted frame, and does not interpolate or move the displayed content of such facets according to the pixels.
  • the displayed content in the predicted frame still uses the displayed content in the real frame, so that these facets can be ensured to be clear in the predicted frame, and the jitter of the displayed content of the two-dimensional facets in the predicted frame is reduced. It can be applied not only to electronic games (such as mobile games) but also to CAD, VR, AR, scientific visualization programs, etc.
  • the electronic device 101 provided in the embodiment of the present application may be a device with a display function, and the electronic device 101 may be mobile or fixed.
  • the electronic device 101 may be deployed on land (e.g. Such as indoors or outdoors, handheld or vehicle-mounted, etc.), it can also be deployed on the water surface (such as ships, etc.), and it can also be deployed in the air (such as airplanes, etc.).
  • the electronic device 101 can be called user equipment (UE), access terminal, terminal unit, subscriber unit, terminal station, mobile station (MS), mobile station, terminal agent or terminal device, etc.
  • the electronic device can be a mobile phone, a tablet computer, a laptop computer, a virtual reality device, an augmented reality device, etc.
  • the embodiments of the present application do not limit the specific type and structure of the electronic device. A possible structure of the electronic device is described below.
  • the electronic device 101 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a power management module 240, a battery 241, a wireless charging coil 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294 and a subscriber identification module (SIM) card interface 295, etc.
  • SIM subscriber identification module
  • the sensor module 280 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 101.
  • the electronic device 101 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently.
  • the components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 210 may include one or more processing units, for example, the processor 210 may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processing unit (CPU), an application processor (AP), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a baseband processor, and a neural-network processing unit (NPU).
  • Different processing units may be independent devices or integrated in one or more processors.
  • the processor 210 may be an application processor AP.
  • the processor 210 may be integrated in a system on chip (SoC). Alternatively, the processor 210 may be integrated in an integrated circuit (IC) chip.
  • the processor 210 may include an analog front end (AFE) and a micro-controller unit (MCU) in an IC chip.
  • the processor 210 may be the nerve center and command center of the electronic device 101.
  • the processor 210 may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.
  • the processor 210 may also be provided with a memory for storing instructions and data.
  • the memory in the processor 210 is a cache memory.
  • the memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.
  • processor 210 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, pulse code modulation (pulse code modulation, PCM) interface, universal asynchronous receiver/transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input/output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface and/or USB interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous receiver/transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • subscriber identity module subscriber identity module
  • SIM subscriber identity module
  • USB interface etc.
  • the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 101.
  • the electronic device 101 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
  • the power management module 240 is used to receive charging input from a charger.
  • the charger may be a wireless charger (such as a wireless charging base of the electronic device 101 or other device that can wirelessly charge the electronic device 101), or a wired charger.
  • the power management module 240 may receive charging input from a wired charger through the USB interface 230.
  • the power management module 240 may receive wireless charging input through the wireless charging coil 242 of the electronic device.
  • the power management module 240 can charge the battery 241 and also power the electronic device.
  • the power management module 240 receives input from the battery 241 and powers the processor 210, the internal memory 221, the external memory interface 220, the display screen 294, the camera 293, and the wireless communication module 260.
  • the power management module 240 can also be used to monitor parameters such as the battery capacity, the number of battery cycles, and the battery health status (leakage, impedance) of the battery 241. In some other embodiments, the power management module 240 can also be set in the processor 210.
  • the wireless communication function of the electronic device 101 can be implemented through the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, the modem processor, the baseband processor, and the like.
  • Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 101 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas.
  • antenna 1 can be reused as a diversity antenna for a wireless local area network.
  • the antenna can be used in combination with a tuning switch.
  • the mobile communication module 250 can provide solutions for wireless communications including 2G/3G/4G/5G, etc., applied to the electronic device 101.
  • the wireless communication module 260 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc., applied to the electronic device 101.
  • WLAN wireless local area networks
  • BT wireless fidelity
  • GNSS global navigation satellite system
  • FM frequency modulation
  • NFC near field communication technology
  • IR infrared technology
  • the electronic device 101 implements the display function through a GPU, a display screen 294, and an application processor.
  • the GPU is a microprocessor for image processing, which connects the display screen 294 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations and is used for GPU rendering.
  • the processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.
  • the display screen 294 is used to display images, videos, etc.
  • the display screen 294 includes a display panel.
  • the electronic device 101 may include 1 or N display screens 294, where N is a positive integer greater than 1.
  • the electronic device 101 can use the ISP, camera 293, video codec, GPU, display screen 294 and The application processor etc. realize the shooting function.
  • the ISP is used to process the data fed back by the camera 293.
  • the ISP can be set in the camera 293.
  • the camera 293 is used to capture static images or videos.
  • the electronic device 101 may include 1 or N cameras 293, where N is a positive integer greater than 1.
  • the camera of the embodiment of the present application includes a wide-angle camera and a main camera.
  • the external memory interface 220 can be used to connect an external memory card, such as a micro SanDisk (Micro SD) card, to expand the storage capacity of the electronic device 101.
  • the external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.
  • the internal memory 221 may be used to store computer executable program codes, which include instructions.
  • the processor 210 executes various functional applications and data processing of the electronic device 101 by running the instructions stored in the internal memory 221.
  • the internal memory 221 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.
  • the memory involved in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories.
  • the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.
  • the volatile memory may be a random access memory (RAM), which is used as an external cache.
  • RAM random access memory
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDR SDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM synchlink DRAM
  • DR RAM direct rambus RAM
  • the electronic device 101 can implement audio functions such as music playing and recording through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone interface 270D and the application processor.
  • the audio module 270 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal.
  • the audio module 270 can be set in the processor 210, or some functional modules of the audio module 270 are set in the processor 210.
  • the speaker 270A also known as the "speaker” is used to convert the audio electrical signal into a sound signal.
  • the receiver 270B also known as the "earpiece”, is used to convert the audio electrical signal into a sound signal.
  • the microphone 270C also known as the "microphone” or “microphone” is used to convert the sound signal into an electrical signal.
  • the electronic device 101 can be provided with at least one microphone 270C.
  • the headphone interface 270D is used to connect a wired headset.
  • the headphone interface 270D can be a USB interface 230, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.
  • the key 290 includes a power button, a volume button, etc.
  • the key 290 can be a mechanical key. It can also be a touch key.
  • the electronic device 101 can receive key input and generate key signal input related to the user settings and function control of the electronic device 101.
  • the motor 291 can generate a vibration prompt.
  • the motor 291 can be used for incoming call vibration prompts, It can also be used for touch vibration feedback.
  • the indicator 292 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc.
  • the SIM card interface 295 is used to connect the SIM card.
  • the SIM card can be connected to and separated from the electronic device 101 by inserting it into the SIM card interface 295 or pulling it out from the SIM card interface 295.
  • the electronic device 101 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.
  • the SIM card interface 295 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.
  • the electronic device 101 uses an embedded SIM (eSIM) card, and the eSIM card can be embedded in the electronic device 101 and cannot be separated from the electronic device 101.
  • eSIM embedded SIM
  • the processor 210 or GPU executes the rendering method provided in the embodiment of the present application by executing the program and instructions stored in the internal memory 221.
  • the program run by the processor 210 can be based on an operating system, such as an Android operating system. Apple (iOS) operating system Windows operating system, etc.
  • the program executed by the processor 210 is based on the Android operating system.
  • the programs run by the processor 210 are layered according to their functions, and may include an application layer, a framework layer, a local layer, a kernel layer, and a driver layer.
  • the application layer can include various applications that require GPU rendering, such as CAD, video games, AR/VR applications, scientific visualization programs, etc.
  • the framework layer is used to provide application programming interfaces (APIs) and system resource services to applications in the application layer, such as a GPU rendering API.
  • APIs application programming interfaces
  • system resource services such as a GPU rendering API.
  • the rendering method involved in the present application can be implemented in the framework layer on the CPU side or in the GPU.
  • the implementation in the framework layer on the CPU side is used for illustration, but it is not intended to be limited to this.
  • the local layer can provide a static link library or a dynamic link library for the API call of the framework layer, for example, providing a static link library or a dynamic link library to the GPU rendering API in the framework layer.
  • the kernel layer includes the operating system (OS) kernel, which manages the system's processes, memory, drivers, file systems, and network systems.
  • OS operating system
  • the driver layer is used to drive the hardware resources of the hardware layer.
  • the driver layer may include a GPU driver.
  • the rendering method provided in the embodiment of the present application includes:
  • the first real frame and the second real frame are adjacent real frames; the first patch and the second patch both refer to two-dimensional patches whose shapes and relative positions of vertices remain unchanged, and this type of patch is referred to as a stable patch in the following text.
  • the relative position of vertices refers to the relative position between vertices in the same patch.
  • the patches such as the text (XXX), icon (smiley face icon), and health bar on the head of the game character in the electronic game shown in FIG8, regardless of whether the displayed game character is zoomed in or out (A), turned around (B), moved (C), or changed posture (D), the shape, relative position of vertices, and viewing angle of these patches will not change, that is, the displayed content remains unchanged.
  • the absolute coordinates of the patch are described above and will not be repeated here.
  • the rendering parameter information of the patch includes at least one of the following information: the texture coordinates mapped to each vertex of the patch, the identifier of the program to which the patch belongs, and the number of vertices of the patch.
  • Step S101 may be performed during the process of rendering a real frame.
  • the target patch is located in a predicted frame obtained based on the first real frame and the second real frame, and describes the same content (eg, the same text, the same icon, the same health bar, etc.) as the first patch and the second patch.
  • the same content eg, the same text, the same icon, the same health bar, etc.
  • the distance threshold i.e.
  • the difference between the absolute coordinates P1 (i.e., gl_Position) of the first patch and the absolute coordinates P2 of the second patch is the movement of the second patch relative to the first patch. Therefore, the absolute coordinates of the first patch and the absolute coordinates of the second patch can be smoothed to obtain the absolute coordinates of the target patch in the predicted frame. This makes the movement of the two-dimensional patch with unchanged shape and unchanged relative position of vertices in the predicted frame and the real frame smoother.
  • W1 and W2 may be equal, in which case the target patch is located between the first patch and the second patch.
  • Step S102 may be performed during the process of rendering a real frame or may be performed during the process of rendering a predicted frame.
  • S103 Render the target face patch according to the absolute coordinates of the target face patch and the rendering parameter information.
  • the rendering parameter information of the first patch is the same as the rendering parameter information of the second patch
  • the rendering parameter information of the target patch also uses the same rendering parameter information.
  • the target patch is rendered according to the absolute coordinates of the target patch and the rendering parameter information, and is drawn to the main scene of the prediction frame. That is, at the absolute coordinates of the target patch, rendering is performed according to the texture coordinates mapped to each vertex of the patch.
  • Step S103 may be performed during the process of rendering a real frame or may be performed during the process of rendering a predicted frame.
  • the rendering method provided in the embodiment of the present application for two-dimensional patches whose shapes and relative positions of vertices remain unchanged (in essence, the displayed content remains unchanged), in the process of obtaining a predicted frame based on two real frames, only the moving position of such patches is predicted in the predicted frame, and the display content of such patches is not interpolated or moved according to pixels.
  • the display content of such patches in the predicted frame still uses the display content in the real frame, which can ensure that such patches are also very clear in the predicted frame, reducing the jitter of the display content of the two-dimensional patches in the predicted frame.
  • the rendering method includes:
  • the GPU rendering API of the framework layer loads the image rendering source code (including the vertex shader source code and the fragment source code) through the shader source code acquisition function (glShaderSource)
  • the real frame information includes but is not limited to the attribute information, absolute coordinates (i.e., gl_Position) and rendering parameter information of each face in the real frame.
  • the absolute coordinates (i.e., gl_Position) and rendering parameter information of the face are described above and will not be repeated here.
  • the vertex shader source code includes the attribute information of the patch, which is used to indicate whether the patch is a two-dimensional patch without depth information (i.e., it will not change with the viewing angle), with an unchanged shape and unchanged relative positions of vertices (i.e., the displayed content), i.e., the stable patch mentioned above. Then, when interpolating two adjacent real frames to obtain a predicted frame, for such a stable patch, it is not necessary to use the existing method of interpolating or moving pixels to obtain the patch in the predicted frame.
  • the attribute information of the face patch can be set to word to indicate that the face patch is drawn with text
  • the attribute information of the face patch can be set to icon to indicate that the face patch is drawn with an icon
  • the attribute information of the face patch can be set to blood to indicate that the face patch is drawn with a health bar, and so on, thereby indicating that the face patch is a stable face patch.
  • the stable patch is usually drawn after the main scene of the real frame or the predicted frame is drawn. Therefore, the patch indicated to be drawn after the main scene drawing is completed in the image rendering source code is the stable patch.
  • each patch is a stable patch based on the patch's attribute information. For stable patches, replace the source code of its vertex shader and compile; for other patches, compile directly.
  • the source code of the vertex shader of other facets in the image rendering source code except for the stable facets is not modified, and is still compiled according to the original logic before rendering.
  • the real frame is rendered according to the rendering method in the prior art, and the two adjacent real frames are interpolated according to the interpolation method in the prior art to obtain the predicted frame and render it, which will not be repeated in this application.
  • the rendering command is passed to the framework layer in the CPU, and then the CPU transmits a drawing initiation (DrawCall) command to the GPU to instruct the GPU to perform rendering (including real frames and predicted frames).
  • DrawCall drawing initiation
  • the real frame and the predicted frame are cached in different frame buffers (frame buffers).
  • frame buffers For example, the real frame is cached in the first frame buffer, and the predicted frame is cached in the second frame buffer.
  • the real frame and the predicted frame are cached in two different areas of the same frame buffer. For example, if the size of a single frame is A*B (A is the width and B is the height), the real frame can be cached in the left half of the 2A*B frame buffer, and the predicted frame can be cached in the right half of the frame buffer.
  • the rendering parameter information and absolute coordinates of each patch are stored in a data list.
  • the data list can be located in the shader storage buffer object (SSBO) of the GPU, and the real frame and the predicted frame are bound to different SSBOs respectively; alternatively, the data list can also be located in the CPU, which is not limited in this application.
  • SSBO shader storage buffer object
  • the prediction workflow sets the rendering flag of the frame through a uniform variable to indicate that the frame is a real frame, so that the rendering workflow can distinguish whether to render a real frame or a predicted frame when rendering.
  • the GPU rendering API renders each patch (including a stable patch) in the real frame through a patch drawing function (glDrawElements).
  • a patch drawing function glDrawElements
  • the viewport can also be set to indicate in which area (e.g., the left half area) the real frame is rendered.
  • the rendering and drawing of the stable patches, as well as the rendering and drawing of other patches can still be done in the original source code without modification.
  • the rendering and drawing of the stable patches can still be done in the original source code without modification.
  • they are drawn onto the main scene of the real frame, thus obtaining the real frame.
  • the predicted frame needs to be predicted.
  • the preset frame can still be obtained by interpolating or moving pixels in the prior art.
  • the rendering parameter information and absolute coordinates of the stable patch in the real frame are stored.
  • the rendering parameter information and absolute coordinates of the stable patch in the real frame can be stored in the data list of the real frame.
  • the target patch in the predicted frame is to be rendered later, if the predicted frame is to be rendered, it is necessary to first obtain stable patches describing the same content in two adjacent real frames. It is assumed that the two adjacent real frames are the first real frame and the second real frame, and the first patch in the first real frame and the second patch in the second real frame are both stable patches.
  • the original source code is run to calculate each vertex of the first face, and the rendering parameter information of the first face in the first real frame and the absolute coordinates of the first face are obtained and saved.
  • the original source code is run to calculate each vertex of the second face, and the rendering parameter information of the second face in the second real frame and the absolute coordinates of the second face are obtained and saved.
  • This step can obtain the rendering parameter information and absolute coordinates of the first facet in the first real frame, and the rendering parameter information and absolute coordinates of the second facet in the second real frame, which corresponds to the aforementioned step S101.
  • S205 Predict the absolute coordinates of the stable patch in the prediction frame.
  • the data list of the previous real frame is traversed to find the first facet that matches the second facet, so as to determine that the first facet and the second facet describe the same content (e.g., the same text, the same icon, the same health bar, etc.) in two adjacent real frames, which corresponds to the target facet in the predicted frame.
  • the rendering parameter information of the first facet is the same as the rendering parameter information of the second facet, and the absolute coordinates of the first facet and the absolute coordinates of the second facet are within a certain range, it can be determined that the first facet and the second facet describe the same content in two adjacent real frames, and the absolute coordinates of the target facet can be determined in the predicted frame.
  • the absolute coordinates of the target facet refer to the description of step S102, which will not be repeated here.
  • This step corresponds to step S102 described above.
  • the target surface is rendered according to the absolute coordinates of the target surface and the rendering parameter information.
  • the prediction workflow sets the rendering flag of the frame through a uniform variable (Uniform) to indicate that the frame is a predicted frame.
  • a uniform variable Uniform
  • the viewport can also be set to indicate which area the rendering is located in.
  • the predicted frame in the region (such as the right half region) is used to render the above stable patches in the predicted frame through the patch drawing function (glDrawElements). This enables the rendering workflow to distinguish whether to render the real frame or the predicted frame when rendering.
  • This step corresponds to step S103 described above.
  • the faces other than the stable face are still rendered in the original way and drawn to the main scene of the predicted frame. Then the rendered target face is overlaid and drawn on the main scene of the predicted frame to obtain a complete predicted frame.
  • the prediction workflow will draw the rendered stable patch onto the main scene of the prediction frame.
  • an embodiment of the present application also provides a chip system.
  • the chip system 60 includes at least one processor 601 and at least one interface circuit 602. At least one processor 601 and at least one interface circuit 602 can be interconnected via lines.
  • the processor 601 is used to support the electronic device to implement the various steps in the above method embodiments, such as the methods shown in Figures 7, 11, and 12, and at least one interface circuit 602 can be used to receive signals from other devices (such as memories), or to send signals to other devices (such as communication interfaces).
  • the chip system may include chips and may also include other discrete devices.
  • An embodiment of the present application also provides a computer-readable storage medium, which includes instructions.
  • the instructions When the instructions are executed on the above-mentioned electronic device, the electronic device executes the various steps in the above-mentioned method embodiment, such as executing the methods shown in Figures 7, 11, and 12.
  • An embodiment of the present application also provides a computer program product including instructions.
  • the instructions When the instructions are executed on the above-mentioned electronic device, the electronic device executes each step in the above-mentioned method embodiment, for example, executes the method shown in Figures 7, 11, and 12.
  • the size of the serial numbers of the above-mentioned processes does not mean the order of execution.
  • the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
  • modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
  • the disclosed systems, devices and methods can be implemented in other ways.
  • the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not performed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, which may be electrical, mechanical or other forms.
  • modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one device or distributed on multiple devices. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional module in each embodiment of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.
  • the computer-readable storage medium may be any available medium that a computer can access or may include one or more servers, data centers and other data storage devices that can be integrated with the medium.
  • the available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Image Generation (AREA)

Abstract

本申请公开了一种渲染方法和电子设备，用于降低预测帧中二维面片显示内容的抖动。渲染方法包括：获取第一真实帧中第一面片的渲染参数信息、第一面片的绝对坐标、第二真实帧中第二面片的渲染参数信息以及第二面片的绝对坐标；第一面片和第二面片指形状不变并且顶点的相对位置不变的二维面片；如果第一面片的渲染参数信息和第二面片的渲染参数信息相同，并且，第一面片的绝对坐标和第二面片的绝对坐标在一定范围内，则根据第一面片的绝对坐标和第二面片的绝对坐标确定目标面片的绝对坐标，目标面片位于基于第一真实帧和第二真实帧得到的预测帧；根据目标面片的绝对坐标以及渲染参数信息对目标面片进行渲染。

Description

渲染方法和电子设备
本申请要求于2023年2月21日提交国家知识产权局、申请号为202310203005.3、发明名称为“渲染方法和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理领域，尤其涉及一种渲染方法和电子设备。
背景技术
随着电子游戏的快速发展，游戏引擎的渲染流程越来越复杂，消耗资源也越来越大，目前主流方式是，在渲染完相邻两个真实帧后，在这两个真实帧之间或之后插入一个预测帧来降低渲染开销。现有技术中，通过对两个真实帧按照像素点进行插值或移动得到预测帧。
对于图像中形状不变并且顶点的相对坐标不变(即显示内容不变)的二维面片(patch)，例如图1所示的电子游戏中游戏人物头上的文字(XXX)、图标(笑脸图标)、血条等，如果也按照像素点进行插值或移动，会导致这些面片在预测帧中模糊而在真实帧中清晰(即出现抖动)，用户能明显察觉到预测帧与真实帧之间的差异，极大的降低了用户体验。
发明内容
本申请实施例提供一种渲染方法和电子设备，用于降低预测帧中二维面片显示内容的抖动。
为达到上述目的，本申请的实施例采用如下技术方案：
第一方面，提供了一种渲染方法，包括：获取第一真实帧中第一面片的渲染参数信息、第一面片的绝对坐标、第二真实帧中第二面片的渲染参数信息以及第二面片的绝对坐标，第一真实帧和第二真实帧为相邻真实帧；第一面片和第二面片指形状不变并且顶点的相对位置不变的二维面片；如果第一面片的渲染参数信息和第二面片的渲染参数信息相同，并且，第一面片的绝对坐标和第二面片的绝对坐标在一定范围内，则根据第一面片的绝对坐标和第二面片的绝对坐标确定目标面片的绝对坐标，目标面片位于基于第一真实帧和第二真实帧得到的预测帧；根据目标面片的绝对坐标以及渲染参数信息对目标面片进行渲染。
本申请实施例提供的渲染方法，对于形状不变并且顶点的相对位置不变(实质就是显示内容不变)的二维面片，在根据两个真实帧得到预测帧的过程中，在预测帧中仅对这类面片移动的位置进行预测，并不对这类面片的显示内容按照像素点进行插值或移动，这类面片在预测帧中显示内容仍采用真实帧中显示内容，这样可以保证这类面片在预测帧中同样很清晰，降低了预测帧中二维面片显示内容的抖动。
在一种可能的实施方式中，第一面片的渲染参数信息或第二面片的渲染参数信息包括以下信息的至少一项：面片的各个顶点映射的纹理坐标、面片所属程序的标识、面片的顶点数。两个面片相同，指的是两个面片的这些参数都相同。
在一种可能的实施方式中，顶点着色器源码中包括面片的属性信息，属性信息用 于指示面片为形状不变并且顶点的相对位置不变的二维面片。一种方式是通过面片的属性信息判断面片是否为形状不变并且顶点的相对位置不变的二维面片。
在一种可能的实施方式中，根据第一面片的绝对坐标和第二面片的绝对坐标确定目标面片的绝对坐标，包括：对第一面片的绝对坐标和第二面片的绝对坐标进行平滑处理，得到目标面片的绝对坐标。使得预测帧和真实帧中，形状不变并且顶点的相对位置不变的二维面片的移动比较平滑。
在一种可能的实施方式中，当预测帧位于第一真实帧和第二真实帧之间时，目标面片的绝对坐标P通过以下公式得到：P＝(P1*W1+P2*W2)/(W1+W2)，其中，P1为第一面片的绝对坐标，P2为第二面片的绝对坐标，W1和W2为权重系数。该实施方式适用于内插帧。
在一种可能的实施方式中，w1和w2相等，此时目标面片位于第一面片和第二面片的中间。
在一种可能的实施方式中，当预测帧位于第一真实帧和第二真实帧之后时，目标面片的绝对坐标P通过以下公式得到：P＝P2+(P2-P1)*W，其中，P1为第一面片的绝对坐标，P2为第二面片的绝对坐标，W为权重系数。该实施方式适用于外插帧。
第二方面，提供了一种电子设备，包括处理器和存储器，存储器中存储指令，当处理器执行指令时，如第一方面及其任一实施方式所述的方法被执行。
第三方面，提供了一种计算机可读存储介质，包括指令，当指令在电子设备上运行时，使得电子设备执行如第一方面及其任一实施方式所述的方法。
第四方面，提供了一种包括指令的计算机程序产品，当指令在上述电子设备上运行时，使得该电子设备执行如第一方面及其任一实施方式所述的方法。
第五方面，提供了一种芯片系统，该芯片系统包括处理器，用于支持电子设备实现上述第一方面中所涉及的功能。在一种可能的设计中，该装置还包括接口电路，接口电路可用于从其它装置(例如存储器)接收信号，或者，向其它装置(例如通信接口)发送信号。该芯片系统可以包括芯片，还可以包括其他分立器件。
第二方面至第五方面的技术效果参照第一方面及其任一实施方式的技术效果，在此不再重复。
附图说明
图1为本申请实施例提供的一种形状不变并且顶点的相对坐标不变的二维面片的示意图；
图2为本申请实施例提供的一种通过多个面片拼接进行三维建模的示意图；
图3为本申请实施例提供的一种纹理映射的示意图；
图4为本申请实施例提供的一种内插帧和外插帧的示意图；
图5为本申请实施例提供的一种电子设备的结构示意图；
图6为本申请实施例提供的一种电子设备运行软件架构的示意图；
图7为本申请实施例提供的一种渲染方法的流程示意图；
图8为本申请实施例提供的一种面片的形状、顶点相对位置、视角都不会发生变化的示意图；
图9为本申请实施例提供的一种内插帧时，目标面片、第一面片、第二面片之间 绝对坐标关系的示意图；
图10为本申请实施例提供的一种外插帧时，目标面片、第一面片、第二面片之间绝对坐标关系的示意图；
图11为本申请实施例提供的另一种渲染方法的流程示意图；
图12为本申请实施例提供的再一种渲染方法的流程示意图；
图13为本申请实施例提供的一种芯片系统的结构示意图。
具体实施方式
首先对本申请涉及的一些概念进行描述。
本申请实施例涉及的术语“第一”、“第二”等仅用于区分同一类型特征的目的，不能理解为用于指示相对重要性、数量、顺序等。
本申请实施例涉及的术语“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
本申请实施例涉及的术语“耦合”、“连接”应做广义理解，例如，可以指物理上的直接连接，也可以指通过电子器件实现的间接连接，例如通过电阻、电感、电容或其他电子器件实现的连接。
图形渲染引擎：图形渲染引擎用于对图中的物体进行建模得到模型，然后对模型进行渲染，例添加颜色、光照、阴影等，最后显示在屏幕上。目前主流的几种图形渲染引擎包括开放图形库(open graphics library，OpenGL)、Metal、Vulkan、DirectX等，本申请以OpenGL为例进行说明，但并不意在限定于此。
开放图形库(open graphics library，OpenGL)是用于渲染二维(two dimensional，2D)或三维(three dimensional，3D)矢量图形的跨语言、跨平台的应用程序编程接口(application programming interface，API)。OpenGL常用于计算机辅助设计(computer aided design，CAD)、虚拟现实(virtual reality，VR)、增强现实(augmented reality，AR)、科学可视化程序和电子游戏开发等。本申请实施例以手机游戏为例，但并不意在限定于此。
着色器(shader)是用来实现图像渲染的可编辑程序，如无特别说明，本申请涉及的着色器均以OpenGL着色器为例进行说明。OpenGL着色器包括顶点着色器(vertex shader)和片段着色器(fragment shader)，顶点着色器主要负责顶点(vertex)的几何关系运算，片段着色器主要负责面片(关于面片见后文描述)颜色的计算。由于着色器具有可编辑性，因此可以实现各种各样的图像效果而不用受显卡的固定渲染管线的限制。
顶点着色器本质是一个程序代码，在每个顶点处被调用以处理每个顶点的数据(例如顶点坐标，法线、颜色和纹理坐标)，例如可以进行顶点坐标变换、纹理坐标生成和变换、灯光、彩色材料应用等。也就是说，每个顶点都要执行一遍顶点着色器的主(main)函数中的程序。
顶点的绝对坐标：在OpenGL中，所有物体都位于3D空间里的，但是屏幕都是2D像素点，所以OpenGL会将顶点的3D坐标通过多种变换转变为适应屏幕的2D坐 标，最后投射到屏幕上去，顶点在屏幕空间的坐标即为顶点的绝对坐标。OpenGL着色器语言为顶点着色器提供以下预定义变量：图形库位置变量(gl_Position)，用于表示顶点在屏幕空间的绝对坐标。
面片的绝对坐标：由于面片包括多个顶点，面片中各个顶点的坐标是以数组的形式进行维护的，所以本申请为了方便描述，示例性的以面片中该数组的首个顶点的坐标做为该面片的绝对坐标。
如图2所示，物体的表面可以通过多个基本形状(例如三角形)拼接而成来进行三维建模，每个基本形状被称为一个面片，面片数量越多则建模越精细。片段着色器是在每个面片上的每个像素处运行的代码，用于计算并填充单个像素的颜色，还可以对像素进行内插值、纹理映射、纹理应用、颜色求和等操作。
纹理、纹理坐标和纹理映射：为使渲染的物体更加逼真，一方面，可以通过采用更多数量的面片来对物体表面进行建模，另一方面，如图3所示，可以将物体表面的面片的顶点的坐标映射至建模好的物体表面的坐标(纹即理坐标)，从而在面片处显示对应的纹理，这个过程即为纹理映射，纹理坐标采用UV坐标系来表示。这样不仅能使渲染的模型表面细节更丰富，而且比较方便高效。
OpenGL着色器以OpenGL着色器语言(OpenGL shading language，GLSL)编写。由于着色器是独立程序，可以将着色器源码存储为单独的脚本供应用程序调用，或者，也可以直接以字符串的形式在内存中调用。着色器的源代码经过编译后得到着色器对象，然后多个着色器对象链接在一起形成一个着色器程序。发送方的着色器声明一个输出类型的变量A，接收方的着色器声明一个输入类型的变量A，当进行编译和链接时，具有相同名称A的两个着色器就会被链接在一起。
统一变量(Uniform)是着色器程序中一种特殊的变量，在整个着色器程序的执行过程中，都可以访问并保持一致。
视口(Viewport)是窗口中用来显示图形的一块矩形区域。
插帧：如图4所示，现有技术中，通过在渲染完相邻两个真实帧后，在这两个真实帧之间(图4中A所示的内插帧)或这两个真实帧之后(图4中B所示的外插帧)插入一个预测帧来降低渲染开销。预测帧是通过对这两个真实帧按照像素点进行插值或移动得到，在这个过程中并不会区分面片的类型，对于图像中形状不变并且顶点的相对位置不变的二维面片，如果也按照像素点进行插值或移动，会导致这些面片在预测帧中模糊而在真实帧中清晰(即出现抖动)，用户能明显察觉到预测帧与真实帧之间的差异，极大的降低了游戏体验。
本申请实施例提供的渲染方法和电子设备，对于形状不变并且顶点的相对位置不变(实质就是显示内容不变)的二维面片，在预测帧中仅对这类面片移动的位置进行预测，并不对这类面片的显示内容按照像素点进行插值或移动，在预测帧中显示内容仍采用真实帧中显示内容，这样可以保证这些面片在预测帧中同样很清晰，降低了预测帧中二维面片显示内容的抖动。不仅可以应用于电子游戏(例如手机游戏)中还可以应用于CAD、VR、AR、科学可视化程序等。
如图5所示，本申请实施例提供的电子设备101可以是一种具有显示功能的设备，电子设备101可以是移动的，也可以是固定的。电子设备101可以部署在陆地上(例 如室内或室外、手持或车载等)，也可以部署在水面上(例如轮船等)，还可以部署在空中(例如飞机等)。该电子设备101可以称为用户设备(user equipment，UE)、接入终端、终端单元、用户单元(subscriber unit)、终端站、移动站(mobile station，MS)、移动台、终端代理或终端装置等。例如，该电子设备可以是手机、平板电脑、笔记本电脑、虚拟现实设备、增强现实设备等。本申请实施例对电子设备的具体类型和结构等不作限定。下面对电子设备的一种可能结构进行说明。
以电子设备为手机为例，该电子设备101可以包括处理器210、外部存储器接口220、内部存储器221、通用串行总线(universal serial bus，USB)接口230、电源管理模块240、电池241、无线充电线圈242、天线1、天线2、移动通信模块250、无线通信模块260、音频模块270、扬声器270A、受话器270B、麦克风270C、耳机接口270D、传感器模块280、按键290、马达291、指示器292、摄像头293、显示屏294以及用户标识模块(subscriber identification module，SIM)卡接口295等。
其中，传感器模块280可以包括压力传感器、陀螺仪传感器、气压传感器、磁传感器、加速度传感器、距离传感器、接近光传感器、指纹传感器、温度传感器、触摸传感器、环境光传感器、骨传导传感器等。
可以理解的是，本发明实施例示意的结构并不构成对电子设备101的具体限定。在本申请另一些实施例中，电子设备101可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器210可以包括一个或多个处理单元，例如：处理器210可以为现场可编程门阵列(field programmable gate array，FPGA)、专用集成电路(application specific integrated circuit，ASIC)、片上系统(system on chip，SoC)、中央处理单元(central processing unit，CPU)、应用处理器(application processor，AP)、网络处理器(network processor，NP)、数字信号处理器(digital signal processor，DSP)、微控制单元(micro controller unit，MCU)、可编程逻辑器件(programmable logic device，PLD)、调制解调处理器、图形处理器(graphics processing unit，GPU)、图像信号处理器(image signal processor，ISP)、控制器、视频编解码器、基带处理器以及神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。例如，处理器210可以是应用处理器AP。或者，上述处理器210可以集成在片上系统(system on chip，SoC)中。或者，上述处理器210可以集成在集成电路(integrated circuit，IC)芯片中。该处理器210可以包括IC芯片中的模拟前端(analog front end，AFE)和微控制单元(micro-controller unit，MCU)。处理器210可以是电子设备101的神经中枢和指挥中心。处理器210可以根据指令操作码和时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器210中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器210中的存储器为高速缓冲存储器。该存储器可以保存处理器210刚用过或循环使用的指令或数据。如果处理器210需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器210的等待时间，因而提高了系统的效率。
在一些实施例中，处理器210可以包括一个或多个接口。接口可以包括集成电路 (inter-integrated circuit，I2C)接口、集成电路内置音频(inter-integrated circuit sound，I2S)接口、脉冲编码调制(pulse code modulation，PCM)接口、通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口、移动产业处理器接口(mobile industry processor interface，MIPI)、通用输入输出(general-purpose input/output，GPIO)接口、用户标识模块(subscriber identity module，SIM)接口和/或USB接口等。
可以理解的是，本发明实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备101的结构限定。在本申请另一些实施例中，电子设备101也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
电源管理模块240用于从充电器接收充电输入。其中，充电器可以是无线充电器(如电子设备101的无线充电底座或者其他可以为电子设备101无线充电的设备)，也可以是有线充电器。例如，电源管理模块240可以通过USB接口230接收有线充电器的充电输入。电源管理模块240可以通过电子设备的无线充电线圈242接收无线充电输入。
其中，电源管理模块240为电池241充电的同时，还可以为电子设备供电。电源管理模块240接收电池241的输入，为处理器210、内部存储器221、外部存储器接口220、显示屏294、摄像头293和无线通信模块260等供电。电源管理模块240还可以用于监测电池241的电池容量、电池循环次数、电池健康状态(漏电、阻抗)等参数。在其他一些实施例中，电源管理模块240也可以设置于处理器210中。
电子设备101的无线通信功能可以通过天线1、天线2、移动通信模块250、无线通信模块260、调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备101中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块250可以提供应用在电子设备101上的包括2G/3G/4G/5G等无线通信的解决方案。无线通信模块260可以提供应用在电子设备101上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)、蓝牙(bluetooth，BT)、全球导航卫星系统(global navigation satellite system，GNSS)、调频(frequency modulation，FM)、近距离无线通信技术(near field communication，NFC)、红外技术(infrared，IR)等无线通信的解决方案。在一些实施例中，电子设备101的天线1和移动通信模块250耦合，天线2和无线通信模块260耦合，使得电子设备101可以通过无线通信技术与网络以及其他设备通信。
电子设备101通过GPU、显示屏294以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏294和应用处理器。GPU用于执行数学和几何计算，用于GPU渲染。处理器210可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏294用于显示图像，视频等。显示屏294包括显示面板。在一些实施例中，电子设备101可以包括1个或N个显示屏294，N为大于1的正整数。
电子设备101可以通过ISP、摄像头293、视频编解码器、GPU、显示屏294以及 应用处理器等实现拍摄功能。ISP用于处理摄像头293反馈的数据。在一些实施例中，ISP可以设置在摄像头293中。摄像头293用于捕获静态图像或视频。在一些实施例中，电子设备101可以包括1个或N个摄像头293，N为大于1的正整数。示例性的，本申请实施例的摄像头包括广角摄像头和主摄像头。
外部存储器接口220可以用于连接外部存储卡，例如微闪迪(micro SanDisk，Micro SD)卡，实现扩展电子设备101的存储能力。外部存储卡通过外部存储器接口220与处理器210通信，实现数据存储功能。例如将音乐，视频等文件保存在外部存储卡中。
内部存储器221可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。处理器210通过运行存储在内部存储器221的指令，从而执行电子设备101的各种功能应用以及数据处理。此外，内部存储器221可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、通用闪存存储器(universal flash storage，UFS)等。
本申请实施例涉及的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic RAM，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。应注意，本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
电子设备101可以通过音频模块270、扬声器270A、受话器270B、麦克风270C、耳机接口270D以及应用处理器等实现音频功能。例如音乐播放，录音等。
音频模块270用于将数字音频信息转换成模拟音频信号输出，也用于将模拟音频输入转换为数字音频信号。在一些实施例中，音频模块270可以设置于处理器210中，或将音频模块270的部分功能模块设置于处理器210中。扬声器270A，也称“喇叭”，用于将音频电信号转换为声音信号。受话器270B，也称“听筒”，用于将音频电信号转换成声音信号。麦克风270C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。电子设备101可以设置至少一个麦克风270C。耳机接口270D用于连接有线耳机。耳机接口270D可以是USB接口230，也可以是3.5mm的开放移动终端平台(open mobile terminal platform，OMTP)标准接口，美国蜂窝电信工业协会(cellular telecommunications industry association of the USA，CTIA)标准接口。
按键290包括开机键、音量键等。按键290可以是机械按键。也可以是触摸式按键。电子设备101可以接收按键输入，产生与电子设备101的用户设置以及功能控制有关的键信号输入。马达291可以产生振动提示。马达291可以用于来电振动提示， 也可以用于触摸振动反馈。指示器292可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息、未接来电、通知等。SIM卡接口295用于连接SIM卡。SIM卡可以通过插入SIM卡接口295，或从SIM卡接口295拔出，实现和电子设备101的接触和分离。电子设备101可以支持1个或N个SIM卡接口，N为大于1的正整数。SIM卡接口295可以支持纳SIN(Nano SIM)卡、微SIM(Micro SIM)卡、SIM卡等。在一些实施例中，电子设备101采用嵌入式(embedded SIM，eSIM)卡，eSIM卡可以嵌在电子设备101中，不能和电子设备101分离。
处理器210或GPU通过执行内部存储器221中存储的程序、指令来执行本申请实施例提供的渲染方法。处理器210运行的程序可以基于操作系统，例如安卓(Android)操作系统苹果(iOS)操作系统视窗(Windows)操作系统等。
如图6所示，以处理器210运行的程序基于安卓操作系统为例，处理器210运行的程序按照功能进行分层，可以包括应用程序层、框架层、本地层、内核层、驱动层。
应用程序层可以包括各种需要进行GPU渲染的应用程序，例如CAD、电子游戏、AR/VR应用、科学可视化程序等。
框架层用于向应用程序层中的应用程序提供应用程序编程接口(application programming interface，API)和系统资源服务，例如GPU渲染API，本申请涉及的渲染方法可以在CPU侧的框架层来实现或者可以在GPU中实现，示例性的，以在CPU侧的框架层来实现为例进行说明，但并不意在限定于此。
本地层可以提供静态链接库或动态链接库，供框架层的API调用。例如向框架层中的GPU渲染API提供静态链接库或动态链接库。
内核层包括操作系统(operation system，OS)内核(kernel)。操作系统内核用于管理系统的进程、内存、驱动程序、文件系统和网络系统。
驱动层用于驱动硬件层的硬件资源。驱动层中可以包括GPU驱动。
如图7所示，对于形状不变并且顶点的相对位置不变(实质就是显示内容不变)的二维面片，本申请实施例提供的渲染方法包括：
S101、获取第一真实帧中第一面片的渲染参数信息、第一面片的绝对坐标、第二真实帧中第二面片的渲染参数信息以及第二面片的绝对坐标。
第一真实帧和第二真实帧为相邻真实帧；第一面片和第二面片均指形状不变并且顶点的相对位置不变的二维面片，后文对这一类面片简称为稳定面片。其中，顶点相对位置指同一面片中各顶点之间的相对位置。例如，对于图8所示的电子游戏中游戏人物头上的文字(XXX)、图标(笑脸图标)、血条等面片，无论显示的游戏人物进行近远景缩放(A)、转身(B)、移动(C)还是改变姿态(D)，这些面片的形状、顶点相对位置、视角都不会发生变化，即显示内容不变。
关于面片的绝对坐标见前文描述，在此不再赘述。面片的渲染参数信息包括以下信息的至少一项：面片的各个顶点映射的纹理坐标、面片所属程序的标识、面片的顶点数。
步骤S101可以在渲染真实帧的过程中执行。
S102、如果第一面片的渲染参数信息和第二面片的渲染参数信息相同，并且，第 一面片的绝对坐标和第二面片的绝对坐标在一定范围内，则根据第一面片的绝对坐标和第二面片的绝对坐标确定目标面片的绝对坐标。
目标面片位于基于第一真实帧和第二真实帧得到的预测帧中，与第一面片和第二面片描述的是同一内容(例如同一文字、同一图标、同一血条等)。
首先，确定第一面片和第二面片在相邻两个真实帧中描述的同一内容。如果第一面片中的各个顶点映射的纹理坐标与第二面片中的各个顶点映射的纹理坐标相同，第一面片所属程序的标识与第二面片所属程序的标识相同，第一面片的顶点数与第二面片的顶点数相同，在此基础上，如果第一面片的绝对坐标P1与第二面片的绝对坐标P2之间的差值小于距离阈值(即|P1-P2|<Th)，则可以确定第一面片和第二面片在相邻两个真实帧中描述的同一内容。
其次，第一面片的绝对坐标P1(即gl_Position)与第二面片的绝对坐标P2之间的差值即为第二面片相对于第一面片的移动量。因此可以对第一面片的绝对坐标和第二面片的绝对坐标进行平滑处理，得到预测帧中目标面片的绝对坐标。使得预测帧和真实帧中，形状不变并且顶点的相对位置不变的二维面片的移动比较平滑。
对于内插帧(即预测帧位于第一真实帧与第二真实帧之间)来说，如图9所示，目标面片的绝对坐标P可以通过以下公式得到：P＝(P1*W1+P2*W2)/(W1+W2)，其中，P1为第一面片的绝对坐标，P2为第二面片的绝对坐标，W1和W2为权重系数。特别地，W1和W2可以相等，此时目标面片位于第一面片和第二面片的中间。
对于外插帧(即预测帧位于第一真实帧与第二真实帧之后)来说，如图10所示，目标面片的绝对坐标P可以通过以下公式得到：P＝P2+(P2-P1)*W，其中，P1为第一面片的绝对坐标，P2为第二面片的绝对坐标，W为权重系数。
步骤S102可以在渲染真实帧的过程中执行或者可以在渲染预测帧的过程中执行。
S103、根据目标面片的绝对坐标以及渲染参数信息对目标面片进行渲染。
其中，由于第一面片的渲染参数信息和第二面片的渲染参数信息相同，所以目标面片的渲染参数信息也采用相同的渲染参数信息。在对目标面片进行渲染时，根据目标面片的绝对坐标以及渲染参数信息对目标面片进行渲染，并绘制到预测帧的主场景上。也就是说，在目标面片的绝对坐标处，根据面片的各个顶点映射的纹理坐标进行渲染。
步骤S103可以在渲染真实帧的过程中执行或者可以在渲染预测帧的过程中执行。
本申请实施例提供的渲染方法，对于形状不变并且顶点的相对位置不变(实质就是显示内容不变)的二维面片，在根据两个真实帧得到预测帧的过程中，在预测帧中仅对这类面片移动的位置进行预测，并不对这类面片的显示内容按照像素点进行插值或移动，这类面片在预测帧中显示内容仍采用真实帧中显示内容，这样可以保证这类面片在预测帧中同样很清晰，降低了预测帧中二维面片显示内容的抖动。
下面结合真实帧和预测帧完整的渲染流程对上述渲染方法进行说明，如图11所示，该渲染方法包括：
S201、获取图像渲染源码。
如图12中S301所示，对于渲染工作流来说，框架层的GPU渲染API通过着色器源码获取函数(glShaderSource)加载图像渲染源码(其中包括顶点着色器源码和片 段着色器源码)，并从中获取并保存真实帧的信息。真实帧的信息包括但不限于真实帧中各面片的属性信息、绝对坐标(即gl_Position)和渲染参数信息。关于面片的绝对坐标(即gl_Position)和渲染参数信息参照前文描述，在此不再赘述。
顶点着色器源码中包括面片的属性信息，用于指示本面片是否为没有深度信息(即不会随视角发生变化)、形状不变并且顶点的相对位置不变(即显示内容不变)的二维面片，即前文所述的稳定面片。那么在对相邻两个真实帧进行插帧得到预测帧时，对于这类稳定面片，就没必要采用现有技术中按照像素点进行插值或移动的方式来得到预测帧中的面片。
在一种实施方式中，图像渲染源码中可以通过对面片的属性信息设置为word来表示面片绘制的是文字，通过对面片的属性信息设置为icon来表示面片绘制的是图标，通过对面片的属性信息设置为blood表示面片绘制的是血条等等，从而指示该面片为稳定面片。
在另一种实施方式中，稳定面片通常是在真实帧或预测帧的主场景绘制完成后才绘制的，所以图像渲染源码中指示完成主场景绘制后，再指示绘制的面片即为稳定面片。
S202、替换稳定面片的顶点着色器的源码，并进行编译。
如图12中S302所示，对于预测工作流来说，先通过面片的属性信息判断各个面片是否为稳定面片，对于稳定面片，替换其顶点着色器的源码，并进行编译；对于其他面片则直接进行编译。
需要说明的是，对于图像渲染源码中除了稳定面片以外其他面片的顶点着色器的源码则不进行修改，仍按照原来的逻辑进行编译后再渲染。例如，按照现有技术中的渲染方式对真实帧进行渲染，按照现有技术中的插帧方式对相邻两个真实帧进行插帧得到预测帧并进行渲染，本申请在此不再赘述。
如图12中S303所示，CPU中的应用程序(例如电子游戏的引擎)启动后向CPU中的框架层传递渲染命令，然后CPU向GPU传输绘画发起(DrawCall)命令，以指示GPU进行渲染(包括真实帧和预测帧)。
S203、如果对真实帧进行渲染，则对真实帧中所有面片进行渲染后绘制到真实帧的主场景上，即得到真实帧。
将真实帧和预测帧缓存到不同的帧缓存(frame buffer)中，示例性的，真实帧缓存到第一帧缓存中，预测帧缓存到第二帧缓存中。或者，将真实帧和预测帧缓存到同一个帧缓存的两个不同区域，例如单个帧的尺寸为A*B(A为宽度B为高度)，则可以将真实帧缓存到2A*B的帧缓存中的左半部分区域，将预测帧缓存到帧缓存中的右半部分区域。在数据列表中存储各个面片的渲染参数信息和绝对坐标等信息，数据列表可以位于GPU的着色器存储缓冲区对象(shader storage buffer object，SSBO)中，并且真实帧和预测帧分别绑定不同的SSBO；或者，数据列表也可以位于CPU中，本申请不作限定。
如图12中S304所示，对于真实帧来说，预测工作流会通过统一变量(Uniform)设置该帧的渲染标志位，以指示该帧为真实帧。使得渲染工作流在进行渲染时能够区分是对真实帧渲染还是对预测帧进行渲染。
如图12中S305所示，对于渲染工作流来说，GPU渲染API通过面片绘制函数(glDrawElements)来渲染真实帧中的各个面片(包括稳定面片)。可选的，当真实帧和预测帧缓存到同一帧缓存的两个不同区域时，还可以通过设置视口(Viewport)来指示渲染位于哪一区域(例如左半部分区域)中的真实帧。
在真实帧中，对于稳定面片的渲染和绘制，以及，对其他面片的渲染和绘制，可以仍按照原来源码的方式，对此不作修改。或者，如图12中S307所示，在对真实帧中所有稳定面片进行渲染完成之后，绘制到真实帧的主场景上，即得到真实帧。
另外，还要对预测帧进行预测，对于真实帧中除了稳定面片以外的其他面片，仍可以采用现有技术中对像素进行插值或移动的方式来得到预设帧。
S204、获取并存储真实帧中稳定面片的渲染参数信息和绝对坐标。
针对每个真实帧，都存储真实帧中稳定面片的渲染参数信息和绝对坐标，示例性的，可以在本真实帧的数据列表中存储本真实帧中稳定面片的渲染参数信息和绝对坐标。
由于后续要渲染预测帧中的目标面片，而如果要渲染预测帧首先要获取相邻的两个真实帧中描述同一内容的稳定面片，假设相邻的两个真实帧为第一真实帧和第二真实帧，第一真实帧中的第一面片和第二真实帧中的第二面片均为稳定面片。
在绘制第一真实帧的过程中，当第一面片的顶点着色器运行后，运行原始源码以计算第一面片的各个顶点，获取并保存第一真实帧中第一面片的渲染参数信息、第一面片的绝对坐标。
在绘制第二真实帧的过程中，当第二面片的顶点着色器运行后，运行原始源码以计算第二面片的各个顶点，获取并保存第二真实帧中第二面片的渲染参数信息以及第二面片的绝对坐标。
该步骤可以获取第一真实帧中第一面片的渲染参数信息、第一面片的绝对坐标、第二真实帧中第二面片的渲染参数信息以及第二面片的绝对坐标，即对应前文所述的步骤S101。
S205、对预测帧中稳定面片的绝对坐标进行预测。
在绘制第二真实帧的过程中，遍历上一个真实帧的数据列表以查找与第二面片匹配的第一面片，以确定第一面片和第二面片在相邻两个真实帧中描述的同一内容(例如同一文字、同一图标、同一血条等)，在预测帧中即对应目标面片。具体的，如果第一面片的渲染参数信息和第二面片的渲染参数信息相同，并且，第一面片的绝对坐标和第二面片的绝对坐标在一定范围内，则可以确定第一面片和第二面片在相邻两个真实帧中描述的同一内容，并且在预测帧中可以确定目标面片的绝对坐标。关于如何确定目标面片的绝对坐标参照步骤S102的描述，在此不再赘述。
该步骤对应前文所述的步骤S102。
S206、对预测帧中稳定面片进行渲染。
具体的，根据目标面片的绝对坐标以及渲染参数信息对目标面片进行渲染。
如图12中S306所示，对于预测帧来说，预测工作流会通过统一变量(Uniform)设置该帧的渲染标志位，以指示该帧为预测帧。可选的，当真实帧和预测帧缓存到同一帧缓存的两个不同区域时，还可以通过设置视口(Viewport)来指示渲染位于哪一 区域(例如右半部分区域)中的预测帧。并通过面片绘制函数(glDrawElements)来渲染预测帧中的上述稳定面片。使得渲染工作流在进行渲染时能够区分是对真实帧渲染还是对预测帧进行渲染。
该步骤对应前文所述的步骤S103。
S207、如果对预测帧进行渲染，则将渲染后的稳定面片绘制到预测帧的主场景上，即得到预测帧。
由于前文已经完成目标面片的渲染，所以在对预测帧进行渲染时，对于除了稳定面片以外的面片，仍采用原有方式进行渲染，并绘制到预测帧的主场景上，然后再将渲染后的目标面片叠加绘制到预测帧的主场景上，即得到完整的预测帧。
如图12中S308所示，预测工作流会在完成预测帧的主场景的渲染后，将渲染完成的稳定面片绘制到预测帧的主场景上。
如图13所示，本申请实施例还提供一种芯片系统。该芯片系统60包括至少一个处理器601和至少一个接口电路602。至少一个处理器601和至少一个接口电路602可通过线路互联。处理器601用于支持电子设备实现上述方法实施例中的各个步骤，例如图7、图11、图12所示的方法，至少一个接口电路602可用于从其它装置(例如存储器)接收信号，或者，向其它装置(例如通信接口)发送信号。该芯片系统可以包括芯片，还可以包括其他分立器件。
本申请实施例还提供一种计算机可读存储介质，该计算机可读存储介质包括指令，当指令在上述电子设备上运行时，使得该电子设备执行上述方法实施例中的各个步骤，例如执行图7、图11、图12所示的方法。
本申请实施例还提供一种包括指令的计算机程序产品，当指令在上述电子设备上运行时，使得该电子设备执行上述方法实施例中的各个步骤，例如执行图7、图11、图12所示的方法。
关于芯片系统、计算机可读存储介质、计算机程序产品的技术效果参照前面方法实施例的技术效果。
应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的模块及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、设备和方法，可以通过其它的方式实现。例如，以上所描述的设备实施例仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个模块或组件可以结合或者可以集成到另一个设备，或一些特征可以忽略，或不执 行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，设备或模块的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个设备，或者也可以分布到多个设备上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能模块可以集成在一个设备中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个设备中。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件程序实现时，可以全部或部分地以计算机程序产品的形式来实现。该计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包括一个或多个可以用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)，光介质(例如，DVD)、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (9)

  1. 一种渲染方法，其特征在于，包括：
    获取第一真实帧中第一面片的渲染参数信息、所述第一面片的绝对坐标、第二真实帧中第二面片的渲染参数信息以及所述第二面片的绝对坐标，所述第一真实帧和第二真实帧为相邻真实帧；所述第一面片和所述第二面片指形状不变并且顶点的相对位置不变的二维面片；
    如果所述第一面片的渲染参数信息和所述第二面片的渲染参数信息相同，并且，所述第一面片的绝对坐标和所述第二面片的绝对坐标在一定范围内，则根据所述第一面片的绝对坐标和所述第二面片的绝对坐标确定目标面片的绝对坐标，所述目标面片位于基于所述第一真实帧和所述第二真实帧得到的预测帧；
    根据所述目标面片的绝对坐标以及所述渲染参数信息对所述目标面片进行渲染。
  2. 根据权利要求1所述的方法，其特征在于，所述第一面片的渲染参数信息或所述第二面片的渲染参数信息包括以下信息的至少一项：面片的各个顶点映射的纹理坐标、面片所属程序的标识、面片的顶点数。
  3. 根据权利要求1或2所述的方法，其特征在于，顶点着色器源码中包括面片的属性信息，所述属性信息用于指示面片为形状不变并且顶点的相对位置不变的二维面片。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述根据所述第一面片的绝对坐标和所述第二面片的绝对坐标确定所述目标面片的绝对坐标，包括：
    对所述第一面片的绝对坐标和所述第二面片的绝对坐标进行平滑处理，得到所述目标面片的绝对坐标。
  5. 根据权利要求4所述的方法，其特征在于，当所述预测帧位于所述第一真实帧和所述第二真实帧之间时，所述目标面片的绝对坐标P通过以下公式得到：P＝(P1*W1+P2*W2)/(W1+W2)，其中，P1为所述第一面片的绝对坐标，P2为所述第二面片的绝对坐标，W1和W2为权重系数。
  6. 根据权利要求5所述的方法，其特征在于，w1和w2相等。
  7. 根据权利要求4所述的方法，其特征在于，当所述预测帧位于所述第一真实帧和所述第二真实帧之后时，所述目标面片的绝对坐标P通过以下公式得到：P＝P2+(P2-P1)*W，其中，P1为所述第一面片的绝对坐标，P2为所述第二面片的绝对坐标，W为权重系数。
  8. 一种电子设备，其特征在于，包括处理器和存储器，所述存储器中存储指令，当所述处理器执行所述指令时，如权利要求1-7任一项所述的方法被执行。
  9. 一种计算机可读存储介质，其特征在于，包括指令，当所述指令在电子设备上执行时，使得所述电子设备执行如权利要求1-7任一项所述的方法。
PCT/CN2023/135910 2023-02-21 2023-12-01 渲染方法和电子设备 Ceased WO2024174655A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP23923818.1A EP4528656A4 (en) 2023-02-21 2023-12-01 RENDERING METHOD AND ELECTRONIC DEVICE
CN202380079933.1A CN120226050A (zh) 2023-02-21 2023-12-01 渲染方法和电子设备

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310203005.3 2023-02-21
CN202310203005.3A CN118537467A (zh) 2023-02-21 2023-02-21 渲染方法和电子设备

Publications (1)

Publication Number Publication Date
WO2024174655A1 true WO2024174655A1 (zh) 2024-08-29

Family

ID=92388429

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/135910 Ceased WO2024174655A1 (zh) 2023-02-21 2023-12-01 渲染方法和电子设备

Country Status (3)

Country Link
EP (1) EP4528656A4 (zh)
CN (2) CN118537467A (zh)
WO (1) WO2024174655A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN121582424B (zh) * 2026-01-29 2026-03-31 瀚博半导体(上海)股份有限公司 一种图像处理的方法、装置、介质和产品

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110199377A1 (en) * 2010-02-12 2011-08-18 Samsung Electronics Co., Ltd. Method, apparatus and computer-readable medium rendering three-dimensional (3d) graphics
CN106997610A (zh) * 2016-01-26 2017-08-01 阿里巴巴集团控股有限公司 一种图像渲染方法、装置及电子设备
WO2022068326A1 (zh) * 2020-09-30 2022-04-07 华为技术有限公司 一种图像帧预测的方法及电子设备
CN114470750A (zh) * 2021-07-06 2022-05-13 荣耀终端有限公司 图像帧流的显示方法、电子设备和存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9111287B2 (en) * 2009-09-30 2015-08-18 Microsoft Technology Licensing, Llc Video content-aware advertisement placement

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110199377A1 (en) * 2010-02-12 2011-08-18 Samsung Electronics Co., Ltd. Method, apparatus and computer-readable medium rendering three-dimensional (3d) graphics
CN106997610A (zh) * 2016-01-26 2017-08-01 阿里巴巴集团控股有限公司 一种图像渲染方法、装置及电子设备
WO2022068326A1 (zh) * 2020-09-30 2022-04-07 华为技术有限公司 一种图像帧预测的方法及电子设备
CN114470750A (zh) * 2021-07-06 2022-05-13 荣耀终端有限公司 图像帧流的显示方法、电子设备和存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4528656A4

Also Published As

Publication number Publication date
EP4528656A1 (en) 2025-03-26
CN120226050A (zh) 2025-06-27
EP4528656A4 (en) 2025-08-20
CN118537467A (zh) 2024-08-23

Similar Documents

Publication Publication Date Title
CN115398907B (zh) 一种图像帧预测的方法及电子设备
CN109993823B (zh) 阴影渲染方法、装置、终端及存储介质
CN109427083B (zh) 三维虚拟形象的显示方法、装置、终端及存储介质
US20250225620A1 (en) Special effect image processing method and apparatus, electronic device, and storage medium
US20210225067A1 (en) Game screen rendering method and apparatus, terminal, and storage medium
CN107247548B (zh) 图像显示方法、图像处理方法及装置
CN109961498B (zh) 图像渲染方法、装置、终端及存储介质
CN108537889A (zh) 增强现实模型的调整方法、装置、存储介质和电子设备
CN108961382B (zh) 一种图像渲染方法、装置和存储介质
CN112001851B (zh) 图像处理方法、装置、计算机设备及存储介质
CN110335200B (zh) 一种虚拟现实反畸变方法、装置以及相关设备
CN114764358A (zh) 一种界面显示方法及电子设备
EP4528656A1 (en) Rendering method and electronic device
WO2025020657A1 (zh) 用于虚拟场景的图像显示方法、设备、介质及程序产品
CN116188724A (zh) 动画渲染的方法、装置、设备及存储介质
WO2023280241A1 (zh) 图像画面渲染方法及电子设备
CN113436348A (zh) 三维模型处理方法、装置、电子设备及存储介质
WO2025021149A1 (zh) 端云协同的渲染方法及相关装置
WO2024103872A1 (zh) 图像处理方法及电子设备
CN116708931A (zh) 图像处理方法及电子设备
CN117131296A (zh) 内容呈现方法、装置、设备及存储介质
CN115970281A (zh) 模型图标生成方法、装置、设备及存储介质
CN118921451B (zh) 图像处理方法、相关装置及存储介质
CN111563956A (zh) 一种二维图片的三维显示方法、装置、设备及介质
CN110201392B (zh) 用户界面生成方法、装置和终端

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23923818

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023923818

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023923818

Country of ref document: EP

Effective date: 20241218

WWE Wipo information: entry into national phase

Ref document number: 202380079933.1

Country of ref document: CN

WWP Wipo information: published in national office

Ref document number: 202380079933.1

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE