WO2023051662A1 - 一种图像渲染方法及其相关设备 - Google Patents

一种图像渲染方法及其相关设备 Download PDF

Info

Publication number
WO2023051662A1
WO2023051662A1 PCT/CN2022/122427 CN2022122427W WO2023051662A1 WO 2023051662 A1 WO2023051662 A1 WO 2023051662A1 CN 2022122427 W CN2022122427 W CN 2022122427W WO 2023051662 A1 WO2023051662 A1 WO 2023051662A1
Authority
WO
WIPO (PCT)
Prior art keywords
material information
reflected
information
reflector
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/122427
Other languages
English (en)
French (fr)
Inventor
余晓峰
石鑫栋
刘畅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to US18/696,019 priority Critical patent/US20240378793A1/en
Priority to EP22875045.1A priority patent/EP4386682A4/en
Priority to CN202280057482.7A priority patent/CN117957576A/zh
Publication of WO2023051662A1 publication Critical patent/WO2023051662A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/06—Ray-tracing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/50—Lighting effects
    • G06T15/80—Shading
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present application relates to the technical field of computer graphics, in particular to an image rendering method and related equipment.
  • a rasterization technology combined with a ray tracing technology is usually used to render a 3D scene, so as to obtain a more realistic image and improve user experience.
  • a three-dimensional scene usually includes reflectors (for example, the ground) and multiple reflected objects (people on the ground, chairs, etc.).
  • the graphics processing unit (GPU) receives the spatial information of the reflector sent by the central processing unit (CPU), the material information of the reflector, the spatial information of multiple reflected objects, and the spatial information of multiple reflected objects.
  • the spatial information of the reflector and the spatial information of multiple reflected objects can be used for intersection calculation, so as to determine multiple intersection points on multiple reflected objects.
  • the GPU can perform coloring based on the spatial information of the intersection point and the material information of the reflected object where the intersection point is located. In this way, after completing the coloring operations corresponding to all intersection points, the reflection in the reflector ( For example, an image of a human figure on the ground). So far, the image rendering based on ray tracing is completed.
  • the cache memory (cache) of the GPU can usually only cache one material information.
  • some of the reflected objects may correspond to the same material information, so the GPU completes the processing of multiple reflected objects in sequence.
  • the same material information is frequently swapped in and out between the memory and the cache, resulting in too much memory bandwidth and unbearable for electronic devices.
  • Embodiments of the present application provide an image rendering method and related equipment, which can effectively reduce the number of times any material information is swapped in and out between the memory and cache, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • the first aspect of the embodiments of the present application provides an image rendering method, the method including:
  • the CPU can obtain the rendering information of the 3D scene. Since the 3D scene usually includes reflectors (such as the ground, etc.) and N reflected objects (such as people, tables, chairs, etc.), the CPU can render the 3D scene The information is divided into rendering information of reflectors and rendering information of N reflected objects. Then, the CPU sends the rendering information of the reflector (including the spatial information and material information of the reflector) and the rendering information of the N reflected objects (including the spatial information and material information of the N reflected objects) to the GPU, so that the GPU
  • the ray-tracing stage of image rendering that is, the following steps are performed:
  • the GPU performs intersection calculation based on ray tracing based on the spatial information of the reflector and the spatial information of the N reflected objects, and obtains M intersection points on the N reflected objects.
  • the N reflected objects are objects that intersect light rays (from the camera) passing through the reflected objects.
  • the N reflected objects there is at least one intersection point on each reflected object, so M ⁇ N ⁇ 1.
  • the 3D scene contains reflected objects such as people, chairs, and tables, and the chairs and tables are made of the same material. Then, the material information of the person is material information 1, and the material information of the chair and the table is material information 2.
  • the image of the reflection of the reflected object in the reflector includes the image of the shadow of the person on the ground, the image of the shadow of the table, the image of the shadow of the chair, etc.).
  • M intersection points on the N reflected objects and N reflected objects can be obtained.
  • Object corresponds to P pieces of material information.
  • the GPU determines all intersection points corresponding to the target material information, and performs coloring according to the spatial information of all intersection points corresponding to the target material information and the target material information, to obtain an image of the reflection of the reflected object in the reflector.
  • the GPU separates and executes the intersection calculation and coloring in ray tracing, that is, after uniformly determining the M intersection points, for a certain material information among the P material information, the GPU can determine the corresponding All the intersection points, and the shading operations corresponding to these intersection points are completed, so the GPU does not need to use the material information in the subsequent shading operations corresponding to the remaining intersection points among the M intersection points. In this way, the number of times any material information is swapped in and out between the memory and the cache can be effectively reduced, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • the target material information is the i-th material information
  • the initial value of i is 1
  • the GPU can determine the intersection point corresponding to the first material information in the P material information among the M intersection points, and according to the spatial information of the intersection point corresponding to the first material information and the first material information Information is colored.
  • the GPU can perform the same operation on the second material information as on the first material information, until the operation on the P-th material information is completed as on the first material information, so the GPU can successfully render Images reflected in reflective objects.
  • M intersection points on the N reflected objects can be obtained, and the N reflected objects correspond to P pieces of material information.
  • the GPU determines the intersection point corresponding to the first material information, and performs coloring according to the spatial information of the intersection point corresponding to the first material information and the first material information.
  • the GPU can perform the same operation on the second material information as on the first material information, until the operation on the P-th material information is completed as on the first material information, then the GPU can successfully render The image of the reflection of the reflected object in the reflector.
  • the GPU separates and executes the intersection calculation and coloring in ray tracing, that is, after uniformly determining the M intersection points, for any one of the P material information, the GPU can determine the corresponding All the intersection points, and the shading operations corresponding to these intersection points are completed, so the GPU does not need to use the material information in the subsequent shading operations corresponding to the remaining intersection points among the M intersection points. In this way, the number of times any material information is swapped in and out between the memory and the cache can be effectively reduced, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • the GPU can process the material information and the spatial information of the M intersection points.
  • the GPU can determine whether the object to be reflected at the first intersection among the M intersection points corresponds to the material information, and if the object to be reflected at the first intersection corresponds to the material information, then according to the first intersection The spatial information and the material information are colored to obtain the first pixel of the reflected image in the reflector. If the reflected object where the first intersection point is located does not correspond to the material information, the process ends. Thereafter, the GPU can perform the same operation on the second intersection point as on the first intersection point until the operation on the Mth intersection point is completed as on the first intersection point. For the rest of the material information in the P pieces of material information, the GPU can also perform the aforementioned process, so as to successfully obtain the reflected image of the reflective object.
  • S1 includes: according to the spatial information of the camera, the spatial information of the reflector, and the hierarchical bounding box BVH tree, it is determined that the light from the camera passes through the reflector and acts on the N reflected objects.
  • the obtained M intersections, the BVH tree is constructed according to the spatial information of N reflected objects.
  • the GPU completely separates the intersection calculation and coloring of ray tracing, and first performs the intersection calculation in a unified manner, so as to obtain M intersection points obtained on N reflected objects, which is conducive to optimizing the entire ray tracing process. .
  • each reflected object has a unique number in the BVH tree
  • S211 includes: if the number of the reflected object where the j-th intersection point is located is located at the first In the number of at least one reflected object corresponding to the i material information, the coloring is performed according to the spatial information of the j intersection point and the i material information.
  • the GPU can determine whether the number of the reflected object where the intersection is located is located in all the reflected objects corresponding to the material information If yes, coloring is performed according to the spatial information of the intersection point and the material information, and if not, the process ends. In this way, it can be accurately judged whether a certain intersection point corresponds to a certain material information.
  • the method further includes: S4: performing rasterization according to the spatial information of the reflector and the material information of the reflector to obtain an image of the reflector; S5: according to the spatial information of N reflected objects Rasterize the material information of the N reflected objects to obtain the images of the N reflected objects; S6: Fusion the image of the reflector, the images of the N reflected objects and the image of the reflection in the reflector to obtain the target image .
  • the rendering information of the reflector including the spatial information of the reflector and the material information of the reflector
  • the rendering information of all reflected objects including the spatial information of all the reflected objects and the material information of all the reflected objects
  • the GPU can also perform rasterization based on these information, so as to obtain the image of the reflector and the images of N reflected objects. Then, the GPU fuses the image of the reflector, the images of the N reflected objects, and the reflection image of the reflector to obtain the target image of the simulated 3D scene. So far, the image rendering of the 3D scene is completed. Thereafter, the CPU can obtain the target image from the buffer, and send the target image to the screen for display by the user.
  • the spatial information includes at least one of a vertex coordinate, a vertex normal, and a triangle index.
  • the material information includes at least one of color, metallicity and roughness.
  • the second aspect of the embodiment of the present application provides an image rendering device, which includes: an intersection module, configured to perform intersection calculation based on ray tracing according to the spatial information of the reflector and the spatial information of the reflected object, to obtain the M intersections on the reflector, the reflected object corresponds to P pieces of material information, M ⁇ 1; the coloring module is used to determine multiple intersections corresponding to the target material information among the M intersections, and according to the number of intersections corresponding to the target material information The spatial information of each intersection point and the target material information are colored to obtain an image of the reflection of the reflected object in the reflector, and the target material information is one of multiple material information.
  • the GPU performs intersection calculation based on ray tracing based on the spatial information of the reflector and the spatial information of the N reflected objects, it can obtain M intersection points on the N reflected objects, and N reflected objects Object corresponds to P pieces of material information.
  • the GPU determines all intersection points corresponding to the target material information, and performs coloring according to the spatial information of all intersection points corresponding to the target material information and the target material information, to obtain an image of the reflection of the reflected object in the reflector.
  • the GPU separates and executes the intersection calculation and coloring in ray tracing, that is, after uniformly determining the M intersection points, for a certain material information among the P material information, the GPU can determine the corresponding All the intersection points, and the shading operations corresponding to these intersection points are completed, so the GPU does not need to use the material information in the subsequent shading operations corresponding to the remaining intersection points among the M intersection points. In this way, the number of times any material information is swapped in and out between the memory and the cache can be effectively reduced, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • the target material information is the i-th material information
  • the initial value of i is 1.
  • the GPU performs intersection calculation based on ray tracing based on the spatial information of the reflector and the spatial information of the N reflected objects
  • M intersection points on the N reflected objects can be obtained, and the N reflected objects correspond to P pieces of material information.
  • the GPU determines the intersection point corresponding to the first material information, and performs coloring according to the spatial information of the intersection point corresponding to the first material information and the first material information.
  • the GPU can also perform the same operation on the second material information as it does on the first material information, until it completes the same operation on the P-th material information as on the first material information, so the GPU can successfully render An image of the reflection of a reflected object in a reflector.
  • the GPU separates and executes the intersection calculation and coloring in ray tracing, that is, after uniformly determining the M intersection points, for any one of the P material information, the GPU can determine the corresponding All the intersection points, and the shading operations corresponding to these intersection points are completed, so the GPU does not need to use the material information in the subsequent shading operations corresponding to the remaining intersection points among the M intersection points. In this way, the number of times any material information is swapped in and out between the memory and the cache can be effectively reduced, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • the intersection module is used to determine, according to the spatial information of the camera, the spatial information of the reflector, and the hierarchical bounding box BVH tree, that the light from the camera will act on the N reflected objects after passing through the reflector.
  • the BVH tree is constructed according to the spatial information of N reflected objects.
  • each reflected object has a unique number in the BVH tree, and the coloring unit is used for the number of the reflected object where the j-th intersection point is located, If it is located in the number of at least one reflected object corresponding to the i-th material information, coloring is performed according to the spatial information of the j-th intersection point and the i-th material information.
  • the device further includes: a first rasterization module, configured to perform rasterization according to the spatial information of the reflector and the material information of the reflector to obtain an image of the reflector; the second rasterization module , for rasterizing according to the spatial information of the N reflected objects and the material information of the N reflected objects, to obtain the images of the N reflected objects; The image of the object is fused with the image reflected in the reflector to obtain the target image.
  • a first rasterization module configured to perform rasterization according to the spatial information of the reflector and the material information of the reflector to obtain an image of the reflector
  • the second rasterization module for rasterizing according to the spatial information of the N reflected objects and the material information of the N reflected objects, to obtain the images of the N reflected objects
  • the image of the object is fused with the image reflected in the reflector to obtain the target image.
  • the spatial information includes at least one of a vertex coordinate, a vertex normal, and a triangle index.
  • the material information includes at least one of color, metallicity and roughness.
  • the third aspect of the embodiments of the present application provides an electronic device, including a memory and a processor; the memory stores codes, and the processor is configured to execute the codes. When the codes are executed, the electronic device performs the same as the first aspect or the second aspect.
  • the fourth aspect of the embodiments of the present application provides a computer-readable storage medium, including computer-readable instructions, when the computer-readable instructions are run on the computer, the computer can execute any one of the first aspect or the first aspect. Possible implementations of the methods described.
  • a fifth aspect of the embodiments of the present application provides a computer program product, including computer-readable instructions.
  • the computer-readable instructions When the computer-readable instructions are run on a computer, the computer is made to perform any one of the possible functions of the first aspect or the first aspect. Implement the method described in the manner.
  • the GPU performs intersection calculation based on ray tracing based on the spatial information of the reflector and the spatial information of N reflected objects, M intersection points on N reflected objects and N reflected objects Corresponding to P pieces of material information.
  • the GPU determines the intersection point corresponding to the first material information, and performs coloring according to the spatial information of the intersection point corresponding to the first material information and the first material information.
  • the GPU can also perform the same operation on the second material information as it does on the first material information, until it completes the same operation on the P-th material information as on the first material information, so the GPU can successfully render An image reflected in a reflective object.
  • the GPU separates and executes the intersection calculation and coloring in ray tracing, that is, after uniformly determining the M intersection points, for any one of the P material information, the GPU can determine the corresponding All the intersection points, and the shading operations corresponding to these intersection points are completed, so the GPU does not need to use the material information in the subsequent shading operations corresponding to the remaining intersection points among the M intersection points. In this way, the number of times any material information is swapped in and out between the memory and the cache can be effectively reduced, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • Figure 1 is a schematic diagram of the principle of ray tracing technology
  • FIG. 2 is a schematic diagram of the principle of rasterization technology
  • FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of an image rendering method provided by an embodiment of the present application.
  • Fig. 5 is a schematic diagram of the BVH tree provided by the embodiment of the present application.
  • FIG. 6 is a schematic diagram of image rendering provided by the embodiment of the present application.
  • Fig. 7 is a schematic diagram of the target area provided by the embodiment of the present application.
  • FIG. 8 is a schematic diagram of an application example of the image rendering method provided by the embodiment of the present application.
  • FIG. 9 is another schematic structural diagram of an image rendering device provided by an embodiment of the present application.
  • Embodiments of the present application provide an image rendering method and related equipment, which can effectively reduce the number of times any material information is swapped in and out between memory and cache, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • rasterization processing is usually used to render a 3D scene, so as to obtain an image capable of displaying the 3D scene.
  • the quality of the image obtained by rendering with rasterization technology is average, and it is often difficult to present a realistic picture. For example, it is usually difficult to truly restore the light reflection, refraction, and shadow effects in the scene in the rendered image.
  • a new rendering technology - ray tracing technology came into being. Both ray tracing technology and rasterization technology are used to achieve image rendering, and their main purpose is to project objects in 3D space to two-dimensional screen space for display through computational shading.
  • FIG. 1 is a schematic diagram of the principle of the ray tracing technology.
  • the principle of ray tracing is: from the position of the camera, through the pixel position on the image plane, emit a beam of light into the three-dimensional scene, find the nearest intersection point between the light and the geometric figure, and then calculate the coloring of the intersection point . If the material of the intersection point is reflective, you can continue to track in the reflection direction of the intersection point, and continue to calculate the coloring of the intersection point after reflection. That is to say, the ray tracing method calculates projection and global illumination by tracing the propagation process of light in a three-dimensional scene, thereby rendering a two-dimensional image.
  • FIG. 2 is a schematic diagram of the principle of the rasterization technology.
  • the principle of rasterization processing is: divide the objects in the 3D scene by triangles, transform the 3D coordinates of the vertices of the triangles into 2D coordinates on the image through coordinate change calculations, and finally create the coordinates in the triangles on the image Fill the texture to achieve the rendering of the image.
  • the processing difficulty is relatively low, and the light and shadow effect provided is also poor.
  • Ray tracing technology achieves realistic effects such as reflection, refraction, shadow, and ambient light occlusion by tracing each light emitted from the camera, so the ray tracing method can provide realistic and realistic light and shadow effects. Therefore, in order to render a more realistic image, current electronic devices usually use a rasterization technology combined with a ray tracing technology to render a three-dimensional scene, so as to improve the user's viewing experience.
  • the GPU receives the spatial information of the reflector (for example, the ground), the material information of the reflector, the spatial information of multiple reflected objects (people on the ground, chairs, etc.) and the spatial information of multiple reflected objects sent by the CPU.
  • the spatial information of the reflector and the spatial information of multiple reflected objects can be used for intersection calculation, so as to determine multiple intersection points on multiple reflected objects.
  • the GPU can perform coloring based on the spatial information of the intersection point and the material information of the reflected object where the intersection point is located. In this way, after completing the coloring operations corresponding to all intersection points, the reflection in the reflector ( For example, an image of a human figure on the ground). At this point, the ray tracing stage in image rendering is completed.
  • the GPU can also use the aforementioned rendering information of the 3D scene to perform rasterization, so as to obtain the image of the reflector and the images of multiple reflected objects. So far, the rasterization stage in image rendering is completed. Finally, the image of the reflection in the reflector, the image of the reflector and the images of multiple reflected objects can be fused to obtain the target image of the simulated 3D scene.
  • the target image includes the images of each object in the 3D scene and some objects Reflected image.
  • the cache memory (cache) of the GPU can usually only cache one material information.
  • some of the reflected objects may correspond to the same material information, so the GPU completes the processing of multiple reflected objects in sequence.
  • the same material information is frequently swapped in and out between the memory and the cache, resulting in too much memory bandwidth and unbearable for electronic devices.
  • an embodiment of the present application provides an image rendering method, which can be executed by an electronic device.
  • the electronic device includes a CPU and a GPU capable of rendering images.
  • the electronic device may be, for example, a mobile phone (mobile phone), a tablet computer, a notebook computer, a PC, a mobile internet device (mobile internet device, MID), a wearable device, a virtual reality (virtual reality, VR) device, an enhanced Augmented reality (AR) devices, wireless electronic devices in industrial control, wireless electronic devices in self driving, wireless electronic devices in remote medical surgery, smart grid ( Wireless electronic devices in smart grid, wireless electronic devices in transportation safety, wireless electronic devices in smart city, wireless electronic devices in smart home, etc.
  • the electronic device can be a device running an Android system, an IOS system, a windows system and other systems.
  • Applications that need to render 3D scenes to obtain 2D images can be run on the electronic device, such as game applications, lock screen applications, map applications, or monitoring applications.
  • an electronic device 3000 may include: a central processing unit 3001 , a graphics processor 3002 , a display device 3003 and a memory 3004 .
  • the electronic device 3000 may further include at least one communication bus (not shown in FIG. 3 ), which is used to realize connection and communication between various components.
  • various components in the electronic device 3000 may also be coupled through other connectors, and other connectors may include various interfaces, transmission lines or buses, and the like.
  • Various components in the electronic device 3000 may also be connected radially with the central processing unit 3001 as the center.
  • coupling refers to being electrically connected or communicating with each other, including direct connection or indirect connection through other devices.
  • the central processing unit 3001 and the graphics processing unit 3002 in the electronic device 3000 may be located on the same chip, or may be independent chips.
  • the functions of the central processing unit 3001 , the graphics processing unit 3002 , the display device 3003 and the memory 3004 are briefly introduced below.
  • the Central processing unit 3001 for running an operating system 3005 and application programs 3006.
  • the application program 3006 may be a graphic application program, such as a game, a video player, and the like.
  • the operating system 3005 provides a system graphics library interface
  • the application program 3006 uses the system graphics library interface and drivers provided by the operating system 3005, such as graphics library user-mode drivers and/or graphics library kernel-mode drivers, to generate graphics or The instruction stream for an image frame, and the associated rendering data required.
  • the system graphics library includes but is not limited to: an embedded open graphics library (open graphics library for embedded system, OpenGL ES), the Kronos platform graphics interface (the khronos platform graphics interface) or Vulkan (a cross-platform graphics application program interface) and other system graphics libraries.
  • the instruction stream includes a series of instructions, which are usually calls to the interface of the system graphics library.
  • the central processing unit 3001 may include at least one of the following types of processors: an application processor, one or more microprocessors, a digital signal processor (digital signal processor, DSP), a microcontroller (microcontroller unit, MCU) or artificial intelligence processor, etc.
  • processors an application processor, one or more microprocessors, a digital signal processor (digital signal processor, DSP), a microcontroller (microcontroller unit, MCU) or artificial intelligence processor, etc.
  • the CPU 3001 may further include necessary hardware accelerators, such as application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or integrated circuits for implementing logic operations.
  • ASICs application specific integrated circuits
  • FPGAs field programmable gate arrays
  • the processor 3001 may be coupled to one or more data buses for transferring data and instructions between the various components of the electronic device 3000 .
  • Graphics processor 3002 for receiving the graphics instruction stream sent by the processor 3001, generating a rendering target through a rendering pipeline (pipeline), and displaying the rendering target to the display device 3003 through the layer composition display module of the operating system.
  • the rendering pipeline may also be referred to as a rendering pipeline, a pixel pipeline or a pixel pipeline, and is a parallel processing unit inside the graphics processor 3002 for processing graphics signals.
  • the graphics processor 3002 may include multiple rendering pipelines, and the multiple rendering pipelines may independently process graphics signals in parallel.
  • the rendering pipeline can perform a series of operations in the process of rendering graphics or image frames. Typical operations can include: vertex processing (Vertex Processing), primitive processing (Primitive Processing), rasterization (Rasterization), fragment processing (Fragment Processing) and so on.
  • the graphics processor 3002 may include a general-purpose graphics processor for executing software, such as a GPU or other types of dedicated graphics processing units.
  • Display device 3003 used to display various images generated by the electronic device 3000, the image may be a graphical user interface (graphical user interface, GUI) of the operating system or image data (including still images and video images) processed by the graphics processor 3002 data).
  • GUI graphical user interface
  • the display device 3003 may include any suitable type of display screen.
  • a liquid crystal display liquid crystal display, LCD
  • a plasma display or an organic light-emitting diode (organic light-emitting diode, OLED) display and the like.
  • OLED organic light-emitting diode
  • the memory 3004 is a transmission channel between the central processing unit 3001 and the graphics processing unit 3002, and may be double data rate synchronous dynamic random access memory (DDR SDRAM) or other types of cache.
  • DDR SDRAM double data rate synchronous dynamic random access memory
  • Offscreen render pass The image rendered through this pass is usually an image that does not need to be displayed on the screen temporarily. This part of the image can be stored in the frame buffer assigned to the off-screen render pass.
  • the frame buffer can be called an off-screen rendering buffer, and is used to store images that do not need to be displayed on the screen temporarily.
  • the off-screen rendering pass can be used to implement ray tracing techniques.
  • the image rendered through this pass is usually the image to be displayed on the screen (that is, the image sent to display), and this part of the image can be stored in the frame buffer assigned to the current screen rendering pass Area (framebuffer), the frame buffer can be called the current screen rendering buffer, used to store the image to be displayed on the screen.
  • the current screen rendering pass can be used to implement the rasterization technology.
  • Fig. 4 is a schematic flow chart of the image rendering method provided by the embodiment of the present application. As shown in Fig. 4, the method includes:
  • the CPU of the electronic device can analyze the rendering information of each object in the 3D scene.
  • the rendering information of each object includes the spatial information of the object and the material information of the object.
  • the spatial information of the object includes information such as the vertex coordinates of the object, the vertex normal of the object, and the triangle index.
  • the material information of the object includes the color of the object. , the metalness of the object and the roughness of the object and other information.
  • the material information of different objects can be the same (type) material information (that is, different objects correspond to the same material information), or multiple (types) material information (that is, different objects correspond to multiple material information) ), it can be seen that the number of objects is generally greater than or equal to the number of material information.
  • the 3D scene contains ground, people, chairs and tables, and the chairs and tables are made of the same material.
  • the material information of the ground is material information A
  • the material information of people is material information B
  • the material information of chairs and tables is material information C.
  • the CPU may determine the category of the object based on the material information of the object. Specifically, after the CPU obtains the material information of an object, it can determine whether the object is a reflective object based on the metallicity of the object. If the metallicity of the object is greater than or equal to a preset threshold, the object can be determined to be a reflective object. object, if the metallicity of the object is less than the preset threshold, it can be determined that the object is a reflected object.
  • the metalness of objects such as the ground, mirrors, and lakes is usually greater than the preset threshold, so these objects can be called reflectors, while the metalness of objects such as people and tables is usually smaller than the preset threshold, so these objects can be called reflectors. called the reflected object.
  • the CPU can classify all objects in the three-dimensional scene into two types of objects, ie reflectors and reflected objects.
  • the 3D scene contains only one reflector and multiple reflected objects for schematic illustration below.
  • the CPU can construct a BVH tree based on the spatial information of all reflected objects, and the BVH tree can be used to implement subsequent ray tracing operations. Specifically, the CPU can construct a BVH tree according to the vertex coordinates, vertex normals, and triangle indexes of the object being reflected.
  • Fig. 5 is a schematic diagram of the BVH tree provided by the embodiment of the present application. As shown in Fig. 5, there are 6 reflected objects in the three-dimensional scene, and the bounding box A (enclosed) can be determined based on the spatial information of the 6 reflected objects.
  • the spatial information of these 6 objects), the spatial information of the bounding box B (surrounding these 4 objects) can be determined based on the spatial information of 4 of the reflected objects, and the bounding box C can be determined based on the spatial information of the remaining 2 reflected objects (enclosing these two objects), manage the spatial information of the bounding box A, the spatial information of the bounding box B, and the spatial information of the bounding box C in a binary tree structure to obtain a BVH tree. It can be seen that the BVH tree can centrally manage the spatial information of the bounding box A, the bounding box B and the bounding box C, which is equivalent to the centralized management of the bounding box A, the bounding box B and the bounding box C.
  • the bounding box A is the largest bounding box among all the bounding boxes. It is worth noting that each of the six reflected objects has a unique number in the BVH tree, as shown in Figure 5, counting from left to right, the number of these six reflected objects is 11 , 12, 13, 14, 21 and 22.
  • the GPU of the electronic device implements ray tracing based on the BVH tree, it is necessary to calculate whether the ray intersects with the reflected object in the three-dimensional scene (ie, intersection calculation). Due to the existence of the BVH tree, the GPU can determine a bounding box surrounding the reflected object based on the BVH tree, and then judge whether the ray intersects the bounding box. If the ray does not touch the bounding box, it means that the ray will not touch the bounding box. Intersect the reflected object in the bounding box; if the ray hits the bounding box, then calculate whether the ray intersects the reflected object in the bounding box.
  • the CPU can also obtain the spatial information of the camera, which is used to shoot the target image of the simulated three-dimensional scene.
  • the spatial information of the camera may include the vertical height of the camera and the coordinates of the camera.
  • the CPU gets the rendering information of the reflector (including the spatial information of the reflector and the material information of the reflector), the rendering information of all the reflected objects (including the spatial information of all the reflected objects and the material information of all the reflected objects), and the BVH tree And the spatial information of the camera, then send this part of information to the GPU, so that the GPU can perform image rendering according to this part of information.
  • intersection points determine the intersection point corresponding to the i-th material information, and perform coloring according to the spatial information of the intersection point corresponding to the i-th material information and the i-th material information, and the initial value of i is 1.
  • the GPU gets the rendering information of the reflector (including the spatial information of the reflector and the material information of the reflector), the rendering information of all the reflected objects (including the spatial information of all the reflected objects and the material information of all the reflected objects), and the BVH tree And after the spatial information of the camera, image rendering can be performed based on this part of the information.
  • the image rendering performed by the GPU includes two stages, that is, the ray tracing stage (ie step 402 to step 404 in this embodiment) and the rasterization stage (ie step 405 to step 406 in this embodiment) .
  • the ray tracing phase includes two sub-phases, the first sub-phase is intersection calculation, and the second sub-phase is coloring.
  • the GPU can completely separate these two sub-phases for execution, thereby optimizing the entire ray tracing process.
  • FIG. 6 is a schematic diagram of image rendering provided by the embodiment of the present application.
  • the GPU is provided with a rendering pipeline 601 for the off-screen rendering pass, and the rendering pipeline 601 includes a vertex shader (vertex shader, VS) 6011, a fragment shader (frag shader, FS) 6012 and a fragment shader 6013.
  • vertex shader vertex shader, VS
  • fragment shader fragment shader, FS
  • the fragment shader 6012 is provided with the part related to the intersection calculation in the ray tracing algorithm
  • the fragment shader 6013 is provided with the part related to the coloring in the ray tracing algorithm
  • the fragment shader 6012 and the fragment shader 6013 can be Commonly used to implement ray tracing.
  • the GPU can process the spatial information of the reflector and the BVH tree through the vertex shader 6011 and the fragment shader 6012, that is, through these two shaders, perform intersection calculation based on ray tracing based on these information, and obtain N
  • the M intersection points on the reflected object it can be understood that the N reflected objects are the reflected objects that have intersection points with the light emitted by the camera among all the reflected objects.
  • each reflected object has at least one intersection point, so M ⁇ N ⁇ 1.
  • the N reflected objects correspond to P pieces (types) of material information, and N ⁇ P ⁇ 1.
  • intersection calculation based on ray tracing is performed on the entire reflector, that is, the camera is used as the light source, and light is emitted to each vertex of the reflector, and then the route calculation and intersection calculation are performed for each ray, the calculation involved
  • the quantity is quite large.
  • not every ray can intersect the object to be reflected, so the calculation based on the invalid ray (that is, the ray that does not intersect the object to be reflected) is an invalid calculation.
  • the reflection in the reflector is actually the influence or mirror image of the reflected object in the reflector, etc., so a certain block area containing the reflection can be determined as the target area in the reflector, if the light hits The target area of the reflector, then this part of the light (effective light) will intersect with the reflected object after passing through this area. If the light hits the reflector, other than the target area, then this part of the light will not intersect with the reflected object after passing through this area. It can be seen that if only the intersection calculation based on ray tracing is performed on the target area, the calculation amount of ray tracing can be effectively reduced, thereby saving the computing power of the GPU and reducing the power consumption of image rendering.
  • the GPU can determine the target area of reflectors in the following ways:
  • the GPU can determine the spatial information of the largest bounding box from the BVH tree, and then determine the spatial information of the target area of the reflector from the spatial information of the reflector according to the spatial information of the largest bounding box and the spatial information of the camera. After obtaining the spatial information of the target area of the reflector, it is equivalent to determining the target area of the reflector. Specifically, any vertex in the largest bounding box is set as the first target point, and any vertex in the target area of the reflector is set as the second target point.
  • H1 is determined based on the coordinates of the camera and the coordinates of the second target point
  • X1 is determined based on the coordinates of the camera and the first The coordinates of the target point are determined.
  • the spatial information of the reflector includes the coordinates of each vertex in the reflector, so the GPU can judge whether the coordinates of each vertex in the reflector satisfy the aforementioned mathematical relationship. If the coordinates of a certain vertex satisfy the aforementioned mathematical relationship, the vertex is a reflector A vertex in the target region of . In this way, the reflector can be divided into the target area and other areas except the target area.
  • the GPU can also divide the spatial information of the reflector into the spatial information of the target area and the spatial information of the remaining areas, but for the target area and the remaining areas, the material information of the two areas is the same (that is, the material information of the reflector , because the material information of the entire reflector is uniform).
  • FIG. 7 is a schematic diagram of the target area provided by the embodiment of the present application.
  • a certain three-dimensional scene is assumed to be a person standing on the ground and the reflection of a person on the ground.
  • the largest bounding box in the scene may be determined first, and the largest bounding box is the bounding box surrounding the character.
  • the target area on the ground can be determined, and the target area includes the reflection of the person.
  • the GPU can input the spatial information of the target area of the reflector (for example, the coordinates of the vertex of the reflector and the normal of the vertex of the reflector) into the vertex shader 6011, and the vertex shader 6011 may perform certain calculation processing on this part of the spatial information (for example, transforming three-dimensional coordinates into two-dimensional coordinates, etc.), and input the processed spatial information into the fragment shader 6012 .
  • the spatial information of the target area of the reflector for example, the coordinates of the vertex of the reflector and the normal of the vertex of the reflector
  • the vertex shader 6011 may perform certain calculation processing on this part of the spatial information (for example, transforming three-dimensional coordinates into two-dimensional coordinates, etc.), and input the processed spatial information into the fragment shader 6012 .
  • the GPU will directly input the camera's spatial information and BVH tree into the fragment shader 6012, so the fragment shader 6012 can perform intersection calculations based on the camera's spatial information, the processed spatial information, and the BVH tree to determine the After the light passes through the reflector, it acts on the M intersection points obtained on the N reflected objects.
  • the GPU can save the number of the reflected object where each intersection point is located and the spatial information of the intersection point (the intersection point is a vertex in nature, so the spatial information of the intersection point includes the coordinates and normal of the vertex, etc.).
  • the GPU can use the fragment shader 6013 to determine the intersection point corresponding to the first material information in the P material information among the M intersection points, and according to the space of the intersection point corresponding to the first material information information and the first material information for coloring. It should be noted that the GPU can also perform the same operation on the second material information as it does on the first material information, until it completes the same operation on the P-th material information as on the first material information, so the GPU can The image of the reflection in the reflective object is successfully rendered, for example, the image of the reflection of the person in Figure 7.
  • intersection points 1, 2, 3, 4, 5 and 6 wherein, intersection 1 and intersection 2 are located on the reflected object 11, intersection 3 and intersection 4 are located on the reflected object 12, and intersection 5 is located on the object to be reflected
  • the intersection 6 is located on the reflected object 22
  • the reflected object 11 corresponds to the material information 1
  • the reflected object 12 corresponds to the material information 2
  • the reflected object 21 and the reflected object 22 correspond to the material information 3 .
  • the GPU can determine that the intersection points corresponding to the material information 1 are the intersection point 1 and the intersection point 2, and perform coloring according to the spatial information of the two intersection points and the material information 1 .
  • the GPU may determine that the intersection points corresponding to the material information 2 are intersection points 3 and 4, and perform coloring according to the spatial information of the two intersection points and the material information 2 . Then, the GPU may determine that the intersection points corresponding to the material information 3 are the intersection points 5 and 6, and perform coloring according to the spatial information of the two intersection points and the material information 3 . In this way, the GPU can obtain images reflected in reflective objects.
  • the GPU can render a pixel based on the information of an intersection (that is, the image of the reflection in the reflector contains M pixels), then the GPU can render the reflection in the reflector in the following way Image:
  • the GPU may input the first material information among the P pieces of material information and the space information of the M intersection points to the fragment shader 6013 . Then, the input fragment shader 6013 can judge whether the object to be reflected at the first intersection point among the M intersection points corresponds to the first material information, if the object to be reflected at the first intersection point corresponds to the first material information, The fragment shader 6013 performs coloring according to the spatial information of the first intersection point and the first material information, and obtains the first pixel of the reflected image in the reflector. If the material information does not correspond, it ends. Thereafter, the fragment shader 6013 may perform the same operation on the second intersection point as on the first intersection point until the operation on the Mth intersection point is completed as on the first intersection point.
  • the GPU can input the second material information of the P material information and the space information of the M intersection points into the fragment shader 6013, so that the fragment shader 6013 can re-execute the above process until the GPU combines the P material information and the M
  • the spatial information of the intersection points is input to the fragment shader 6013, so that the above process is re-executed on the fragment shader 6013. So far, the fragment shader 6013 can obtain M pixels of the image reflected in the reflector, which is equivalent to obtaining the image reflected in the reflector.
  • fragment shader 6013 can determine whether the reflected object where a certain intersection point is located corresponds to a certain material information in the following manner:
  • the fragment shader 6013 can determine whether the number of the reflected object where the intersection point is located (which can be input to the fragment shader 6013 by the GPU) is located in this In the number of all reflected objects corresponding to the material information (which can be input by the GPU to the fragment shader 6013 ), if yes, perform coloring according to the space information of the intersection point and the material information, if not, end.
  • the GPU After getting the image of the reflection in the reflector, the GPU can complete the ray tracing stage of image rendering.
  • the GPU After obtaining the rendering information of the reflector (including the spatial information of the reflector and the material information of the reflector) and the rendering information of all the reflected objects (including the spatial information of all the reflected objects and the material information of all the reflected objects), the GPU also The rasterization stage of image rendering can be implemented based on this information.
  • the GPU is provided with two rendering pipelines for the current screen rendering pass, which are respectively a rendering pipeline 602 and a rendering pipeline 603 .
  • the rendering pipeline 602 includes a vertex shader 6021 and a fragment shader 6022
  • the rendering pipeline 603 includes a vertex shader 6031 and a fragment shader 6032
  • the fragment shader 6022 and the fragment shader 6032 are original fragment shaders (ie No algorithm is set inside), so both the fragment shader 6022 and the fragment shader 6032 can be used to perform the rasterization step, where the fragment shader 6022 is used to perform the rasterization of the reflector, and the fragment shader 6032 is used to perform the reflected Rasterization of objects.
  • the GPU can input the rendering information of the reflective object into the rendering pipeline 602, so that the rendering pipeline 602 performs rasterization according to the rendering information of the reflective object, that is, rasterizes the reflective object to obtain an image of the reflective object.
  • the GPU can input the spatial information of the reflector (for example, the coordinates of the vertices of the reflector) into the vertex shader, and the vertex shader 6021 can perform certain calculation processing on this part of the spatial information, and input the processed spatial information Fragment shader 6022.
  • the GPU will directly input the material information of the reflector into the fragment shader 6022, so the fragment shader 6022 can perform rasterization operations based on the processed spatial information and the material information of the reflector to obtain the image in the reflector.
  • the GPU can also input the rendering information of all the reflected objects into the rendering pipeline 603, so that the rendering pipeline 603 performs rasterization according to the rendering information of all the reflected objects, that is, rasterizes all the reflected objects to obtain images of all the reflected objects.
  • the GPU can input the spatial information of all reflected objects (for example, the coordinates of vertices of all reflected objects) into the vertex shader, and the vertex shader 6031 can perform certain calculation processing on this part of the spatial information, and the processed The spatial information of is input to the fragment shader 6032.
  • the GPU will directly input the material information of all the reflected objects into the fragment shader 6032, so the fragment shader 6032 can perform rasterization operations based on the processed spatial information and the material information of all the reflected objects, so as to obtain all the reflected objects images in objects. So far, the GPU has successfully rendered the images of the reflector and all the reflected objects, for example, the image of the ground and the images of people in Figure 7 .
  • the GPU can fuse the image of the reflector, the images of all the reflected objects and the image of the reflection in the reflector to obtain the target for display
  • the image for example, the target image may present a three-dimensional scene as shown in FIG. 7 , that is, a person standing on the ground and a reflection of a person on the ground.
  • the CPU can acquire the target image from the current screen rendering buffer, and send the target image to the screen for display by the user.
  • the ray tracing phase and the rasterization phase may be performed synchronously or asynchronously, which is not limited here.
  • step 402 and step 404 (or step 405) can be performed simultaneously.
  • step 402 may be performed prior to step 404 (or step 405).
  • step 402 may be performed after step 404 (or step 405).
  • step 404 and step 405 in the rasterization stage may be performed synchronously or asynchronously, which is not limited here.
  • step 404 and step 405 can be performed simultaneously.
  • step 404 may be performed prior to step 405 .
  • step 404 may be performed after step 405 .
  • step 401 to step 407 can be performed on the reflector and all reflected objects in the three-dimensional scene, and the simulation of the three-dimensional scene can also be obtained. target image.
  • each material information in the P pieces of material information corresponds to at least one of the N reflected objects for a schematic illustration, and the quantity of material information and the number of reflected objects are not discussed.
  • Each of the reflected objects corresponds to at least one piece of material information among the P pieces of material information, then N ⁇ P, and P>1.
  • the GPU performs intersection calculation based on ray tracing based on the spatial information of the reflector and the spatial information of N reflected objects, M intersection points on N reflected objects and N reflected objects Corresponding to P pieces of material information.
  • the GPU determines the intersection point corresponding to the first material information, and performs coloring according to the spatial information of the intersection point corresponding to the first material information and the first material information.
  • the GPU can also perform the same operation on the second material information as it does on the first material information, until it completes the same operation on the P-th material information as on the first material information, so the GPU can successfully render An image reflected in a reflective object.
  • the GPU separates and executes the intersection calculation and coloring in ray tracing, that is, after uniformly determining the M intersection points, for any one of the P material information, the GPU can determine the corresponding All the intersection points, and the shading operations corresponding to these intersection points are completed, so the GPU does not need to use the material information in the subsequent shading operations corresponding to the remaining intersection points among the M intersection points. In this way, the number of times any material information is swapped in and out between the memory and the cache can be effectively reduced, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • Fig. 8 is a schematic diagram of an application example of the image rendering method provided by the embodiment of the present application. As shown in Fig. 8, in this application example, an additional plug-in can be added to the Unity program, and the plug-in is used to realize the 4 shows the method.
  • the program when the program is executing the initialization (awake) stage, the original data in the 3D scene can be passed into the plug-in.
  • the program When the program is executing the update stage of a certain frame, it can update the rendering information of the reflector corresponding to the frame, the rendering information of the reflected object, the spatial information of the camera, etc. based on the original data, and based on the Rendering information builds a BVH tree, and then updates this part of information to the plug-in.
  • the plug-in When the program executes the rendering (on post render) stage, the plug-in is called through the plug-in interface, and the plug-in transmits the aforementioned information to the GPU, and the GPU uses the rendering method provided by the plug-in to complete the rendering to obtain the final target image for display.
  • Fig. 9 is another schematic structural diagram of an image rendering device provided by an embodiment of the present application. As shown in Fig. 9, the image rendering device includes:
  • the intersection module 901 is used to perform intersection calculation based on ray tracing according to the spatial information of the reflector and the spatial information of the reflected object, to obtain M intersection points on the reflected object, and the reflected object corresponds to P material information, M ⁇ 1;
  • the coloring module 902 is configured to determine a plurality of intersection points corresponding to the target material information among the M intersection points, and perform coloring according to the spatial information of the plurality of intersection points corresponding to the target material information and the target material information, so as to obtain the reflected object in the reflector Reflected image, the target material information is one of multiple material information.
  • the GPU performs intersection calculation based on ray tracing based on the spatial information of the reflector and the spatial information of the reflected object
  • M intersection points on the reflected object can be obtained, and the reflected object corresponds to P pieces of material information.
  • the GPU determines all intersection points corresponding to the target material information, and performs coloring according to the spatial information of all intersection points corresponding to the target material information and the target material information, to obtain an image of the reflection of the reflected object in the reflector.
  • the GPU separates and executes the intersection calculation and shading in ray tracing, that is, after uniformly determining the M intersection points, for a certain material information among the P material information, the GPU can determine the corresponding All the intersection points, and the shading operations corresponding to these intersection points are completed, so the GPU does not need to use the material information in the subsequent shading operations corresponding to the remaining intersection points among the M intersection points. In this way, the number of times any material information is swapped in and out between the memory and the cache can be effectively reduced, thereby reducing memory bandwidth and optimizing the performance of electronic devices.
  • the plurality of material information is P
  • the target material information is the i-th material information
  • the initial value of i is 1
  • the intersection module is configured to determine, according to the spatial information of the camera, the spatial information of the reflector, and the hierarchical bounding box BVH tree, that the light from the camera acts on the reflected object after passing through the reflector.
  • the obtained M intersection points, the BVH tree is constructed according to the spatial information of the reflected object.
  • each reflected object has a unique number in the BVH tree, and the coloring unit is used for the number of the reflected object where the j-th intersection point is located at the i-th In the number of at least one reflected object corresponding to the material information, coloring is performed according to the spatial information of the j-th intersection point and the i-th material information.
  • the device further includes: a first rasterization module, configured to perform rasterization according to the spatial information of the reflector and the material information of the reflector to obtain an image of the reflector; a second rasterization module, It is used to perform rasterization according to the spatial information of the reflected object and the material information of the reflected object to obtain the image of the reflected object; the fusion module is used to combine the image of the reflected object, the image of the reflected object and the image reflected in the reflective object Fusion is performed to obtain the target image.
  • a first rasterization module configured to perform rasterization according to the spatial information of the reflector and the material information of the reflector to obtain an image of the reflector
  • a second rasterization module It is used to perform rasterization according to the spatial information of the reflected object and the material information of the reflected object to obtain the image of the reflected object
  • the fusion module is used to combine the image of the reflected object, the image of the reflected object and the image reflected
  • the spatial information includes at least one of a vertex coordinate, a vertex normal, and a triangle index.
  • the material information includes at least one of color, metallicity and roughness.
  • the embodiment of the present application also relates to a computer-readable storage medium, including computer-readable instructions.
  • the computer-readable instructions When the computer-readable instructions are run on the computer, the computer is made to execute the method described in the embodiment shown in FIG. 4 .
  • the embodiment of the present application also relates to a computer program product, including computer-readable instructions, and when the computer-readable instructions are run on a computer, the computer is made to execute the method described in the embodiment shown in FIG. 4 .
  • the disclosed system, device and method can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Generation (AREA)

Abstract

本申请提供一种图像渲染方法及其相关设备，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。本申请的方法包括：S1：根据反射物的空间信息以及被反射物的空间信息进行基于光线追踪的求交计算，得到被反射物上的M个交点，被反射物对应多个材质信息，M≥1；S2：在M个交点中，确定目标材质信息对应的多个交点，并根据目标材质信息对应的多个交点的空间信息以及目标材质信息进行着色，得到反射物中被反射物的倒影的图像，目标材质信息为多个材质信息中的一个。

Description

一种图像渲染方法及其相关设备
本申请要求于2021年09月30日提交中国专利局、申请号为202111163949.X、发明名称为“一种图像渲染方法及其相关设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机图形学技术领域，尤其涉及一种图像渲染方法及其相关设备。
背景技术
随着计算机行业的迅速发展，用户对图像的要求日益增高。当前，通常采用光栅化技术结合光线追踪技术对三维场景进行渲染，从而得到较为逼真的图像，以提高用户体验。
一般地，三维场景中通常包含反射物(例如，地面)和多个被反射物(地面上的人、椅子等等)。图形处理器(graphics processing unit，GPU)接收中央处理器(central processing unit，CPU)发送的反射物的空间信息、反射物的材质信息、多个被反射物的空间信息和多个被反射物的材质信息等三维场景的渲染信息后，可利用反射物的空间信息和多个被反射物的空间信息进行求交计算，从而确定位于多个被反射物上的多个交点。当每确定一个交点时，GPU可基于该交点的空间信息以及该交点所在的被反射物的材质信息进行着色，如此一来，在完成所有交点对应的着色操作后，可得到反射物中倒影(例如，地面上的人影)的图像。至此，则完成了基于光线追踪的图像渲染。
然而，GPU的高速缓冲存储器(cache)通常只能缓存一个材质信息，在多个被反射物中，可能有一部分被反射物对应同一个材质信息，故GPU在依次完成多个被反射物上的多个交点对应的着色操作时，往往同一个材质信息会被频繁地在内存和cache之间换入换出，导致内存带宽太大，电子设备难以承受。
发明内容
本申请实施例提供了一种图像渲染方法及其相关设备，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
本申请实施例的第一方面提供了一种图像渲染方法，该方法包括：
CPU可获取三维场景的渲染信息，由于三维场景中通常包含反射物(例如，地面等等)以及N个被反射物(例如，人、桌子以及椅子等等)，故CPU可将三维场景的渲染信息划分为反射物的渲染信息以及N个被反射物的渲染信息。然后，CPU将反射物的渲染信息(包含反射物的空间信息和材质信息)以及N个被反射物的渲染信息(包含N个被反射物的空间信息和材质信息)发送至GPU，以使得GPU进行图像渲染的光线追踪阶段，即执行以下步骤：
S1：GPU根据反射物的空间信息以及N个被反射物的空间信息进行基于光线追踪的求交计算，得到N个被反射物上的M个交点。需要说明的是，这N个被反射物为与经过反射物后的光线(来源于相机)相交的物体。在这N个被反射物中，每个被反射物上存在至少一个交点，故M≥N≥1。进一步地，N个被反射物对应P个(种)材质信息，若P个材质信息中的每个材质信息对应N个被反射物中的至少一个被反射物，则N＞P＞1，若P个材质信息中的 每个材质信息对应N个被反射物中的一个被反射物，则N＝P＞1，若N个被反射物中的每个被反射物对应P个材质信息中的至少一个材质信息，则N＜P，且P＞1。例如，设三维场景中包含人、椅子和桌子等被反射物，椅子和桌子是同种材料制作的。那么，人的材质信息为材质信息1，椅子和桌子的材质信息均为材质信息2。
S2：在M个交点中，确定目标材质信息对应的多个交点，并根据目标材质信息对应的多个交点的空间信息以及目标材质信息进行着色，得到反射物中被反射物的倒影的图像。(例如，反射物中被反射物的倒影的图像包含地面上人的影子的图像、桌子的影子的图像和椅子的影子的图像等等)。
从上述方法可以看出：GPU根据反射物的空间信息以及N个被反射物的空间信息进行基于光线追踪的求交计算后，可得到N个被反射物上的M个交点，N个被反射物对应P个材质信息。接着，在M个交点中，GPU确定目标材质信息对应的所有交点，并根据目标材质信息对应的所有交点的空间信息以及目标材质信息进行着色，得到反射物中被反射物的倒影的图像。前述过程中，GPU将光线追踪中的求交计算和着色剥离开来执行，即在统一确定M个交点后，对于P个材质信息中的某一个材质信息，GPU可确定该材质信息所对应的所有交点，并完成这些交点所对应的着色操作，故后续在M个交点中其余交点对应的着色操作中，GPU不需再用到该材质信息。如此一来，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
在一种可能的实现的方式中，目标材质信息为第i个材质信息，i的初始值为1，S2包括：S21：在M个交点中，确定第i个材质信息对应的多个交点，并根据第i个材质信息对应的多个交点的空间信息以及第i个材质信息进行着色；S22：令i＝i+1，并重新执行S21，直至i＝P，得到反射物中被反射物的倒影的图像。具体地，得到M个交点后，GPU可在M个交点中，确定P个材质信息中第1个材质信息对应的交点，并根据第1个材质信息对应的交点的空间信息以及第1个材质信息进行着色。然后，GPU可对第2个材质信息也可执行如同对第1个材质信息所执行的操作，直至对第P个材质信息完成如同对第1个材质信息所执行的操作，故GPU可成功渲染出反射物中倒影的图像。前述实现方式中，GPU根据反射物的空间信息以及N个被反射物的空间信息进行基于光线追踪的求交计算后，可得到N个被反射物上的M个交点，N个被反射物对应P个材质信息。接着，在M个交点中，GPU确定第1个材质信息对应的交点，并根据第1个材质信息对应的交点的空间信息以及第1个材质信息进行着色。然后，GPU对第2个材质信息也可执行如同对第1个材质信息所执行的操作，直至对第P个材质信息完成如同对第1个材质信息所执行的操作，那么，GPU可成功渲染出反射物中被反射物的倒影的图像。前述过程中，GPU将光线追踪中的求交计算和着色剥离开来执行，即在统一确定M个交点后，对于P个材质信息中的任意一个材质信息，GPU可确定该材质信息所对应的所有交点，并完成这些交点所对应的着色操作，故后续在M个交点中其余交点对应的着色操作中，GPU不需再用到该材质信息。如此一来，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
在一种可能的实现的方式中，S21包括：S211：若第j个交点所在的被反射物与第i个材质信息对应，则根据第j个交点的空间信息与第i个材质信息进行着色，得到反射物中倒影的图像的第j个像素点，j的初始值为1；S212：令j＝j+1，并重新执行S21，直至j＝M。前 述实现方式中，对于P个材质信息中的任意一个材质信息，GPU可对该材质信息和M个交点的空间信息进行处理。具体地，GPU可判断M个交点中的第1个交点所在的被反射物是否与该材质信息对应，若第1个交点所在的被反射物与该材质信息对应，则根据第1个交点的空间信息与该材质信息进行着色，得到反射物中倒影的图像的第1个像素点，若第1个交点所在的被反射物与该材质信息不对应，则结束。此后，GPU可对第2交点执行如同对第1个交点所执行的操作，直至对第M个交点完成如同第1个交点的操作。对于P个材质信息中的其余材质信息，GPU也可执行如前述的过程，从而顺利得到反射物中倒影的图像。
在一种可能的实现的方式中，S1包括：根据相机的空间信息、反射物的空间信息以及层次包围盒BVH树，确定来自相机的光线经过反射物后，作用于N个被反射物上所得到的M个交点，BVH树根据N个被反射物的空间信息构建。前述实现方式中，GPU将光线追踪的求交计算和着色完全剥离开来，先统一进行求交计算，从而得到N个被反射物上所得到的M个交点，有利于优化整个光线追踪的过程。
在一种可能的实现的方式中，在N个被反射物中，每个被反射物在BVH树中具有唯一的编号，S211包括：若第j个交点所在的被反射物的编号，位于第i个材质信息对应的至少一个被反射物的编号中，则根据第j个交点的空间信息与第i个材质信息进行着色。前述实现方式中，对于P个材质信息中的任意一个材质信息以及M个交点中的任意一个交点，GPU可判断该交点所在的被反射物的编号，是否位于该材质信息对应的所有被反射物的编号中，若是，则根据该交点的空间信息与该材质信息进行着色，若否，则结束。通过这种方式，可准确判断某个交点是否与某个材质信息相对应。
在一种可能的实现的方式中，该方法还包括：S4：根据反射物的空间信息和反射物的材质信息进行光栅化，得到反射物的图像；S5：根据N个被反射物的空间信息和N个被反射物的材质信息进行光栅化，得到N个被反射物的图像；S6：将反射物的图像、N个被反射物的图像和反射物中倒影的图像进行融合，得到目标图像。前述实现方式中，得到反射物的渲染信息(包含反射物的空间信息和反射物的材质信息)和所有被反射物体的渲染信息(包含所有被反射物的空间信息和所有被反射物的材质信息)后，GPU还可基于这些信息进行光栅化，从而得到反射物的图像和N个被反射物的图像。然后，GPU再将反射物的图像、N个被反射物的图像和反射物中倒影的图像进行融合，得到模拟三维场景的目标图像，至此，则完成了三维场景的图像渲染。此后，CPU可从缓冲区中获取目标图像，并将目标图像发送至屏幕中显示，以供用户观看。
在一种可能的实现的方式中，空间信息包含顶点坐标、顶点法线和三角形索引中的至少一种。
在一种可能的实现的方式中，材质信息包含颜色、金属度以及粗糙度中的至少一种。
本申请实施例的第二方面提供了一种图像渲染装置，该装置包括：求交模块，用于根据反射物的空间信息以及被反射物的空间信息进行基于光线追踪的求交计算，得到被反射物上的M个交点，被反射物对应P个材质信息，M≥1；着色模块，用于在M个交点中，确定目标材质信息对应的多个交点，并根据目标材质信息对应的多个交点的空间信息以及目标材质信息进行着色，得到反射物中被反射物的倒影的图像，目标材质信息为多个材质信息中的一个。
从上述装置可以看出：GPU根据反射物的空间信息以及N个被反射物的空间信息进行基于光线追踪的求交计算后，可得到N个被反射物上的M个交点，N个被反射物对应P个材质信息。接着，在M个交点中，GPU确定目标材质信息对应的所有交点，并根据目标材质信息对应的所有交点的空间信息以及目标材质信息进行着色，得到反射物中被反射物的倒影的图像。前述过程中，GPU将光线追踪中的求交计算和着色剥离开来执行，即在统一确定M个交点后，对于P个材质信息中的某一个材质信息，GPU可确定该材质信息所对应的所有交点，并完成这些交点所对应的着色操作，故后续在M个交点中其余交点对应的着色操作中，GPU不需再用到该材质信息。如此一来，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
在一种可能的实现的方式中，目标材质信息为第i个材质信息，i的初始值为1，着色模块包括：着色子模块：在M个交点中，确定第i个材质信息对应的多个交点，并根据第i个材质信息对应的多个交点的空间信息以及第i个材质信息进行着色；循环子模块：令i＝i+1，并重新触发着色子模块，直至i＝P，得到反射物中被反射物的倒影的图像。前述实现方式中，GPU根据反射物的空间信息以及N个被反射物的空间信息进行基于光线追踪的求交计算后，可得到N个被反射物上的M个交点，N个被反射物对应P个材质信息。接着，在M个交点中，GPU确定第1个材质信息对应的交点，并根据第1个材质信息对应的交点的空间信息以及第1个材质信息进行着色。然后，GPU对第2个材质信息也可执行如同对第1个材质信息所执行的操作，直至对第P个材质信息完成如同对第1个材质信息所执行的操作，故GPU可成功渲染出反射物中被反射物体的倒影的图像。前述过程中，GPU将光线追踪中的求交计算和着色剥离开来执行，即在统一确定M个交点后，对于P个材质信息中的任意一个材质信息，GPU可确定该材质信息所对应的所有交点，并完成这些交点所对应的着色操作，故后续在M个交点中其余交点对应的着色操作中，GPU不需再用到该材质信息。如此一来，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
在一种可能的实现的方式中，着色子模块，包括：着色单元，用于若第j个交点所在的被反射物与第i个材质信息对应，则根据第j个交点的空间信息与第i个材质信息进行着色，得到反射物中倒影的图像的第j个像素点，j的初始值为1；循环单元，用于令j＝j+1，并重新触发着色单元，直至j＝M。
在一种可能的实现的方式中，求交模块，用于根据相机的空间信息、反射物的空间信息以及层次包围盒BVH树，确定来自相机的光线经过反射物后，作用于N个被反射物上所得到的M个交点，BVH树根据N个被反射物的空间信息构建。
在一种可能的实现的方式中，在N个被反射物中，每个被反射物在BVH树中具有唯一的编号，着色单元，用于若第j个交点所在的被反射物的编号，位于第i个材质信息对应的至少一个被反射物的编号中，则根据第j个交点的空间信息与第i个材质信息进行着色。
在一种可能的实现的方式中，该装置还包括：第一光栅化模块，用于根据反射物的空间信息和反射物的材质信息进行光栅化，得到反射物的图像；第二光栅化模块，用于根据N个被反射物的空间信息和N个被反射物的材质信息进行光栅化，得到N个被反射物的图像；融合模块，用于将反射物的图像、N个被反射物的图像和反射物中倒影的图像进行融合，得到 目标图像。
在一种可能的实现的方式中，空间信息包含顶点坐标、顶点法线和三角形索引中的至少一种。
在一种可能的实现的方式中，材质信息包含颜色、金属度以及粗糙度中的至少一种。
本申请实施例的第三方面提供了一种电子设备，包括存储器和处理器；存储器存储有代码，处理器被配置为执行代码，当该代码被执行时，电子设备执行如第一方面或第一方面的任意一种可能的实现方式所述的方法。
本申请实施例的第四方面提供了一种计算机可读存储介质，包括计算机可读指令，当计算机可读指令在计算机上运行时，使得计算机执行如第一方面或第一方面的任意一种可能的实现方式所述的方法。
本申请实施例的第五方面提供了一种计算机程序产品，包括计算机可读指令，当计算机可读指令在计算机上运行时，使得计算机执行如第一方面或第一方面的任意一种可能的实现方式所述的方法。
本申请实施例中，GPU根据反射物的空间信息以及N个被反射物的空间信息进行基于光线追踪的求交计算后，可得到N个被反射物上的M个交点，N个被反射物对应P个材质信息。接着，在M个交点中，GPU确定第1个材质信息对应的交点，并根据第1个材质信息对应的交点的空间信息以及第1个材质信息进行着色。然后，GPU对第2个材质信息也可执行如同对第1个材质信息所执行的操作，直至对第P个材质信息完成如同对第1个材质信息所执行的操作，故GPU可成功渲染出反射物中倒影的图像。前述过程中，GPU将光线追踪中的求交计算和着色剥离开来执行，即在统一确定M个交点后，对于P个材质信息中的任意一个材质信息，GPU可确定该材质信息所对应的所有交点，并完成这些交点所对应的着色操作，故后续在M个交点中其余交点对应的着色操作中，GPU不需再用到该材质信息。如此一来，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
附图说明
图1为光线追踪技术的原理示意图；
图2为光栅化技术的原理示意图；
图3为本申请实施例提供的电子设备的一个结构示意图；
图4为本申请实施例提供的图像渲染方法的一个流程示意图；
图5为本申请实施例提供的BVH树的一种示意图；
图6为本申请实施例提供的图像渲染的一个示意图；
图7为本申请实施例提供的目标区域的一个示意图；
图8为本申请实施例提供的图像渲染的方法的一个应用例示意图；
图9为本申请实施例提供的图像渲染装置的另一结构示意图。
具体实施方式
本申请实施例提供了一种图像渲染方法及其相关设备，可有效减少任意一个材质信息在 内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或模块的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或模块，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或模块。在本申请中出现的对步骤进行的命名或者编号，并不意味着必须按照命名或者编号所指示的时间/逻辑先后顺序执行方法流程中的步骤，已经命名或者编号的流程步骤可以根据要实现的技术目的变更执行次序，只要能达到相同或者相类似的技术效果即可。
随着计算机技术的发展，越来越多的应用程序，例如游戏应用或者视频应用等应用程序，要求在电子设备上显示画质精美的图像。这些图像通常是由电子设备基于三维(three dimensional，3D)场景中的模型进行渲染得到的。
在传统的图像处理方法中，通常采用光栅化处理对3D场景进行渲染，以得到能够显示3D场景的图像。然而，采用光栅化技术进行渲染所得到的图像的品质一般，往往难以呈现逼真的画面。例如，渲染得到的图像中通常难以真实还原场景中的光线反射、折射以及阴影等效果。有鉴于此，一种新的渲染技术--光线追踪技术应运而生。光线追踪技术与光栅化技术都是用于实现图像渲染的方法，其主要目的是通过计算着色，将3D空间的物体，投影到二维的屏幕空间进行显示。
图1为光线追踪技术的原理示意图。如图1所示，光线追踪的原理为：从相机的位置，通过图像平面上的像素位置，发射一束光线到三维场景中，求光线和几何图形间最近的交点，再求该交点的着色。如果该交点的材质是反射性的，可以在该交点向的反射方向继续追踪，并且继续求取反射后的交点的着色。也就是说，光线追踪方法是通过追踪光线在三维场景中的传播过程，来计算投影与全局光照，从而渲染得到二维图像。
图2为光栅化技术的原理示意图。如图2所示，光栅化处理的原理为：将三维场景中的物体采用三角形进行分割，通过坐标变化计算将三角形顶点的三维坐标变换为图像上的二维坐标，最后在图像上的三角形内填充纹理，以实现图像的渲染。
由于光栅化技术是将屏幕空间上可见的内容直接投影至屏幕空间上，而得到相应的图像，其处理难度较低，所提供的光影效果也较差。光线追踪技术则是通过追踪从相机发出的每条光线来实现如反射、折射、阴影以及环境光遮蔽等真实效果，因此光线追踪方法能够提供真实逼真的光影效果。因此，为了渲染出较为逼真的图像，当前电子设备通常采用光栅化技术结合光线追踪技术对三维场景进行渲染，以提高用户的观看体验。
具体地，GPU接收CPU发送的反射物(例如，地面)的空间信息、反射物的材质信息、多个被反射物(地面上的人、椅子等等)的空间信息和多个被反射物的材质信息等三维场景的渲染信息后，可利用反射物的空间信息和多个被反射物的空间信息进行求交计算，从而确定位于多个被反射物上的多个交点。当每确定一个交点时，GPU可基于该交点的空间信息以及该交点所在的被反射物的材质信息进行着色，如此一来，在完成所有交点对应的着色操作后，可得到反射物中倒影(例如，地面上的人影)的图像。至此，则完成了图像渲染中的光 线追踪阶段。
进一步地，GPU还可利用前述的三维场景的渲染信息进行光栅化，从而得到反射物的图像和多个被反射物的图像，至此，则完成了图像渲染中的光栅化阶段。最后，可将反射物中倒影的图像、反射物的图像和多个被反射物的图像进行融合，从而得到模拟三维场景的目标图像，该目标图像包含三维场景中各个物体的图像以及某些物体中倒影的图像。
然而，GPU的高速缓冲存储器(cache)通常只能缓存一个材质信息，在多个被反射物中，可能有一部分被反射物对应同一个材质信息，故GPU在依次完成多个被反射物上的多个交点对应的着色操作时，往往同一个材质信息会被频繁地在内存和cache之间换入换出，导致内存带宽太大，电子设备难以承受。
为了解决上述问题，本申请实施例提供了一种图像渲染方法，该方法可以由电子设备来执行。该电子设备中包括CPU和GPU，能够对图像进行渲染处理。示例性地，该电子设备例如可以是手机(mobile phone)、平板电脑、笔记本电脑、PC、移动互联网设备(mobile internet device，MID)、可穿戴设备，虚拟现实(virtual reality，VR)设备、增强现实(augmented reality，AR)设备、工业控制(industrial control)中的无线电子设备、无人驾驶(self driving)中的无线电子设备、远程手术(remote medical surgery)中的无线电子设备、智能电网(smart grid)中的无线电子设备、运输安全(transportation safety)中的无线电子设备、智慧城市(smart city)中的无线电子设备、智慧家庭(smart home)中的无线电子设备等。该电子设备可以是运行安卓系统、IOS系统、windows系统以及其他系统的设备。在该电子设备中可以运行有需要对3D场景进行渲染而得到二维图像的应用程序，例如游戏应用、锁屏应用、地图应用或监控应用等应用。
为了便于理解，下面结合图3对电子设备的具体结构进行详细的介绍。可以参阅图3，图3为本申请实施例提供的电子设备的一个结构示意图。如图3所示，电子设备3000可以包括：中央处理器3001、图形处理器3002、显示设备3003和存储器3004。可选地，该电子设备3000还可以包括至少一个通信总线(图3中未示出)，用于实现各个组件之间的连接通信。
应当理解，电子设备3000中的各个组件还可以通过其他连接器相耦合，其他连接器可包括各类接口、传输线或总线等。电子设备3000中的各个组件还可以是以中央处理器3001为中心的放射性连接方式。在本申请的各个实施例中，耦合是指通过相互电连接或连通，包括直接相连或通过其他设备间接相连。
中央处理器3001和图形处理器3002的连接方式也有多种，并不局限于图2所示的方式。电子设备3000中的中央处理器3001和图形处理器3002可以位于同一个芯片上，也可以分别为独立的芯片。
下面对中央处理器3001、图形处理器3002、显示设备3003和存储器3004的作用进行简单的介绍。
中央处理器3001：用于运行操作系统3005和应用程序3006。应用程序3006可以为图形类应用程序，比如游戏、视频播放器等等。操作系统3005提供了系统图形库接口，应用程序3006通过该系统图形库接口，以及操作系统3005提供的驱动程序，比如图形库用户态驱动和/或图形库内核态驱动，生成用于渲染图形或图像帧的指令流，以及所需的相关渲染数据。其中，系统图形库包括但不限于：嵌入式开放图形库(open graphics library for embedded  system，OpenGL ES)、柯罗诺斯平台图形界面(the khronos platform graphics interface)或Vulkan(一个跨平台的绘图应用程序接口)等系统图形库。指令流包含一些列的指令，这些指令通常为对系统图形库接口的调用指令。
可选地，中央处理器3001可以包括以下至少一种类型的处理器：应用处理器、一个或多个微处理器、数字信号处理器(digital signal processor，DSP)、微控制器(microcontroller unit，MCU)或人工智能处理器等。
中央处理器3001还可进一步包括必要的硬件加速器，如专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field programmable gate array，FPGA)、或者用于实现逻辑运算的集成电路。处理器3001可以被耦合到一个或多个数据总线，用于在电子设备3000的各个组件之间传输数据和指令。
图形处理器3002：用于接收处理器3001发送的图形指令流，通过渲染管线(pipeline)生成渲染目标，并通过操作系统的图层合成显示模块将渲染目标显示到显示设备3003。其中，渲染管线也可以称为渲染流水线、像素流水线或像素管线，是图形处理器3002内部用于处理图形信号的并行处理单元。图形处理器3002中可以包括多个渲染管线，多个渲染管线之间可以相互独立地并行处理图形信号。例如，渲染管线可以在渲染图形或图像帧的过程中执行一些列操作，典型的操作可以包括：顶点处理(Vertex Processing)、图元处理(Primitive Processing)、光栅化(Rasterization)、片段处理(Fragment Processing)等等。
可选地，图形处理器3002可以包括执行软件的通用图形处理器，如GPU或其他类型的专用图形处理单元等。
显示设备3003：用于显示由电子设备3000生成的各种图像，该图像可以为操作系统的图形用户界面(graphical user interface，GUI)或由图形处理器3002处理的图像数据(包括静止图像和视频数据)。
可选地，显示设备3003可以包括任何合适类型的显示屏。例如液晶显示器(liquid crystal display，LCD)或等离子显示器或有机发光二极管(organic light-emitting diode，OLED)显示器等。
存储器3004，是中央处理器3001和图形处理器3002之间的传输通道，可以为双倍速率同步动态随机存储器(double data rate synchronous dynamic random access memory，DDR SDRAM)或者其它类型的缓存。
以上介绍了本申请实施例提供的图像渲染方法所应用的电子设备的具体结构，以下将详细介绍本申请实施例所提供的图像渲染方法的流程。首先对下文涉及的若干个名词进行解释：
离屏渲染通道(offscreen render pass)：通过该通道渲染得到的图像，通常为暂时不需在屏幕中显示的图像，这部分图像可存储于分配给离屏渲染通道的帧缓冲区(framebuffer)，将该帧缓冲区可称为离屏渲染缓冲区，用于存储暂时不需在屏幕中显示的图像。在本申请中，离屏渲染通道可用于实现光线追踪技术。
当前屏幕渲染通道(onscreen render pass)：通过该通道渲染得到的图像，通常为即将在屏幕中显示的图像(即送显的图像)，这部分图像可存储于分配给当前屏幕渲染通道的帧缓冲区(framebuffer)，将该帧缓冲区可称为当前屏幕渲染缓冲区，用于存储即将在屏幕中显示的图像。在本申请中，当前屏幕渲染通道即可用于实现光栅化技术。
图4为本申请实施例提供的图像渲染方法的一个流程示意图，如图4所示，该方法包括：
401、获取反射物的空间信息、反射物的材质信息、N个被反射物的空间信息以及N个被反射物的材质信息；
电子设备的CPU在获取某个三维场景的模型文件后，可从中解析出三维场景中各个物体的渲染信息。每个物体的渲染信息包含该物体的空间信息以及该物体的材质信息，其中，物体的空间信息包含物体的顶点坐标、物体的顶点法线以及三角形索引等信息，物体的材质信息包含物体的颜色、物体的金属度以及物体的粗糙度等信息。可以理解的是，不同物体的材质信息既可以是同一个(种)材质信息(即不同物体对应同一个材质信息)，也可以是多个(种)材质信息(即不同物体对应多个材质信息)，可见，物体的数量一般大于或等于材质信息的数量。例如，设三维场景中包含地面、人、椅子和桌子，椅子和桌子是同种材料制作的。那么，地面的材质信息为材质信息A，人的材质信息为材质信息B，椅子和桌子的材质信息均为材质信息C。
CPU在得到物体的材质信息后，可基于物体的材质信息确定物体的类别。具体地，CPU在得到某个物体的材质信息后，可基于该物体的金属度判断该物体是否为反射物，若该物体的金属度大于或等于预置的阈值，则可确定该物体为反射物，若该物体的金属度小于预置的阈值，则可确定该物体为被反射物。例如，地面、镜子以及湖泊等物体的金属度通常大于预置的阈值，故这部分物体可称为反射物，而人以及桌子等物体的金属度通常小于预置的阈值，故这部分物体可称为被反射物。如此一来，CPU可将三维场景中的所有物体分为两类物体，即反射物和被反射物。
为了便于说明，下文以该三维场景中仅包含一个反射物以及多个被反射物进行示意性说明。确定该三维场景中的反射物和所有被反射物后，CPU可基于所有被反射物的空间信息构建BVH树，BVH树可用于实现后续的光线追踪操作。具体地，CPU可根据被反射物的顶点坐标、顶点法线以及三角形索引构建BVH树，可以理解的是，BVH树包含多个包围盒的空间信息，每个包围盒的空间信息包含该包围盒(长方体)的8个顶点的坐标以及8个顶点的垂直高度，每个包围盒用于包围至少一个被反射物。为了进一步理解前述的BVH树，下文结合图5作进一步的介绍。图5为本申请实施例提供的BVH树的一种示意图，如图5所示，设三维场景中存在6个被反射物，基于这6个被反射物的空间信息可确定包围盒A(包围这6个物)的空间信息，基于其中4个被反射物的空间信息可确定包围盒B(包围这4个物)的空间信息，基于剩余2个被反射物的空间信息可确定包围盒C(包围这2个物)的空间信息，并将包围盒A的空间信息、包围盒B的空间信息和包围盒C的空间信息以二叉树的结构进行管理，得到BVH树。可见，BVH树可集中管理包围盒A、包围盒B和包围盒C的空间信息，相当于集中管理包围盒A、包围盒B和包围盒C。其中，包围盒A即为所有包围盒中的最大包围盒。值得注意的是，这6个被反射物中的每个被反射物在BVH树中具有唯一的编号，如图5所示，从左往右数，这6个被反射物的编号依次为11、12、13、14、21和22。
在电子设备的GPU基于BVH树实现光线追踪的时候，需要计算光线是否和三维场景中的被反射物是否相交(即求交计算)。由于BVH树的存在，GPU可基于BVH树确定包围被反射物的某个包围盒，然后判断光线是否和该包围盒相交，若光线没有碰到该包围盒，表明光线一定不会和该包围盒里的被反射物相交；如果光线碰到该包围盒，那么再计算光线是否 和该包围盒中的被反射物相交。例如，在检测到光线与二叉树中的B包围盒不相交时，则表示该光线一定不会和B包围盒中的四个被反射物相交，因此，可以免去检测光线是否与B包围盒中的四个被反射物相交的步骤，从而仅检测光线是否与C包围盒中的两个被反射物相交。
此外，CPU还可获取相机的空间信息，该相机用于拍摄模拟三维场景的目标图像。其中，相机的空间信息可包括相机的垂直高度以及相机的坐标。
CPU得到反射物的渲染信息(包含反射物的空间信息和反射物的材质信息)、所有被反射物体的渲染信息(包含所有被反射物的空间信息和所有被反射物的材质信息)、BVH树以及相机的空间信息后，则将这部分信息发送至GPU，以使得GPU根据这部分信息进行图像渲染。
402、根据反射物的空间信息以及N个被反射物的空间信息进行基于光线追踪的求交计算，得到N个被反射物上的M个交点，N个被反射物对应P个材质信息，M≥N≥P≥1。
403、在M个交点中，确定第i个材质信息对应的交点，并根据第i个材质信息对应的交点的空间信息以及第i个材质信息进行着色，i的初始值为1。
404、令i＝i+1，并重新执行403，直至i＝P，得到反射物中倒影的图像。
GPU得到反射物的渲染信息(包含反射物的空间信息和反射物的材质信息)、所有被反射物体的渲染信息(包含所有被反射物的空间信息和所有被反射物的材质信息)、BVH树以及相机的空间信息后，可基于这部分信息进行图像渲染。
需要说明的是，GPU所进行的图像渲染包含两个阶段，即光线追踪阶段(即本实施例中的步骤402至步骤404)和光栅化阶段(即本实施例中的步骤405至步骤406)。其中，光线追踪阶段包含两个子阶段，第一个子阶段为求交计算，第二个子阶段为着色，GPU可将这两个子阶段完全剥离开来执行，从而优化整个光线追踪的过程。
为了便于理解GPU实现光线追踪的过程，下文结合图6对前述光线追踪的过程做具体的介绍。图6为本申请实施例提供的图像渲染的一个示意图。如图6所示，GPU为离屏渲染通道设置有一个渲染管线601，渲染管线601包含顶点着色器(vertex shader，VS)6011、片段着色器(frag shader，FS)6012和片段着色器6013。其中，片段着色器6012内设置有光线追踪算法中与求交计算相关的部分，片段着色器6013内设置有光线追踪算法中与着色相关的部分，那么，片段着色器6012和片段着色器6013可共同用于实现光线追。
基于此，下文将分别对光线追踪的两个子阶段进行介绍：
(1)GPU可通过顶点着色器6011和片段着色器6012，对反射物的空间信息和BVH树进行处理，即通过这两个着色器根据这些信息进行基于光线追踪的求交计算，得到N个被反射物上的M个交点，可以理解的是，该N个被反射物为所有被反射物中，与相机发射的光线存在交点的被反射物。在这N个被反射物中，每个被反射物存在至少一个交点，故M≥N≥1。进一步地，基于前述物体与材质信息之间的对应关系可知，该N个被反射物对应P个(种)材质信息，且N≥P≥1。
然而，若对整个反射物进行基于光线追踪的求交计算，即以相机为光源，对反射物的每个顶点均发射光线，然后对每条光线均进行路线计算和求交计算，涉及的计算量相当大。而且，并不是每条光线均能与被反射物相交，那么基于无效光线(即不与被反射物相交的光线)所进行的计算则为无效的计算。可以理解的是，反射物中的倒影实际上就是被反射物在反射 物中的影响或镜像等等，故可在反射物中将包含倒影的某一个块区域确定为目标区域，若光线射向反射物的目标区域，那么这部分光线(有效光线)在经过该区域后，则会与被反射物相交。若光线射向反射物中，除目标区域之外的其余区域，那么这部分光线在经过该区域后，则不会与被反射物相交。由此可见，若只对目标区域进行基于光线追踪的求交计算，则可以有效减小光线追踪的计算量，从而节省GPU的算力，降低图像渲染的功耗。
值得注意的是，GPU可通过如下方式确定反射物的目标区域：
GPU可从BVH树中确定最大包围盒的空间信息，然后根据最大包围盒的空间信息以及相机的空间信息，在反射物的空间信息中确定反射物的目标区域的空间信息。得到反射物的目标区域的空间信息后，则相当于确定反射物的目标区域。具体地，设最大包围盒中的任意一个顶点为第一目标点，设反射物的目标区域中的任意一个顶点为第二目标点。那么，相机的垂直高度(H1)，第一目标点的垂直高度(H2)，第二目标点与相机之间的距离(X1)，以及相机与第一目标点之间的距离(X2)，这四者之间存在如下的数学关系：H1/X1＝H2/(X2-X1)，需要说明的是，X1基于相机的坐标以及第二目标点的坐标确定，X2基于相机的坐标以及第一目标点的坐标确定。
由于相机的坐标、相机的垂直高度、第一目标点的坐标以及第一目标点的垂直高度均为已知值，只有第二目标点的坐标为未知数。反射物的空间信息包含反射物中每一个顶点的坐标，故GPU可判断反射物中每一个顶点的坐标是否满足前述数学关系，若某一顶点的坐标满足前述数学关系，则该顶点为反射物的目标区域中的一个顶点。如此一来，则可将反射物划分为目标区域，以及除目标区域外的其余区域。相应地，GPU也可将反射物的空间信息划分为目标区域的空间信息以及其余区域的空间信息，但对于目标区域和其余区域，两个区域的材质信息是相同的(即反射物的材质信息，因为整个反射物的材质信息是统一的)。
下面结合图7对上述确定反射物的目标区域的过程作进一步的介绍。图7为本申请实施例提供的目标区域的一个示意图，如图7所示，设某个三维场景为某个人物站在地面，地面上有人的倒影。可先确定该场景中的最大包围盒，该最大包围盒即围住人物的包围盒。基于最大包围盒的空间信息以及相机的空间信息，则可确定出地面的目标区域，该目标区域包含人物的倒影。
在得到反射物的目标区域的空间信息后，GPU可将反射物的目标区域的空间信息(例如，反射物的顶点的坐标以及反射物的顶点的法线)输入顶点着色器6011，顶点着色器6011可对这部分空间信息进行一定的计算处理(例如，将三维的坐标转换为二维的坐标等等)，并将处理后的空间信息输入片段着色器6012。此外，GPU还会将相机的空间信息和BVH树直接输入片段着色器6012，故片段着色器6012可基于相机的空间信息、处理后的空间信息以及BVH树进行求交计算，从而确定来自相机的光线经过反射物后，作用于N个被反射物上所得到的M个交点。
GPU在确定M个交点后，可保存每个交点所在的被反射物的编号以及该交点的空间信息(交点本质上是顶点，故交点的空间信息包含该顶点的坐标和法线等等)。
(2)得到M个交点后，GPU可通过片段着色器6013在M个交点中，确定P个材质信息中的第1个材质信息对应的交点，并根据第1个材质信息对应的交点的空间信息以及第1个材质信息进行着色。需要说明的是，GPU对第2个材质信息也可执行如同对第1个材质信 息所执行的操作，直至对第P个材质信息完成如同对第1个材质信息所执行的操作，故GPU可成功渲染出反射物中倒影的图像，例如，图7中人物的倒影的图像。
为了理解前述过程，下文结合一个例子作进一步的介绍。设存在交点1、交点2、交点3、交点4、交点5和交点6，其中，交点1和交点2位于被反射物11上，交点3和交点4位于被反射物12上，交点5位于被反射物21上，交点6位于被反射物22上，被反射物11对应材质信息1，被反射物12对应材质信息2，被反射物21和被反射物22对应材质信息3。那么，GPU可确定材质信息1对应的交点为交点1和交点2，并根据这两个交点的空间信息和材质信息1进行着色。接着，GPU可确定材质信息2对应的交点为交点3和交点4，并根据这两个交点的空间信息和材质信息2进行着色。然后，GPU可确定材质信息3对应的交点为交点5和交点6，并根据这两个交点的空间信息和材质信息3进行着色。如此一来，GPU可得到反射物中倒影的图像。
值得注意的是，在M个交点中，GPU基于一个交点的信息可渲染出一个像素点(即反射物中倒影的图像包含M个像素点)，那么，GPU可通过以下方式渲染反射物中倒影的图像：
GPU可将P个材质信息中的第1个材质信息和M个交点的空间信息输入片段着色器6013。那么，输入片段着色器6013可判断M个交点中的第1个交点所在的被反射物是否与第1个材质信息对应，若第1个交点所在的被反射物与第1个材质信息对应，片段着色器6013则根据第1个交点的空间信息与第1个材质信息进行着色，得到反射物中倒影的图像的第1个像素点，若第1个交点所在的被反射物与第1个材质信息不对应，则结束。此后，片段着色器6013可对第2交点执行如同对第1个交点所执行的操作，直至对第M个交点完成如同第1个交点的操作。
然后，GPU可将P个材质信息中的第2个材质信息和M个交点的空间信息输入片段着色器6013，以使得片段着色器6013重新执行以上过程，直至GPU将第P个材质信息和M个交点的空间信息输入片段着色器6013，以使得对片段着色器6013重新执行以上过程。至此，片段着色器6013可得到反射物中倒影的图像的M个像素点，相当于得到反射物中倒影的图像。
值得注意的是，片段着色器6013可通过以下方式确定某一个交点所在的被反射物是否与某一个材质信息对应：
对于P个材质信息中的任意一个材质信息以及M个交点中的任意一个交点，片段着色器6013可判断该交点所在的被反射物的编号(可由GPU输入至片段着色器6013)，是否位于该材质信息对应的所有被反射物的编号(可由GPU输入至片段着色器6013)中，若是，则根据该交点的空间信息与该材质信息进行着色，若否，则结束。
得到反射物中倒影的图像后，GPU可完成了图像渲染的光线追踪阶段。
405、根据反射物的空间信息和反射物的材质信息进行光栅化，得到反射物的图像；
406、根据N个被反射物的空间信息和N个被反射物的材质信息进行光栅化，得到N个被反射物的图像；
得到反射物的渲染信息(包含反射物的空间信息和反射物的材质信息)和所有被反射物体的渲染信息(包含所有被反射物的空间信息和所有被反射物的材质信息)后，GPU还可基于这些信息实现图像渲染的光栅化阶段。
为了便于理解GPU实现光栅化的过程，下文结合图6对前述光栅化的过程做具体的介绍。如图6所示，GPU为当前屏幕渲染通道设置有两个渲染管线，分别为渲染管线602和渲染管线603。渲染管线602包含一个顶点着色器6021和一个片段着色器6022，渲染管线603包含一个顶点着色器6031和一个片段着色器6032，片段着色器6022和片段着色器6032均为原始的片段着色器(即内部未设置任何算法)，故片段着色器6022和片段着色器6032均可用于执行光栅化的步骤，其中，片段着色器6022用于执行反射物的光栅化，片段着色器6032用于执行被反射物的光栅化。
因此，GPU可将反射物的渲染信息输入渲染管线602，使得渲染管线602根据反射物的渲染信息进行光栅化，即对反射物进行光栅化，得到反射物的图像。具体地，GPU可将反射物的空间信息(例如，反射物的顶点的坐标)输入顶点着色器，顶点着色器6021可对这部分空间信息进行一定的计算处理，并将处理后的空间信息输入片段着色器6022。此外，GPU还会将反射物的材质信息直接输入片段着色器6022，故片段着色器6022可基于处理后的空间信息以及反射物的材质信息进行光栅化操作，从而得到反射物中的图像。
GPU还可将所有被反射物的渲染信息输入渲染管线603，使得渲染管线603根据所有被反射物的渲染信息进行光栅化，即对所有被反射物进行光栅化，得到所有被反射物的图像。具体地，GPU可将所有被反射物的空间信息(例如，所有被反射物的顶点的坐标)输入顶点着色器，顶点着色器6031可对这部分空间信息进行一定的计算处理，并将处理后的空间信息输入片段着色器6032。此外，GPU还会将所有被反射物的材质信息直接输入片段着色器6032，故片段着色器6032可基于处理后的空间信息以及所有被反射物的材质信息进行光栅化操作，从而得到所有被反射物中的图像。至此，GPU成功渲染出反射物的图像以及所有被反射物的图像，例如，图7中地面的图像以及人的图像。
407、将反射物的图像、N个被反射物的图像和反射物中倒影的图像进行融合，得到目标图像。
得到反射物的图像、所有被反射物的图像和反射物中倒影的图像后，GPU可将反射物的图像、所有被反射物的图像和反射物中倒影的图像进行融合，得到送显的目标图像，例如，目标图像可呈现出图7所示的三维场景，即人物站在地面，且地面上有人的倒影。此后，CPU可从当前屏幕渲染缓冲区中获取目标图像，并将目标图像发送至屏幕中显示，以供用户观看。
应理解，本实施例中，光线追踪阶段和光栅化阶段可同步进行，也可异步进行，此处不做限制。例如，步骤402和步骤404(或步骤405)可同时进行。又如，步骤402可先于步骤404(或步骤405)进行。再如，步骤402可后于步骤404(或步骤405)进行。
还应理解，本实施例中，光栅化阶段中的步骤404和步骤405可同步进行，也可异步进行，此处不做限制。例如，步骤404和步骤405可同时进行。又如，步骤404可先于步骤405进行。再如，步骤404可后于步骤405进行。
还应理解，本实施例中，仅以该三维场景包含一个反射物进行示意性说明，并不对该三维场景中反射物的数量构成限制，该三维场景中也可包含多个反射物，那么，对于该三维场景中的任意一个反射物而言，可对该反射物和该三维场景中的所有被反射物，均执行如同步骤401至步骤407中的操作，则同样能够得到模拟该三维场景的目标图像。
还应理解，本实施例中，仅以P个材质信息中的每个材质信息对应N个被反射物中的至少一个被反射物为例进行示意性说明，并不对材质信息的数量和被反射物的数量之间的对应关系构成限制，例如，若P个材质信息中的每个材质信息对应N个被反射物中的一个被反射物，则N＝P＞1，又如，若N个被反射物中的每个被反射物对应P个材质信息中的至少一个材质信息，则N＜P，且P＞1。
本申请实施例中，GPU根据反射物的空间信息以及N个被反射物的空间信息进行基于光线追踪的求交计算后，可得到N个被反射物上的M个交点，N个被反射物对应P个材质信息。接着，在M个交点中，GPU确定第1个材质信息对应的交点，并根据第1个材质信息对应的交点的空间信息以及第1个材质信息进行着色。然后，GPU对第2个材质信息也可执行如同对第1个材质信息所执行的操作，直至对第P个材质信息完成如同对第1个材质信息所执行的操作，故GPU可成功渲染出反射物中倒影的图像。前述过程中，GPU将光线追踪中的求交计算和着色剥离开来执行，即在统一确定M个交点后，对于P个材质信息中的任意一个材质信息，GPU可确定该材质信息所对应的所有交点，并完成这些交点所对应的着色操作，故后续在M个交点中其余交点对应的着色操作中，GPU不需再用到该材质信息。如此一来，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
此外，还可将结合一个具体的应用例，对本申请实施例提供的图像渲染的方法作进一步的介绍。图8为本申请实施例提供的图像渲染的方法的一个应用例示意图，如图8所示，该应用例中，可在Unity程序，加入能够实现一个额外的插件，该插件用于实现如图4所示的方法。
具体地，当程序在执行初始化(awake)阶段，可将三维场景中的原始数据传入插件中。当程序在执行某帧画面的更新(update)阶段，可基于原始数据，更新该帧画面对应的反射物的渲染信息、被反射物的渲染信息、相机的空间信息等，并基于被反射物的渲染信息构建BVH树，然后将这部分信息更新至插件中。当程序执行渲染(on post render)阶段时，通过插件的接口调用插件，插件将前述信息传入GPU，GPU使用插件提供的渲染方式，完成渲染得到最终送显的目标图像。
以上是对本申请实施例提供的图像渲染的方法所进行的详细说明，以下将对本申请实施例提供的图像渲染装置进行介绍。图9为本申请实施例提供的图像渲染装置的另一结构示意图，如图9所示，该图像渲染装置包括：
求交模块901，用于根据反射物的空间信息以及被反射物的空间信息进行基于光线追踪的求交计算，得到被反射物上的M个交点，被反射物对应P个材质信息，M≥1；
着色模块902，用于在M个交点中，确定目标材质信息对应的多个交点，并根据目标材质信息对应的多个交点的空间信息以及目标材质信息进行着色，得到反射物中被反射物的倒影的图像，目标材质信息为多个材质信息中的一个。
本申请实施例中，GPU根据反射物的空间信息以及被反射物的空间信息进行基于光线追踪的求交计算后，可得到被反射物上的M个交点，被反射物对应P个材质信息。接着，在M个交点中，GPU确定目标材质信息对应的所有交点，并根据目标材质信息对应的所有交点的空间信息以及目标材质信息进行着色，得到反射物中被反射物的倒影的图像。前述过程中， GPU将光线追踪中的求交计算和着色剥离开来执行，即在统一确定M个交点后，对于P个材质信息中的某一个材质信息，GPU可确定该材质信息所对应的所有交点，并完成这些交点所对应的着色操作，故后续在M个交点中其余交点对应的着色操作中，GPU不需再用到该材质信息。如此一来，可有效减少任意一个材质信息在内存和cache之间换入换出的次数，从而降低内存带宽，优化电子设备的性能。
在一种可能的实现方式中，多个材质信息为P个，目标材质信息为第i个材质信息，i的初始值为1，着色模块902包括：着色子模块：在M个交点中，确定第i个材质信息对应的多个交点，并根据第i个材质信息对应的多个交点的空间信息以及第i个材质信息进行着色；循环子模块：令i＝i+1，并重新触发着色子模块，直至i＝P，得到反射物中被反射物的倒影的图像。
在一种可能的实现方式中，着色子模块，包括：着色单元，用于若第j个交点所在的被反射物与第i个材质信息对应，则根据第j个交点的空间信息与第i个材质信息进行着色，得到反射物中倒影的图像的第j个像素点，j的初始值为1；循环单元，用于令j＝j+1，并重新触发着色单元，直至j＝M。
在一种可能的实现方式中，求交模块，用于根据相机的空间信息、反射物的空间信息以及层次包围盒BVH树，确定来自相机的光线经过反射物后，作用于被反射物上所得到的M个交点，BVH树根据被反射物的空间信息构建。
在一种可能的实现方式中，在被反射物中，每个被反射物在BVH树中具有唯一的编号，着色单元，用于若第j个交点所在的被反射物的编号，位于第i个材质信息对应的至少一个被反射物的编号中，则根据第j个交点的空间信息与第i个材质信息进行着色。
在一种可能的实现方式中，该装置还包括：第一光栅化模块，用于根据反射物的空间信息和反射物的材质信息进行光栅化，得到反射物的图像；第二光栅化模块，用于根据被反射物的空间信息和被反射物的材质信息进行光栅化，得到被反射物的图像；融合模块，用于将反射物的图像、被反射物的图像和反射物中倒影的图像进行融合，得到目标图像。
在一种可能的实现方式中，空间信息包含顶点坐标、顶点法线和三角形索引中的至少一种。
在一种可能的实现方式中，材质信息包含颜色、金属度以及粗糙度中的至少一种。
需要说明的是，上述装置各模块/单元之间的信息交互、执行过程等内容，由于与本申请方法实施例基于同一构思，其带来的技术效果与本申请方法实施例相同，具体内容可参考本申请实施例前述所示的方法实施例中的叙述，此处不再赘述。
本申请实施例还涉及一种计算机可读存储介质，包括计算机可读指令，当计算机可读指令在计算机上运行时，使得计算机执行如图4所示实施例中所述的方法。
本申请实施例还涉及一种计算机程序产品，包括计算机可读指令，当计算机可读指令在计算机上运行时，使得计算机执行如图4所示实施例中所述的方法。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分， 仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。

Claims (19)

  1. 一种图像渲染方法，其特征在于，所述方法包括：
    S1：根据反射物的空间信息以及被反射物的空间信息进行基于光线追踪的求交计算，得到所述被反射物上的M个交点，所述被反射物对应多个材质信息，M≥1；
    S2：在所述M个交点中，确定目标材质信息对应的多个交点，并根据所述目标材质信息对应的多个交点的空间信息以及所述目标材质信息进行着色，得到所述反射物中所述被反射物的倒影的图像，所述目标材质信息为所述多个材质信息中的一个。
  2. 根据权利要求1所述的方法，其特征在于，所述多个材质信息为P个，所述目标材质信息为第i个材质信息，i的初始值为1，所述S2包括：
    S21：在所述M个交点中，确定所述第i个材质信息对应的多个交点，并根据所述第i个材质信息对应的所述多个交点的空间信息以及所述第i个材质信息进行着色；
    S22：令i＝i+1，并重新执行所述S21，直至i＝P，得到所述反射物中所述被反射物的倒影的图像。
  3. 根据权利要求2所述的方法，其特征在于，所述S21包括：
    S211：若第j个交点所在的被反射物与第i个材质信息对应，则根据所述第j个交点的空间信息与所述第i个材质信息进行着色，得到所述反射物中倒影的图像的第j个像素点，j的初始值为1；
    S212：令j＝j+1，并重新执行所述S211，直至j＝M。
  4. 根据权利要求3所述的方法，其特征在于，所述S1包括：
    根据相机的空间信息、反射物的空间信息以及层次包围盒BVH树，确定来自所述相机的光线经过所述反射物后，作用于所述被反射物上所得到的M个交点，所述BVH树根据被反射物的空间信息构建。
  5. 根据权利要求4所述的方法，其特征在于，在所述被反射物中，每个被反射物在所述BVH树中具有唯一的编号，所述S211包括：
    若第j个交点所在的被反射物的编号，位于第i个材质信息对应的至少一个被反射物的编号中，则根据所述第j个交点的空间信息与所述第i个材质信息进行着色。
  6. 根据权利要求1至5任意一项所述的方法，其特征在于，所述方法还包括：
    S4：根据所述反射物的空间信息和所述反射物的材质信息进行光栅化，得到所述反射物的图像；
    S5：根据所述被反射物的空间信息和所述被反射物的材质信息进行光栅化，得到所述被反射物的图像；
    S6：将所述反射物的图像、所述被反射物的图像和所述反射物中倒影的图像进行融合，得到目标图像。
  7. 根据权利要求1至6任意一项所述的方法，其特征在于，所述空间信息包含顶点坐标、顶点法线和三角形索引中的至少一种。
  8. 根据权利要求1至7任意一项所述的方法，其特征在于，所述材质信息包含颜色、金属度以及粗糙度中的至少一种。
  9. 一种图像渲染装置，其特征在于，所述装置包括：
    求交模块，用于根据反射物的空间信息以及被反射物的空间信息进行基于光线追踪的求交计算，得到所述被反射物上的M个交点，所述被反射物对应P个材质信息，M≥1；
    着色模块，用于在所述M个交点中，确定目标材质信息对应的多个交点，并根据所述目标材质信息对应的多个交点的空间信息以及所述目标材质信息进行着色，得到所述反射物中所述被反射物的倒影的图像，所述目标材质信息为所述多个材质信息中的一个。
  10. 根据权利要求9所述的装置，其特征在于，所述多个材质信息为P个，所述目标材质信息为第i个材质信息，i的初始值为1，所述着色模块包括：
    着色子模块：在所述M个交点中，确定所述第i个材质信息对应的多个交点，并根据所述第i个材质信息对应的所述多个交点的空间信息以及所述第i个材质信息进行着色；
    循环子模块：令i＝i+1，并重新触发所述着色子模块，直至i＝P，得到所述反射物中所述被反射物的倒影的图像。
  11. 根据权利要求10所述的装置，其特征在于，所述着色子模块，包括：
    着色单元，用于若第j个交点所在的被反射物与第i个材质信息对应，则根据所述第j个交点的空间信息与所述第i个材质信息进行着色，得到所述反射物中倒影的图像的第j个像素点，j的初始值为1；
    循环单元，用于令j＝j+1，并重新触发所述着色单元，直至j＝M。
  12. 根据权利要求11所述的装置，其特征在于，所述求交模块，用于根据相机的空间信息、反射物的空间信息以及层次包围盒BVH树，确定来自所述相机的光线经过所述反射物后，作用于所述被反射物上所得到的M个交点，所述BVH树根据被反射物的空间信息构建。
  13. 根据权利要求12所述的装置，其特征在于，在所述被反射物中，每个被反射物在所述BVH树中具有唯一的编号，所述着色单元，用于若第j个交点所在的被反射物的编号，位于第i个材质信息对应的至少一个被反射物的编号中，则根据所述第j个交点的空间信息与所述第i个材质信息进行着色。
  14. 根据权利要求9至13任意一项所述的装置，其特征在于，所述装置还包括：
    第一光栅化模块，用于根据所述反射物的空间信息和所述反射物的材质信息进行光栅化，得到所述反射物的图像；
    第二光栅化模块，用于根据所述被反射物的空间信息和所述被反射物的材质信息进行光栅化，得到所述被反射物的图像；
    融合模块，用于将所述反射物的图像、所述被反射物的图像和所述反射物中倒影的图像进行融合，得到目标图像。
  15. 根据权利要求9至14任意一项所述的装置，其特征在于，所述空间信息包含顶点坐标、顶点法线和三角形索引中的至少一种。
  16. 根据权利要求9至15任意一项所述的装置，其特征在于，所述材质信息包含颜色、金属度以及粗糙度中的至少一种。
  17. 一种电子设备，其特征在于，包括存储器和处理器；所述存储器存储有代码，所述处理器被配置为执行所述代码，当所述代码被执行时，所述电子设备执行如权利要求1至8任一所述的方法。
  18. 一种计算机可读存储介质，其特征在于，包括计算机可读指令，当所述计算机可读指 令在计算机上运行时，使得所述计算机执行如权利要求1至8中任一项所述的方法。
  19. 一种计算机程序产品，其特征在于，包括计算机可读指令，当所述计算机可读指令在计算机上运行时，使得所述计算机执行如权利要求1至8任一项所述的方法。
PCT/CN2022/122427 2021-09-30 2022-09-29 一种图像渲染方法及其相关设备 Ceased WO2023051662A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US18/696,019 US20240378793A1 (en) 2021-09-30 2022-09-29 Image rendering method and related device
EP22875045.1A EP4386682A4 (en) 2021-09-30 2022-09-29 IMAGE DISPLAY METHOD AND ASSOCIATED DEVICE THEREFOR
CN202280057482.7A CN117957576A (zh) 2021-09-30 2022-09-29 一种图像渲染方法及其相关设备

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111163949.X 2021-09-30
CN202111163949.XA CN115908683A (zh) 2021-09-30 2021-09-30 一种图像渲染方法及其相关设备

Publications (1)

Publication Number Publication Date
WO2023051662A1 true WO2023051662A1 (zh) 2023-04-06

Family

ID=85739430

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/122427 Ceased WO2023051662A1 (zh) 2021-09-30 2022-09-29 一种图像渲染方法及其相关设备

Country Status (4)

Country Link
US (1) US20240378793A1 (zh)
EP (1) EP4386682A4 (zh)
CN (2) CN115908683A (zh)
WO (1) WO2023051662A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025015971A1 (zh) * 2023-07-14 2025-01-23 优三缔科技(上海)有限公司 用于多应用的非集成的单引擎后台渲染方法、系统以及设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120451355A (zh) * 2024-02-08 2025-08-08 华为技术有限公司 一种光线追踪方法、装置和图像渲染系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5966134A (en) * 1996-06-28 1999-10-12 Softimage Simulating cel animation and shading
CN105261059A (zh) * 2015-09-18 2016-01-20 浙江大学 一种基于在屏幕空间计算间接反射高光的渲染方法
CN106504185A (zh) * 2016-10-26 2017-03-15 腾讯科技（深圳）有限公司 一种渲染优化方法和装置
CN112396684A (zh) * 2020-11-13 2021-02-23 贝壳技术有限公司 光线追踪方法、装置及机器可读存储介质
CN112799773A (zh) * 2021-02-23 2021-05-14 京东方科技集团股份有限公司 一种数据可视化方法、终端设备、系统和存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10565776B2 (en) * 2015-12-12 2020-02-18 Adshir Ltd. Method for fast generation of path traced reflections on a semi-reflective surface
WO2019224804A1 (en) * 2018-05-21 2019-11-28 Adshir Ltd. Fast generation of ray traced reflections of virtual objects in real world environment
US10825230B2 (en) * 2018-08-10 2020-11-03 Nvidia Corporation Watertight ray triangle intersection
CN109961498B (zh) * 2019-03-28 2022-12-13 腾讯科技（深圳）有限公司 图像渲染方法、装置、终端及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5966134A (en) * 1996-06-28 1999-10-12 Softimage Simulating cel animation and shading
CN105261059A (zh) * 2015-09-18 2016-01-20 浙江大学 一种基于在屏幕空间计算间接反射高光的渲染方法
CN106504185A (zh) * 2016-10-26 2017-03-15 腾讯科技（深圳）有限公司 一种渲染优化方法和装置
CN112396684A (zh) * 2020-11-13 2021-02-23 贝壳技术有限公司 光线追踪方法、装置及机器可读存储介质
CN112799773A (zh) * 2021-02-23 2021-05-14 京东方科技集团股份有限公司 一种数据可视化方法、终端设备、系统和存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4386682A4

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025015971A1 (zh) * 2023-07-14 2025-01-23 优三缔科技(上海)有限公司 用于多应用的非集成的单引擎后台渲染方法、系统以及设备

Also Published As

Publication number Publication date
EP4386682A1 (en) 2024-06-19
EP4386682A4 (en) 2024-11-13
CN115908683A (zh) 2023-04-04
US20240378793A1 (en) 2024-11-14
CN117957576A (zh) 2024-04-30

Similar Documents

Publication Publication Date Title
US12148088B2 (en) Method for continued bounding volume hierarchy traversal on intersection without shader intervention
US11302057B2 (en) Rendering of soft shadows
CN113808241B (zh) 共享顶点的射线追踪图元的硬件加速
US20230316633A1 (en) Image processing method and related apparatus
CN114758051B (zh) 一种图像渲染方法及其相关设备
CN107408312B (zh) 光线追踪应用中的有效层次结构遍历
CN117157676A (zh) 加速实时光线追踪的三角形可见性测试
US10628995B2 (en) Anti-aliasing of graphical elements defined based on functions
KR20240001021A (ko) 이미지 렌더링 방법, 장치, 전자 설비 및 저장 매체
CN110827389A (zh) 严密的光线三角形相交
WO2024027237A1 (zh) 渲染的优化方法、电子设备和计算机可读存储介质
US20130127895A1 (en) Method and Apparatus for Rendering Graphics using Soft Occlusion
US20110043523A1 (en) Graphics processing apparatus for supporting global illumination
CN117726496A (zh) 使用光线剪裁减少假阳性光线遍历
KR102151443B1 (ko) 하이브리드 gpu 아키텍처 기반의 그래픽 처리 장치
WO2023051662A1 (zh) 一种图像渲染方法及其相关设备
CN117726732A (zh) 减少包围体层次结构中的假阳性光线遍历
CN103426199A (zh) 一种三维几何场景的低噪声实时全局光照绘制方法
CN103679818B (zh) 一种基于虚拟面光源的实时场景绘制方法
TW202141429A (zh) 使用陰影資訊進行渲染
WO2024002130A1 (zh) 一种图像渲染方法及其相关设备
CN117726743A (zh) 使用点退化剔除减少假阳性光线遍历
CN121392104A (zh) 渲染处理方法、装置、设备、存储介质及程序产品
HK40095360A (zh) 图像渲染方法、装置、计算机设备和存储介质
KR20080077748A (ko) 컴퓨터 그래픽 시스템에서의 동영상의 반사 효과 처리 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22875045

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280057482.7

Country of ref document: CN

ENP Entry into the national phase

Ref document number: 2022875045

Country of ref document: EP

Effective date: 20240311

WWE Wipo information: entry into national phase

Ref document number: 18696019

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE