WO2023284503A1 - 全景图像的色调映射方法和装置 - Google Patents

全景图像的色调映射方法和装置 Download PDF

Info

Publication number
WO2023284503A1
WO2023284503A1 PCT/CN2022/100495 CN2022100495W WO2023284503A1 WO 2023284503 A1 WO2023284503 A1 WO 2023284503A1 CN 2022100495 W CN2022100495 W CN 2022100495W WO 2023284503 A1 WO2023284503 A1 WO 2023284503A1
Authority
WO
WIPO (PCT)
Prior art keywords
segmented
mapping
panoramic video
area
metadata information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/100495
Other languages
English (en)
French (fr)
Inventor
徐巍炜
余全合
陈虎
王弋川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to EP22841143.5A priority Critical patent/EP4358018A4/en
Publication of WO2023284503A1 publication Critical patent/WO2023284503A1/zh
Priority to US18/411,529 priority patent/US20240153051A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/16—Spatio-temporal transformations, e.g. video cubism
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20208—High dynamic range [HDR] image processing

Definitions

  • the present application relates to the field of image processing, in particular to a tone mapping method and device for a panoramic image.
  • Dynamic range is used in many fields to express the ratio of the maximum value to the minimum value of a variable.
  • dynamic range is used to express the ratio between the maximum gray value and the minimum gray value within the displayable range of the image.
  • the dynamic range is usually between 10-3 candela/square meter (cd/m 2 ) and 106 cd/m 2 , which is called high dynamic range (HDR).
  • HDR high dynamic range
  • LDR low dynamic range
  • mapping of the dynamic range can be applied to the adaptation of the front-end HDR signal and the HDR display device of the display terminal.
  • the front-end collects a 4000cd/m 2 light signal
  • the HDR display capability of the HDR display device of the display terminal is 500cd/m2 2.
  • Mapping the light signal of 4000cd/m 2 to the display device of 500cd/m 2 is a tone mapping (TM) process from high to low.
  • Dynamic range mapping can also be applied to the adaptation of the front-end SDR signal and the HDR display device at the display end.
  • the front-end collects a 100cd/m 2 light signal
  • the HDR display capability of the HDR display device at the display end is 2000cd/m2.
  • m 2 mapping the light signal of 100cd/m 2 to the display device of 2000cd/m 2 is a TM process from low to high.
  • mapping methods of the dynamic range can be divided into two types: static and dynamic.
  • the static mapping method is based on the same video content or the same hard disk content, and the overall TM process is performed from a single data, that is, for various scenes, the mapping curve is usually the same.
  • the advantage of this method is that the image needs to carry less data, and the processing flow is relatively simple; the disadvantage is that all scenes use the same mapping curve for TM, which will lead to information loss in some scenes, for example, if the key protection of the mapping curve is In the bright area, some details will be lost in some extremely dark scenes, or they will not be seen at all, which will affect the display effect of the image.
  • the dynamic mapping method is to dynamically adjust the mapping curve for each scene or the content of each frame according to a specific area.
  • the advantage of this method is that it can realize differential processing of different scenes or different frames; Each frame or each scene carries relevant scene information, and the image needs to carry a large amount of data.
  • the present application provides a method and device for tone mapping of a panoramic image, so as to improve the processing efficiency of the panoramic image.
  • the present application provides a tone mapping method for a panoramic image, including: determining one or more target metadata information units of a first pixel from a plurality of metadata information units, the plurality of metadata information units It is obtained by parsing the code stream, the first pixel point is any pixel point in the two-dimensional plane projection map of the panoramic video to be processed, and the plurality of metadata information units and the three-dimensional spherical expression panorama of the panoramic video include multiple Corresponding to the divided regions, the panoramic video two-dimensional plane projection map and the panoramic video three-dimensional spherical expression panorama have a mapping relationship; according to the pixel value of the first pixel point, the one or more target metadata information units Perform tone mapping to obtain a target tone mapping value of the first pixel.
  • the metadata information unit guides the display terminal to generate a tone mapping curve corresponding to the segmented area, and then realizes the tone mapping from the image to the display, which can not only determine the matching tone mapping parameters for different dynamic range areas on the panoramic image, but also It can conform to the display dynamic range of the display terminal, and improve the processing efficiency of the panoramic image.
  • the method before determining one or more target metadata information units of the first pixel from the multiple metadata information units, the method further includes: acquiring the multiple segmented regions.
  • the acquiring the plurality of segmented regions includes: dividing the panoramic video three-dimensional spherical expression panorama according to a preset division manner to obtain the plurality of segmented regions.
  • the obtaining the plurality of segmented areas includes: dividing the panoramic video three-dimensional spherical expression panorama according to the division method obtained by parsing the code stream to obtain the plurality of segmented regions. area.
  • the multiple segmented regions are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video with a preset viewing angle interval rule; or,
  • the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from the latitude direction; and/or, the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from obtained by dividing in the longitude direction.
  • the dividing the three-dimensional spherical representation panorama of the panoramic video to obtain the plurality of segmented regions includes: a plurality of pixels included in the two-dimensional planar projection image of the panoramic video The points are clustered to obtain a plurality of pixel point sets; the plurality of pixel points are respectively mapped to the three-dimensional spherical surface of the panoramic video to express the panorama; the pixels included in the first pixel point set are expressed on the three-dimensional spherical surface of the panoramic video
  • the area formed by the corresponding mapping points on the panorama is used as the first segmented area, the first pixel point set is one of the plurality of pixel point sets, and the first segmented area is one of the plurality of segmented areas One.
  • the dividing the three-dimensional spherical representation panorama of the panoramic video to obtain the plurality of segmented regions includes: dividing a plurality of pixels included in the two-dimensional planar projection image of the panoramic video The points are respectively mapped to the three-dimensional spherical expression panorama of the panoramic video to obtain a plurality of mapping points; the plurality of mapping points are clustered to obtain a plurality of mapping point sets; the mapping points included in the first mapping point set are formed.
  • a region is used as a second segmented area, the first set of mapping points is one of the plurality of sets of mapping points, and the second segmented area is one of the plurality of segmented areas.
  • the acquiring the multiple segmented areas includes: acquiring the multiple segmented areas according to indication information of the multiple segmented areas obtained by parsing the code stream.
  • the present application can use one of the above-mentioned multiple division methods to divide the panoramic video three-dimensional spherical expression panorama to obtain multiple segmentation regions, which can be adapted to the pixel distribution of the panoramic video two-dimensional planar projection map, so that tone mapping can be divided into
  • the area is used as a unit to achieve tone mapping in a small dynamic range and improve the efficiency of tone mapping.
  • the determining one or more target metadata information units of the first pixel from the plurality of metadata information units includes: determining the plurality of metadata information units and the plurality of divisions Correspondence between regions, one metadata information unit corresponds to one or more of the segmented regions; one or more target segmented regions are determined according to the set mapping points; when there is only one target segmented region, the one Determine the metadata information unit corresponding to the target segmented area as the one target metadata information unit; or, when there are multiple target segmented areas, determine the metadata information units corresponding to the multiple target segmented areas as the A number of object metadata information units.
  • the determining the correspondence between the plurality of metadata information units and the plurality of segmented regions includes: extracting from the plurality of metadata information units in a first preset order The current metadata information unit; extracting the current segmented area from the plurality of segmented areas according to a second preset order; establishing a corresponding relationship between the current segmented area and the current metadata information unit.
  • the determining the correspondence between the plurality of metadata information units and the plurality of segmented regions includes: extracting from the plurality of metadata information units in a first preset order The current metadata information unit; extracting the current segmented area from the plurality of segmented areas according to the traversal order obtained by parsing the code stream; establishing a corresponding relationship between the current segmented area and the current metadata information unit.
  • the determining the correspondence between the plurality of metadata information units and the plurality of segmented regions includes: extracting from the plurality of metadata information units in a first preset order The current metadata information unit; obtain one or more coordinates included in the current metadata information unit; determine one or more mapping points on the three-dimensional spherical expression panorama of the panoramic video according to the one or more coordinates; when When there is only one mapping point, establish a corresponding relationship between the segmentation area to which the one mapping point belongs and the current metadata information unit; when there are multiple mapping points, at least one segmentation area to which the multiple mapping points belong and The current metadata information unit establishes a corresponding relationship.
  • the present application adopts one of the above-mentioned multiple corresponding relationship acquisition methods to determine the corresponding relationship between multiple segmented regions and multiple metadata information units, and does not need to transmit the corresponding relationship to the display terminal through the code stream, which can reduce the number of errors in the code stream. information carried.
  • the set mapping point is the first mapping point of the first pixel point on the panoramic video three-dimensional spherical expression panorama;
  • a target segmentation area including: determining a first coverage area on the panoramic video three-dimensional spherical expression panorama with the first mapping point as the center; when the first coverage area includes a segmentation area, the one A segmented area is determined as the one target segmented area; or, when the first coverage area includes multiple segmented areas, the multiple segmented areas are determined as the multiple target segmented areas.
  • the set mapping point is the center of the viewpoint on the panorama expressed on the three-dimensional spherical surface of the panoramic video; the determining one or more target segmentation regions according to the set mapping point includes: determining the A second segmented area to which the viewpoint center belongs, the plurality of segmented areas include the second segmented area; and determining the second segmented area as the one target segmented area.
  • the set mapping point is the center of the viewpoint on the panorama expressed on the three-dimensional spherical surface of the panoramic video; the determining one or more target segmentation regions according to the set mapping point includes: Determine a second coverage area on the three-dimensional spherical expression panorama of the panoramic video with the visual center as the center; when the second coverage area includes a segmented area, determine the segmented area as the target segmented area; Or, when the second coverage range includes multiple segmented areas, determine the multiple segmented areas as the multiple target segmented areas.
  • performing tone mapping on the pixel value of the first pixel according to the one or more target metadata information units to obtain the target tone mapping value of the first pixel includes: obtaining one or more tone mapping curves according to the one or more target metadata information units; when there is only one color bar mapping curve, performing a pixel value of the first pixel point according to the one tone mapping curve Tone mapping to obtain the target tone mapping value; or, when there are multiple color stripe mapping curves, respectively perform tone mapping on the pixel value of the first pixel according to the multiple tone mapping curves to obtain the second A plurality of intermediate tone values of a pixel; acquiring the target tone mapping value according to the plurality of intermediate tone values.
  • the acquiring the target tone mapping value according to the multiple intermediate tone values includes: determining an intermediate value among the multiple intermediate tone values as the target tone mapping value; Or, performing weighted average on the multiple intermediate hue values to obtain the target tone mapping value, the weighting coefficients of the multiple intermediate hue values are preset, or the weighting coefficients of the multiple intermediate hue values are equal to The distance between the first pixel point and the center point of each segmented area is related, or the weighting coefficients of the plurality of mid-tone values are related to the area proportions of the segmented areas respectively corresponding to the plurality of target metadata information units.
  • This application adopts one of the above-mentioned methods to obtain the target tone mapping value of the first pixel, and can refer to the metadata information unit of one or more segmented regions to generate the tone mapping curve of the first pixel to improve the accuracy of tone mapping Rate.
  • the present application provides a tone mapping method for a panoramic image, including: acquiring at least one mapping point included in a first segmented area, the first segmented area being a plurality of segmented areas included in a three-dimensional spherical expression panorama of a panoramic video
  • One of the three-dimensional spherical expression panoramas of the panoramic video and the two-dimensional planar projection of the panoramic video to be processed have a mapping relationship, and the at least one mapping point and at least one pixel point on the two-dimensional planar projection of the panoramic video
  • the metadata information unit guides the display terminal to generate a tone mapping curve corresponding to the segmented area, and then realizes the tone mapping from the image to the display, which can not only determine the matching tone mapping parameters for different dynamic range areas on the panoramic image, but also It can conform to the display dynamic range of the display terminal, and improve the processing efficiency of the panoramic image.
  • the method further includes: when the first segmented area and the second segmented area When the histogram and/or brightness of the first segmented area meet the set condition, the metadata information unit of the first segmented area and the second metadata information unit of the second segmented area are fused to obtain the first segmented area and the second metadata information unit of the second segmented area.
  • the present application fuses the metadata information units of two or more segmented areas with small differences in histograms and/or luminance, thereby reducing the data amount of the metadata information units carried in the code stream.
  • the method before acquiring at least one mapping point included in the first segmented area, the method further includes: mapping the panoramic video 2D planar projection image to the panoramic video 3D spherical surface to express the panorama Figure: dividing the three-dimensional spherical expression panorama of the panoramic video to obtain the plurality of segmented regions.
  • the dividing the three-dimensional spherical expression panorama of the panoramic video to obtain the plurality of segmented regions includes: dividing the three-dimensional spherical expression panorama of the panoramic video according to a preset division method Dividing is performed to obtain the plurality of divided regions.
  • the dividing the three-dimensional spherical surface of the panoramic video to obtain the plurality of segmented regions includes: dividing the three-dimensional spherical surface of the panoramic video according to a division method determined based on preset rules The expression panorama is divided to obtain the plurality of divided regions.
  • the multiple segmented regions are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video with a preset viewing angle interval rule; or,
  • the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from the latitude direction; and/or, the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from obtained by dividing in the longitude direction.
  • the dividing the three-dimensional spherical representation panorama of the panoramic video to obtain the plurality of segmented regions includes: a plurality of pixel points included in the two-dimensional planar projection image of the panoramic video Perform clustering to obtain a plurality of pixel point sets; respectively map the plurality of pixel points to the three-dimensional spherical surface of the panoramic video to express the panorama; express the panorama on the three-dimensional spherical surface of the panoramic video with the pixels included in the first set of pixel points
  • the area formed by the corresponding mapping points on the figure is used as the first segmented area, the first pixel point set is one of the plurality of pixel point sets, and the first segmented area is one of the plurality of segmented areas .
  • the dividing the three-dimensional spherical representation panorama of the panoramic video to obtain the plurality of segmented regions includes: dividing the multiple pixel points included in the two-dimensional planar projection image of the panoramic video Respectively map to the three-dimensional spherical expression panorama of the panoramic video to obtain a plurality of mapping points; cluster the plurality of mapping points to obtain a plurality of mapping point sets; form an area formed by the mapping points included in the first mapping point set As the second segmented area, the first set of mapping points is one of the plurality of sets of mapping points, and the second segmented area is one of the plurality of segmented areas.
  • the present application can use one of the above-mentioned multiple division methods to divide the panoramic video three-dimensional spherical expression panorama to obtain multiple segmentation regions, which can be adapted to the pixel distribution of the panoramic video two-dimensional planar projection map, so that tone mapping can be divided into
  • the area is used as a unit to achieve tone mapping in a small dynamic range and improve the efficiency of tone mapping.
  • the present application provides a display device including: a division module, a determination module and a mapping module.
  • a determining module configured to determine one or more target metadata information units of a first pixel from a plurality of metadata information units, the plurality of metadata information units are obtained by parsing the code stream, and the first pixel is any pixel in the two-dimensional plane projection map of the panoramic video to be processed, and the plurality of metadata information units correspond to the multiple segmentation regions included in the three-dimensional spherical expression panorama of the panoramic video, and the two-dimensional plane projection of the panoramic video There is a mapping relationship between the picture and the panoramic video three-dimensional spherical expression panorama; the mapping module is configured to tone-map the pixel value of the first pixel according to the one or more target metadata information units to obtain the second pixel. The target tonemap value for a pixel.
  • the dividing module is configured to acquire the plurality of segmented regions.
  • the dividing module is specifically configured to divide the three-dimensional spherical expression panorama of the panoramic video according to a preset dividing manner, so as to obtain the plurality of divided regions.
  • the division module is specifically configured to divide the three-dimensional spherical expression panorama of the panoramic video according to a division manner obtained by parsing the code stream, so as to obtain the plurality of division regions.
  • the multiple segmented regions are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video with a preset viewing angle interval rule; or,
  • the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from the latitude direction; and/or, the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from obtained by dividing in the longitude direction.
  • the division module is specifically configured to cluster a plurality of pixel points included in the panoramic video two-dimensional planar projection image to obtain a plurality of pixel point sets;
  • the points are respectively mapped to the panoramic video three-dimensional spherical expression panorama;
  • the area formed by the corresponding mapping points of the pixels included in the first set of pixel points on the panoramic video three-dimensional spherical expression panorama is used as the first segmented area,
  • the first set of pixel points is one of the plurality of sets of pixel points, and the first segmented area is one of the plurality of segmented areas.
  • the division module is specifically configured to respectively map a plurality of pixels included in the panoramic video two-dimensional planar projection image to the panoramic video three-dimensional spherical expression panorama to obtain multiple mappings points; cluster the multiple mapping points to obtain multiple mapping point sets; use the area formed by the mapping points included in the first mapping point set as the second segmentation area, and the first mapping point set is the multiple one of a set of mapping points, and the second segmented area is one of the plurality of segmented areas.
  • the division module is specifically configured to acquire the plurality of segmented areas according to indication information of the plurality of segmented areas obtained by parsing the code stream.
  • the determining module is specifically configured to determine the correspondence between the plurality of metadata information units and the plurality of segmented regions, and one metadata information unit corresponds to one or more of the The above segmented area; determine one or more target segmented areas according to the set mapping point; when there is only one target segmented area, determine the metadata information unit corresponding to the one target segmented area as the one target metadata information unit; Alternatively, when there are multiple target segmented areas, the metadata information units respectively corresponding to the multiple target segmented areas are determined as the multiple target metadata information units.
  • the determining module is specifically configured to extract the current metadata information unit from the plurality of metadata information units in a first preset order; extract the current metadata information unit from the plurality of metadata information units in a second preset order. Extract the current segmented area from a segmented area; establish a corresponding relationship between the current segmented area and the current metadata information unit.
  • the determining module is specifically configured to extract the current metadata information unit from the plurality of metadata information units according to a first preset order; according to the traversal order obtained by parsing the code stream Extracting a current segmented area from the plurality of segmented areas; establishing a corresponding relationship between the current segmented area and the current metadata information unit.
  • the determining module is specifically configured to extract the current metadata information unit from the plurality of metadata information units in a first preset order; acquire the One or more coordinates; according to the one or more coordinates, determine one or more mapping points on the three-dimensional spherical expression panorama of the panoramic video; when there is only one mapping point, the division area to which the one mapping point belongs Establishing a corresponding relationship with the current metadata information unit; when there are multiple mapping points, establishing a corresponding relationship between at least one segment area to which the multiple mapping points belong and the current metadata information unit.
  • the set mapping point is the first mapping point of the first pixel point on the panorama expressed by the three-dimensional spherical surface of the panoramic video; the determining module is specifically configured to use the Determine the first coverage area on the three-dimensional spherical expression panorama of the panoramic video with the first mapping point as the center; when the first coverage area includes a segmented area, determine the segmented area as the target segmented area ; or, when the first coverage range includes multiple segmented areas, determining the multiple segmented areas as the multiple target segmented areas.
  • the set mapping point is the viewpoint center on the panorama expressed by the three-dimensional spherical surface of the panoramic video; the determining module is specifically configured to determine the second segmented area to which the viewpoint center belongs, The plurality of segmented areas include the second segmented area; and the second segmented area is determined as the one target segmented area.
  • the set mapping point is the viewpoint center on the panorama expressed on the three-dimensional spherical surface of the panoramic video; Determining a second coverage area on the three-dimensional spherical expression panorama; when the second coverage area includes a segmented area, determining the segmented area as the target segmented area; or, when the second coverage area includes When there are multiple segmented areas, the multiple segmented areas are determined as the multiple target segmented areas.
  • the mapping module is specifically configured to acquire one or more tone mapping curves according to the one or more target metadata information units; when there is only one color bar mapping curve, according to the A tone mapping curve performs tone mapping on the pixel value of the first pixel point to obtain the target tone mapping value; or, when there are multiple color bar mapping curves, the The pixel value of the first pixel is tone-mapped to obtain a plurality of intermediate tone values of the first pixel; and the target tone mapping value is obtained according to the plurality of intermediate tone values.
  • the mapping module is specifically configured to determine an intermediate value among the plurality of intermediate tone values as the target tone mapping value; or, weight the plurality of intermediate tone values To obtain the target tone mapping value by averaging, the weighting coefficients of the plurality of intermediate tone values are preset, or the weighting coefficients of the plurality of intermediate tone values are related to the first pixel point and the center of each segmented area The distance of the points is related, or, the weighting coefficients of the plurality of intermediate hue values are related to the area proportions of the divided regions respectively corresponding to the plurality of target metadata information units.
  • the present application provides a front-end device including: a division module, an acquisition module and a generation module. in,
  • An acquisition module configured to acquire at least one mapping point included in a first segmented area, the first segmented area being one of multiple segmented areas included in a three-dimensional spherical expression panorama of a panoramic video, the three-dimensional spherical expression panorama of a panoramic video It has a mapping relationship with the to-be-processed panoramic video two-dimensional planar projection map, and the at least one mapping point corresponds to at least one pixel on the panoramic video two-dimensional planar projection map; the generating module is configured to use the at least one pixel generating the metadata information unit of the first segmented area; writing the metadata information unit of the first segmented area into a code stream.
  • the generating module is further configured to, when the histograms and/or brightness of the first segmented area and the second segmented area meet the set condition,
  • the metadata information unit is fused with the second metadata information unit of the second segment area to obtain the metadata information unit of the first segment area and the second segment area, and the second segment area is the One of multiple partitioned regions.
  • the division module is configured to map the 2D planar projection image of the panoramic video to the 3D spherical expression panorama of the panoramic video; divide the 3D spherical expression panorama of the panoramic video to obtain the plurality of divided regions.
  • the dividing module is specifically configured to divide the panoramic video three-dimensional spherical expression panorama according to a preset dividing manner to obtain the plurality of divided regions.
  • the division module is specifically configured to divide the panoramic video three-dimensional spherical expression panorama according to a division manner determined based on preset rules to obtain the plurality of division regions.
  • the multiple segmented regions are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video with a preset viewing angle interval rule; or,
  • the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from the latitude direction; and/or, the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from obtained by dividing in the longitude direction.
  • the division module is specifically configured to cluster a plurality of pixel points included in the panoramic video two-dimensional planar projection image to obtain a plurality of pixel point sets;
  • the points are respectively mapped to the panoramic video three-dimensional spherical expression panorama;
  • the area formed by the corresponding mapping points of the pixels included in the first set of pixel points on the panoramic video three-dimensional spherical expression panorama is used as the first segmented area,
  • the first set of pixel points is one of the plurality of sets of pixel points, and the first segmented area is one of the plurality of segmented areas.
  • the division module is specifically configured to respectively map a plurality of pixels included in the panoramic video two-dimensional planar projection image to the panoramic video three-dimensional spherical expression panorama to obtain multiple mappings points; cluster the multiple mapping points to obtain multiple mapping point sets; use the area formed by the mapping points included in the first mapping point set as the second segmentation area, and the first mapping point set is the multiple one of a set of mapping points, and the second segmented area is one of the plurality of segmented areas.
  • the present application provides a terminal device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors , so that the one or more processors implement the method described in any one of the first to second aspects above.
  • the present application provides a computer-readable storage medium, including a computer program.
  • the computer program When the computer program is executed on a computer, the computer executes the method described in any one of the first to second aspects above.
  • the present application provides a computer program product, the computer program product includes computer program code, and when the computer program code is run on a computer, the computer is made to execute any one of the above-mentioned first to second aspects. Methods.
  • Figure 1 is an example diagram of dynamic range mapping during real-world imaging
  • Fig. 2 is a schematic diagram of a curve of the PQ photoelectric transfer function
  • Fig. 3 is a schematic diagram of a curve of HLG photoelectric transfer function
  • Fig. 4 is a schematic diagram of a curve of SLF photoelectric transfer function
  • Fig. 5 is a schematic diagram of the sigmoidal curve
  • Fig. 6 is a schematic diagram of Bezier curve
  • Fig. 7 is a schematic diagram of the S-shaped curve
  • Fig. 8 is an exemplary block diagram of the panoramic video playing system of the present application.
  • FIG. 9 is a schematic structural diagram of an exemplary terminal device
  • Figure 10a is an example map of latitude and longitude map mapping
  • Figure 10b is a schematic diagram of a spherical surface mapped to a regular polyhedron and the corresponding expanded image
  • Figure 10c and Figure 10d are schematic diagrams of the mapping relationship between a sphere and a hexahedron
  • Figure 10e is a schematic diagram of the mapping relationship between a sphere, a hexahedron and an octahedron;
  • Figures 11a to 11d are exemplary schematic diagrams of several segmented areas of the panoramic video three-dimensional spherical representation panorama in the present application;
  • FIG. 12 is a flowchart of a process 1200 of the tone mapping method for a panoramic image of the present application.
  • FIG. 13 is a flowchart of a process 1300 of the tone mapping method for a panoramic image of the present application
  • FIG. 14 is an exemplary structural diagram of a display device 1400 of the present application.
  • FIG. 15 is an exemplary structural diagram of a front-end device 1500 of the present application.
  • At least one (item) means one or more, and “multiple” means two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, “A and/or B” can mean: only A exists, only B exists, and A and B exist at the same time , where A and B can be singular or plural.
  • the character “/” generally indicates that the contextual objects are an “or” relationship.
  • At least one of the following” or similar expressions refer to any combination of these items, including any combination of single or plural items.
  • At least one item (piece) of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c ", where a, b, c can be single or multiple.
  • Dynamic range is used in many fields to express the ratio of the maximum value to the minimum value of a variable.
  • dynamic range is used to express the ratio between the maximum gray value and the minimum gray value within the displayable range of the image.
  • the brightness of the night scene under the starry sky is about 0.001cd/m 2
  • the brightness of the sun itself is as high as 10 9 cd/m 2
  • the dynamic range is usually between 10 - 3 cd/m 2 and 10 6 cd/m 2 It is called high dynamic range (HDR).
  • the gray levels of the three channels of red (R), green (G) and blue (B) are stored in one byte respectively, that is, the gray levels of the three channels of R, G and B
  • the degree ranges are 0-255 respectively, and the 0-255 is the dynamic range of the image, which is called low dynamic range (LDR).
  • OETF Optical-electro transfer function
  • FIG 1 is an example diagram of dynamic range mapping in the imaging process of the real world.
  • the brightness of moonlight in the real world is 1cd/m 2
  • the brightness of indoor lighting is The brightness is 100cd/m 2
  • the brightness is 500cd/m 2 when the outdoor is cloudy
  • the brightness is 2000cd/m 2 when the outdoor is sunny.
  • the brightness range of 100cd/m 2 to 2000cd/m 2 is mapped to the brightness range of 1cd/m 2 to 200cd/m 2 corresponding to the storage method of the display device.
  • an electro-optical transfer function (EOTF) is needed to express the brightness in the real world as the brightness corresponding to the storage method of the display device.
  • the brightness in the real world is 10000cd/m 2
  • the display device uses 10 bits to store brightness information
  • the maximum value that can be stored in this storage mode is 1023, so 10000cd/m 2 can be expressed as 1023.
  • the early display device was a cathode ray tube (CRT) display, and its photoelectric transfer function was the Gamma function.
  • the Gamma function is defined in the ITU-R Recommendation BT.1886 standard:
  • the illuminance range of the display device is continuously increasing.
  • the illuminance of the existing HDR display reaches 600cd/m 2
  • the illuminance of the high-end HDR display can reach 2000cd/m 2 . Therefore, an improved photoelectric transfer function is needed to adapt to the upgrade of display devices.
  • Common photoelectric transfer functions at this stage include perception quantization (PQ) photoelectric transfer function and hybrid log-gamma (HLG) photoelectric transfer function.
  • scene luminance fidelity scene luminance fidelity, SLF
  • PQ photoelectric transfer function Different from the traditional Gamma function, the PQ photoelectric transfer function is proposed according to the contrast perception model of the human eye under different brightness.
  • the PQ photoelectric transfer function represents the conversion relationship from the linear signal value of the image frame pixel to the nonlinear signal value in the PQ domain.
  • Figure 2 is a schematic diagram of a curve of the PQ photoelectric transfer function.
  • the PQ photoelectric transfer function can be expressed as:
  • L represents the linear signal value of the image frame pixel, and its value is normalized to [0,1]
  • L' represents the nonlinear signal value in the PQ domain, and its value range is [0,1]
  • m 1 , m 2 , c 1 , c 2 and c 3 are all PQ photoelectric transfer coefficients
  • HLG photoelectric transfer function It is improved on the basis of the traditional Gamma curve.
  • the HLG photoelectric transfer function applies the traditional Gamma curve in the low section, and supplements the log curve in the high section.
  • the HLG photoelectric transfer function represents the conversion relationship from the linear signal value of the pixel of the image frame to the nonlinear signal value in the HLG domain, and FIG. 3 is a schematic diagram of a curve of the HLG photoelectric transfer function.
  • the HLG photoelectric transfer function can be expressed as:
  • L represents the linear signal value of the image frame pixel, and its value range is [0,12];
  • L' represents the nonlinear signal value in the HLG domain, and its value range is [0,1];
  • SLF photoelectric transfer function under the premise of satisfying the optical characteristics of the human eye, it is obtained according to the brightness distribution of the HDR scene.
  • the SLF photoelectric transfer function represents the conversion relationship from the linear signal value of the image frame pixel to the nonlinear signal value in the SLF domain.
  • Figure 4 is a schematic diagram of the SLF photoelectric transfer function.
  • the SLF photoelectric transfer function can be expressed as:
  • L represents the linear signal value of the image frame pixel, and its value is normalized to [0,1]
  • L' represents the nonlinear signal value in the PQ domain, and its value range is [0,1]
  • the dynamic range mapping can be applied to the adaptation of the front-end HDR signal and the HDR display device of the display terminal.
  • the front-end collects a 4000cd/m 2 light signal, and the HDR display capability of the HDR display device of the display terminal is 500cd/m2 2.
  • Mapping the light signal of 4000cd/m 2 to the display device of 500cd/m 2 is a tone mapping (TM) process from high to low.
  • Dynamic range mapping can also be applied to the adaptation of the front-end SDR signal and the HDR display device at the display end.
  • the front-end collects a 100cd/m 2 light signal, and the HDR display capability of the HDR display device at the display end is 2000cd/m2.
  • m 2 mapping the light signal of 100cd/m 2 to the display device of 2000cd/m 2 is a TM process from low to high.
  • mapping methods of the dynamic range can be divided into two types: static and dynamic.
  • the static mapping method is based on the same video content or the same hard disk content, and the overall TM process is performed from a single data, that is, for various scenes, the mapping curve is usually the same.
  • the advantage of this method is that the video needs to carry less data, and the processing flow is relatively simple; the disadvantage is that all scenes use the same mapping curve for TM, which will lead to information loss in some scenes, for example, if the key protection of the mapping curve is In the bright area, some details will be lost or completely invisible in some extremely dark scenes, which will affect the display effect of the video.
  • the dynamic mapping method is to dynamically adjust the mapping curve for each scene or the content of each frame according to a specific area.
  • the advantage of this method is that it can realize differential processing of different scenes or different frames; Each frame or each scene carries relevant scene information, and the video needs to carry a large amount of data.
  • Fig. 5 is a schematic diagram of a sigmoidal curve.
  • FIG. 6 is a schematic diagram of a Bezier curve.
  • Figure 7 is a schematic diagram of an S-curve, and the photoelectric transfer function corresponding to the S-curve can be expressed as:
  • L and L' are normalized electrical signals or optical signals respectively, and their value ranges are [0,1]; the value range of a is [0,1]; the value range of b is [0 ,1]; the value ranges of p, n, and m are all [0,N], and N is a rational number greater than 0.1; k 1 , k 2 , and k 3 are all rational numbers.
  • Panoramic video can provide an immersive visual experience. Mainly: (1) It provides description of spherical scene content of 360 degrees horizontally and 180 degrees vertically; (2) The VR head-mounted device can freely choose to watch the window in the specified direction. Compared with the narrow viewing angle range of ordinary 2D video of 33 degrees horizontally and 19 degrees vertically, panoramic video covers a larger range.
  • the panoramic video introduces a more serious problem of insufficient dynamic range displayed by the device.
  • the viewing angle range of 2D video is relatively small, the lighting conditions are relatively similar, and its dynamic range is still within the controllable range; while the viewing angle range of panoramic video is relatively large, for example, the video content is a daytime scene, which is likely to include both A very bright area like the sun, which also includes darker areas such as building interiors or shadows, has a large dynamic range, which brings great challenges to description and display.
  • the front end (video capture and/or production) will carry parameters related to the mapping curve in the metadata information sent to the display end (video display).
  • the definition of metadata information related to the sigmoidal curve proposed by St2094-10 not only includes the maximum value (Maximum PQ-encoded maxRGB), minimum value (Minimum PQ-encoded maxRGB), and average value (Average PQ-encoded maxRGB) of video brightness
  • Statistical values such as tone mapping offset (tone mapping offset), tone mapping gain (tone mapping gain) and tone mapping gamma (tone mapping gamma) and other parameters related to sigmoid curves are also included, which are used to directly generate sigmoidal curves.
  • Metadata information based on Bezier curves proposed by St2094-40 includes histogram information (Distribution MaxRGB) and parameters related to Bezier curves (bezier curve anchors), which are used to directly generate Bezier Seoul curve.
  • the standards of the ST2094 series include the target system display maximum (peak) luminance (targeted_system_display_maximum_luminance, TSDAPL).
  • Metadata information can include statistical values such as the maximum value, minimum value, and average value of video brightness, and can also contain parameters related to S-shaped curves such as p, m, a, b, n, K1, K2, and K3.
  • FIG. 8 is an exemplary block diagram of the panoramic video playing system of the present application.
  • the panoramic video playback system is divided into a front end and a display end, wherein the front end includes a module for collecting and/or producing panoramic video, and the display end includes an HDR display module and/or an SDR display module.
  • the front end transmits the collected panoramic video data to the display terminal after preprocessing, and also carries the metadata information of the panoramic video data.
  • the display terminal enhances the multiple image frames contained in the panoramic video frame by frame, and obtains an image with excellent color, brightness and contrast, and displays it.
  • the front end and the display end may be independent and different physical devices.
  • the front end may be a video camera, a camera, an image plotter, etc.
  • the display end may be a virtual reality (VR) device. Glasses, mobile phones, tablets and other devices with panoramic video playback function.
  • a wireless connection or a wired connection can be established between the front end and the display terminal, wherein the wireless connection can be used to include, for example, long term evolution (long term evolution, LTE), fifth generation (5th generation, 5G) mobile communication, and future mobile communication, etc. technology.
  • the wireless connection may also include technologies such as wireless-fidelity (wireless-fidelity, WiFi), Bluetooth, and near field communication (Near Field Communication, NFC). Wired connections may include Ethernet connections, local area network connections, and the like. This is not specifically limited.
  • the functions of the front end and the display end can also be integrated on the same physical device, for example, terminal devices such as mobile phones and tablets with video shooting functions.
  • some functions of the front end and some functions of the display end can also be integrated on the same physical device. This is not specifically limited.
  • Fig. 9 is a schematic structural diagram of an exemplary terminal device.
  • a terminal device 900 includes: an application processor 901, a microcontroller unit (microcontroller unit, MCU) 902, a memory 903, a modem (modem) 904, a radio frequency (radio frequency, RF) module 905, a wireless fidelity (Wireless-Fidelity, Wi-Fi for short) module 906, Bluetooth module 907, sensor 908, input/output (input/output, I/O) device 909, positioning module 910 and other components.
  • These components may communicate over one or more communication buses or signal lines.
  • the foregoing communication bus or signal line may be the CAN bus provided by the present application.
  • the terminal device 900 may include more or fewer components than shown in the figure, or combine certain components, or arrange different components.
  • Each component of the terminal device 900 is specifically introduced below in conjunction with FIG. 9:
  • the application processor 901 is the control center of the terminal device 900 and connects various components of the terminal device 900 through various interfaces and buses.
  • processor 901 may include one or more processing units.
  • Computer programs such as an operating system 911 and application programs 912 shown in FIG. 9 are stored in the memory 903 .
  • the application processor 901 is configured to execute the computer program in the memory 903 to realize the functions defined by the computer program.
  • the application processor 901 executes the operating system 911 to realize various functions of the operating system on the terminal device 900 .
  • the memory 903 also stores data other than computer programs, such as data generated during the running of the operating system 911 and application programs 912 .
  • the storage 903 is a non-volatile storage medium, generally including internal memory and external storage.
  • Memory includes, but is not limited to, random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), or cache (cache).
  • External memory includes but not limited to flash memory (flash memory), hard disk, optical disk, universal serial bus (universal serial bus, USB) disk, etc.
  • Computer programs are usually stored in external memory, and the processor loads the program from external memory into memory before executing the computer program.
  • the memory 903 may be independent and connected to the application processor 901 through a bus; the memory 903 and the application processor 901 may also be integrated into a chip subsystem.
  • the MCU 902 is a coprocessor for acquiring and processing data from the sensor 908.
  • the processing capability and power consumption of the MCU 902 are smaller than that of the application processor 901, but it has the feature of "always on” and can be used in the application processor.
  • MCU 902 may be a sensor hub chip.
  • Sensors 908 may include light sensors, motion sensors.
  • the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display 9091 according to the brightness of the ambient light, and the proximity sensor may turn off the power of the display screen when the terminal device 900 moves to the ear. .
  • an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary; the sensor 908 can also include a gyroscope, a barometer, a hygrometer, Other sensors such as thermometers and infrared sensors will not be described in detail here.
  • the MCU 902 and the sensor 908 can be integrated on the same chip, or they can be separate components connected by a bus.
  • the modem 904 and the radio frequency module 905 constitute the communication subsystem of the terminal device 900, which is used to realize the main functions of the wireless communication standard protocol. Among them, modem 904 is used for codec, signal modulation and demodulation, equalization, etc.
  • the radio frequency module 905 is used for receiving and sending wireless signals, and the radio frequency module 905 includes but is not limited to an antenna, at least one amplifier, a coupler, a duplexer, and the like.
  • the radio frequency module 905 cooperates with the modem 904 to realize the wireless communication function.
  • the modem 904 can be used as an independent chip, or can be combined with other chips or circuits to form a system-on-chip or integrated circuit. These chips or integrated circuits can be applied to all terminal devices that implement wireless communication functions, including: mobile phones, computers, notebooks, tablets, routers, wearable devices, automobiles, home appliances, etc.
  • the terminal device 900 can also use the Wi-Fi module 906, the Bluetooth module 907, etc. to perform wireless communication.
  • the Wi-Fi module 906 is used to provide the terminal device 900 with network access complying with Wi-Fi related standard protocols.
  • the terminal device 900 can access a Wi-Fi access point through the Wi-Fi module 906, and then access the Internet.
  • the Wi-Fi module 906 can also serve as a Wi-Fi wireless access point, and can provide Wi-Fi network access for other terminal devices.
  • the Bluetooth module 907 is used to implement short-distance communication between the terminal device 900 and other terminal devices (such as mobile phones, smart watches, etc.).
  • the Wi-Fi module 906 in this embodiment of the application may be an integrated circuit or a Wi-Fi chip, and the Bluetooth module 907 may be an integrated circuit or a Bluetooth chip.
  • the positioning module 910 is used to determine the geographic location of the terminal device 900 . It can be understood that the positioning module 910 may specifically be a receiver of a global positioning system (global position system, GPS) or Beidou satellite navigation system, Russian GLONASS and other positioning systems.
  • a global positioning system global position system, GPS
  • Beidou satellite navigation system Russian GLONASS and other positioning systems.
  • the Wi-Fi module 906, the Bluetooth module 907 and the positioning module 910 may be separate chips or integrated circuits, or may be integrated together.
  • the Wi-Fi module 906, the Bluetooth module 907 and the positioning module 910 can be integrated on the same chip.
  • the Wi-Fi module 906, the Bluetooth module 907, the positioning module 910 and the MCU 902 can also be integrated into the same chip.
  • the input/output device 909 includes, but is not limited to: a display 9091, a touch screen 9092, and an audio circuit 9093 and the like.
  • the touch screen 9092 can collect touch events of the user of the terminal device 900 on or near it (for example, the user uses any suitable object such as a finger or a stylus to operate on the touch screen 9092 or near the touch screen 9092), and Send the collected touch events to other devices (such as the application processor 901).
  • the user's operation near the touch screen 9092 can be called floating touch; through the floating touch, the user can select, move or drag an object (such as an icon, etc.) without directly touching the touch screen 9092 .
  • the touch screen 9092 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave.
  • the display (also referred to as a display screen) 9091 is used to display information input by the user or information presented to the user.
  • the display may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.
  • the touch screen 9092 can be covered on the display 9091. When the touch screen 9092 detects a touch event, it sends it to the application processor 901 to determine the type of the touch event, and then the application processor 901 can provide corresponding information on the display 9091 according to the type of the touch event. visual output.
  • the touch screen 9092 and the display 9091 are used as two independent components to realize the input and output functions of the terminal device 900, in some embodiments, the touch screen 9092 and the display 9091 can be integrated to realize the terminal device 900. input and output functions.
  • the touch screen 9092 and the display 9091 may be configured on the front of the terminal device 900 in the form of a full panel, so as to realize a frameless structure.
  • the audio circuit 9093 , the speaker 9094 and the microphone 9095 can provide an audio interface between the user and the terminal device 900 .
  • the audio circuit 9093 can transmit the electrical signal converted from the received audio data to the speaker 9094, and the speaker 9094 converts it into an audio signal for output; After being received, it is converted into audio data, and then the audio data is sent to another terminal device through the modem 904 and the radio frequency module 905, or the audio data is output to the memory 903 for further processing.
  • the terminal device 900 may also have a fingerprint recognition function.
  • a fingerprint collection device may be configured on the back of the terminal device 900 (for example, under the rear camera), or a fingerprint collection device may be configured on the front of the terminal device 900 (for example, under the touch screen 9092 ).
  • a fingerprint collection device can be configured in the touch screen 9092 to realize the fingerprint recognition function, that is, the fingerprint collection device can be integrated with the touch screen 9092 to realize the fingerprint recognition function of the terminal device 900 .
  • the fingerprint collection device is configured in the touch screen 9092 , may be a part of the touch screen 9092 , or may be configured in the touch screen 9092 in other ways.
  • the main component of the fingerprint collection device in the embodiment of the present application is a fingerprint sensor, which can use any type of sensing technology, including but not limited to optical, capacitive, piezoelectric or ultrasonic sensing technologies.
  • the operating system 911 carried by the terminal device 900 may be or other operating systems, which is not limited in this embodiment of the present application.
  • the terminal device 900 can be logically divided into a hardware layer, an operating system 911 , and an application layer.
  • the hardware layer includes hardware resources such as the application processor 901, the MCU 902, the memory 903, the modem 904, the Wi-Fi module 906, the sensor 908, and the positioning module 910 as described above.
  • the application layer includes one or more application programs, such as the application program 912, and the application program 912 may be any type of application program such as a social application, an e-commerce application, or a browser.
  • the operating system 911 is a software middleware between the hardware layer and the application layer, and is a computer program that manages and controls hardware and software resources.
  • the operating system 911 includes a kernel, a hardware abstraction layer (hardware abstraction layer, HAL), libraries and runtime (libraries and runtime), and a framework (framework).
  • the kernel is used to provide underlying system components and services, such as: power management, memory management, thread management, hardware drivers, etc.; hardware drivers include Wi-Fi drivers, sensor drivers, positioning module drivers, etc.
  • the hardware abstraction layer is the encapsulation of the kernel driver, provides an interface to the framework, and shields the implementation details of the low layer.
  • the hardware abstraction layer runs in user space, while the kernel driver runs in kernel space.
  • libraries and runtimes are also called runtime libraries, which provide the required library files and execution environment for executable programs at runtime.
  • the library and runtime include Android runtime (android runtime, ART), library, and scene package runtime.
  • ART is a virtual machine or virtual machine instance that converts an application's bytecode into machine code.
  • a library is a program library that provides support for an executable program at runtime, including a browser engine (such as webkit), a script execution engine (such as a JavaScript engine), a graphics processing engine, and the like.
  • the runtime of the scene package is the running environment of the scene package, mainly including the page execution environment (page context) and the script execution environment (script context).
  • the execution environment parses and executes codes or executable files implemented by scripting languages such as JavaScript by calling corresponding function libraries.
  • the framework is used to provide various basic public components and services for applications in the application layer, such as window management, location management, and so on.
  • the framework includes a geofence service, a policy service, a notification manager, and the like.
  • each component of the operating system 911 described above can be implemented by the application processor 901 executing programs stored in the memory 903 .
  • the terminal device 900 may include fewer or more components than those shown in FIG. 9 .
  • the terminal device shown in FIG. 9 only includes components that are more relevant to the multiple implementations disclosed in this application. .
  • the panoramic video can provide a spherical scene content of 360 degrees horizontally and 180 degrees vertically.
  • the user's head can be used as the center point, and the user's head can rotate 360 degrees horizontally and 180 degrees up and down. Pitching, so the user's viewing area can reach 360 degrees horizontally and 180 degrees vertically. It can be seen that if the panoramic video is simulated as a spherical surface, the user's head can be rotated through the head as long as it is at the center of the sphere. and/or tilt to see the full panorama video.
  • any point on the panoramic video three-dimensional spherical expression panoramic image can adopt spherical coordinates to represent, and on the 3D panoramic image
  • Any pixel point can find a corresponding point on the three-dimensional spherical expression panorama of the panoramic video, and this point can be called the mapping point of any pixel point mentioned above.
  • mapping The operation of expanding a 3D panoramic image to obtain a 2D panoramic image is called mapping.
  • mapping there are various mapping methods, and correspondingly, there are various 2D panoramic image formats.
  • the most common 2D panoramic image format is called longitude and latitude map.
  • the content is that the 3D panoramic image of the earth is expanded into a 2D panoramic image, and the images of the adjacent north and south poles are obtained by stretching, which has serious distortion and data redundancy. Remain.
  • the 3D panoramic image can be projected onto the surface of a regular polyhedron, so that the spherical surface can be expressed as a number of triangles, squares or pentagons of equal size.
  • polyhedron For example, a regular tetrahedron (as shown in (a) in Figure 10b), a regular hexahedron (as shown in (b) in Figure 10b), a regular octahedron (as shown in (c) in Figure 10b), a regular decahedron Dihedron (as shown in (d) in Figure 10b), regular icosahedron (as shown in (e) in Figure 10b), etc.
  • the present application does not limit the polyhedron to be a regular polyhedron, and a sphere may also be expressed as a polyhedron composed of several polygons of unequal sizes.
  • the polyhedron is expanded into a 2D panoramic image
  • (a) in Figure 10b is expanded into a 2D panoramic image shown in (f) in Figure 10b
  • (b) in Figure 10b is expanded into (g) in Figure 10b 2D panoramic image
  • (c) in Figure 10b is expanded into a 2D panoramic image shown in (h) in Figure 10b
  • (d) in Figure 10b is expanded into a 2D panoramic image shown in (i) in Figure 10b
  • (e) in Figure 10b is expanded into a 2D panoramic image shown in (j) in Figure 10b.
  • These 2D panoramic images may be collectively referred to as panoramic images in a polyhedron format.
  • the specific projection process is to place the sphere into the polyhedron so that it becomes the inscribed sphere of the polyhedron.
  • the center of the sphere (or the center of the polyhedron) is connected with a point on the sphere and extended to intersect the surface of the polyhedron.
  • the pixel value at this point on the sphere is the pixel value at the intersection point on the surface of the polyhedron.
  • the sphere is inscribed in the hexahedron ABCDEFGH as shown in Figure 10c.
  • the pixel value at point M is the pixel value at point M'.
  • the pixel values of all pixel positions in the ABCD plane on the plane A'B'C'D' can be obtained.
  • the pixel points in the ABCD plane constitute the surface image of the ABCD plane (face), and the plane A'B'C'D' It is called the projection plane of the ABCD plane.
  • the surface images of other surfaces of the hexahedron and the corresponding projection planes can be obtained.
  • the surface image referred to in this application is an N ⁇ N array of spatially adjacent pixels, and these pixels are in the same projection plane during the process of mapping a spherical surface to a polyhedron surface, and these pixels The pixel value of the point location is known.
  • each surface image on the polyhedron surface becomes an image of a region in the 2D panoramic image or a sub-image of the 2D panoramic image
  • the surface image of the polyhedron surface refers to the image of this region or the sub-image In the image
  • this face of the polyhedron surface refer to this area in the 2D panoramic image or the coverage area of this sub-image.
  • the hexahedral surface of (a) in Figure 10d is unfolded into the 2D panoramic image of (b) in Figure 10d, and the surface image of the Top surface on the hexahedral surface will become the child in the upper left corner of (b) in Figure 10d.
  • the Top surface image of the 2D panoramic image we call this sub-image the Top surface image of the 2D panoramic image, and the Top surface in the 2D panoramic image refers to the area covered by the Top surface image.
  • the Top surface is called the surface where the pixel is located.
  • the rest of the irregular 2D image can be filled with default content, such as all gray, all black or all white, etc.
  • (b) in Figure 10e is obtained by expanding (a) in Figure 10e
  • (d) in Figure 10e is obtained by expanding (c) in Figure 10e
  • the values of the pixels in the rest are set to gray corresponding to The 2D rectangular image after the pixel values of .
  • these faces can also be directly stitched into a regular rectangle, thereby avoiding the filling operation.
  • the processed image is generally a rectangular image.
  • the 3D panoramic image can be projected into a two-dimensional plane projection map of the panoramic video, and then the two-dimensional plane projection map of the panoramic video can be mapped to a three-dimensional spherical expression panorama of the panoramic video, that is, the panorama
  • the multiple pixel points included in the video two-dimensional plane projection map are respectively mapped to the panoramic video three-dimensional spherical expression panorama to obtain multiple mapping points, and the processing of the panoramic video two-dimensional plane projection map is converted into the panoramic video three-dimensional spherical expression panorama. deal with.
  • the three-dimensional spherical representation panorama of the panoramic video can be divided to obtain multiple divided regions.
  • Multiple segmentation regions can be obtained in any of the following ways:
  • the multiple segmented areas are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video with a preset viewing angle interval rule.
  • the three-dimensional spherical expression panorama of the panoramic video can be divided into a plurality of segmented areas with preset viewing angle intervals.
  • the horizontal angles corresponding to the regions are equal, and the pitch angles corresponding to the two adjacent division regions in longitude are equal.
  • the multiple segmentation areas are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video in the latitude direction; and/or, the multiple segmentation areas are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video in the longitude direction of.
  • the three-dimensional spherical expression panorama of the panoramic video can be divided into multiple segmentation areas from the latitude direction.
  • the three-dimensional spherical expression panorama of the panoramic video is evenly divided into a plurality of segmentation areas from the longitude direction, for example, as shown in Figure 11c, the three-dimensional spherical expression panorama of the panoramic video is divided into two equal-sized segmentation areas on the left and right;
  • the three-dimensional spherical expression panorama of the panoramic video can be divided into multiple segmentation regions from the latitude direction, and the three-dimensional spherical expression panorama of the panoramic video can be evenly divided into multiple segmentation regions from the longitude direction, for example, as shown in Figure 11d , divide the panoramic video three-dimensional spherical expression panorama into four segmentation regions of equal size up and down and equal in size left and right.
  • the division methods in the above examples do not limit the division methods, and the panoramic video three-dimensional spherical expression panorama can also be divided into two parts of equal size, and then the two parts are evenly quartered;
  • the three-dimensional spherical expression panorama of the panoramic video is divided into two parts of equal size, the upper and lower parts, and then the two parts are evenly quartered, and finally each quartered area is divided into areas with equal viewing angle intervals.
  • the division of the panorama expressed by the three-dimensional spherical surface of the panoramic video may not be equally divided, or not all division steps are equally divided, and this application does not specifically limit this.
  • the area formed by the corresponding mapping points of the pixels included in the first pixel point set on the three-dimensional spherical expression panorama of the panoramic video is used as the first segmented area, and the first pixel point set is one of a plurality of pixel point sets.
  • a segmented area is one of a plurality of segmented areas.
  • the area formed by the mapping points included in the first mapping point set is used as the second segmented area, the first mapping point set is one of the multiple mapping point sets, and the second segmented area is one of the multiple segmented areas.
  • the above clustering methods include:
  • the initial division method may also include any division method in the above-mentioned 1 or 2, which is not specifically limited;
  • mapping points respectively included in the plurality of divided regions.
  • the above mapping process obtains multiple mapping points on the three-dimensional spherical representation panorama of the panoramic video, and these mapping points can be determined to which segmented area they belong according to their spherical coordinates.
  • Each of the multiple segmentation regions may contain mapping points, or some of the segmentation regions may not contain mapping points;
  • the color component value can refer to at least one of the components Y, U, and V in the color format YUV, and the color component value can also refer to at least one of the components R, G, B and max(R, G, B) in the color format RBG.
  • the color component value may also refer to at least one of L, a and b in the color format Lab.
  • the average color component value of the i-th segmented area may refer to an average value of color component values of a plurality of mapping points included in the i-th segmented area.
  • the i-th segmented area is one of the plurality of segmented areas, and the processing process of each segmented area is described here by taking the processing process of the i-th segmented area as an example. It should be understood that the present application may also obtain other feature information of the i-th segmented area, for example, the color component value of a specified mapping point of the i-th segmented area, which is not specifically limited.
  • the average value avgx of the x-coordinates and the average value avgy of the y-coordinates of all mapped points in the i-th segmented area can be calculated, using coordinates (avgx ,avgy) to replace the above-mentioned mapping point at the center;
  • Dj represents the clustering distortion value of the jth mapping point
  • Qc represents the color component value of the mapping point located in the central position in the ith segmented area
  • Q j represents the color component value of the jth mapping point
  • (x c , y c , z c ) represent the coordinates of the mapping point located at the center of the i-th divided region
  • (x j , y j , z j ) represent the coordinates of the j-th mapping point.
  • the normalization (Q c -Q j ) can use the maximum value range of Q c and Q j to normalize them to between 0 and 1.0.
  • the average area of the segmented area (that is, the spherical area divided by the number of segmented areas) can be used. Assuming that the segmented area is circular or square, the radius (corresponding to the circular segmented area) or side length (corresponding to the Divide the calculated Euclidean distance by the radius or the side length of the square, and if it is greater than 1, set it to 0, which can be normalized to between 0 and 1.0;
  • D j represents the clustering distortion value of the j-th mapping point
  • Qa c represents the average color component value of the i-th segmented area
  • Q j represents the color component value of the j-th mapping point
  • (x c , y c , z c ) represents the coordinates of the mapping point at the center position in the i-th segmented area
  • (x j , y j , z j ) represents the coordinates of the j-th mapping point.
  • mapping point located in the center of the i-th segmented area is replaced by the value of the neighboring segmented area of the i-th segmented area
  • the mapping point located at the center position of and then determine whether the jth mapping point belongs to the neighbor segmentation area according to the clustering distortion value obtained again, and so on until the segmentation area to which the jth mapping point belongs is determined.
  • mapping points belonging to the same segmentation area (such as the i-th segmentation area) after clustering (the multiple mapping points constitute one of the above The area formed by the set of mapping points) is the final segmentation area.
  • the difference between the above division method 3 and division method 4 is that the division method 3 is to use the above clustering method to cluster the multiple pixels included in the panoramic image to obtain a plurality of pixel sets, and then map the pixel points to the three-dimensional panoramic video respectively.
  • the spherical surface expresses the panorama, and the area formed by the corresponding mapping points of the pixels included in a set of pixels is the final segmented area.
  • FIG. 12 is a flowchart of a process 1200 of the tone mapping method for a panoramic image of the present application.
  • the process 1200 may be executed by the front end in the panoramic video playing system shown in FIG. 8 .
  • the process 1200 is described as a series of steps or operations. It should be understood that the process 1200 may be performed in various orders and/or concurrently, and is not limited to the order of execution shown in FIG. 12 .
  • Process 1200 may include:
  • Step 1201. Obtain at least one mapping point included in the first segmented area.
  • the front end may use the above method to divide the three-dimensional spherical representation panorama of the panoramic video to obtain multiple divided regions.
  • the front-end can adopt a preset division method. For example, if the first division method above is preset, the front-end can divide the three-dimensional spherical expression panorama of the panoramic video into multiple divisions with equal viewing angle intervals. area. At this time, the preset division method can be applied to most panoramic images, so as to improve image processing efficiency. Alternatively, the front-end can also adopt a division method determined based on preset rules.
  • the front-end after the front-end acquires the panoramic image to be processed, it can perform feature analysis on the panoramic image to determine the division method, which is more in line with the panoramic image. features to improve image processing efficiency.
  • the front end may also provide a user selection interface, on which the user selects or inputs a desired division method, so that the division method can be more in line with the dynamic range of the panoramic image.
  • the first segmented area is one of the multiple segmented areas obtained after the front end divides the panorama expressed by the three-dimensional spherical surface of the panoramic video.
  • the mapping relationship between the panoramic image and the panoramic video three-dimensional spherical expression panorama multiple pixels included in the panoramic image can be respectively mapped to the panoramic video three-dimensional spherical expression panorama to obtain multiple mapping points, multiple pixel points and multiple Mapping points correspond one to one. Based on this, after the range of the first segmented area is determined, at least one mapping point contained therein may be obtained.
  • Step 1202 generating metadata information units of the first segmented area according to at least one pixel point.
  • panoramic video data includes video data and metadata information
  • video data can be related data of a plurality of image frames included in panoramic video, for example, pixel data of each image frame; metadata information can include data related to video data and mapping curves related data.
  • the data related to video data can include the format and characteristics of panoramic video data
  • the target system displays maximum luminance (targeted_system_display_maximum_luminance), the maximum luminance MaxSource, the minimum luminance MinSource, and the average luminance AvgSource of panoramic video in memory, which can be stored.
  • Variation range, etc. the principle of the storable variation range is similar to the variance or distribution confidence interval, which is used to describe the brightness aggregation range of the video signal.
  • the data related to the mapping curve may include parameters related to the mapping curve such as shrinkage factor, offset factor, bending shape factor, etc.
  • the data related to the S-curve includes a, b, p, n, m, k1, k2 and k3 A total of eight parameters. It should be noted that the panoramic video data includes but not limited to the above content, which is not specifically limited in this application.
  • the ith segmented area is taken as an example for illustration.
  • n the number of mapping points contained in the i-th segmentation region. It should be noted that the luminance value of the mapping point is actually the luminance value of the pixel in the panoramic image corresponding to the mapping point, and the same applies below.
  • n the number of mapping points included in the i-th segmented area. It should be noted that the histogram of all the mapping points is actually the histogram of multiple pixels in the panoramic image corresponding to all the mapping points, and the same applies below.
  • the front end may The second metadata information unit is fused to obtain the metadata information unit of the first segmented area and the second segmented area, and the second segmented area is one of the plurality of segmented areas.
  • the front end obtains the histogram and/or brightness value of multiple segmented regions, if the histogram and/or brightness value of more than two segmented regions are similar, for example, the histogram and/or brightness value of the two or more segmented regions are distorted If the value is less than the set threshold, it can be considered that the dynamic range similarity of the image regions corresponding to the two or more segmented regions is relatively high, so the same metadata information unit can be determined for the two or more segmented regions.
  • the metadata information unit may be obtained by weighting and averaging the metadata information units of the two or more segmented regions obtained in the above steps, or the two or more segmented regions obtained in the above steps may be used as a segmented region again Calculate metadata information unit.
  • the following method may be used to determine whether the distortion values of the histograms of more than two segmented regions are smaller than a set threshold:
  • the front end can add indication information to the metadata information unit to indicate which segmented areas the metadata information unit corresponds to. For example, multiple spherical coordinates are written in the metadata information unit, and the multiple spherical coordinates are respectively It is located in the above two or more division areas, so that when the display terminal reads the metadata information unit, it determines the division area to which the spherical coordinates belong according to the spherical coordinates included in it, so as to determine it as the division corresponding to the metadata information unit area.
  • the front end when the front end generates the metadata information unit of each segmented area, it fully considers the characteristics of the pixels corresponding to the mapping points contained in the segmented area, and the features of the pixels can reflect the dynamic range of the segmented area, so the display terminal is in When the tone mapping curve of the corresponding segmented area is obtained based on the metadata information unit, a tone mapping curve matching the dynamic range characteristics of the segmented area is obtained, thereby realizing more accurate tone mapping processing for the panoramic image.
  • Step 1203 write the metadata information unit of the first divided area into the code stream.
  • the metadata information unit can be written into the code stream for the display terminal to perform corresponding tone mapping when displaying the panoramic image. It should be understood that the above methods may be used to determine corresponding metadata units for other segmented areas except the first segmented area among the plurality of segmented areas, which will not be repeated here.
  • the front end may traverse multiple divided areas in a set order, and then write its corresponding metadata information unit into the code stream each time a divided area is traversed.
  • the front-end can determine the order of traversing multiple divided areas according to preset rules, and then write the corresponding metadata information unit into the code stream for each divided area according to the order. At the same time, the front-end also writes the Write the bitstream in a definite order.
  • the above order may be a top-down or front-to-back order of expressing the panorama from the three-dimensional spherical surface of the panoramic video.
  • the front-end can interleave the process of generating metadata information units and writing code streams, that is, the front-end traverses multiple divided areas according to the above sequence, and uses the method in step 1202 to generate its corresponding metadata information unit, and then write the metadata information unit into the code stream.
  • the metadata information unit guides the display terminal to generate a tone mapping curve corresponding to the segmented area, and then realizes the tone mapping from the image to the display, which can not only determine the matching tone mapping parameters for different dynamic range areas on the panoramic image, but also It can conform to the display dynamic range of the display terminal, and improve the processing efficiency of the panoramic image.
  • FIG. 13 is a flow chart of a process 1300 of the tone mapping method for a panoramic image of the present application.
  • the process 1300 may be executed by the display terminal in the panoramic video playing system shown in FIG. 8 .
  • the process 1300 is described as a series of steps or operations. It should be understood that the process 1300 may be performed in various orders and/or concurrently, and is not limited to the order of execution shown in FIG. 13 .
  • Process 1300 may include:
  • Step 1301. Determine one or more target metadata information units of the first pixel from multiple metadata information units.
  • the display terminal may use the above method to divide the three-dimensional spherical representation panorama of the panoramic video to obtain multiple divided regions.
  • the display terminal may adopt a preset division method, or may adopt a division method obtained by parsing the code stream.
  • the display terminal can also parse the code stream to obtain multiple metadata information units.
  • the display terminal can analyze the number information of multiple metadata information units from the code stream.
  • the number information can identify multiple metadata information units, and on the other hand, it can indicate the sequence of multiple metadata information units; or , the display terminal may determine the order in which the plurality of metadata information units are parsed out as the sequence of the plurality of metadata information units.
  • the display terminal can determine the corresponding relationship between multiple segmentation areas and multiple metadata information units through the following methods:
  • the display terminal can extract the current metadata information unit from multiple metadata information units according to the first preset order, extract the current segmented area from multiple segmented areas according to the second preset order, and combine the current segmented area and the current metadata information unit
  • the data information unit establishes a corresponding relationship.
  • the display terminal can extract the current metadata information unit from multiple metadata information units according to the first preset order, extract the current segmented area from multiple segmented areas according to the traversal order obtained by parsing the code stream, and combine the current segmented area and The current metadata information unit establishes a corresponding relationship.
  • the display terminal can extract the current metadata information unit from a plurality of metadata information units according to the first preset order, obtain one or more coordinates included in the current metadata information unit, and display the three-dimensional information in the panoramic video according to one or more coordinates.
  • One or more mapping points are determined on the spherical representation panorama. When there is only one mapping point, establish a corresponding relationship between the segmentation area to which one mapping point belongs and the current metadata information unit; when there are multiple mapping points, at least one segmentation area to which multiple mapping points belong and the current metadata information unit Create correspondence.
  • the current metadata information unit can be regarded as the first among multiple metadata information units. As mentioned above, there is an order among multiple metadata information units. The order of the data information units is to extract the metadata information unit at the top, which is the current metadata information unit. Once the correspondence between the current metadata information unit and one or more segmentation units is determined, it will be Remove from the first sequence, determine the next metadata information unit as the new current metadata information unit, and iterate multiple times until the corresponding relationship between multiple divided regions and multiple metadata information units is determined.
  • the display terminal may determine one or more target segmented areas according to the set mapping points, and when there is only one target segmented area, determine the metadata information unit corresponding to one target segmented area as a target metadata information unit; or , when there are multiple target segmented areas, the metadata information units respectively corresponding to the multiple target segmented areas are determined as the multiple target metadata information units.
  • the mapping point is set to be the first mapping point, and the first coverage area is determined on the three-dimensional spherical expression panorama of the panoramic video centered on the first mapping point, when the first coverage area includes a segmented area , determine one segmented area as one target segmented area; or, when the first coverage range includes multiple segmented areas, determine the multiple segmented areas as multiple target segmented areas.
  • the display terminal can use the first mapping point as the center and the first length as the radius to determine the first coverage area.
  • the first coverage area is a circular area, or, the display terminal can also use the first mapping point as the center and the first length as the side length to determine the first coverage area. In this case, the first coverage area is a square area.
  • the first length may be a preset length, or the first length may be specified in the first metadata information unit corresponding to the first division area to which the first mapping point belongs.
  • the plurality of divided areas includes a first divided area, and the plurality of metadata information units includes a first metadata information unit. It should be understood that the first coverage area may also be an area of other shapes, and the relevant information of its shape and size may be preset or specified in the aforementioned first metadata information unit, which is not covered by this application. Specific limits.
  • the first coverage area may only cover the segmented area to which the first mapping point belongs.
  • the segmented area to which the first mapping point belongs is a target Divide the area.
  • the first coverage range may also cover the division area to which the first mapping point belongs and one or more division areas adjacent to it, then the division area to which the first mapping point belongs and one or more division areas adjacent to it are multiple target segmentation area.
  • the first coverage area may only cover a part of the aforementioned adjacent segmented area, so the adjacent segmented area may also be used as the target segmented area.
  • the mapping point is set to be the viewpoint center of the panorama expressed on the three-dimensional spherical surface of the panoramic video, and the second segmented area to which the viewpoint center belongs is determined, the plurality of segmented areas include the second segmented area, and the second segmented area Determined as a target segmentation region.
  • the mapping point is set to be the viewpoint center of the panoramic video three-dimensional spherical expression panorama, and the second coverage area is determined on the panoramic video three-dimensional spherical expression panorama centered on the visual center, when the second coverage area
  • one segmented area is determined as one target segmented area; or, when the second coverage range includes multiple segmented areas, the multiple segmented areas are determined as multiple target segmented areas.
  • the display end can use the visual center as the center and the second length as the radius/side length to determine the second coverage area.
  • the second coverage area is a circle.
  • the display terminal may also center on the visual center and use the second length as the side length to determine the second coverage area.
  • the second coverage area is a square area.
  • the second length may be a preset length, or the second length may be specified in the second metadata information unit corresponding to the second segmented area to which the viewpoint center belongs.
  • the plurality of divided areas includes a second divided area, and the plurality of metadata information units includes a second metadata information unit. It should be understood that the second coverage area can also be an area of other shapes, and the related information of its shape and size can be preset or specified in the aforementioned second metadata information unit, which is not covered by this application. Specific limits.
  • the difference from determining the first coverage area with the first mapping point as the center and the first length as the radius is that at this time, the visual center is used as the center and the second length is the radius to determine the second coverage area, and then the The segmented area covered by the second coverage area is determined as the target segmented area.
  • the second coverage may only cover the segmented area to which the visual center belongs.
  • the segmented area to which the visual center belongs is a target segmented area
  • the second coverage may also cover the segmented area to which the visual center belongs and an adjacent or multiple segmented areas, then the segmented area to which the visual center belongs and one or more adjacent segmented areas are multiple target segmented areas.
  • the second coverage range may only cover a part of the aforementioned adjacent segmented area, so the adjacent segmented area may also be used as the target segmented area.
  • the metadata information units corresponding to one or more target segmented areas determined in the above steps can be determined as one or more target elements of the first pixel Data information unit.
  • Step 1302 Perform tone mapping on the pixel value of the first pixel point according to one or more target metadata information units to obtain a target tone mapping value of the first pixel point.
  • the display terminal may acquire one or more tone mapping curves according to one or more target metadata information units.
  • the pixel values of the first pixel are respectively tone-mapped to obtain a plurality of intermediate tone values of the first pixel, and then a target tone mapping value is obtained according to the plurality of intermediate tone values.
  • the metadata information unit includes the relevant parameters of the tone mapping curve, so the corresponding tone mapping curve can be obtained based on the parameters. It should be noted that the present application does not specifically limit the parameters of the tone mapping curve and the tone mapping curve .
  • a tone mapping curve can be obtained for the first pixel; if the first pixel has multiple target metadata information units, then for the first pixel Points can get multiple tone mapping curves.
  • the target tone-mapped value of the first pixel can be directly obtained by performing tone-mapping on the pixel value of the first pixel by using the above-mentioned tone-mapping curve.
  • multiple tone intermediate values of the first pixel point can be obtained, and the multiple tone intermediate values are further processed, for example, taking multiple tone intermediate values
  • the middle value among the values for example, if the middle values of multiple hues include 1, 2, and 3, then the middle value is 2), or the result of performing weighted average processing on multiple middle values of hues is the target mid-tone of the first pixel value.
  • the weighting coefficients corresponding to the intermediate values of each tone in the aforementioned weighted average may be related to the distance between the first mapping point and the center point of each divided region.
  • the distance (such as Euclidean distance) between the first mapping point and the center point of each segmented area in the multiple target segmented areas
  • the ratio of the sum is used as the metadata information unit of the corresponding target segmented area to obtain the weighting coefficient of the intermediate value of the hue; or, the weighting coefficient corresponding to each intermediate value of the hue in the aforementioned weighted average may correspond to multiple target metadata information units respectively
  • the area proportion of the segmented area is related.
  • the tone mapping curve parameters a, b, p, m, n are obtained from the target metadata information unit of the first pixel, and then the tone mapping curve can be obtained:
  • L and L' may be normalized optical signals or electrical signals, which are not specifically limited in this application.
  • the normalization calculation can be the nonlinear space of PQ, or the linear space of normalization 0 ⁇ 1. Normalization can be 0-10000nit or 0.001-100000nit. This application does not specifically limit the scope and process of data normalization.
  • the normalized HDR display data L' is inversely normalized to between the maximum display capability and the minimum display capability of the display device, the first pixel
  • the target hue value of the point is the HDR display data after inverse normalization.
  • the reverse normalization calculation may be a nonlinear space of PQ, or a linear space of normalization 0-1.
  • the reverse normalization can be 0-10000nit or 0.001-100000nit. This application does not specifically limit the scope and process of data denormalization.
  • the metadata information unit guides the display terminal to generate a tone mapping curve corresponding to the segmented area, and then realizes the tone mapping from the image to the display, and can determine matching tone mapping parameters for different dynamic range areas on the panoramic image, It can also conform to the display dynamic range of the display terminal, and improve the processing efficiency of the panoramic image.
  • the display end can parse the code stream to obtain multiple metadata information units.
  • the display end can parse the code stream to obtain the number information of multiple metadata information units.
  • the number information can identify Multiple metadata information units, on the other hand, can indicate the sequence of multiple metadata information units; or, the display terminal can parse the code stream to obtain multiple metadata information units, and determine the order in which the metadata information units are parsed out as The sequence of multiple metadata information units.
  • the display terminal may divide the three-dimensional spherical representation panorama of the panoramic video to obtain multiple divided regions.
  • the display terminal may obtain a preset division method, and then divide the panoramic video three-dimensional spherical expression panorama according to the preset division method to obtain multiple division regions.
  • Users can set the division method in advance based on historical data or laboratory data, so that the division method can be applied to most panoramic images to improve image processing efficiency.
  • the user may also select or input a desired division method on the user interface provided by the panoramic image processing system, so that the division method can be more in line with the dynamic range of the panoramic image.
  • the display end may parse the code stream to obtain the division mode set by the front end.
  • the front end After acquiring the panorama image to be processed, the front end can analyze the features of the panorama image to determine a division method, which is more in line with the characteristics of the panorama image, so as to improve image processing efficiency. In order to be consistent with the display end, the front end can write the determined division method into the code stream. After the display terminal parses the code stream, the aforementioned division method can be obtained.
  • the display terminal can determine that the division method is division according to the viewing angle rules.
  • the display terminal can divide the three-dimensional spherical expression panorama of the panoramic video into a plurality of division areas with equal viewing angle intervals. For example, as shown in FIG.
  • the display terminal can divide the panorama video three-dimensional spherical expression panorama from the latitude direction into multiple division areas, for example, as shown in Figure 12b
  • the display end divides the three-dimensional spherical expression panorama of the panoramic video into two division areas of equal size up and down; or, divides the three-dimensional spherical expression panorama of the panoramic video into multiple divisional areas uniformly from the longitude direction, for example, as shown in Figure 12c
  • the display end divides the three-dimensional spherical expression panorama of the panoramic video into two divided areas of equal size on the left and right; or, the display end can divide the three-dimensional spherical expression panorama of the panoramic video into multiple divided areas from the latitude direction, and
  • the three-dimensional spherical expression panorama of the panoramic video is evenly divided into multiple segmentation areas from the longitude direction. area.
  • the display terminal can also divide the three-dimensional spherical expression panorama of the panoramic video into two parts of equal size up and down, and then divide the two parts into equal quarters; display The terminal can also divide the three-dimensional spherical expression panorama of the panoramic video into two parts of equal size, the upper and lower parts, and then divide the two parts into quarters, and finally divide each quartered area into areas with equal viewing angle intervals.
  • the division of the panoramic video three-dimensional spherical expression panorama by the display end may not be equally divided, or not all division steps are equally divided, which is not specifically limited.
  • the display terminal can also analyze the code stream and directly obtain multiple segmented regions included in the three-dimensional spherical expression panorama of the panoramic video.
  • the front-end divides the three-dimensional spherical expression panorama of the panoramic video to obtain multiple segmentation areas, it can write the division information of the multiple segmentation areas into the code stream, for example, the shape, size, and coordinates of the upper left corner of the segmentation area, etc., and for example , the description information of the dividing line, so that the display terminal can analyze the code stream to obtain the dividing information of the above-mentioned multiple dividing areas, and then determine the multiple dividing areas.
  • FIG. 14 is an exemplary structural diagram of a display device 1400 of the present application.
  • the display device 1400 of this embodiment can be applied to the display terminal in the above embodiment, and can also be applied to the terminal in the above embodiment. equipment.
  • the display device 1400 may include: a division module 1401 , a determination module 1402 and a mapping module 1403 . in,
  • a determining module 1402 configured to determine one or more target metadata information units of a first pixel from a plurality of metadata information units, the plurality of metadata information units are obtained by parsing the code stream, and the first pixel
  • the point is any pixel point in the two-dimensional plane projection map of the panoramic video to be processed, and the plurality of metadata information units correspond to the multiple segmentation regions included in the three-dimensional spherical expression panorama of the panoramic video, and the two-dimensional plane of the panoramic video
  • the projection image and the panoramic video three-dimensional spherical expression panorama have a mapping relationship;
  • the mapping module 1403 is configured to perform tone mapping on the pixel value of the first pixel point according to the one or more target metadata information units to obtain the The target tone mapping value of the first pixel.
  • the dividing module 1401 is configured to acquire the plurality of segmented regions.
  • the division module 1401 is specifically configured to divide the panoramic video three-dimensional spherical expression panorama according to a preset division method, so as to obtain the plurality of division regions.
  • the division module 1401 is specifically configured to divide the panoramic video three-dimensional spherical expression panorama according to a division manner obtained by parsing the code stream, so as to obtain the plurality of division regions.
  • the multiple segmented regions are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video with a preset viewing angle interval rule; or,
  • the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from the latitude direction; and/or, the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from obtained by dividing in the longitude direction.
  • the division module 1401 is specifically configured to cluster a plurality of pixel points included in the panoramic video two-dimensional plane projection map to obtain a plurality of pixel point sets;
  • the pixels are respectively mapped to the three-dimensional spherical expression panorama of the panoramic video; the area formed by the corresponding mapping points of the pixels included in the first set of pixel points on the three-dimensional spherical expression panorama of the panoramic video is used as the first segmented area, so
  • the first set of pixel points is one of the plurality of sets of pixel points, and the first segmented area is one of the plurality of segmented areas.
  • the division module 1401 is specifically configured to map multiple pixels included in the panoramic video two-dimensional planar projection image to the panoramic video three-dimensional spherical expression panorama to obtain multiple Mapping points; clustering the multiple mapping points to obtain multiple mapping point sets; using the area formed by the mapping points included in the first mapping point set as a second segmented area, the first mapping point set is the One of multiple mapping point sets, the second segmented area is one of the multiple segmented areas.
  • the division module 1401 is specifically configured to acquire the multiple segmented areas according to the indication information of the multiple segmented areas obtained by parsing the code stream.
  • the determining module 1402 is specifically configured to determine the correspondence between the plurality of metadata information units and the plurality of segmented regions, and one metadata information unit corresponds to one or more The segmented area: determining one or more target segmented areas according to the set mapping point; when there is only one target segmented area, determining the metadata information unit corresponding to the one target segmented area as the one target metadata information unit or, when there are multiple target segmented areas, determining the metadata information units corresponding to the multiple target segmented areas as the multiple target metadata information units.
  • the determining module 1402 is specifically configured to extract the current metadata information unit from the plurality of metadata information units in a first preset order; extract the current metadata information unit from the plurality of metadata information units in a second preset order; Extracting a current segmented area from a plurality of segmented areas; establishing a corresponding relationship between the current segmented area and the current metadata information unit.
  • the determining module 1402 is specifically configured to extract the current metadata information unit from the plurality of metadata information units in a first preset order; according to the traversal obtained by parsing the code stream Sequentially extracting the current segmented area from the plurality of segmented areas; establishing a corresponding relationship between the current segmented area and the current metadata information unit.
  • the determining module 1402 is specifically configured to extract the current metadata information unit from the plurality of metadata information units in a first preset order; obtaining the current metadata information unit includes One or more coordinates; according to the one or more coordinates, one or more mapping points are determined on the three-dimensional spherical expression panorama of the panoramic video; when there is only one mapping point, the segmentation to which the one mapping point belongs Establish a corresponding relationship between the area and the current metadata information unit; when there are multiple mapping points, establish a corresponding relationship between at least one segmented area to which the multiple mapping points belong and the current metadata information unit.
  • the set mapping point is the first mapping point of the first pixel point on the panorama expressed by the three-dimensional spherical surface of the panoramic video; the determining module 1402 is specifically configured to use the Determine the first coverage area on the panoramic video three-dimensional spherical expression panorama with the first mapping point as the center; when the first coverage area includes a segmented area, determine the segmented area as the target segment or, when the first coverage range includes multiple segmented areas, determining the multiple segmented areas as the multiple target segmented areas.
  • the set mapping point is the center of the viewpoint on the panorama expressed on the three-dimensional spherical surface of the panoramic video; the determining module 1402 is specifically configured to determine the second segmented area to which the center of the viewpoint belongs , the multiple segmented areas include the second segmented area; determining the second segmented area as the one target segmented area.
  • the set mapping point is the viewpoint center on the panoramic video three-dimensional spherical expression panorama; the determining module 1402 is specifically configured to center the visual center on the panoramic Determine the second coverage area on the video three-dimensional spherical expression panorama; when the second coverage area includes a segmented area, determine the segmented area as the target segmented area; or, when the second coverage area When a plurality of segmented regions are included, the plurality of segmented regions are determined as the plurality of target segmented regions.
  • the mapping module 1403 is specifically configured to obtain one or more tone mapping curves according to the one or more target metadata information units; when there is only one color bar mapping curve, according to the The one tone mapping curve performs tone mapping on the pixel value of the first pixel point to obtain the target tone mapping value; or, when there are multiple color stripe mapping curves, each performing tone mapping on the pixel value of the first pixel to obtain a plurality of intermediate tone values of the first pixel; and obtaining the target tone mapping value according to the plurality of intermediate tone values.
  • the mapping module 1403 is specifically configured to determine an intermediate value among the plurality of intermediate tone values as the target tone mapping value; or, perform weighted average to obtain the target tone mapping value, the weighting coefficients of the plurality of intermediate tone values are preset, or the weighting coefficients of the plurality of intermediate tone values are combined with the first pixel point and each segmented area The distance of the central point is related, or, the weighting coefficients of the plurality of mid-tone values are related to the area proportions of the divided regions respectively corresponding to the plurality of target metadata information units.
  • the device of this embodiment can be used to implement the technical solution of the method embodiment shown in FIG. 13 , and its implementation principle and technical effect are similar, and will not be repeated here.
  • FIG. 15 is an exemplary structural diagram of a front-end device 1500 of the present application.
  • the front-end device 1500 of this embodiment can be applied to the front-end in the above-mentioned embodiment, and can also be applied to the terminal equipment in the above-mentioned embodiment .
  • the front-end device 1500 may include: a division module 1501 , an acquisition module 1502 and a generation module 1503 . in,
  • An acquisition module 1502 configured to acquire at least one mapping point included in a first segmented area, where the first segmented area is one of multiple segmented areas included in a panoramic video three-dimensional spherical expression panorama, and the panoramic video three-dimensional spherical expression panorama There is a mapping relationship between the graph and the 2D planar projection map of the panoramic video to be processed, and the at least one mapping point corresponds to at least one pixel on the 2D planar projection map of the panoramic video; the generation module 1503 is configured to use the at least generating a metadata information unit of the first segmented area by one pixel; writing the metadata information unit of the first segmented area into a code stream.
  • the generating module 1503 is further configured to, when the histograms and/or brightness of the first segmented area and the second segmented area meet the set condition, The metadata information unit of the second segmented area is fused with the second metadata information unit of the second segmented area to obtain the metadata information unit of the first segmented area and the second segmented area, and the second segmented area is the one of the plurality of divided regions.
  • the division module 1501 is configured to map the 2D plane projection image of the panoramic video to the 3D spherical expression panorama of the panoramic video; divide the 3D spherical expression panorama of the panoramic video to The plurality of segmented regions are obtained.
  • the dividing module 1501 is specifically configured to divide the panoramic video three-dimensional spherical expression panorama according to a preset dividing manner to obtain the plurality of divided regions.
  • the division module 1501 is specifically configured to divide the panoramic video three-dimensional spherical expression panorama according to a division method determined based on preset rules to obtain the plurality of division regions.
  • the multiple segmented regions are obtained by dividing the three-dimensional spherical expression panorama of the panoramic video with a preset viewing angle interval rule; or,
  • the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from the latitude direction; and/or, the plurality of segmented areas are obtained by dividing the panoramic video three-dimensional spherical expression panorama from obtained by dividing in the longitude direction.
  • the division module 1501 is specifically configured to cluster a plurality of pixel points included in the panoramic video two-dimensional plane projection map to obtain a plurality of pixel point sets;
  • the pixels are respectively mapped to the three-dimensional spherical expression panorama of the panoramic video; the area formed by the corresponding mapping points of the pixels included in the first set of pixel points on the three-dimensional spherical expression panorama of the panoramic video is used as the first segmented area, so
  • the first set of pixel points is one of the plurality of sets of pixel points, and the first segmented area is one of the plurality of segmented areas.
  • the division module 1501 is specifically configured to map multiple pixels included in the panoramic video two-dimensional planar projection image to the panoramic video three-dimensional spherical expression panorama to obtain multiple Mapping points; clustering the multiple mapping points to obtain multiple mapping point sets; using the area formed by the mapping points included in the first mapping point set as a second segmented area, the first mapping point set is the One of multiple mapping point sets, the second segmented area is one of the multiple segmented areas.
  • the device of this embodiment can be used to implement the technical solution of the method embodiment shown in FIG. 12 , and its implementation principle and technical effect are similar, and will not be repeated here.
  • each step of the above-mentioned method embodiments may be completed by an integrated logic circuit of hardware in a processor or instructions in the form of software.
  • the processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other possible Program logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the steps of the methods disclosed in the embodiments of the present application may be directly implemented by a hardware coded processor, or executed by a combination of hardware and software modules in the coded processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register.
  • the storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.
  • the memories mentioned in the above embodiments may be volatile memories or nonvolatile memories, or may include both volatile and nonvolatile memories.
  • the non-volatile memory can be read-only memory (read-only memory, ROM), programmable read-only memory (programmable ROM, PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically programmable Erases programmable read-only memory (electrically EPROM, EEPROM) or flash memory.
  • Volatile memory can be random access memory (RAM), which acts as external cache memory.
  • RAM random access memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • DRAM synchronous dynamic random access memory
  • SDRAM double data rate synchronous dynamic random access memory
  • ESDRAM enhanced synchronous dynamic random access memory
  • SLDRAM direct memory bus random access memory
  • direct rambus RAM direct rambus RAM
  • the disclosed systems, devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (personal computer, server, or network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供一种全景图像的色调映射方法和装置。本申请全景图像的色调映射方法，包括：从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元，所述多个元数据信息单元是解析码流得到的，所述第一像素点是待处理的全景视频二维平面投影图中的任意一个像素点，所述多个元数据信息单元和全景视频三维球面表达全景图包括的多个分割区域对应，所述全景视频二维平面投影图和所述全景视频三维球面表达全景图具有映射关系；根据所述一个或多个目标元数据信息单元对所述第一像素点的像素值进行色调映射以得到所述第一像素点的目标色调映射值。本申请可以提高全景图像的处理效率。

Description

全景图像的色调映射方法和装置
本申请要求于2021年7月14日提交中国专利局、申请号为202110794014.5、申请名称为“全景图像的色调映射方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理领域，特别涉及一种全景图像的色调映射方法和装置。
背景技术
动态范围(dynamic range)在很多领域用来表示某个变量的最大值和最小值的比率。在数字图像中，动态范围用于表示在图像可显示的范围内最大灰度值和最小灰度值之间的比率。真实世界的同一场景下，其动态范围通常在10-3坎德拉/平方米(cd/m 2)到106cd/m 2之间，称作高动态范围(high dynamic range，HDR)。目前大部分的彩色数字图像中，0～255就是图像的动态范围，称作低动态范围(low dynamic range，LDR)。
由于显示设备的照度范围和真实世界的动态范围存在差异，因此需要将真实世界的动态范围映射至显示设备的照度范围，称作动态范围的映射。动态范围的映射可以应用在前端HDR信号和显示端HDR的显示设备的适配上，例如，前端采集到4000cd/m 2的光照信号，而显示端HDR的显示设备的HDR显示能力为500cd/m 2，把4000cd/m 2的光照信号映射到500cd/m 2的显示设备上，是一种从高到低的色调映射(tone mapping，TM)过程。动态范围的映射也可以应用在前端SDR信号和显示端HDR的显示设备的适配上，例如，前端采集到100cd/m 2的光照信号，而显示端HDR的显示设备的HDR显示能力为2000cd/m 2，把100cd/m 2的光照信号映射到2000cd/m 2的显示设备上，是一种从低到高的TM过程。
目前动态范围的映射方法可以分为静态和动态两种。静态的映射方法是根据同一个视频内容或者同一个硬盘内容，由单一的数据进行整体的TM过程，也就是针对各种场景，映射曲线通常是一样的。这种方法的优点是图像需要携带的数据较少，并且处理流程比较简单；缺点是所有场景都使用同一映射曲线进行TM，这样在有一些场景会导致信息丢失，例如如果映射曲线重点保护的是亮区，那么在一些极其暗的场景会丢掉一些细节，或者干脆看不见，这样会影响图像的显示效果。动态的映射方法是根据特定区域，针对每一个场景或者每一帧的内容进行映射曲线的动态调整，这种方法的优点是可以实现不同场景或者不同帧的差异化处理；缺点是需要在每一帧或者每一场景中携带相关的场景信息，图像需要携带的数据量较大。
但是，无论是上述哪种方法都无法兼容全景视频的动态范围的映射。
发明内容
本申请提供一种全景图像的色调映射方法和装置，以提高全景图像的处理效率。
第一方面，本申请提供一种全景图像的色调映射方法，包括：从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元，所述多个元数据信息单元是解析码 流得到的，所述第一像素点是待处理的全景视频二维平面投影图中的任意一个像素点，所述多个元数据信息单元和全景视频三维球面表达全景图包括的多个分割区域对应，所述全景视频二维平面投影图和所述全景视频三维球面表达全景图具有映射关系；根据所述一个或多个目标元数据信息单元对所述第一像素点的像素值进行色调映射以得到所述第一像素点的目标色调映射值。
本申请，通过将与全景图像对应的全景视频三维球面表达全景图划分成多个分割区域，根据每个分割区域包含的映射点所对应的像素点生成该分割区域的元数据信息单元，从而通过元数据信息单元指导显示端生成对应于该分割区域的色调映射曲线，进而实现从图像到显示的色调映射，既可以针对性的对全景图像上的不同动态范围区域确定匹配的色调映射参数，又可以符合显示端的显示动态范围，提高全景图像的处理效率。
在一种可能的实现方式中，所述从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元之前，所述方法还包括：获取所述多个分割区域。
在一种可能的实现方式中，所述获取所述多个分割区域，包括：根据预设的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述获取所述多个分割区域，包括：根据解析所述码流得到的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
在一种可能的实现方式中，所述对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域，包括：对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；将所述多个像素点分别映射至所述全景视频三维球面表达全景图；将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域，包括：将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；对所述多个映射点进行聚类以得到多个映射点集合；将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述获取所述多个分割区域，包括：根据解析所述码流得到的所述多个分割区域的指示信息获取所述多个分割区域。
本申请可以采用上述多个划分方法中的一个对全景视频三维球面表达全景图进行划分以得到多个分割区域，可以适应于全景视频二维平面投影图的像素分布，从而使得色调映射可以以分割区域为单位，在较小的动态范围内实现色调映射，提高色调映射效率。
在一种可能的实现方式中，所述从多个元数据信息单元中确定第一像素点的一个或多 个目标元数据信息单元，包括：确定所述多个元数据信息单元和多个分割区域之间的对应关系，一个所述元数据信息单元对应一个或多个所述分割区域；根据设定映射点确定一个或多个目标分割区域；当只有一个目标分割区域时，将所述一个目标分割区域对应的元数据信息单元确定为所述一个目标元数据信息单元；或者，当有多个目标分割区域时，将所述多个目标分割区域分别对应的元数据信息单元确定为所述多个目标元数据信息单元。
在一种可能的实现方式中，所述确定所述多个元数据信息单元和多个分割区域之间的对应关系，包括：按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；按照第二预设顺序从所述多个分割区域中提取当前分割区域；将所述当前分割区域和所述当前元数据信息单元建立对应关系。
在一种可能的实现方式中，所述确定所述多个元数据信息单元和多个分割区域之间的对应关系，包括：按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；按照解析所述码流得到的遍历顺序从所述多个分割区域中提取当前分割区域；将所述当前分割区域和所述当前元数据信息单元建立对应关系。
在一种可能的实现方式中，所述确定所述多个元数据信息单元和多个分割区域之间的对应关系，包括：按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；获取所述当前元数据信息单元包括的一个或多个坐标；根据所述一个或多个坐标在所述全景视频三维球面表达全景图上确定一个或多个映射点；当只有一个映射点时，将所述一个映射点所属的分割区域和所述当前元数据信息单元建立对应关系；当有多个映射点时，将所述多个映射点所属的至少一个分割区域和所述当前元数据信息单元建立对应关系。
本申请采用上述多种对应关系获取方法的其中一个确定多个分割区域和多个元数据信息单元之间的对应关系，不需要将该对应关系通过码流传输给显示端，可以减少码流中携带的信息。
在一种可能的实现方式中，所述设定映射点为所述第一像素点在所述全景视频三维球面表达全景图上的第一映射点；所述根据设定映射点确定一个或多个目标分割区域，包括：以所述第一映射点为中心在所述全景视频三维球面表达全景图上确定第一覆盖范围；当所述第一覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；或者，当所述第一覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
在一种可能的实现方式中，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述根据设定映射点确定一个或多个目标分割区域，包括：确定所述视点中心所属的第二分割区域，所述多个分割区域包括所述第二分割区域；将所述第二分割区域确定为所述一个目标分割区域。
在一种可能的实现方式中，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述根据设定映射点确定一个或多个目标分割区域，包括：以所述视觉中心为中心在所述全景视频三维球面表达全景图上确定第二覆盖范围；当所述第二覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；或者，当所述第二覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
在一种可能的实现方式中，所述根据所述一个或多个目标元数据信息单元对所述第一像素点的像素值进行色调映射以得到所述第一像素点的目标色调映射值，包括：根据所述 一个或多个目标元数据信息单元获取一条或多条色调映射曲线；当只有一条色条映射曲线时，根据所述一条色调映射曲线对所述第一像素点的像素值进行色调映射以得到所述目标色调映射值；或者，当有多条色条映射曲线时，根据所述多条色调映射曲线分别对所述第一像素点的像素值进行色调映射以得到所述第一像素点的多个色调中间值；根据所述多个色调中间值获取所述目标色调映射值。
在一种可能的实现方式中，所述根据所述多个色调中间值获取所述目标色调映射值，包括：将所述多个色调中间值中的中间值确定为所述目标色调映射值；或者，对所述多个色调中间值进行加权平均以得到所述目标色调映射值，所述多个色调中间值的加权系数是预先设定的，或者所述多个色调中间值的加权系数与所述第一像素点与各个分割区域的中心点的距离相关，或者，所述多个色调中间值的加权系数与所述多个目标元数据信息单元分别对应的分割区域的面积比重相关。
本申请采用上述多种方法的其中之一得到第一像素点的目标色调映射值，可以参考一个或多个分割区域的元数据信息单元生成第一像素点的色调映射曲线，提高色调映射的准确率。
第二方面，本申请提供一种全景图像的色调映射方法，包括：获取第一分割区域包括的至少一个映射点，所述第一分割区域是全景视频三维球面表达全景图包括的多个分割区域中的一个，所述全景视频三维球面表达全景图和待处理的全景视频二维平面投影图具有映射关系，所述至少一个映射点和所述全景视频二维平面投影图上的至少一个像素点对应；根据所述至少一个像素点生成所述第一分割区域的元数据信息单元；将所述第一分割区域的元数据信息单元写入码流。
本申请，通过将与全景图像对应的全景视频三维球面表达全景图划分成多个分割区域，根据每个分割区域包含的映射点所对应的像素点生成该分割区域的元数据信息单元，从而通过元数据信息单元指导显示端生成对应于该分割区域的色调映射曲线，进而实现从图像到显示的色调映射，既可以针对性的对全景图像上的不同动态范围区域确定匹配的色调映射参数，又可以符合显示端的显示动态范围，提高全景图像的处理效率。
在一种可能的实现方式中，所述根据所述至少一个像素点生成所述第一分割区域的元数据信息单元之后，所述方法还包括：当所述第一分割区域和第二分割区域的直方图和/或亮度满足设定条件时，对所述第一分割区域的元数据信息单元和所述第二分割区域的第二元数据信息单元进行融合以得到所述第一分割区域和所述第二分割区域的元数据信息单元，所述第二分割区域是所述多个分割区域中的一个。
本申请对于直方图和/或亮度差异较小的两个以上分割区域的元数据信息单元进行融合，从而减少码流中携带的元数据信息单元的数据量。
在一种可能的实现方式中，所述获取第一分割区域包括的至少一个映射点之前，所述方法还包括：将所述全景视频二维平面投影图映射至所述全景视频三维球面表达全景图；对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域，包括：根据预设的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述对所述全景视频三维球面表达全景图进行划分以得到 所述多个分割区域，包括：根据基于预设规则确定的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
在一种可能的实现方式中，所述对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域，包括：对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；将所述多个像素点分别映射至所述全景视频三维球面表达全景图；将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域，包括：将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；对所述多个映射点进行聚类以得到多个映射点集合；将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
本申请可以采用上述多个划分方法中的一个对全景视频三维球面表达全景图进行划分以得到多个分割区域，可以适应于全景视频二维平面投影图的像素分布，从而使得色调映射可以以分割区域为单位，在较小的动态范围内实现色调映射，提高色调映射效率。
第三方面，本申请提供一种显示装置包括：划分模块、确定模块和映射模块。其中，
确定模块，用于从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元，所述多个元数据信息单元是解析码流得到的，所述第一像素点是待处理的全景视频二维平面投影图中的任意一个像素点，所述多个元数据信息单元和全景视频三维球面表达全景图包括的多个分割区域对应，所述全景视频二维平面投影图和所述全景视频三维球面表达全景图具有映射关系；映射模块，用于根据所述一个或多个目标元数据信息单元对所述第一像素点的像素值进行色调映射以得到所述第一像素点的目标色调映射值。
在一种可能的实现方式中，划分模块，用于获取所述多个分割区域。
在一种可能的实现方式中，所述划分模块，具体用于根据预设的划分方式对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域。
在一种可能的实现方式中，所述划分模块，具体用于根据解析所述码流得到的划分方式对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域。
在一种可能的实现方式中，所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
在一种可能的实现方式中，所述划分模块，具体用于对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；将所述多个像素点分别映射至所述全景视频三维球面表达全景图；将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述划分模块，具体用于将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；对所述多个映射点进行聚类以得到多个映射点集合；将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述划分模块，具体用于根据解析所述码流得到的所述多个分割区域的指示信息获取所述多个分割区域。
在一种可能的实现方式中，所述确定模块，具体用于确定所述多个元数据信息单元和多个分割区域之间的对应关系，一个所述元数据信息单元对应一个或多个所述分割区域；根据设定映射点确定一个或多个目标分割区域；当只有一个目标分割区域时，将所述一个目标分割区域对应的元数据信息单元确定为所述一个目标元数据信息单元；或者，当有多个目标分割区域时，将所述多个目标分割区域分别对应的元数据信息单元确定为所述多个目标元数据信息单元。
在一种可能的实现方式中，所述确定模块，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；按照第二预设顺序从所述多个分割区域中提取当前分割区域；将所述当前分割区域和所述当前元数据信息单元建立对应关系。
在一种可能的实现方式中，所述确定模块，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；按照解析所述码流得到的遍历顺序从所述多个分割区域中提取当前分割区域；将所述当前分割区域和所述当前元数据信息单元建立对应关系。
在一种可能的实现方式中，所述确定模块，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；获取所述当前元数据信息单元包括的一个或多个坐标；根据所述一个或多个坐标在所述全景视频三维球面表达全景图上确定一个或多个映射点；当只有一个映射点时，将所述一个映射点所属的分割区域和所述当前元数据信息单元建立对应关系；当有多个映射点时，将所述多个映射点所属的至少一个分割区域和所述当前元数据信息单元建立对应关系。
在一种可能的实现方式中，所述设定映射点为所述第一像素点在所述全景视频三维球面表达全景图上的第一映射点；所述确定模块，具体用于以所述第一映射点为中心在所述全景视频三维球面表达全景图上确定第一覆盖范围；当所述第一覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；或者，当所述第一覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
在一种可能的实现方式中，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述确定模块，具体用于确定所述视点中心所属的第二分割区域，所述多个分割区域包括所述第二分割区域；将所述第二分割区域确定为所述一个目标分割区域。
在一种可能的实现方式中，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述确定模块，具体用于以所述视觉中心为中心在所述全景视频三维球面表达全景图上确定第二覆盖范围；当所述第二覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；或者，当所述第二覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
在一种可能的实现方式中，所述映射模块，具体用于根据所述一个或多个目标元数据信息单元获取一条或多条色调映射曲线；当只有一条色条映射曲线时，根据所述一条色调映射曲线对所述第一像素点的像素值进行色调映射以得到所述目标色调映射值；或者，当有多条色条映射曲线时，根据所述多条色调映射曲线分别对所述第一像素点的像素值进行色调映射以得到所述第一像素点的多个色调中间值；根据所述多个色调中间值获取所述目标色调映射值。
在一种可能的实现方式中，所述映射模块，具体用于将所述多个色调中间值中的中间值确定为所述目标色调映射值；或者，对所述多个色调中间值进行加权平均以得到所述目标色调映射值，所述多个色调中间值的加权系数是预先设定的，或者所述多个色调中间值的加权系数与所述第一像素点与各个分割区域的中心点的距离相关，或者，所述多个色调中间值的加权系数与所述多个目标元数据信息单元分别对应的分割区域的面积比重相关。
第四方面，本申请提供一种前端装置包括：划分模块、获取模块和生成模块。其中，
获取模块，用于获取第一分割区域包括的至少一个映射点，所述第一分割区域是全景视频三维球面表达全景图包括的多个分割区域中的一个，所述全景视频三维球面表达全景图和待处理的全景视频二维平面投影图具有映射关系，所述至少一个映射点和所述全景视频二维平面投影图上的至少一个像素点对应；生成模块，用于根据所述至少一个像素点生成所述第一分割区域的元数据信息单元；将所述第一分割区域的元数据信息单元写入码流。
在一种可能的实现方式中，所述生成模块，还用于当所述第一分割区域和第二分割区域的直方图和/或亮度满足设定条件时，对所述第一分割区域的元数据信息单元和所述第二分割区域的第二元数据信息单元进行融合以得到所述第一分割区域和所述第二分割区域的元数据信息单元，所述第二分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，划分模块，用于将所述全景视频二维平面投影图映射至所述全景视频三维球面表达全景图；对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述划分模块，具体用于根据预设的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述划分模块，具体用于根据基于预设规则确定的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
在一种可能的实现方式中，所述划分模块，具体用于对所述全景视频二维平面投影图 包括的多个像素点进行聚类以得到多个像素点集合；将所述多个像素点分别映射至所述全景视频三维球面表达全景图；将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述划分模块，具体用于将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；对所述多个映射点进行聚类以得到多个映射点集合；将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
第五方面，本申请提供一种终端设备，包括：一个或多个处理器；存储器，用于存储一个或多个程序；当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如上述第一至二方面中任一项所述的方法。
第六方面，本申请提供一种计算机可读存储介质，包括计算机程序，所述计算机程序在计算机上被执行时，使得所述计算机执行上述第一至二方面中任一项所述的方法。
第七方面，本申请提供一种计算机程序产品，所述计算机程序产品包括计算机程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述第一至二方面中任一项所述的方法。
附图说明
图1为真实世界的成像过程中动态范围映射的一个示例图；
图2为PQ光电转移函数的一个曲线示意图；
图3为HLG光电转移函数的一个曲线示意图；
图4为SLF光电转移函数的一个曲线示意图；
图5为sigmoidal曲线的一个示意图；
图6为贝泽尔曲线的一个示意图；
图7为S型曲线的一个示意图；
图8为本申请的全景视频播放系统的一个示例性的框图；
图9为终端设备的一个示例性的结构示意图；
图10a为经纬图映射示例图；
图10b为球面映射为正多面体以及对应的展开图像示意图；
图10c和图10d为球面和六面体之间的映射关系示意图；
图10e为球面和六面体、八面体之间的映射关系示意图；
图11a～图11d为本申请对全景视频三维球面表达全景图的几个分割区域的示例性的示意图；
图12为本申请全景图像的色调映射方法的过程1200的流程图；
图13为本申请全景图像的色调映射方法的过程1300的流程图；
图14为本申请显示装置1400的一个示例性的结构图；
图15为本申请前端装置1500的一个示例性的结构图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请中的附图，对本申请中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书实施例和权利要求书及附图中的术语“第一”、“第二”等仅用于区分描述的目的，而不能理解为指示或暗示相对重要性，也不能理解为指示或暗示顺序。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元。方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
应当理解，在本申请中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。
以下是本申请实施例涉及的一些相关技术：
一、动态范围(dynamic range)
动态范围在很多领域用来表示某个变量的最大值和最小值的比率。在数字图像中，动态范围用于表示在图像可显示的范围内最大灰度值和最小灰度值之间的比率。
在自然界中，星空下的夜景亮度约为0.001cd/m 2，太阳本身亮度高达10 9cd/m 2，动态范围可达到10 9/0.001＝10 12cd/m 2量级。但是在自然界的真实世界中，太阳的亮度和星光的亮度不会同时得到，因此对真实世界的同一场景来说，其动态范围通常在10 - 3cd/m 2到10 6cd/m 2之间，称作高动态范围(high dynamic range，HDR)。目前大部分的彩色数字图像中，红色(R)、绿色(G)和蓝色(B)三个通道的灰度分别使用一个字节来存储，亦即R、G和B三个通道的灰度范围分别是0～255，该0～255就是图像的动态范围，称作低动态范围(low dynamic range，LDR)。
二、光电转移函数(optical-electro transfer function，OETF)
数码相机的成像过程实际上就是真实世界的高动态范围到数字图像的低动态范围的映射。图1为真实世界的成像过程中动态范围映射的一个示例图，如图1所示，真实世界中除星光的亮度和太阳的亮度外，还包括月光的亮度为1cd/m 2，室内照明的亮度为100cd/m 2，室外多云天气时的亮度为500cd/m 2，室外晴天时的亮度为2000cd/m 2。真实世界中亮度范围100cd/m 2到2000cd/m 2和显示设备的存储的方式所对应的亮度范围1cd/m 2到200cd/m 2成映射关系。
由于显示设备的存储方式并不能达到真实世界的高亮度，因此需要光电转移函数(electro-optical transfer function，EOTF)，将真实世界中的亮度表示为显示设备的存储方式所对应的亮度。例如，真实世界中的亮度为10000cd/m 2，显示设备用10比特存储亮度信息，那么该存储方式下可存储的最大值为1023，因此可以将10000cd/m 2表示为1023。
早期的显示设备是阴极射线显像管(cathode ray tube，CRT)显示器，其光电转移函数就是伽马(Gamma)函数。在ITU-R Recommendation BT.1886标准当中定义了该Gamma函数：
Figure PCTCN2022100495-appb-000001
随着显示设备的升级，显示设备的照度范围不断增加，现有的HDR显示器的照度达到600cd/m 2，高端的HDR显示器的照度能达到2000cd/m 2。因此需要改进的光电转移函数以适应显示设备的升级，现阶段常见的光电转移函数有感知量化(perception quantization，PQ)光电转移函数、混合对数伽马(hybrid log-gamma，HLG)光电转移函数和场景亮度保真(scene luminance fidelity，SLF)光电转移函数三种。
(1)PQ光电转移函数：不同于传统的Gamma函数，根据人眼的在不同亮度下的对比度感知模型，提出了PQ光电转移函数。PQ光电转移函数表示图像帧像素的线性信号值到PQ域非线性信号值的转换关系，图2为PQ光电转移函数的一个曲线示意图。PQ光电转移函数可以表示为：
Figure PCTCN2022100495-appb-000002
上述公式中对应于R、G和B的参数可以通过公式
Figure PCTCN2022100495-appb-000003
计算，其中，L表示图像帧像素的线性信号值，其值归一化为[0,1]；L’表示PQ域非线性信号值，其取值范围为[0,1]；m 1、m 2、c 1、c 2和c 3均为PQ光电转移系数，
Figure PCTCN2022100495-appb-000004
Figure PCTCN2022100495-appb-000005
Figure PCTCN2022100495-appb-000006
Figure PCTCN2022100495-appb-000007
Figure PCTCN2022100495-appb-000008
(2)HLG光电转移函数：是在传统的Gamma曲线的基础上改进得到的，HLG光电转移函数在低段应用传统的Gamma曲线，在高段补充了log曲线。HLG光电转移函数表示图像帧像素的线性信号值到HLG域非线性信号值的转换关系，图3为HLG光电转移函数的一个曲线示意图。HLG光电转移函数可以表示为：
Figure PCTCN2022100495-appb-000009
其中，L表示图像帧像素的线性信号值，其取值范围为[0,12]；L’表示HLG域非线性信号值，其取值范围为[0,1]；a、b和c均为HLG光电转移系数，a＝0.17883277，b＝0.28466892，c＝0.55991073。
(3)SLF光电转移函数：在满足人眼光学特性的前提下，根据HDR场景亮度分布得 到。SLF光电转移函数表示图像帧像素的线性信号值到SLF域非线性信号值的转换关系，图4为SLF光电转移函数的一个曲线示意图。SLF光电转移函数可以表示为：
Figure PCTCN2022100495-appb-000010
上述公式中对应于R、G和B的参数可以通过公式
Figure PCTCN2022100495-appb-000011
计算，其中，L表示图像帧像素的线性信号值，其值归一化为[0,1]；L’表示PQ域非线性信号值，其取值范围为[0,1]；p、m、a和b均为SLF光电转移系数，p＝2.3，m＝0.14，a＝1.12762，b＝-0.12762。
三、动态范围的映射
动态范围的映射可以应用在前端HDR信号和显示端HDR的显示设备的适配上，例如，前端采集到4000cd/m 2的光照信号，而显示端HDR的显示设备的HDR显示能力为500cd/m 2，把4000cd/m 2的光照信号映射到500cd/m 2的显示设备上，是一种从高到低的色调映射(tone mapping，TM)过程。动态范围的映射也可以应用在前端SDR信号和显示端HDR的显示设备的适配上，例如，前端采集到100cd/m 2的光照信号，而显示端HDR的显示设备的HDR显示能力为2000cd/m 2，把100cd/m 2的光照信号映射到2000cd/m 2的显示设备上，是一种从低到高的TM过程。
目前动态范围的映射方法可以分为静态和动态两种。静态的映射方法是根据同一个视频内容或者同一个硬盘内容，由单一的数据进行整体的TM过程，也就是针对各种场景，映射曲线通常是一样的。这种方法的优点是视频需要携带的数据较少，并且处理流程比较简单；缺点是所有场景都使用同一映射曲线进行TM，这样在有一些场景会导致信息丢失，例如如果映射曲线重点保护的是亮区，那么在一些极其暗的场景会丢掉一些细节，或者完全看不见，这样会影响视频的显示效果。动态的映射方法是根据特定区域，针对每一个场景或者每一帧的内容进行映射曲线的动态调整，这种方法的优点是可以实现不同场景或者不同帧的差异化处理；缺点是需要在每一帧或者每一场景中携带相关的场景信息，视频需要携带的数据量较大。
四、TM技术
(1)基于sigmoidal曲线的TM过程
图5为sigmoidal曲线的一个示意图。
(2)基于贝泽尔曲线的TM过程
图6为贝泽尔曲线的一个示意图。
(3)基于人眼感知的S型曲线的TM过程
图7为S型曲线的一个示意图，S曲线对应的光电转移函数可以表示为：
Figure PCTCN2022100495-appb-000012
其中，L和L’分别为归一化的电信号或者光信号，其取值范围均为[0,1]；a的取值范围为[0,1]；b的取值范围为[0,1]；p、n和m的取值范围均为[0,N]，N为大于0.1的有理数；k 1、k 2、k 3均为有理数。
五、全景视频
全景视频能提供沉浸式的视觉体验。主要在于：(1)提供了描述水平360度、竖直180度的球面场景内容；(2)通过VR头戴式设备能够自由的选择观看指定方向的视窗。对比普通二维视频水平33度、竖直19度的狭小视角范围，全景视频覆盖的范围更大。
相应的，全景视频引入了设备显示更严重的动态范围不足的问题。因为二维视频的视角范围较小，光照情况比较相似，其动态范围还在可控范围内；而全景视频的视角范围较大，例如，视频内容是白天的场景，很可能既包含天空中的太阳这种非常亮的区域，又包含建筑内部或阴影等比较暗的区域，其动态范围会很大，给描述和显示带来了极大的挑战。
六、元数据信息
前端(视频采集和/或制作)会在发送给显示端(视频显示)的元数据信息中携带与映射曲线相关的参数。
(1)sigmoidal曲线
在St2094-10提出的sigmoidal曲线相关的元数据信息定义中不单包含了视频亮度的最大值(Maximum PQ-encoded maxRGB)、最小值(Minimum PQ-encoded maxRGB)、平均值(Average PQ-encoded maxRGB)等统计值，还包含了色调映射偏移(tone mapping offset)、色调映射增益(tone mapping gain)和色调映射伽马(tone mapping gamma)等sigmoid曲线相关的参数，用于直接生成sigmoidal曲线。
但是上述参数的生成方式比较固定，元数据信息包含的参数无法给曲线生成提供更多的灵活性。
(2)贝泽尔曲线
在St2094-40提出的基于贝泽尔曲线相关的元数据信息定义中包含了直方图信息(Distribution MaxRGB)，还包含了贝泽尔曲线相关的参数(bezier curve anchors)，用于直接生成贝泽尔曲线。
另外，ST2094系列的标准中都包含了目标系统显示最大(峰值)亮度(targeted_system_display_maximum_luminance，TSDAPL)。
但是上述参数在前端生成时对应于TSDAPL，而在显示端不同的显示设备使用相同的曲线，无法达到最好的显示效果。
(3)S型曲线
元数据信息中可以包含视频亮度的最大值、最小值、平均值等统计值，也可以包含p、m、a、b、n、K1、K2、K3等S型曲线相关的参数。
图8为本申请的全景视频播放系统的一个示例性的框图。如图8所示，全景视频播放系统分为前端和显示端，其中，前端包括采集和/或制作全景视频的模块，显示端包括HDR显示模块和/或SDR显示模块。前端将采集到的全景视频数据经预处理后传输给显示端，同时还会携带全景视频数据的元数据信息。显示端根据全景视频数据，结合对应的元数据信息对全景视频包含的多个图像帧逐帧进行增强处理，得到颜色、亮度及对比度等俱佳的图像，并显示出来。
本申请中前端和显示端可以是各自独立的、不同的物理设备，例如，前端可以是摄像机、相机、图像绘制机等具有视频采集功能的设备，显示端可以是虚拟现实(virtual reality，VR)眼镜、手机、平板等具有全景视频播放功能的设备。此时前端和显示端之间可以建立 无线连接或有线连接，其中无线连接可以采用包括例如长期演进(long term evolution，LTE)、第五代(5th generation，5G)移动通信以及未来的移动通信等技术。无线连接还可以包括无线保真(wireless-fidelity，WiFi)、蓝牙、近场通信(Near Field Communication，NFC)等技术。有线连接可以包括以太网连接、局域网连接等。对此不作具体限定。
本申请还可以将前端的功能和显示端的功能集成在同一个物理设备上，例如，具有视频拍摄功能的手机、平板等终端设备。本申请还可以将前端的部分功能和显示端的部分功能集成在同一个物理设备上。对此不作具体限定。
图9为终端设备的一个示例性的结构示意图。如图9所示，终端设备900包括：应用处理器901、微控制器单元(microcontroller unit，MCU)902、存储器903、调制解调器(modem)904、射频(radio frequency，RF)模块905、无线保真(Wireless-Fidelity，简称Wi-Fi)模块906、蓝牙模块907、传感器908、输入/输出(input/output，I/O)设备909、定位模块910等部件。这些部件可通过一根或多根通信总线或信号线进行通信。前述通信总线或信号线可以是本申请提供的CAN总线。本领域技术人员可以理解，终端设备900可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
下面结合图9对终端设备900的各个部件进行具体的介绍：
应用处理器901是终端设备900的控制中心，利用各种接口和总线连接终端设备900的各个部件。在一些实施例中，处理器901可包括一个或多个处理单元。
存储器903中存储有计算机程序，诸如图9所示的操作系统911和应用程序912。应用处理器901被配置用于执行存储器903中的计算机程序，从而实现该计算机程序定义的功能，例如应用处理器901执行操作系统911从而在终端设备900上实现操作系统的各种功能。存储器903还存储有除计算机程序之外的其他数据，诸如操作系统911和应用程序912运行过程中产生的数据。存储器903为非易失性存储介质，一般包括内存和外存。内存包括但不限于随机存取存储器(random access memory，RAM)，只读存储器(read-only memory，ROM)，或高速缓存(cache)等。外存包括但不限于闪存(flash memory)、硬盘、光盘、通用串行总线(universal serial bus，USB)盘等。计算机程序通常被存储在外存上，处理器在执行计算机程序前会将该程序从外存加载到内存。
存储器903可以是独立的，通过总线与应用处理器901相连接；存储器903也可以和应用处理器901集成到一个芯片子系统。
MCU 902是用于获取并处理来自传感器908的数据的协处理器，MCU 902的处理能力和功耗小于应用处理器901，但具有“永久开启(always on)”的特点，可以在应用处理器901处于休眠模式时持续收集以及处理传感器数据，以极低的功耗保障传感器的正常运行。在一个实施例中，MCU 902可以为sensor hub芯片。传感器908可以包括光传感器、运动传感器。具体地，光传感器可包括环境光传感器及接近传感器，其中，环境光传感器可根据环境光线的明暗来调节显示器9091的亮度，接近传感器可在终端设备900移动到耳边时，关闭显示屏的电源。作为运动传感器的一种，加速计传感器可检测各个方向上(一般为三轴)加速度的大小，静止时可检测出重力的大小及方向；传感器908还可以包括陀螺仪、气压计、湿度计、温度计、红外线传感器等其它传感器，在此不再赘述。MCU 902和传感器908可以集成到同一块芯片上，也可以是分离的元件，通过总线连接。
modem 904以及射频模块905构成了终端设备900通信子系统，用于实现无线通信标 准协议的主要功能。其中，modem 904用于编解码、信号的调制解调、均衡等。射频模块905用于无线信号的接收和发送，射频模块905包括但不限于天线、至少一个放大器、耦合器、双工器等。射频模块905配合modem 904实现无线通信功能。modem 904可以作为单独的芯片，也可以与其他芯片或电路在一起形成系统级芯片或集成电路。这些芯片或集成电路可应用于所有实现无线通信功能的终端设备，包括：手机、电脑、笔记本、平板、路由器、可穿戴设备、汽车、家电设备等。
终端设备900还可以使用Wi-Fi模块906，蓝牙模块907等来进行无线通信。Wi-Fi模块906用于为终端设备900提供遵循Wi-Fi相关标准协议的网络接入，终端设备900可以通过Wi-Fi模块906接入到Wi-Fi接入点，进而访问互联网。在其他一些实施例中，Wi-Fi模块906也可以作为Wi-Fi无线接入点，可以为其他终端设备提供Wi-Fi网络接入。蓝牙模块907用于实现终端设备900与其他终端设备(例如手机、智能手表等)之间的短距离通信。本申请实施例中的Wi-Fi模块906可以是集成电路或Wi-Fi芯片等，蓝牙模块907可以是集成电路或者蓝牙芯片等。
定位模块910用于确定终端设备900的地理位置。可以理解的是，定位模块910具体可以是全球定位系统(global position system，GPS)或北斗卫星导航系统、俄罗斯GLONASS等定位系统的接收器。
Wi-Fi模块906，蓝牙模块907和定位模块910分别可以是单独的芯片或集成电路，也可以集成到一起。例如，在一个实施例中，Wi-Fi模块906，蓝牙模块907和定位模块910可以集成到同一芯片上。在另一个实施例中，Wi-Fi模块906，蓝牙模块907、定位模块910以及MCU 902也可以集成到同一芯片中。
输入/输出设备909包括但不限于：显示器9091、触摸屏9092，以及音频电路9093等等。
其中，触摸屏9092可采集终端设备900的用户在其上或附近的触摸事件(比如用户使用手指、触控笔等任何适合的物体在触摸屏9092上或在触控屏触摸屏9092附近的操作)，并将采集到的触摸事件发送给其他器件(例如应用处理器901)。其中，用户在触摸屏9092附近的操作可以称之为悬浮触控；通过悬浮触控，用户可以在不直接接触触摸屏9092的情况下选择、移动或拖动目的(例如图标等)。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型来实现触摸屏9092。
显示器(也称为显示屏)9091用于显示用户输入的信息或展示给用户的信息。可以采用液晶显示屏、有机发光二极管等形式来配置显示器。触摸屏9092可以覆盖在显示器9091之上，当触摸屏9092检测到触摸事件后，传送给应用处理器901以确定触摸事件的类型，随后应用处理器901可以根据触摸事件的类型在显示器9091上提供相应的视觉输出。虽然在图9中，触摸屏9092与显示器9091是作为两个独立的部件来实现终端设备900的输入和输出功能，但是在某些实施例中，可以将触摸屏9092与显示器9091集成而实现终端设备900的输入和输出功能。另外，触摸屏9092和显示器9091可以以全面板的形式配置在终端设备900的正面，以实现无边框的结构。
音频电路9093、扬声器9094、麦克风9095可提供用户与终端设备900之间的音频接口。音频电路9093可将接收到的音频数据转换后的电信号，传输到扬声器9094，由扬声器9094转换为声音信号输出；另一方面，麦克风9095将收集的声音信号转换为电信号， 由音频电路9093接收后转换为音频数据，再通过modem 904和射频模块905将音频数据发送给比如另一终端设备，或者将音频数据输出至存储器903以便进一步处理。
另外，终端设备900还可以具有指纹识别功能。例如，可以在终端设备900的背面(例如后置摄像头的下方)配置指纹采集器件，或者在终端设备900的正面(例如触摸屏9092的下方)配置指纹采集器件。又例如，可以在触摸屏9092中配置指纹采集器件来实现指纹识别功能，即指纹采集器件可以与触摸屏9092集成在一起来实现终端设备900的指纹识别功能。在这种情况下，该指纹采集器件配置在触摸屏9092中，可以是触摸屏9092的一部分，也可以以其他方式配置在触摸屏9092中。本申请实施例中的指纹采集器件的主要部件是指纹传感器，该指纹传感器可以采用任何类型的感测技术，包括但不限于光学式、电容式、压电式或超声波传感技术等。
进一步地，终端设备900搭载的操作系统911可以为
Figure PCTCN2022100495-appb-000013
或者其它操作系统，本申请实施例对此不作任何限制。
以搭载
Figure PCTCN2022100495-appb-000014
操作系统的终端设备900为例，终端设备900从逻辑上可划分为硬件层、操作系统911，以及应用层。硬件层包括如上所述的应用处理器901、MCU 902、存储器903、modem 904、Wi-Fi模块906、传感器908、定位模块910等硬件资源。应用层包括一个或多个应用程序，比如应用程序912，应用程序912可以为社交类应用、电子商务类应用、浏览器等任意类型的应用程序。操作系统911作为硬件层和应用层之间的软件中间件，是管理和控制硬件与软件资源的计算机程序。
在一个实施例中，操作系统911包括内核，硬件抽象层(hardware abstraction layer，HAL)、库和运行时(libraries and runtime)以及框架(framework)。其中，内核用于提供底层系统组件和服务，例如：电源管理、内存管理、线程管理、硬件驱动程序等；硬件驱动程序包括Wi-Fi驱动、传感器驱动、定位模块驱动等。硬件抽象层是对内核驱动程序的封装，向框架提供接口，屏蔽低层的实现细节。硬件抽象层运行在用户空间，而内核驱动程序运行在内核空间。
库和运行时也叫做运行时库，它为可执行程序在运行时提供所需要的库文件和执行环境。在一个实施例中，库与运行时包括安卓运行时(android runtime，ART)，库，以及场景包运行时。ART是能够把应用程序的字节码转换为机器码的虚拟机或虚拟机实例。库是为可执行程序在运行时提供支持的程序库，包括浏览器引擎(比如webkit)、脚本执行引擎(比如JavaScript引擎)、图形处理引擎等。场景包运行时是场景包的运行环境，主要包括页面执行环境(page context)和脚本执行环境(script context)，其中，页面执行环境通过调用相应的库解析html、css等格式的页面代码，脚本执行环境通过调用相应的功能库解析执行JavaScript等脚本语言实现的代码或可执行文件。
框架用于为应用层中的应用程序提供各种基础的公共组件和服务，比如窗口管理、位置管理等等。在一个实施例中，框架包括地理围栏服务，策略服务，通知管理器等。
以上描述的操作系统911的各个组件的功能均可以由应用处理器901执行存储器903中存储的程序来实现。
所属领域的技术人员可以理解终端设备900可包括比图9所示的更少或更多的部件，图9所示的该终端设备仅包括与本申请所公开的多个实现方式更加相关的部件。
如上所述，全景视频可以提供水平360度、竖直180度的球面场景内容，用户观看该 全景视频，将用户的头部作为中心点，该用户的头部可以水平360度转动、上下180度俯仰，因此用户的视区范围可以达到水平360度、竖直180度，可见，如果将全景视频模拟成一个球体的球面，用户的头部只要处于球体的球心位置，即可通过头部转动和/或俯仰观看到完整的全景视频。
因此可以根据全景视频包含的多帧三维(3D)全景图像虚拟用户的全景视频三维球面表达全景图，全景视频三维球面表达全景图上的任意一个点可以采用球面坐标表示，而3D全景图像上的任意一个像素点可以在全景视频三维球面表达全景图上找到一个对应的点，该点可以称作前述任意一个像素点的映射点。
由于球面格式的三维(3D)全景图像无法方便的表示、存储和索引，因此相关技术通常是将3D全景图像展开成平面，得到二维(2D)全景图像(本申请中，2D全景图像可以称作全景视频二维平面投影图)，再对该2D全景图像进行压缩、处理、存储、传输等。将3D全景图像展开得到2D全景图像的操作称为映射。目前存在多种映射方法，相应的，存在多种2D全景图像格式。
1、最常见的2D全景图像格式称为经纬图，如图10a所示，内容为地球的3D全景图像展开成2D全景图像，邻近南北两极区域图像经过拉伸获得，存在严重的失真和数据冗余。
2、为了减轻经纬图中存在的强失真，可通过将3D全景图像投影到一个正多面体表面上，从而将球面表示为由若干个等大小的三角形面、正方形面或者五边形面等构成的多面体。例如，正四面体(如图10b中的(a)所示)，正六面体(如图10b中的(b)所示)，正八面体(如图10b中的(c)所示)，正十二面体(如图10b中的(d)所示)，正二十面体(如图10b中的(e)所示)等。需要说明的是，本申请不限定多面体为正多面体，球面也可以表示为由若干个非等大小的多边形构成的多面体。
再将多面体展开为2D全景图像，图10b中的(a)展开成图10b中的(f)所示的2D全景图像，图10b中的(b)展开成图10b中的(g)所示的2D全景图像，图10b中的(c)展开成图10b中的(h)所示的2D全景图像，图10b中的(d)展开成图10b中的(i)所示的2D全景图像，图10b中的(e)展开成图10b中的(j)所示的2D全景图像。这些2D全景图像可统称为多面体格式的全景图像。
具体的投影过程为，将球体放置到多面体中，使其成为多面体的内切球。将球心(或多面体体心)与球面上某点连线并延长相交于多面体表面，球面上此点位置的像素值为多面体表面上交点位置处的像素值。结合插值方法，便可以得到多面体上所有像素点位置的像素值。另外，为了得到多面体上某像素点位置的像素值，也可以直接将此像素点与球心连线，交于球面上的某点，将球面上交点位置的像素值作为此点位置的像素值。
以六面体投影过程为例，将球面内切于如图10c所示的六面体ABCDEFGH中，为了得到多面体上M’点位置处的像素值，将球心O与M’连线，并交于球面M点，则M点处的像素值即为M’点处的像素值。按照此方法可以得到平面A’B’C’D’上ABCD面内所有像素点位置的像素值，ABCD面内像素点构成ABCD面(face)的面图像，平面A’B’C’D’称为ABCD面的投影平面(projection plane)。同理可得六面体其他面的面图像以及对应投影平面。需要说明的是，本申请所说的面图像为一些空间相邻像素点组成的N×N大小的阵列，这些像素点在球面映射为多面体表面的过程中，处于同一个投影平面，且这些像素点 位置的像素值已知。
将多面体表面展开为2D图像之后，多面体表面上的每个面图像成为2D全景图像中一个区域的图像或者2D全景图像的一个子图像，用多面体表面的面图像指代这个区域的图像或者这个子图像，用多面体表面的这个面指代2D全景图像中的这个区域或者这个子图像的覆盖区域。例如，图10d中的(a)的六面体表面展开成图10d中的(b)的2D全景图像，六面体表面上的Top面的面图像将会成为图10d中的(b)中左上角的子图像，我们称这个子图像为2D全景图像的Top面的面图像，2D全景图像中的Top面指代Top面的面图像覆盖的区域。对于Top面中的任意一个像素点，Top面称为此像素点的所在面。
在压缩等图像处理中，可直接处理图10b中的(f)～(j)所示的不规则2D图像，也可选择将其包围在内的最小面积矩形区域作为处理对象，该矩形区域内不规则2D图像以外的其余部分可填充为默认内容，例如全灰，全黑或全白等。例如，图10e中的(b)是图10e中的(a)展开得到，图10e中的(d)是图10e中的(c)展开得到，并且其余部分中像素点的值设置为灰色对应的像素值之后的二维矩形图像。除此之外，还可以直接将这些面拼接成一个规则的矩形，进而避免填充操作。在进行实际编码处理时，处理的图像一般为矩形图像。
使用上述方法或其它未在本申请中列举的方法，可以将3D全景图像投影为全景视频二维平面投影图，再将全景视频二维平面投影图映射至全景视频三维球面表达全景图，即将全景视频二维平面投影图包括的多个像素点分别映射至全景视频三维球面表达全景图得到多个映射点，将对全景视频二维平面投影图的处理转换成对全景视频三维球面表达全景图的处理。由此可见，无论是采用哪种格式表示的全景视频二维平面投影图，均可以映射至全景视频三维球面表达全景图，这样对全景图像的色调映射不必再受限于2D全景视频的格式。
本申请中，可以对全景视频三维球面表达全景图进行划分以得到多个分割区域。多个分割区域可以采用以下任意一种方式得到：
1、多个分割区域是通过对全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的。
本申请中，可以将全景视频三维球面表达全景图划分为预设视角间隔的多个分割区域，例如，如图11a所示，划分的到的多个分割区域中，纬度相邻的两个分割区域对应的水平夹角相等，经度相邻的两个分割区域对应的俯仰夹角相等。
2、多个分割区域是通过对全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，多个分割区域是通过对全景视频三维球面表达全景图从经度方向上进行划分得到的。
本申请中，可以将全景视频三维球面表达全景图从纬度方向上划分为多个分割区域，例如，如图11b所示，将全景视频三维球面表达全景图划分为上下等大的两个分割区域；或者，将全景视频三维球面表达全景图从经度方向上均匀划分为多个分割区域，例如，如图11c所示，将全景视频三维球面表达全景图划分为左右等大的两个分割区域；或者，可以将全景视频三维球面表达全景图从纬度方向上划分为多个分割区域，并将全景视频三维球面表达全景图从经度方向上均匀划分为多个分割区域，例如，如图11d所示，将全景视频三维球面表达全景图划分为上下等大且左右等大的四个分割区域。
应当理解的，上述示例的几种划分方式并不对划分方式构成限定，还可以将全景视频三维球面表达全景图划分为上下等大的两部分，再将该两部分分别均匀四等分；还可以将全景视频三维球面表达全景图划分为上下等大的两部分，再将该两部分分别均匀四等分，最后将每个四分区域划分为等视角间隔的区域。另外，对全景视频三维球面表达全景图的划分也可以不是等分的，或者不是全部的划分步骤都是等分的，本申请对此不做具体限定。
3、对全景图像包括的多个像素点进行聚类以得到多个像素点集合，将多个像素点分别映射至全景视频三维球面表达全景图。将第一像素点集合包括的像素点在全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，该第一像素点集合是多个像素点集合中的一个，该第一分割区域是多个分割区域中的一个。
4、将全景图像包括的多个像素点分别映射至全景视频三维球面表达全景图以得到多个映射点，对多个映射点进行聚类以得到多个映射点集合。将第一映射点集合包括的映射点形成的区域作为第二分割区域，该第一映射点集合是多个映射点集合中的一个，该第二分割区域是多个分割区域中的一个。
上述聚类的方法包括：
a.按照预设的初始划分方式将全景视频三维球面表达全景图划分为等视角间隔的多个分割区域。应当理解的，初始划分方式还可以包括上述1或2中的任意一种划分方式，对此不做具体限定；
b.确定多个分割区域中分别包括的映射点。上述映射过程得到了全景视频三维球面表达全景图上的多个映射点，这些映射点可以根据其球面坐标，确定是属于哪一个分割区域。多个分割区域中可能每个分割区域都包含了映射点，也可能部分分割区域没有包含映射点；
c.确定第i个分割区域中位于的中心位置的映射点的颜色分量值或者第i个分割区域的平均颜色分量值。颜色分量值可以指颜色格式YUV中的分量Y、U、V中的至少一个，颜色分量值也可以指颜色格式RBG中的分量R、G、B和max(R,G,B)中的至少一个，颜色分量值还可以指颜色格式Lab中的L、a和b中的至少一个。第i个分割区域的平均颜色分量值可以指第i个分割区域中包括的多个映射点的颜色分量值的平均值。第i个分割区域是多个分割区域的其中之一，此处以第i个分割区域的处理过程为例说明每个分割区域的处理过程。应当理解的，本申请还可以获取第i个分割区域的其它特征信息，例如，第i个分割区域的指定映射点的颜色分量值，对此不做具体限定。需要说明的是，如果第i个分割区域没有位于的中心位置的映射点，可以计算第i个分割区域内所有映射点的x坐标的平均值avgx，y坐标的平均值avgy，用坐标(avgx,avgy)替代上述位于中心位置的映射点；
d1.计算第i个分割区域中其它映射点(下文以第j个映射点为例说明)分别与第i个分割区域中位于的中心位置的映射点的聚类失真值：
Figure PCTCN2022100495-appb-000015
其中，D j表示第j个映射点的聚类失真值，Q c表示第i个分割区域中位于的中心位置的映射点的颜色分量值，Q j表示第j个映射点的颜色分量值，(x c,y c,z c)表示第i个分割区域中位于的中心位置的映射点的坐标，(x j,y j,z j)表示第j个映射点的坐标。归一化(Q c-Q j)可 以使用Q c和Q j的最大取值范围，将其归一化到0～1.0之间。归一化
Figure PCTCN2022100495-appb-000016
可以使用分割区域的平均面积(即球面面积除以分割区域的个数)，假设分割区域为圆形或者正方形，可以估计得到的分割区域的半径(对应于圆形分割区域)或者边长(对应于正方形分割区域)，用计算得到的欧氏距离除以半径或者正方形边长，大于1的置为0，即可归一化为0～1.0之间；
d2.计算第i个分割区域中其它映射点(下文以第j个映射点为例说明)分别与第i个分割区域中位于的中心位置的映射点的聚类失真值：
Figure PCTCN2022100495-appb-000017
其中，D j表示第j个映射点的聚类失真值，Qa c表示第i个分割区域的平均颜色分量值，Q j表示第j个映射点的颜色分量值，(x c,y c,z c)表示第i个分割区域中位于的中心位置的映射点的坐标，(x j,y j,z j)表示第j个映射点的坐标。
上述d1和d2是二选一的步骤。
e.根据第i个分割区域包含的多个映射点的聚类失真值确定该多个映射点归属的分割区域。例如，若第j个映射点的聚类失真值D j小于或等于预设阈值，则确定第j个映射点属于第i个分割区域；若第j个映射点的聚类失真值D j大于预设阈值，则确定第j个映射点不属于第i个分割区域。进而采用上述d1和d2的方法再次计算第j个映射点的聚类失真值，区别在于，将第i个分割区域中位于的中心位置的映射点换成第i个分割区域的邻居分割区域的位于的中心位置的映射点，然后根据再次得到的聚类失真值确定第j个映射点是否属于邻居分割区域，以此类推，直到确定第j个映射点所属的分割区域。
f.在采用上述步骤确定出所有映射点所属的分割区域后，可以根据聚类后属于同一分割区域(例如第i个分割区域)的多个映射点(该多个映射点构成上文的一个映射点集合)形成的区域即为最终的分割区域。
上述划分方式3和划分方式4的区别在于，划分方式3是采用上述聚类方法对全景图像包括的多个像素点聚类得到多个像素点集合，然后将像素点映分别射至全景视频三维球面表达全景图，一个像素点集合包括的像素点各自对应的映射点形成的区域即为最终的分割区域。
应当理解的，除了上述4种划分方式外，还可以采用划分方式，本申请对此不做具体限定。
图12为本申请全景图像的色调映射方法的过程1200的流程图。过程1200可以由图8所示的全景视频播放系统中的前端执行。过程1200描述为一系列的步骤或操作，应当理解的是，过程1200可以以各种顺序执行和/或同时发生，不限于图12所示的执行顺序。过程1200可以包括：
步骤1201、获取第一分割区域包括的至少一个映射点。
本申请中，前端可以采用上述方法对全景视频三维球面表达全景图进行划分以得到多个分割区域。对于具体采用哪种划分方式，前端可以采用预设的划分方式，例如，预先设定采用上述第1种划分方式，那么前端可以将全景视频三维球面表达全景图划分成多个等视角间隔的分割区域。此时预设的划分方式可以适用于大多数的全景图像，以提高图像处 理效率。或者，前端也可以采用基于预设规则确定的划分方式，例如，前端在获取到待处理的全景图像后，可以对该全景图像进行特征分析，从而确定划分方式，该划分方式更符合全景图像的特征，以提高图像处理效率。或者，前端还可以提供用户选择接口，由用户在该接口上选择或者输入期望采用的划分方式，这样划分方式可以更符合全景图像的动态范围。
应当理解的，除了上述3种确定划分方式的方法外，还可以采用其它方法确定划分方式，本申请对此不做具体限定。
第一分割区域是前端划分全景视频三维球面表达全景图后得到的多个分割区域中的一个。根据全景图像和全景视频三维球面表达全景图之间的映射关系，可以将全景图像包括的多个像素点分别映射至全景视频三维球面表达全景图，得到多个映射点，多个像素点和多个映射点一一对应。基于此，可以在确定了第一分割区域的范围后，获取其包含的至少一个映射点。
步骤1202、根据至少一个像素点生成第一分割区域的元数据信息单元。
通常全景视频数据包括视频数据和元数据信息，视频数据可以是全景视频包括的多个图像帧的相关数据，例如，各个图像帧的像素数据；元数据信息可以包括视频数据相关的数据和映射曲线相关的数据。其中，视频数据相关的数据例如可以包括全景视频数据的格式和特征，目标系统显示最大亮度(targeted_system_display_maximum_luminance)，全景视频在存储器中可存储的最大亮度MaxSource、最小亮度MinSource、平均亮度AvgSource，可储存的变化范围等等，可储存的变化范围的原理类似于方差或者分布置信区间，用于描述视频信号的亮度聚集范围。映射曲线相关的数据例如可以包括收缩因子、偏移因子、弯曲形状因子等与映射曲线相关的参数，例如，S型曲线相关的数据包括a、b、p、n、m、k1、k2和k3共八个参数。需要说明的是，全景视频数据包括但不限于上述内容，本申请对此不做具体限定。
示例性的，以第i个分割区域为例说明。
1、生成元数信息单元中的“最大亮度值”：
a、计算第i个分割区域包含的映射点的亮度值Y[i](可选的，此处也可以计算映射点的RGB三个分量中的最大值)，i∈[0,n-1]，n表示第i个分割区域包含的映射点的个数。需要说明的是，映射点的亮度值实际上是该映射点对应的全景图像中的像素点的亮度值，以下同理。
b、获取第i个分割区域包含的映射点的亮度值中的最大亮度值MaxY＝max(Y[i])。
c、根据预设风格或者参数对MaxY进行调整得到MaxY1。
d、将MaxY1作为“曲线信息值”写入第i个分割区域的元数据信息单元中。
2、生成元数据信息单元中的“曲线信息值”
a、计算第i个分割区域包含的所有映射点的直方图his[i]，i∈[0,n-1]，n表示第i个分割区域包含的映射点的个数。需要说明的是，所有映射点的直方图实际上是该所有映射点对应的全景图像中的多个像素点的直方图，以下同理。
b、对直方图his[i]进行处理(包括不操作或者超过一定大小后截断)，生成his1[i]。
c、根据处理后直方图his1[i]生成映射曲线，Ratio[i]＝his1[i]/Sum of(his1[i])。
d、根据预设的风格或者算法参数调整Ratio[i]得到Ratio1[i]＝w[i]×Ratio[i]。
e、将Ratio1[n]作为“曲线信息值”写入第i个分割区域的元数据信息单元中。
在一种可能的实现方式中，当第一分割区域和第二分割区域的直方图和/或亮度满足设定条件时，前端可以对第一分割区域的元数据信息单元和第二分割区域的第二元数据信息单元进行融合以得到第一分割区域和第二分割区域的元数据信息单元，第二分割区域是多个分割区域中的一个。
应当理解的，上述示例性的描述了两种元数据信息单元中的内容的生成方法，但并不构成对元数据信息单元的生成方式、所包含的内容的限定，本申请对此不做具体限定。
前端获取多个分割区域的直方图和/或者亮度值，如果有两个以上分割区域的直方图和/或者亮度值相似，例如，该两个以上分割区域的直方图和/或者亮度值的失真值小于设定阈值，可以认为该两个以上分割区域对应的图像区域的动态范围相似度较高，因此可以对该两个以上分割区域确定同一个元数据信息单元。而该元数据信息单元可以是对上述步骤中得到的该两个以上分割区域各自的元数据信息单元加权平均得到的，或者，将上述步骤中得到的该两个以上分割区域作为一个分割区域重新计算元数据信息单元。
示例性的，可以采用以下方法判断两个以上分割区域的直方图的失真值是否小于设定阈值：
1、计算第一分割区域包含的所有映射点的直方图hisA[n]。
2、计算第二分割区域包含的所有映射点的直方图hisB[n]。
3、计算hisDiff[i]＝(hisA[i]-hisB[i])×(hisA[i]-hisB[i])。
4、根据预设权重计算总的失真值hisDiffSum＝∑hisDiff[i]×w[i]
5、若hisDiffSum小于预设阈值T，则认为“第一分割区域和第二分割区域的直方图满足设定条件”。
此时，前端可以在元数据信息单元中加入指示信息，以指示该元数据信息单元和哪几个分割区域对应，例如，元数据信息单元中写入多个球面坐标，该多个球面坐标分别位于前述两个以上分割区域内，这样显示端在读取元数据信息单元时，根据其中包括的球面坐标，确定球面坐标所属的分割区域，从而将其确定为该元数据信息单元所对应的分割区域。
由此可见，前端在生成各个分割区域的元数据信息单元时，充分考虑了分割区域包含的映射点对应的像素点的特征，而像素点的特征可以反应分割区域的动态范围，因此显示端在基于元数据信息单元获取其对应的分割区域的色调映射曲线时，得到的是与该分割区域的动态范围特征相匹配的色调映射曲线，从而实现针对全景图像的更精确的色调映射处理。
步骤1203、将第一分割区域的元数据信息单元写入码流。
前端确定了第一分割区域及其对应的元数据信息单元后，可以将该元数据信息单元写入码流，以供显示端在显示全景图像时，进行相应的色调映射。应当理解的，多个分割区域中除第一分割区域外的其它分割区域均可以采用上述方法确定对应的元数据单元，此处不再赘述。
可选的，前端可以按照设定的顺序遍历多个分割区域，然后每遍历一个分割区域，将其对应的元数据信息单元写入码流。
可选的，前端可以根据预设规则确定遍历多个分割区域的顺序，然后根据该顺序每遍历一个分割区域，将其对应的元数据信息单元写入码流，与此同时，前端还要将确定的顺 序写入码流。
上述顺序可以是从全景视频三维球面表达全景图的自顶向下或者自前向后等顺序。
在一种可能的实现方式中，前端可以将生成元数据信息单元和写码流的过程交叉进行，即前端根据上述顺序遍历多个分割区域，每遍历一个分割区域，采用步骤1202的方法生成其对应的元数据信息单元，然后将该元数据信息单元写入码流。
本申请，通过将与全景图像对应的全景视频三维球面表达全景图划分成多个分割区域，根据每个分割区域包含的映射点所对应的像素点生成该分割区域的元数据信息单元，从而通过元数据信息单元指导显示端生成对应于该分割区域的色调映射曲线，进而实现从图像到显示的色调映射，既可以针对性的对全景图像上的不同动态范围区域确定匹配的色调映射参数，又可以符合显示端的显示动态范围，提高全景图像的处理效率。
图13为本申请全景图像的色调映射方法的过程1300的流程图。过程1300可以由图8所示的全景视频播放系统中的显示端执行。过程1300描述为一系列的步骤或操作，应当理解的是，过程1300可以以各种顺序执行和/或同时发生，不限于图13所示的执行顺序。过程1300可以包括：
步骤1301、从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元。
本申请中，显示端可以采用上述方法对全景视频三维球面表达全景图进行划分以得到多个分割区域。对于具体采用哪种划分方式，显示端可以采用预设的划分方式，也可以采用解析码流得到的划分方式。
应当理解的，除了上述2种确定划分方式的方法外，还可以采用其它方法确定划分方式，本申请对此不做具体限定。
显示端还可以解析码流得到多个元数据信息单元。另外显示端可以从码流中解析得到多个元数据信息单元的编号信息，该编号信息一方面可以标识多个元数据信息单元，另一方面可以表示多个元数据信息单元的先后顺序；或者，显示端可以将多个元数据信息单元解析出来的顺序确定为多个元数据信息单元的先后顺序。
显示端可以通过以下几种方法确定多个分割区域和多个元数据信息单元之间的对应关系：
1、显示端可以按照第一预设顺序从多个元数据信息单元中提取当前元数据信息单元，按照第二预设顺序从多个分割区域中提取当前分割区域，将当前分割区域和当前元数据信息单元建立对应关系。
2、显示端可以按照第一预设顺序从多个元数据信息单元中提取当前元数据信息单元，按照解析码流得到的遍历顺序从多个分割区域中提取当前分割区域，将当前分割区域和当前元数据信息单元建立对应关系。
3、显示端可以按照第一预设顺序从多个元数据信息单元中提取当前元数据信息单元，获取当前元数据信息单元包括的一个或多个坐标，根据一个或多个坐标在全景视频三维球面表达全景图上确定一个或多个映射点。当只有一个映射点时，将一个映射点所属的分割区域和当前元数据信息单元建立对应关系；当有多个映射点时，将多个映射点所属的至少一个分割区域和当前元数据信息单元建立对应关系。
当前元数据信息单元可以认为是多个元数据信息单元中排在最前面的一个，如上所述， 多个元数据信息单元之间具有先后顺序，上述第一预设顺序可以指按照多个元数据信息单元的先后顺序提取排在最前面的元数据信息单元即为当前元数据信息单元，一旦确定了该当前元数据信息单元和其中一个或多个分割单元之间的对应关系，就将其从最先顺序撤掉，将其下一个元数据信息单元确定为新的当前元数据信息单元，多次迭代，直到确定多个分割区域和多个元数据信息单元之间的对应关系。
本申请中，显示端可以根据设定映射点确定一个或多个目标分割区域，当只有一个目标分割区域时，将一个目标分割区域对应的元数据信息单元确定为一个目标元数据信息单元；或者，当有多个目标分割区域时，将多个目标分割区域分别对应的元数据信息单元确定为多个目标元数据信息单元。
在一种可能的实现方式中，设定映射点是第一映射点，以第一映射点为中心在全景视频三维球面表达全景图上确定第一覆盖范围，当第一覆盖范围包括一个分割区域时，将一个分割区域确定为一个目标分割区域；或者，当第一覆盖范围包括多个分割区域时，将多个分割区域确定为多个目标分割区域。
设定映射点为第一像素点在全景视频三维球面表达全景图上的第一映射点，显示端可以以第一映射点为圆心，以第一长度为半径确定第一覆盖范围，此时第一覆盖范围是圆形区域，或者，显示端也可以以第一映射点为中心，以第一长度为边长确定第一覆盖范围，此时第一覆盖范围是方形区域。第一长度可以是预设长度，或者，第一长度可以是第一映射点所属的第一分割区域对应的第一元数据信息单元中指定的。多个分割区域包括第一分割区域，多个元数据信息单元包括第一元数据信息单元。应当理解的，第一覆盖范围还可以是其它形状的区域，其形状和尺寸的相关信息可以是预先设定的，也可以是前述第一元数据信息单元中指定的，本申请对此不做具体限定。
例如，以第一映射点为中心，以第一长度为半径/边长确定的第一覆盖范围可能只覆盖第一映射点所属的分割区域，此时第一映射点所属的分割区域为一个目标分割区域。第一覆盖范围也可能覆盖第一映射点所属的分割区域和与其相邻的一个或多个分割区域，那么第一映射点所属的分割区域和与其相邻的一个或多个分割区域为多个目标分割区域。第一覆盖范围可能只覆盖了前述相邻分割区域的部分区域，那么该相邻分割区域也可以作为目标分割区域。
在一种可能的实现方式中，设定映射点是全景视频三维球面表达全景图的视点中心，确定视点中心所属的第二分割区域，多个分割区域包括第二分割区域，将第二分割区域确定为一个目标分割区域。
设定映射点为全景视频三维球面表达全景图上的视点中心，无论第一像素点在全景图像中的哪个位置，均可以参照全景视频三维球面表达全景图上的视点中心，将其所属的第二分割区域确定为一个目标分割区域。
在一种可能的实现方式中，设定映射点是全景视频三维球面表达全景图的视点中心，以视觉中心为中心在全景视频三维球面表达全景图上确定第二覆盖范围，当第二覆盖范围包括一个分割区域时，将一个分割区域确定为一个目标分割区域；或者，当第二覆盖范围包括多个分割区域时，将多个分割区域确定为多个目标分割区域。
设定映射点为全景视频三维球面表达全景图上的视点中心，显示端可以以视觉中心为中心，以第二长度为半径/边长确定第二覆盖范围，此时第二覆盖范围是圆形区域，或者， 显示端也可以以视觉中心为中心，以第二长度为边长确定第二覆盖范围，此时第二覆盖范围是方形区域。第二长度可以是预设长度，或者，第二长度可以是视点中心所属的第二分割区域对应的第二元数据信息单元中指定的。多个分割区域包括第二分割区域，多个元数据信息单元包括第二元数据信息单元。应当理解的，第二覆盖范围还可以是其它形状的区域，其形状和尺寸的相关信息可以是预先设定的，也可以是前述第二元数据信息单元中指定的，本申请对此不做具体限定。
例如，与以第一映射点为圆心，以第一长度为半径确定第一覆盖范围的不同在于，此时是以视觉中心为圆心，以第二长度为半径确定第二覆盖范围，之后再将第二覆盖范围覆盖的分割区域确定为目标分割区域。同理，第二覆盖范围可能只覆盖视觉中心所属的分割区域，此时视觉中心所属的分割区域为一个目标分割区域，第二覆盖范围也可能覆盖视觉中心所属的分割区域和与其相邻的一个或多个分割区域，那么视觉中心所属的分割区域和与其相邻的一个或多个分割区域为多个目标分割区域。第二覆盖范围可能只覆盖了前述相邻分割区域的部分区域，那么该相邻分割区域也可以作为目标分割区域。
基于多个分割区域和多个元数据信息单元之间的对应关系，可以将上述步骤确定的一个或多个目标分割区域对应的元数据信息单元确定为第一像素点的一个或多个目标元数据信息单元。
步骤1302、根据一个或多个目标元数据信息单元对第一像素点的像素值进行色调映射以得到第一像素点的目标色调映射值。
本申请中，显示端可以根据一个或多个目标元数据信息单元获取一条或多条色调映射曲线。当只有一条色条映射曲线时，根据一条色调映射曲线对第一像素点的像素值进行色调映射以得到目标色调映射值；或者，当有多条色条映射曲线时，根据多条色调映射曲线分别对第一像素点的像素值进行色调映射以得到第一像素点的多个色调中间值，再根据多个色调中间值获取目标色调映射值。
如上所述，元数据信息单元中包括色调映射曲线的相关参数，因此基于该参数可以得到对应的色调映射曲线，需要说明的是，本申请对色调映射曲线的参数和色调映射曲线均不作具体限定。
基于此，如果第一像素点只有一个目标元数据信息单元，那么对该第一像素点可以得到一条色调映射曲线，如果第一像素点有多个目标元数据信息单元，那么对该第一像素点可以得到多条色调映射曲线。
相应的，
用上述一条色调映射曲线对第一像素点的像素值进行色调映射可以直接得到第一像素点的目标色调映射值。
用上述多条色调映射曲线分别对第一像素点的像素值进行色调映射可以得到第一像素点的多个色调中间值，对该多个色调中间值做进一步处理，例如，取多个色调中间值中的中间值(例如，多个色调中间值包括1、2、3，那么中间值为2)，或者对多个色调中间值进行加权平均处理的结果即为第一像素点的目标色调中间值。前述加权平均中对应于各个色调中间值的加权系数可以与第一映射点与各个分割区域的中心点的距离相关。即计算第一映射点分别与多个目标分割区域中的每个分割区域的中心点的距离(例如欧氏距离)，计算多个目标分割区域分别对应的距离之和，将各个距离分别与距离之和的比值作为对应 的目标分割区域的元数据信息单元求得到色调中间值的加权系数；或者，前述加权平均中对应于各个色调中间值的加权系数可以与多个目标元数据信息单元分别对应的分割区域的面积比重相关。即计算上述第一覆盖范围或者第二覆盖范围内包含的多个分割区域的面积，如果分割区域被前述第一覆盖范围或者第二覆盖范围完全覆盖，则计算该分割区域的全面积，如果分割区域被前述第一覆盖范围或者第二覆盖范围覆盖部分区域，则计算该分割区域的部分区域的面积，计算多个目标分割区域的面积之和，将各个分割区域的面积分别与面积之和的比值作为对应的目标分割区域的元数据信息单元求得到色调中间值的加权系数。
示例性的，根据从第一像素点的目标元数据信息单元获取色调映射曲线参数a,b,p,m,n，进而可以获取色调映射曲线：
Figure PCTCN2022100495-appb-000018
根据上述色调映射曲线可以获得从归一化的第一像素点的像素值映射到归一化的HDR显示数据之间的映射关系。需要说明的是，L和L’可以是归一化的光信号或者电信号，本申请不做具体限定。归一化计算可以是PQ的非线性空间，也可以是归一化0～1的线性空间。归一化可以是0-10000nit，也可以是0.001-100000nit。本申请对数据的归一化范围和过程不做具体限定。
根据HDR显示设备的最大显示能力和最小显示能力(通常为0)，将归一化的HDR显示数据L’反向归一化到显示设备的最大显示能力和最小显示能力之间，第一像素点的目标色调值为反向归一化后的HDR显示数据。需要说明的是，反向归一化计算可以是PQ的非线性空间，也可以是归一化0～1的线性空间。反向归一化可以是0-10000nit，也可以是0.001-100000nit。本申请对数据的反归一化范围和过程不做具体限定。
本申请，通过将与全景图像对应的全景视频三维球面表达全景图划分成多个分割区域，根据每个分割区域包含的映射点所对应的像素点生成该分割区域的元数据信息单元，从而通过元数据信息单元指导显示端生成对应于该分割区域的色调映射曲线，进而实现从图像到显示的色调映射，既可以针对性的对全景图像上的不同动态范围区域确定匹配的色调映射参数，，又可以符合显示端的显示动态范围，提高全景图像的处理效率。
在一种可能的实现方式中，显示端可以解析码流得到多个元数据信息单元，另外显示端可以从码流中解析得到多个元数据信息单元的编号信息，该编号信息一方面可以标识多个元数据信息单元，另一方面可以表示多个元数据信息单元的先后顺序；或者，显示端可以解析码流得到多个元数据信息单元，并且将元数据信息单元解析出来的顺序确定为多个元数据信息单元的先后顺序。
本申请中，显示端可以对全景视频三维球面表达全景图进行划分以得到多个分割区域。
在一种可能的实现方式中，显示端可以获取预设的划分方式，然后根据预设的划分方式对全景视频三维球面表达全景图进行划分以得到多个分割区域。
用户可以根据历史数据或者实验室数据提前设定好划分方式，这样划分方式可以适用于大多数的全景图像，以提高图像处理效率。或者，用户也可以通过全景图像处理系统提供的用户界面，在其上选择或者输入期望采用的划分方式，这样划分方式可以更符合全景 图像的动态范围。
在一种可能的实现方式中，显示端可以解析码流以得到由前端设置的划分方式。
前端在获取到待处理的全景图像后，可以对该全景图像进行特征分析，从而确定划分方式，该划分方式更符合全景图像的特征，以提高图像处理效率。为了与显示端保持一致，前端可以将确定的划分方式写入码流。显示端解析码流后即可得到前述划分方式。
通过上述方法，显示端可以确定划分方式为依视角规则划分。显示端可以将全景视频三维球面表达全景图划分为等视角间隔的多个分割区域，例如，如图12a所示，划分的到的多个分割区域中，纬度相邻的两个分割区域对应的水平夹角相等，经度相邻的两个分割区域对应的俯仰夹角相等；或者，显示端可以将全景视频三维球面表达全景图从纬度方向上划分为多个分割区域，例如，如图12b所示，显示端将全景视频三维球面表达全景图划分为上下等大的两个分割区域；或者，将全景视频三维球面表达全景图从经度方向上均匀划分为多个分割区域，例如，如图12c所示，显示端将全景视频三维球面表达全景图划分为左右等大的两个分割区域；或者，显示端可以将全景视频三维球面表达全景图从纬度方向上划分为多个分割区域，并将全景视频三维球面表达全景图从经度方向上均匀划分为多个分割区域，例如，如图12d所示，显示端将全景视频三维球面表达全景图划分为上下等大且左右等大的四个分割区域。
应当理解的，上述示例的几种划分方式并不对划分方式构成限定，显示端还可以将全景视频三维球面表达全景图划分为上下等大的两部分，然后将两部分分别均匀四等分；显示端还可以将全景视频三维球面表达全景图划分为上下等大的两部分，然后将两部分分别均匀四等分，最后将每个四分区域划分为等视角间隔的区域。另外，显示端对全景视频三维球面表达全景图的划分也可以不是等分的，或者不是全部的划分步骤都是等分的，对此不做具体限定。
本申请中，显示端也可以解析码流直接获取到全景视频三维球面表达全景图包括的多个分割区域。前端在对全景视频三维球面表达全景图划分得到多个分割区域后，可以将该多个分割区域的划分信息写入码流，例如，分割区域的形状、尺寸、左上角的坐标等，又例如，分割线的描述信息，这样显示端可以解析码流得到上述多个分割区域的划分信息，进而确定多个分割区域。
图14为本申请显示装置1400的一个示例性的结构图，如图14所示，本实施例的显示装置1400可以应用于上述实施例中的显示端，也可以应用于上述实施例中的终端设备。显示装置1400可以包括：划分模块1401、确定模块1402和映射模块1403。其中，
确定模块1402，用于从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元，所述多个元数据信息单元是解析码流得到的，所述第一像素点是待处理的全景视频二维平面投影图中的任意一个像素点，所述多个元数据信息单元和全景视频三维球面表达全景图包括的多个分割区域对应，所述全景视频二维平面投影图和所述全景视频三维球面表达全景图具有映射关系；映射模块1403，用于根据所述一个或多个目标元数据信息单元对所述第一像素点的像素值进行色调映射以得到所述第一像素点的目标色调映射值。
在一种可能的实现方式中，划分模块1401，用于获取所述多个分割区域。
在一种可能的实现方式中，所述划分模块1401，具体用于根据预设的划分方式对所述 全景视频三维球面表达全景图进行划分，以得到所述多个分割区域。
在一种可能的实现方式中，所述划分模块1401，具体用于根据解析所述码流得到的划分方式对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域。
在一种可能的实现方式中，所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
在一种可能的实现方式中，所述划分模块1401，具体用于对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；将所述多个像素点分别映射至所述全景视频三维球面表达全景图；将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述划分模块1401，具体用于将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；对所述多个映射点进行聚类以得到多个映射点集合；将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述划分模块1401，具体用于根据解析所述码流得到的所述多个分割区域的指示信息获取所述多个分割区域。
在一种可能的实现方式中，所述确定模块1402，具体用于确定所述多个元数据信息单元和多个分割区域之间的对应关系，一个所述元数据信息单元对应一个或多个所述分割区域；根据设定映射点确定一个或多个目标分割区域；当只有一个目标分割区域时，将所述一个目标分割区域对应的元数据信息单元确定为所述一个目标元数据信息单元；或者，当有多个目标分割区域时，将所述多个目标分割区域分别对应的元数据信息单元确定为所述多个目标元数据信息单元。
在一种可能的实现方式中，所述确定模块1402，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；按照第二预设顺序从所述多个分割区域中提取当前分割区域；将所述当前分割区域和所述当前元数据信息单元建立对应关系。
在一种可能的实现方式中，所述确定模块1402，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；按照解析所述码流得到的遍历顺序从所述多个分割区域中提取当前分割区域；将所述当前分割区域和所述当前元数据信息单元建立对应关系。
在一种可能的实现方式中，所述确定模块1402，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；获取所述当前元数据信息单元包括的一个或多个坐标；根据所述一个或多个坐标在所述全景视频三维球面表达全景图上确定一个或多个映射点；当只有一个映射点时，将所述一个映射点所属的分割区域和所述当前元数据信息单元建立对应关系；当有多个映射点时，将所述多个映射点所属的至少一个分割区域和所述当前元数据信息单元建立对应关系。
在一种可能的实现方式中，所述设定映射点为所述第一像素点在所述全景视频三维球面表达全景图上的第一映射点；所述确定模块1402，具体用于以所述第一映射点为中心在所述全景视频三维球面表达全景图上确定第一覆盖范围；当所述第一覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；或者，当所述第一覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
在一种可能的实现方式中，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述确定模块1402，具体用于确定所述视点中心所属的第二分割区域，所述多个分割区域包括所述第二分割区域；将所述第二分割区域确定为所述一个目标分割区域。
在一种可能的实现方式中，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述确定模块1402，具体用于以所述视觉中心为中心在所述全景视频三维球面表达全景图上确定第二覆盖范围；当所述第二覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；或者，当所述第二覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
在一种可能的实现方式中，所述映射模块1403，具体用于根据所述一个或多个目标元数据信息单元获取一条或多条色调映射曲线；当只有一条色条映射曲线时，根据所述一条色调映射曲线对所述第一像素点的像素值进行色调映射以得到所述目标色调映射值；或者，当有多条色条映射曲线时，根据所述多条色调映射曲线分别对所述第一像素点的像素值进行色调映射以得到所述第一像素点的多个色调中间值；根据所述多个色调中间值获取所述目标色调映射值。
在一种可能的实现方式中，所述映射模块1403，具体用于将所述多个色调中间值中的中间值确定为所述目标色调映射值；或者，对所述多个色调中间值进行加权平均以得到所述目标色调映射值，所述多个色调中间值的加权系数是预先设定的，或者所述多个色调中间值的加权系数与所述第一像素点与各个分割区域的中心点的距离相关，或者，所述多个色调中间值的加权系数与所述多个目标元数据信息单元分别对应的分割区域的面积比重相关。
本实施例的装置，可以用于执行图13所示方法实施例的技术方案，其实现原理和技术效果类似，此处不再赘述。
图15为本申请前端装置1500的一个示例性的结构图，如图15所示，本实施例的前端装置1500可以应用于上述实施例中的前端，也可以应用于上述实施例中的终端设备。前端装置1500可以包括：划分模块1501、获取模块1502和生成模块1503。其中，
获取模块1502，用于获取第一分割区域包括的至少一个映射点，所述第一分割区域是全景视频三维球面表达全景图包括的多个分割区域中的一个，所述全景视频三维球面表达全景图和待处理的全景视频二维平面投影图具有映射关系，所述至少一个映射点和所述全景视频二维平面投影图上的至少一个像素点对应；生成模块1503，用于根据所述至少一个像素点生成所述第一分割区域的元数据信息单元；将所述第一分割区域的元数据信息单元写入码流。
在一种可能的实现方式中，所述生成模块1503，还用于当所述第一分割区域和第二分割区域的直方图和/或亮度满足设定条件时，对所述第一分割区域的元数据信息单元和所述第二分割区域的第二元数据信息单元进行融合以得到所述第一分割区域和所述第二分 割区域的元数据信息单元，所述第二分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，划分模块1501，用于将所述全景视频二维平面投影图映射至所述全景视频三维球面表达全景图；对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述划分模块1501，具体用于根据预设的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述划分模块1501，具体用于根据基于预设规则确定的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
在一种可能的实现方式中，所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
在一种可能的实现方式中，所述划分模块1501，具体用于对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；将所述多个像素点分别映射至所述全景视频三维球面表达全景图；将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
在一种可能的实现方式中，所述划分模块1501，具体用于将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；对所述多个映射点进行聚类以得到多个映射点集合；将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
本实施例的装置，可以用于执行图12所示方法实施例的技术方案，其实现原理和技术效果类似，此处不再赘述。
在实现过程中，上述方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。处理器可以是通用处理器、数字信号处理器(digital signal processor,DSP)、特定应用集成电路(application-specific integrated circuit，ASIC)、现场可编程门阵列(field programmable gate array,FPGA)或其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。本申请实施例公开的方法的步骤可以直接体现为硬件编码处理器执行完成，或者用编码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。
上述各实施例中提及的存储器可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(read-only memory，ROM)、可编程只读存储器(programmable ROM，PROM)、可擦除可编程只读存储器(erasable PROM，EPROM)、电可擦除可编程只读存储器(electrically EPROM，EEPROM) 或闪存。易失性存储器可以是随机存取存储器(random access memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(static RAM，SRAM)、动态随机存取存储器(dynamic RAM，DRAM)、同步动态随机存取存储器(synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(double data rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(direct rambus RAM，DR RAM)。应注意，本文描述的系统和方法的存储器旨在包括但不限于这些和任意其它适合类型的存储器。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (52)

  1. 一种全景图像的色调映射方法，其特征在于，包括：
    从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元，所述多个元数据信息单元是解析码流得到的，所述第一像素点是待处理的全景视频二维平面投影图中的任意一个像素点，所述多个元数据信息单元和全景视频三维球面表达全景图包括的多个分割区域对应，所述全景视频二维平面投影图和所述全景视频三维球面表达全景图具有映射关系；
    根据所述一个或多个目标元数据信息单元对所述第一像素点的像素值进行色调映射,以得到所述第一像素点的目标色调映射值。
  2. 根据权利要求1所述的方法，其特征在于，所述从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元之前，所述方法还包括：
    获取所述多个分割区域。
  3. 根据权利要求2所述的方法，其特征在于，所述获取所述多个分割区域，包括：
    根据预设的划分方式对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域。
  4. 根据权利要求2所述的方法，其特征在于，所述获取所述多个分割区域，包括：
    根据解析所述码流得到的划分方式对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域。
  5. 根据权利要求3或4所述的方法，其特征在于，
    所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
    所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
  6. 根据权利要求3或4所述的方法，其特征在于，所述对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域，包括：
    对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；
    将所述多个像素点分别映射至所述全景视频三维球面表达全景图；
    将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
  7. 根据权利要求3或4所述的方法，其特征在于，所述对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域，包括：
    将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；
    对所述多个映射点进行聚类以得到多个映射点集合；
    将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
  8. 根据权利要求2所述的方法，其特征在于，所述获取所述多个分割区域，包括：
    根据解析所述码流得到的所述多个分割区域的指示信息获取所述多个分割区域。
  9. 根据权利要求1-8中任一项所述的方法，其特征在于，所述从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元，包括：
    确定所述多个元数据信息单元和多个分割区域之间的对应关系，一个所述元数据信息单元对应一个或多个所述分割区域；
    根据设定映射点确定一个或多个目标分割区域；
    当只有一个目标分割区域时，将所述一个目标分割区域对应的元数据信息单元确定为所述一个目标元数据信息单元；
    或者，当有多个目标分割区域时，将所述多个目标分割区域分别对应的元数据信息单元确定为所述多个目标元数据信息单元。
  10. 根据权利要求9所述的方法，其特征在于，所述确定所述多个元数据信息单元和多个分割区域之间的对应关系，包括：
    按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；
    按照第二预设顺序从所述多个分割区域中提取当前分割区域；
    将所述当前分割区域和所述当前元数据信息单元建立对应关系。
  11. 根据权利要求9所述的方法，其特征在于，所述确定所述多个元数据信息单元和多个分割区域之间的对应关系，包括：
    按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；
    按照解析所述码流得到的遍历顺序从所述多个分割区域中提取当前分割区域；
    将所述当前分割区域和所述当前元数据信息单元建立对应关系。
  12. 根据权利要求9所述的方法，其特征在于，所述确定所述多个元数据信息单元和多个分割区域之间的对应关系，包括：
    按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；
    获取所述当前元数据信息单元包括的一个或多个坐标；
    根据所述一个或多个坐标在所述全景视频三维球面表达全景图上确定一个或多个映射点；
    当只有一个映射点时，将所述一个映射点所属的分割区域和所述当前元数据信息单元建立对应关系；
    当有多个映射点时，将所述多个映射点所属的至少一个分割区域和所述当前元数据信息单元建立对应关系。
  13. 根据权利要求9-12中任一项所述的方法，其特征在于，所述设定映射点为所述第一像素点在所述全景视频三维球面表达全景图上的第一映射点；所述根据设定映射点确定一个或多个目标分割区域，包括：
    以所述第一映射点为中心在所述全景视频三维球面表达全景图上确定第一覆盖范围；
    当所述第一覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；
    或者，当所述第一覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
  14. 根据权利要求9-12中任一项所述的方法，其特征在于，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述根据设定映射点确定一个或多个目标分割区域，包括：
    确定所述视点中心所属的第二分割区域，所述多个分割区域包括所述第二分割区域；
    将所述第二分割区域确定为所述一个目标分割区域。
  15. 根据权利要求9-12中任一项所述的方法，其特征在于，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述根据设定映射点确定一个或多个目标分割区域，包括：
    以所述视觉中心为中心在所述全景视频三维球面表达全景图上确定第二覆盖范围；
    当所述第二覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；
    或者，当所述第二覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
  16. 根据权利要求1-15中任一项所述的方法，其特征在于，所述根据所述一个或多个目标元数据信息单元对所述第一像素点的像素值进行色调映射以得到所述第一像素点的目标色调映射值，包括：
    根据所述一个或多个目标元数据信息单元获取一条或多条色调映射曲线；
    当只有一条色条映射曲线时，根据所述一条色调映射曲线对所述第一像素点的像素值进行色调映射以得到所述目标色调映射值；
    或者，当有多条色条映射曲线时，根据所述多条色调映射曲线分别对所述第一像素点的像素值进行色调映射以得到所述第一像素点的多个色调中间值；
    根据所述多个色调中间值获取所述目标色调映射值。
  17. 根据权利要求16所述的方法，其特征在于，所述根据所述多个色调中间值获取所述目标色调映射值，包括：
    将所述多个色调中间值中的中间值确定为所述目标色调映射值；
    或者，对所述多个色调中间值进行加权平均以得到所述目标色调映射值，所述多个色调中间值的加权系数是预先设定的，或者所述多个色调中间值的加权系数与所述第一像素点与各个分割区域的中心点的距离相关，或者，所述多个色调中间值的加权系数与所述多个目标元数据信息单元分别对应的分割区域的面积比重相关。
  18. 一种全景图像的色调映射方法，其特征在于，包括：
    获取第一分割区域包括的至少一个映射点，所述第一分割区域是全景视频三维球面表达全景图包括的多个分割区域中的一个，所述全景视频三维球面表达全景图和待处理的全景视频二维平面投影图具有映射关系，所述至少一个映射点和所述全景视频二维平面投影图上的至少一个像素点对应；
    根据所述至少一个像素点生成所述第一分割区域的元数据信息单元；
    将所述第一分割区域的元数据信息单元写入码流。
  19. 根据权利要求18所述的方法，其特征在于，所述根据所述至少一个像素点生成所述第一分割区域的元数据信息单元之后，所述方法还包括：
    当所述第一分割区域和第二分割区域的直方图和/或亮度满足设定条件时，对所述第 一分割区域的元数据信息单元和所述第二分割区域的第二元数据信息单元进行融合以得到所述第一分割区域和所述第二分割区域的元数据信息单元，所述第二分割区域是所述多个分割区域中的一个。
  20. 根据权利要求18或19所述的方法，其特征在于，所述获取第一分割区域包括的至少一个映射点之前，所述方法还包括：
    将所述全景视频二维平面投影图映射至所述全景视频三维球面表达全景图；
    对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
  21. 根据权利要求20所述的方法，其特征在于，所述对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域，包括：
    根据预设的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
  22. 根据权利要求20所述的方法，其特征在于，所述对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域，包括：
    根据基于预设规则确定的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
  23. 根据权利要求21或22所述的方法，其特征在于，
    所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
    所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
  24. 根据权利要求21或22所述的方法，其特征在于，所述对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域，包括：
    对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；
    将所述多个像素点分别映射至所述全景视频三维球面表达全景图；
    将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
  25. 根据权利要求21或22所述的方法，其特征在于，所述对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域，包括：
    将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；
    对所述多个映射点进行聚类以得到多个映射点集合；
    将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
  26. 一种显示装置，其特征在于，包括：
    确定模块，用于从多个元数据信息单元中确定第一像素点的一个或多个目标元数据信息单元，所述多个元数据信息单元是解析码流得到的，所述第一像素点是待处理的全景视频二维平面投影图中的任意一个像素点，所述多个元数据信息单元和全景视频三维球面表 达全景图包括的多个分割区域对应，所述全景视频二维平面投影图和所述全景视频三维球面表达全景图具有映射关系；
    映射模块，用于根据所述一个或多个目标元数据信息单元对所述第一像素点的像素值进行色调映射以得到所述第一像素点的目标色调映射值。
  27. 根据权利要求26所述的装置，其特征在于，还包括：
    划分模块，用于获取所述多个分割区域。
  28. 根据权利要求27所述的装置，其特征在于，所述划分模块，具体用于根据预设的划分方式对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域。
  29. 根据权利要求27所述的装置，其特征在于，所述划分模块，具体用于根据解析所述码流得到的划分方式对所述全景视频三维球面表达全景图进行划分，以得到所述多个分割区域。
  30. 根据权利要求28或29所述的装置，其特征在于，所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
    所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
  31. 根据权利要求28或29所述的装置，其特征在于，所述划分模块，具体用于对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；将所述多个像素点分别映射至所述全景视频三维球面表达全景图；将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
  32. 根据权利要求28或29所述的装置，其特征在于，所述划分模块，具体用于将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；对所述多个映射点进行聚类以得到多个映射点集合；将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
  33. 根据权利要求27所述的装置，其特征在于，所述划分模块，具体用于根据解析所述码流得到的所述多个分割区域的指示信息获取所述多个分割区域。
  34. 根据权利要求26-33中任一项所述的装置，其特征在于，所述确定模块，具体用于确定所述多个元数据信息单元和多个分割区域之间的对应关系，一个所述元数据信息单元对应一个或多个所述分割区域；根据设定映射点确定一个或多个目标分割区域；当只有一个目标分割区域时，将所述一个目标分割区域对应的元数据信息单元确定为所述一个目标元数据信息单元；或者，当有多个目标分割区域时，将所述多个目标分割区域分别对应的元数据信息单元确定为所述多个目标元数据信息单元。
  35. 根据权利要求34所述的装置，其特征在于，所述确定模块，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；按照第二预设顺序从所述多个分割区域中提取当前分割区域；将所述当前分割区域和所述当前元数据信息单元建立对应关系。
  36. 根据权利要求34所述的装置，其特征在于，所述确定模块，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；按照解析所述码流得到的遍历顺序从所述多个分割区域中提取当前分割区域；将所述当前分割区域和所述当前元数据信息单元建立对应关系。
  37. 根据权利要求34所述的装置，其特征在于，所述确定模块，具体用于按照第一预设顺序从所述多个元数据信息单元中提取当前元数据信息单元；获取所述当前元数据信息单元包括的一个或多个坐标；根据所述一个或多个坐标在所述全景视频三维球面表达全景图上确定一个或多个映射点；当只有一个映射点时，将所述一个映射点所属的分割区域和所述当前元数据信息单元建立对应关系；当有多个映射点时，将所述多个映射点所属的至少一个分割区域和所述当前元数据信息单元建立对应关系。
  38. 根据权利要求34-37中任一项所述的装置，其特征在于，所述设定映射点为所述第一像素点在所述全景视频三维球面表达全景图上的第一映射点；所述确定模块，具体用于以所述第一映射点为中心在所述全景视频三维球面表达全景图上确定第一覆盖范围；当所述第一覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；或者，当所述第一覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
  39. 根据权利要求34-37中任一项所述的装置，其特征在于，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述确定模块，具体用于确定所述视点中心所属的第二分割区域，所述多个分割区域包括所述第二分割区域；将所述第二分割区域确定为所述一个目标分割区域。
  40. 根据权利要求34-37中任一项所述的装置，其特征在于，所述设定映射点为所述全景视频三维球面表达全景图上的视点中心；所述确定模块，具体用于以所述视觉中心为中心在所述全景视频三维球面表达全景图上确定第二覆盖范围；当所述第二覆盖范围包括一个分割区域时，将所述一个分割区域确定为所述一个目标分割区域；或者，当所述第二覆盖范围包括多个分割区域时，将所述多个分割区域确定为所述多个目标分割区域。
  41. 根据权利要求26-40中任一项所述的装置，其特征在于，所述映射模块，具体用于根据所述一个或多个目标元数据信息单元获取一条或多条色调映射曲线；当只有一条色条映射曲线时，根据所述一条色调映射曲线对所述第一像素点的像素值进行色调映射以得到所述目标色调映射值；或者，当有多条色条映射曲线时，根据所述多条色调映射曲线分别对所述第一像素点的像素值进行色调映射以得到所述第一像素点的多个色调中间值；根据所述多个色调中间值获取所述目标色调映射值。
  42. 根据权利要求41所述的装置，其特征在于，所述映射模块，具体用于将所述多个色调中间值中的中间值确定为所述目标色调映射值；或者，对所述多个色调中间值进行加权平均以得到所述目标色调映射值，所述多个色调中间值的加权系数是预先设定的，或者所述多个色调中间值的加权系数与所述第一像素点与各个分割区域的中心点的距离相关，或者，所述多个色调中间值的加权系数与所述多个目标元数据信息单元分别对应的分割区域的面积比重相关。
  43. 一种前端装置，其特征在于，包括：
    获取模块，用于获取第一分割区域包括的至少一个映射点，所述第一分割区域是全景 视频三维球面表达全景图包括的多个分割区域中的一个，所述全景视频三维球面表达全景图和待处理的全景视频二维平面投影图具有映射关系，所述至少一个映射点和所述全景视频二维平面投影图上的至少一个像素点对应；
    生成模块，用于根据所述至少一个像素点生成所述第一分割区域的元数据信息单元；将所述第一分割区域的元数据信息单元写入码流。
  44. 根据权利要求43所述的装置，其特征在于，所述生成模块，还用于当所述第一分割区域和第二分割区域的直方图和/或亮度满足设定条件时，对所述第一分割区域的元数据信息单元和所述第二分割区域的第二元数据信息单元进行融合以得到所述第一分割区域和所述第二分割区域的元数据信息单元，所述第二分割区域是所述多个分割区域中的一个。
  45. 根据权利要求43或44所述的装置，其特征在于，还包括：
    划分模块，用于将所述全景视频二维平面投影图映射至所述全景视频三维球面表达全景图；对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
  46. 根据权利要求45所述的装置，其特征在于，所述划分模块，具体用于根据预设的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
  47. 根据权利要求45所述的装置，其特征在于，所述划分模块，具体用于根据基于预设规则确定的划分方式对所述全景视频三维球面表达全景图进行划分以得到所述多个分割区域。
  48. 根据权利要求46或47所述的装置，其特征在于，所述多个分割区域是通过对所述全景视频三维球面表达全景图以预设视角间隔规则进行划分得到的；或者，
    所述多个分割区域是通过对所述全景视频三维球面表达全景图从纬度方向上进行划分得到的；和/或，所述多个分割区域是通过对所述全景视频三维球面表达全景图从经度方向上进行划分得到的。
  49. 根据权利要求46或47所述的装置，其特征在于，所述划分模块，具体用于对所述全景视频二维平面投影图包括的多个像素点进行聚类以得到多个像素点集合；将所述多个像素点分别映射至所述全景视频三维球面表达全景图；将第一像素点集合包括的像素点在所述全景视频三维球面表达全景图上对应的映射点形成的区域作为第一分割区域，所述第一像素点集合是所述多个像素点集合中的一个，所述第一分割区域是所述多个分割区域中的一个。
  50. 根据权利要求46或47所述的装置，其特征在于，所述划分模块，具体用于将所述全景视频二维平面投影图包括的多个像素点分别映射至所述全景视频三维球面表达全景图以得到多个映射点；对所述多个映射点进行聚类以得到多个映射点集合；将第一映射点集合包括的映射点形成的区域作为第二分割区域，所述第一映射点集合是所述多个映射点集合中的一个，所述第二分割区域是所述多个分割区域中的一个。
  51. 一种终端设备，其特征在于，包括：
    一个或多个处理器；
    存储器，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-25中任一项所述的方法。
  52. 一种计算机可读存储介质，其特征在于，包括计算机程序，所述计算机程序在计算机上被执行时，使得所述计算机执行权利要求1-25中任一项所述的方法。
PCT/CN2022/100495 2021-07-14 2022-06-22 全景图像的色调映射方法和装置 Ceased WO2023284503A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP22841143.5A EP4358018A4 (en) 2021-07-14 2022-06-22 METHOD AND APPARATUS FOR TONE MATCHING FOR PANORAMIC IMAGE
US18/411,529 US20240153051A1 (en) 2021-07-14 2024-01-12 Tone mapping method and apparatus for panoramic image

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110794014.5 2021-07-14
CN202110794014.5A CN115619648A (zh) 2021-07-14 2021-07-14 全景图像的色调映射方法和装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/411,529 Continuation US20240153051A1 (en) 2021-07-14 2024-01-12 Tone mapping method and apparatus for panoramic image

Publications (1)

Publication Number Publication Date
WO2023284503A1 true WO2023284503A1 (zh) 2023-01-19

Family

ID=84854862

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/100495 Ceased WO2023284503A1 (zh) 2021-07-14 2022-06-22 全景图像的色调映射方法和装置

Country Status (4)

Country Link
US (1) US20240153051A1 (zh)
EP (1) EP4358018A4 (zh)
CN (1) CN115619648A (zh)
WO (1) WO2023284503A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116542659A (zh) * 2023-04-10 2023-08-04 北京城市网邻信息技术有限公司 资源分配方法、装置、电子设备及存储介质
CN119991795A (zh) * 2024-12-28 2025-05-13 中国电子科技集团公司第十五研究所 用于动态监控的数据处理方法和装置

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
PL4028987T3 (pl) * 2020-05-08 2025-10-20 Huawei Technologies Co., Ltd. Koder, dekoder, system i sposób określania parametrów krzywej mapowania tonalnego
JP2024098411A (ja) * 2023-01-10 2024-07-23 キヤノン株式会社 画像処理装置および画像処理方法
CN118967686B (zh) * 2024-10-16 2024-12-17 北京迈迪斯医疗技术有限公司 医学图像融合数据处理方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106899840A (zh) * 2017-03-01 2017-06-27 北京大学深圳研究生院 全景图像映射方法
US20170256039A1 (en) * 2016-03-07 2017-09-07 Novatek Microelectronics Corp. Method of Processing High Dynamic Range Images Using Dynamic Metadata
CN107230180A (zh) * 2016-10-10 2017-10-03 华为技术有限公司 一种全景图像的映射方法、装置和设备
CN113052923A (zh) * 2021-03-31 2021-06-29 维沃移动通信（深圳）有限公司 色调映射方法、装置、电子设备和存储介质

Family Cites Families (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6753987B1 (en) * 2000-02-25 2004-06-22 Xerox Corporation Systems and methods to determine a contrast and a brightness adjusted system tone reproduction curve
US7492375B2 (en) * 2003-11-14 2009-02-17 Microsoft Corporation High dynamic range image viewing on low dynamic range displays
WO2008136629A1 (en) * 2007-05-03 2008-11-13 Mtekvision Co., Ltd. Image brightness controlling apparatus and method thereof
CN101458824B (zh) * 2009-01-08 2011-06-15 浙江大学 一种基于web的全景图的光照渲染方法
WO2012147303A1 (ja) * 2011-04-25 2012-11-01 Hayashi Mitsuo 全方位画像編集プログラム及び全方位画像編集装置
CN103024300B (zh) * 2012-12-25 2015-11-25 华为技术有限公司 一种高动态范围图像显示方法及装置
JP2016058848A (ja) * 2014-09-08 2016-04-21 ソニー株式会社 画像処理装置及び画像処理方法
GB2537822A (en) * 2015-04-21 2016-11-02 Sharp Kk Liquid crystal display device and a method of driving thereof
US10217189B2 (en) * 2015-09-16 2019-02-26 Google Llc General spherical capture methods
US20170127045A1 (en) * 2015-10-28 2017-05-04 Toppano Co., Ltd. Image calibrating, stitching and depth rebuilding method of a panoramic fish-eye camera and a system thereof
TWI555379B (zh) * 2015-11-06 2016-10-21 輿圖行動股份有限公司 一種全景魚眼相機影像校正、合成與景深重建方法與其系統
US9621767B1 (en) * 2015-11-24 2017-04-11 Intel Corporation Spatially adaptive tone mapping for display of high dynamic range (HDR) images
US10977764B2 (en) * 2015-12-29 2021-04-13 Dolby Laboratories Licensing Corporation Viewport independent image coding and rendering
CN108111851B (zh) * 2016-11-25 2020-12-22 华为技术有限公司 一种去块滤波方法及终端
CN108235031B (zh) * 2016-12-15 2019-11-05 华为技术有限公司 一种运动矢量解码方法及解码器
CN108243333A (zh) * 2016-12-23 2018-07-03 上海微湃电子科技有限公司 全景视频及图片的投影方法和反投影方法
CN108616747A (zh) * 2016-12-23 2018-10-02 上海微湃电子科技有限公司 全景视频及图片的播放方法
US10958942B2 (en) * 2017-02-27 2021-03-23 Koninklijke Kpn N.V. Processing spherical video data
US10643301B2 (en) * 2017-03-20 2020-05-05 Qualcomm Incorporated Adaptive perturbed cube map projection
US10957044B2 (en) * 2017-03-22 2021-03-23 Qualcomm Incorporated Sphere pole projections for efficient compression of 360-degree video
US20180276800A1 (en) * 2017-03-23 2018-09-27 Gopro, Inc. Apparatus and methods for source dynamic range processing of panoramic content
KR102628508B1 (ko) * 2017-05-04 2024-01-24 한국전자통신연구원 영상 처리 장치 및 방법
EP3622483B1 (en) * 2017-05-09 2025-06-25 Koninklijke KPN N.V. Coding spherical video data
US10402952B2 (en) * 2017-06-02 2019-09-03 Apple Inc. Perceptual tone mapping of SDR images for an HDR display
US11049219B2 (en) * 2017-06-06 2021-06-29 Gopro, Inc. Methods and apparatus for multi-encoder processing of high resolution content
WO2019009667A1 (ko) * 2017-07-06 2019-01-10 가온미디어 주식회사 동기화된 영상의 처리 방법 및 그 장치
CN109547766B (zh) * 2017-08-03 2020-08-14 杭州海康威视数字技术股份有限公司 一种全景图像生成方法及装置
CN108520232B (zh) * 2017-08-16 2020-07-24 威盛电子股份有限公司 立体环景影片产生方法及装置
CN107622474B (zh) * 2017-09-26 2021-03-30 北京大学深圳研究生院 基于主视点的全景视频映射方法
EP3462743A1 (en) * 2017-09-27 2019-04-03 Thomson Licensing Device and method for dynamic range expansion in a virtual reality scene
CN108124193A (zh) * 2017-12-25 2018-06-05 中兴通讯股份有限公司 视频处理方法及装置
CN110458755B (zh) * 2018-05-07 2023-01-13 华为技术有限公司 图像处理方法、相关设备及计算机存储介质
CN110956583B (zh) * 2018-09-26 2022-05-10 华为技术有限公司 球面图像处理方法、装置及服务器
US11032607B2 (en) * 2018-12-07 2021-06-08 At&T Intellectual Property I, L.P. Methods, devices, and systems for embedding visual advertisements in video content
US10715783B1 (en) * 2019-03-01 2020-07-14 Adobe Inc. Stereo-aware panorama conversion for immersive media
CN110058777B (zh) * 2019-03-13 2022-03-29 华为技术有限公司 快捷功能启动的方法及电子设备
US11109067B2 (en) * 2019-06-26 2021-08-31 Gopro, Inc. Methods and apparatus for maximizing codec bandwidth in video applications
US11481863B2 (en) * 2019-10-23 2022-10-25 Gopro, Inc. Methods and apparatus for hardware accelerated image processing for spherical projections
US10999527B1 (en) * 2020-02-14 2021-05-04 Gopro, Inc. Generation of enhanced panoramic visual content
WO2021163845A1 (en) * 2020-02-17 2021-08-26 Intel Corporation Enhancing 360-degree video using convolutional neural network (cnn) -based filter
CN115883810B (zh) * 2020-04-30 2024-11-26 华为技术有限公司 映射曲线参数的获取方法和装置
US12372370B2 (en) * 2020-11-23 2025-07-29 Electronics And Telecommunications Research Institute Method and apparatus for generating a map for autonomous driving and recognizing location
KR20220128800A (ko) * 2021-03-15 2022-09-22 에스케이하이닉스 주식회사 파노라마 이미지 생성 장치 및 방법
US20240163549A1 (en) * 2021-03-23 2024-05-16 Kanta SHIMIZU Imaging device, imaging method, and information processing device
CN116309083A (zh) * 2021-12-17 2023-06-23 华为技术有限公司 全景视频的动态范围映射方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170256039A1 (en) * 2016-03-07 2017-09-07 Novatek Microelectronics Corp. Method of Processing High Dynamic Range Images Using Dynamic Metadata
CN107230180A (zh) * 2016-10-10 2017-10-03 华为技术有限公司 一种全景图像的映射方法、装置和设备
CN106899840A (zh) * 2017-03-01 2017-06-27 北京大学深圳研究生院 全景图像映射方法
CN113052923A (zh) * 2021-03-31 2021-06-29 维沃移动通信（深圳）有限公司 色调映射方法、装置、电子设备和存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4358018A4

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116542659A (zh) * 2023-04-10 2023-08-04 北京城市网邻信息技术有限公司 资源分配方法、装置、电子设备及存储介质
CN116542659B (zh) * 2023-04-10 2024-06-04 北京城市网邻信息技术有限公司 资源分配方法、装置、电子设备及存储介质
CN119991795A (zh) * 2024-12-28 2025-05-13 中国电子科技集团公司第十五研究所 用于动态监控的数据处理方法和装置

Also Published As

Publication number Publication date
CN115619648A (zh) 2023-01-17
EP4358018A1 (en) 2024-04-24
US20240153051A1 (en) 2024-05-09
EP4358018A4 (en) 2024-10-30

Similar Documents

Publication Publication Date Title
US20240153051A1 (en) Tone mapping method and apparatus for panoramic image
CN109993823B (zh) 阴影渲染方法、装置、终端及存储介质
CN113596428B (zh) 映射曲线参数的获取方法和装置
CN111932463B (zh) 图像处理方法、装置、设备及存储介质
CN111080656B (zh) 一种图像处理的方法、图像合成的方法以及相关装置
US20170358126A1 (en) Image processing apparatus and method
WO2024011976A1 (zh) 一种扩展图像动态范围的方法及电子设备
CN113126937A (zh) 一种显示终端调整方法及显示终端
US20180241943A1 (en) Electronic device and method for displaying 360-degree image in the electronic device
CN113052923B (zh) 色调映射方法、装置、电子设备和存储介质
WO2023273465A1 (zh) 地图显示方法、装置与终端设备
CN108712641A (zh) 用于基于多面体提供vr图像的电子设备及其图像提供方法
CN110956571B (zh) 基于slam进行虚实融合的方法及电子设备
AU2017320166A1 (en) Image streaming method and electronic device for supporting the same
CN120807567A (zh) 图像分割模型训练方法、图像分割方法、装置及设备
CN117131213B (zh) 图像处理方法及相关设备
CN116453131B (zh) 文档图像矫正方法、电子设备及存储介质
CN114076939B (zh) 一种定位方法、装置及系统
WO2023109442A1 (zh) 全景视频的动态范围映射方法及装置
WO2025140290A1 (zh) 一种图形绘制方法及相关装置
CN120144034A (zh) 一种显示方法和电子设备
CN121707900A (zh) 新视角图像生成方法及电子设备
CN115802072A (zh) 一种视频处理方法、设备及可读存储介质
WO2025140290A9 (zh) 一种图形绘制方法及相关装置
CN120765825A (zh) 图像显示方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22841143

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022841143

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022841143

Country of ref document: EP

Effective date: 20240118

NENP Non-entry into the national phase

Ref country code: DE