WO2022262546A1 - 一种数据处理方法、装置、电子设备和存储介质 - Google Patents

一种数据处理方法、装置、电子设备和存储介质 Download PDF

Info

Publication number
WO2022262546A1
WO2022262546A1 PCT/CN2022/095290 CN2022095290W WO2022262546A1 WO 2022262546 A1 WO2022262546 A1 WO 2022262546A1 CN 2022095290 W CN2022095290 W CN 2022095290W WO 2022262546 A1 WO2022262546 A1 WO 2022262546A1
Authority
WO
WIPO (PCT)
Prior art keywords
attribute
prediction table
attribute information
value
point cloud
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/095290
Other languages
English (en)
French (fr)
Inventor
梁凡
徐贵
吴钊
吴平
蔡品隆
谢绍伟
高莹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Sun Yat Sen University
Original Assignee
ZTE Corp
Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp, Sun Yat Sen University filed Critical ZTE Corp
Priority to EP22824026.3A priority Critical patent/EP4358516A4/en
Priority to US18/570,629 priority patent/US20240289993A1/en
Publication of WO2022262546A1 publication Critical patent/WO2022262546A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/002—Image coding using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/004—Predictors, e.g. intraframe, interframe coding

Definitions

  • the present application relates to the technical field of image processing, and in particular to a data processing method, device, electronic equipment and storage medium.
  • point cloud data describing the 3D structure of objects has been widely used in surveying and mapping, car driving, agriculture, planning and design, archaeology and cultural relic protection, medical treatment and game entertainment.
  • the point cloud data has the characteristics of fast and accurate acquisition, which makes point cloud data especially valued in the field of image processing technology.
  • the amount of point cloud data generated after scanning is often greater than tens of thousands of bits or even higher, which results in the storage and transmission of point cloud data requiring a lot of resources, and the point cloud data needs to be compressed.
  • the current common point cloud data compression methods can be divided into video-based point cloud coding (Video-based Point Cloud Coding, V-PCC) and geometry-based point cloud coding (Geometry-based Point Cloud Coding, G-PCC).
  • V-PCC Video-based Point Cloud Coding
  • G-PCC geometry-based point cloud coding
  • the compression method of G-PCC is usually to convert the point cloud data into geometric information and attribute information, etc., and then encode the geometric information and attribute information into a code stream.
  • the encoding of attribute information is mainly divided into three categories: transformation-based encoding method , mapping-based coding methods, and prediction-based coding methods. Transformation-based encoding methods use the reconstructed geometric information to design attribute transformations to remove the correlation between attribute information.
  • the mapping-based coding method uses the same projection method as the mapping-based geometric coding, and uses video coding technology to code the attribute video after recoloring.
  • the prediction-based coding method is to use the existing attribute information to predict the current attribute information and reduce the coding cost of the current attribute information.
  • the main purpose of the embodiments of the present application is to provide a data processing method, device, electronic device and storage medium, which aims to improve the coding efficiency of three-dimensional point cloud data.
  • An embodiment of the present application provides a data processing method, which includes the following steps: determining a prediction table corresponding to the attribute information of the point cloud; determining a residual and a predicted value index of the attribute information according to the prediction table; encoding the The residual and the predicted value index are sent or stored as a code stream.
  • the embodiment of the present application also provides a data processing method, the method includes the following steps: receiving a code stream and obtaining a residual and a predicted value index in the code stream; determining the corresponding residual and the predicted value index A prediction table: determine the attribute information of the point cloud according to the prediction table, the residual and the predicted value index.
  • the embodiment of the present application also provides a data processing device, the device includes: a prediction table module, used to determine the prediction table corresponding to the attribute information of the point cloud; a residual determination module, used to determine the attribute according to the prediction table Information residual and predictive value index; a data encoding module, used to encode the residual and the predictive value index to form a code stream for transmission or storage.
  • a prediction table module used to determine the prediction table corresponding to the attribute information of the point cloud
  • a residual determination module used to determine the attribute according to the prediction table Information residual and predictive value index
  • a data encoding module used to encode the residual and the predictive value index to form a code stream for transmission or storage.
  • the embodiment of the present application also provides a data processing device, which includes: a code stream receiving module, configured to receive the code stream and obtain the residual and predicted value index in the code stream; a prediction table module, configured to determine the A prediction table corresponding to the residual and the predicted value index; a data decoding module configured to determine the attribute information of the point cloud according to the prediction table, the residual and the predicted value index.
  • a code stream receiving module configured to receive the code stream and obtain the residual and predicted value index in the code stream
  • a prediction table module configured to determine the A prediction table corresponding to the residual and the predicted value index
  • a data decoding module configured to determine the attribute information of the point cloud according to the prediction table, the residual and the predicted value index.
  • the embodiment of the present application also provides an electronic device, the electronic device includes: one or more processors; a memory for storing one or more programs; when the one or more programs are used by the one or more The processor executes, so that the one or more processors implement the data processing method described in any one of the embodiments of the present application.
  • An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein the computer program is executed by a processor to implement the data processing method described in any one of the embodiments of the present application.
  • the prediction table of the point cloud attribute information by determining the prediction table of the point cloud attribute information, processing the attribute information according to the prediction table to obtain the residual and the predicted value index, adding the residual and the predicted value index to the code stream for transmission or storage, according to the point cloud attribute.
  • the characteristics of information Obtain the prediction table, and determine the residual and prediction value index according to the prediction table, which can reduce the coding cost of point cloud attribute information and improve the coding performance of attribute information.
  • FIG. 1 is a flow chart of a data processing method provided in an embodiment of the present application
  • FIG. 2 is a flow chart of a data processing method provided by an embodiment of the present application.
  • FIG. 3 is a flow chart of a data processing method provided by an embodiment of the present application.
  • FIG. 4 is an example diagram of a data processing method provided by an embodiment of the present application.
  • Fig. 5 is a flow chart of another data processing method provided by the embodiment of the present application.
  • Fig. 6 is an example diagram of another data processing method provided by the embodiment of the present application.
  • Fig. 7 is an example diagram of a prediction table transmission provided by an embodiment of the present application.
  • Fig. 8 is a flow chart of another data processing method provided by the embodiment of the present application.
  • FIG. 9 is an example diagram of a data processing method provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a data processing device provided in an embodiment of the present application.
  • Fig. 11 is a schematic structural diagram of a data processing device provided by an embodiment of the present application.
  • Fig. 12 is an example diagram of a data processing device provided by an embodiment of the present application.
  • FIG. 13 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • Fig. 1 is a flow chart of a data processing method provided by the embodiment of the present application.
  • the embodiment of the present application is applicable to the case where the attribute information of the three-dimensional point cloud is encoded, and the method can be executed by the data processing device provided by the embodiment of the present application.
  • the device can be realized by software and/or hardware, referring to Figure 1, the method provided by the embodiment of the present application includes the following steps:
  • Step 110 determine the prediction table corresponding to the attribute information of the point cloud.
  • the point cloud can be composed of a set of discrete point sets randomly distributed in space, expressing the spatial structure and surface properties of three-dimensional objects or scenes.
  • the points in the point cloud also include some additional attributes, such as color, reflectivity, etc. These additional attributes can be called attribute information. Due to the different ways of collecting point clouds, they can be divided into point clouds obtained according to the principle of laser measurement.
  • the attribute information of point clouds can include three-dimensional coordinates (XYZ) and laser reflection intensity (Reflectance); According to the point cloud obtained by the photogrammetry principle, the attribute information of the point cloud can include three-dimensional coordinates and color information (RGB); for the point cloud obtained by combining laser measurement and photogrammetry principles, the attribute information of the point cloud can include three-dimensional coordinates, laser reflection intensity and color information.
  • the prediction table may be an information reference table used when attribute information is compressed, and the prediction table may include predicted values and predicted value indexes of attribute information.
  • the attribute information of the point cloud can be extracted, the features of the attribute information can be extracted, and the corresponding prediction table can be obtained by using the features.
  • the attribute information of the point cloud can be extracted, the value law of the attribute information can be counted, the prediction table corresponding to the value law can be obtained according to the value law, and all the attribute information of the point cloud can be input
  • the prediction result output by the neural network model is used as the prediction table.
  • Step 120 determine the residual of the attribute information and the index of the predicted value according to the prediction table.
  • the residual can be the difference between each attribute information and the predicted value in the prediction table, and the residual can be used for encoding and compression of attribute information
  • the predicted value index can identify the unique identification number of each predicted value in the prediction table
  • the predicted value index can be Used when encoding attribute information.
  • the attribute information can be compared with the corresponding predicted value in the prediction table, the difference between the predicted value and the value of the attribute information can be used as the residual, and the unique identification number of the predicted value can be used as the predicted value index.
  • Step 130 encode the residual and the index of the predicted value to form a code stream for transmission or storage.
  • the code stream may be data formed after encoding and compressing video or image data, and the code stream may be sent between the sending end and the receiving end.
  • the residual and the predicted value index may be compressed and encoded to form a code stream, and the code stream may be sent or stored.
  • the prediction table by determining the prediction table of the point cloud attribute information, processing the attribute information according to the prediction table to obtain the residual and the predicted value index, adding the residual and the predicted value index to the code stream, sending or storing the code According to the characteristics of point cloud attribute information, the prediction table is obtained, and the residual error and prediction value index used for attribute information encoding are determined according to the prediction table, which can reduce the encoding cost of point cloud attribute information and improve the encoding performance of attribute information.
  • Fig. 2 is a flow chart of a data processing method provided by the embodiment of the present application.
  • the embodiment of the present application is based on the embodiment of the above application. Referring to Fig. 2, the method provided by the embodiment of the present application includes the following steps:
  • Step 210 determine point cloud attribute distribution features according to the value of each attribute information.
  • the point cloud attribute distribution feature can be a feature that represents the value distribution of attribute information in the point cloud.
  • the point cloud attribute distribution feature can reflect the dispersion of values.
  • the point cloud attribute distribution feature can use neural networks or statistical histograms, etc. way to determine.
  • the value of all attribute information of the point cloud can be collected, or the value of attribute information of part of the point cloud can be collected, and the corresponding point cloud attribute distribution characteristics can be determined according to the analysis of the value, for example, A statistical histogram of the value of each attribute information is generated, and the distribution of each value in the statistical histogram is used as the point cloud attribute distribution feature.
  • Step 220 generating a prediction table according to the attribute distribution characteristics of the point cloud.
  • the prediction table can be generated according to the point cloud attribute distribution characteristics.
  • the corresponding weights can be generated for the values of each attribute information according to the point cloud attribute distribution characteristics, and the values of one or more attribute information can be filled into the prediction table according to the weights and Generate the predicted value index of each value.
  • the point cloud attribute distribution feature can be input into the pre-trained neural network model, and the predicted value output by the neural network model can be used to form a prediction table.
  • Step 230 determine the residual of the attribute information and the index of the predicted value according to the prediction table.
  • Step 240 encode the residual and the predicted value index to form a code stream for transmission or storage.
  • the attribute distribution characteristics of the point cloud are determined through the value of each attribute information of the point cloud, a prediction table is generated according to the attribute distribution characteristics of the point cloud, and each attribute information is processed according to the prediction table to determine the corresponding residual and prediction value index , which can reduce the encoding cost of point cloud attribute information and improve the encoding performance of attribute information.
  • the point cloud attribute distribution features include at least one of the number of occurrences of attribute values, the mean value of attribute values, and the variance of attribute values.
  • the number of occurrences of the attribute value may represent the total number of occurrences of the value of the attribute information
  • the mean value of the attribute value may be the mean value of the values of the attribute information
  • the variance of the attribute value may be the variance of the values of the attribute information.
  • one or more of the attribute value occurrence times, attribute value mean value, and attribute value variance of each attribute information can be counted as point cloud attribute distribution characteristics.
  • the prediction table of this embodiment is generated using other statistical characteristics.
  • the mean value of the attribute information can be used as the statistical characteristics of the prediction table generation, such as selecting several values near the mean value to form the prediction table;
  • the variance of the attribute information can be used as the statistical characteristic of the prediction table generation, such as sorting according to the variance of the attribute information from small to large, Select the first few attribute values with smaller variance to form the prediction table.
  • Different combinations of statistical properties can be used to generate forecast tables.
  • Fig. 3 is a flow chart of a data processing method provided by the embodiment of the present application.
  • the embodiment of the present application is based on the embodiment of the above application. Referring to Fig. 3, the method provided by the embodiment of the present application includes the following steps:
  • Step 310 Count the number of occurrences of at least one attribute value in various types of attribute information of the point cloud, and use each occurrence number as a point cloud attribute distribution feature.
  • the attribute values of all or part of the attribute information of the point cloud can be extracted, and the number of occurrences of each attribute value can be counted in each attribute information, and the number of occurrences of each attribute value can be used as a point cloud. attribute distribution characteristics.
  • Step 320 among various types of attribute information, sort each attribute value according to the value of each occurrence frequency from high to low.
  • the attribute values may be sorted from high to low according to the appearance and storage of the attribute values in each attribute information. It can be understood that when there are multiple types of attribute information, there may be multiple sorts of attribute values.
  • Step 330 selecting a preset number of attribute values in order from high to low during sorting to fill in the prediction table.
  • the prediction table can be an empty table, and the prediction table can be used to store attribute values.
  • the prediction table can store attribute values of multiple types of attribute information, and the storage upper limit of attribute values of each type of attribute information in the prediction table It can be a preset number, which can be set by the user. After sorting the attribute values of each attribute information, you can select a preset number of attribute values in each sort to fill in the prediction table, and fill in the prediction table for each The attribute value of the prediction table generates a unique identification number as the index of the prediction value.
  • Step 340 determine the residual of the attribute information and the index of the predicted value according to the prediction table.
  • Step 350 encode the residual and the predicted value index to form a code stream for transmission or storage.
  • generating a prediction table according to the point cloud attribute distribution characteristics may also include:
  • the attribute values of attribute information can be sorted from high to low, and attribute values can be selected to fill in the prediction table according to the number of occurrences of each attribute value. For example, the attribute values with more occurrences will be filled in the prediction table. .
  • FIG. 4 is an example diagram of a data processing method provided by the embodiment of the present application.
  • the attribute information that appears different times forms a prediction table, and then finds the predicted value for each attribute information in the point cloud in the prediction table, subtracts the value of the attribute information at the current point from the predicted value to generate a residual, and finally, the predicted value Indexes and residuals in the table are encoded.
  • the processing procedure of the embodiment of the present application includes the following steps:
  • Step S101 Count the occurrence times of attribute information
  • the point cloud refers to a set of data that fully characterizes the spatial structure or attributes of a three-dimensional object or scene, and may be static point cloud data or data at a certain moment in a dynamic point cloud. This data includes geometric information and attribute information.
  • the attribute information is expressed in integer form, for example, the value range of the component value of the color attribute is 0 to 255, then, the number of times 0 to 255 appears in the attribute information of the point cloud is counted.
  • the floating point number can be rounded first, and then the statistical frequency operation can be performed.
  • the rounding operation may adopt a rounding method or a tail removal method.
  • Step S102 Select a number of times attribute information to generate a prediction table
  • the frequency of occurrence of different attribute information can be sorted from high to low, and the top N values with the highest frequency can be selected to generate a prediction table.
  • the attribute information can also be divided into data segments, and M values are selected in different segments to combine to generate a prediction table. Select 14 values from 150 to 150, and select 16 values from 151 to 255, and these 32 values form the prediction table.
  • the prediction table includes an index number and a predicted value, and may also include the number of occurrences of the predicted value.
  • Step S103 Select the predicted value in the prediction table and generate the residual
  • a prediction value is selected in the prediction table for each attribute information of the point cloud, and subtracted to generate a residual.
  • the selection rule can be the closest predicted value. For example, if the attribute information has only one component, then the value of the current attribute information can be subtracted from all the predicted values in the prediction table to obtain the absolute value, and the one with the smallest absolute value is selected as Predictive value. If the attribute information is multiple components, such as the color attribute, then all the components of the current attribute information can be respectively subtracted from the components of all predicted values in the prediction table to take the absolute value and then added, and the one with the smallest sum value is the predicted value .
  • the attribute information is multiple components, such as color attributes
  • only one component of the current attribute information (such as the brightness component) can be selected to subtract the corresponding components of all predicted values in the prediction table to obtain the absolute value, and the one with the smallest absolute value is selected as Predictive value.
  • the attribute information is multiple components, such as color attributes
  • the components of the current attribute information can be weighted and combined first, and then subtracted from the component weights of all predicted values in the prediction table to obtain the absolute value. The weighting method is the same, and the absolute value is selected. The smallest is the predicted value.
  • the selection rule may also be any other selection rule.
  • Step S104 Coding residual and predictor index
  • the generated residual and predicted value can be coded together with other information, such as geometric information, to generate a code stream or a file, or a code stream or file can be generated independently.
  • the encoded data can be generated as a file for storage, or as a code stream for transmission.
  • Fig. 5 is a flow chart of another data processing method provided by the embodiment of the present application.
  • the embodiment of the present application is based on the embodiment of the above application. Referring to Fig. 5, the method provided by the embodiment of the present application includes the following steps:
  • Step 410 extracting current attribute information from the attribute information set of the point cloud.
  • the encoding process of the prediction table and the attribute information can be processed simultaneously, and an unprocessed information can be selected in the attribute information set of the point cloud as the current attribute information.
  • Step 420 determine whether the attribute value of the current attribute information already exists in the prediction table.
  • Step 430 if it exists, update the count value of the attribute value in the prediction table.
  • the count value may reflect the information of the number of occurrences of the predicted value in the prediction table, and the count value may be weighted, for example, the value of the count value may be increased by 2 whenever a predicted value appears.
  • the weights of the count values of different attribute information can be the same or different. When the weights are different, if the attribute value of attribute information 1 appears once, the count value is added by 2, and the attribute value of attribute information 2 appears once, then the count value The value can be increased by 3.
  • the count value of the attribute value in the prediction table may be updated for updating. It can be understood that the count value may be updated according to the weighted value of the current attribute information.
  • Step 440 if not, update the attribute value to the prediction table if the prediction table is not full.
  • the prediction table can be considered to be full; otherwise, the prediction table is not full.
  • the forecast value can be added to the forecast table.
  • Step 450 Determine the residual of the attribute information and the index of the predicted value according to the prediction table.
  • Step 460 encode the residual and the predicted value index to form a code stream for transmission or storage.
  • updating the attribute value to the prediction table includes at least one of the following:
  • the threshold time may be a time period for caching attribute values, and the threshold time may be set by a user.
  • the attribute value that occurs most frequently within a period of time can be updated to the prediction table. This period of time can be determined by Determined by how the user sets the length of time for the threshold time. It is also possible to replace the attribute value with the longest stored prediction value in the prediction table according to the first-in-first-out principle.
  • the attribute value stored in the prediction table can also be replaced with the predicted value, and the attribute value whose count value is the attribute value with the smallest count value in the prediction table.
  • the prediction table may not be generated after traversing all attribute information of the point cloud, but may be dynamically generated while selecting attribute information one by one, which can save time for generating the prediction table.
  • Fig. 6 is an example diagram of another data processing method provided by the embodiment of the present application. Referring to Fig. 6, the data processing method may include the following steps:
  • Step S201 Select an attribute information
  • Step S202 Whether the prediction table has this attribute information value
  • this step is to dynamically update the prediction table.
  • Step S203 Whether the prediction table is full
  • Pre-set the maximum number of predicted values contained in the prediction table If the current attribute value is not in the prediction table, then according to whether the number of predictions currently contained in the prediction table is greater than the maximum number of predictions, if it is greater than the maximum number of predictions, then The prediction table is full, otherwise it is not full.
  • Step S204 Add current attribute information to the prediction table
  • step S203 judges that the prediction table is not full, then add the current attribute value into the prediction table and record the count.
  • Step S205 Update the count of this attribute information value in the prediction table
  • step S202 judges that the prediction table contains the current attribute information value, then the count of the attribute information value in the prediction table is updated.
  • the count update can be the number plus one, or a different number can be added according to the importance of the current attribute information, for example, the count plus two for important attribute information.
  • the prediction table can adopt different update principles, for example, regardless of the number of occurrences of the prediction value, the queue principle of first-in-first-out is adopted, and only the prediction value of the largest number of predictions that occurs recently is saved, or it can also be combined with prediction The number of times the value appears.
  • the new attribute information value only replaces the predicted value with the least count in the table.
  • Step S206 Select predicted value
  • step S202 judges that the current attribute information is the predicted value, you can directly use the current attribute information as the predicted value, otherwise, you need to select the predicted value in the prediction table, the selection rule is the same as the selection in step S103 Same rules.
  • Step S207 Calculate the residual
  • Step S208 Whether there is any attribute information
  • step S209 can be performed; otherwise, step S201 can be performed.
  • Step S209 Coding residual and index
  • step S104 It is also possible to encode the current attribute information residual and index frame after step S207 is performed, instead of waiting for all attribute information to be traversed before performing residual and index encoding.
  • the prediction table corresponding to the attribute information of the point cloud is determined, including:
  • the attribute information of the point cloud can be divided into multiple attribute information groups, and each attribute information group has its own corresponding prediction table.
  • the attribute information of the point cloud can be divided into multiple attribute information groups according to the type of attribute information, and each attribute information group can generate a corresponding prediction table according to the characteristics of the respective attribute information.
  • the attribute information of the point cloud can be divided into multiple attribute information groups according to the value of the attribute information.
  • the attribute information with similar variance can be divided into the same attribute information group.
  • Each attribute information group has its own prediction table.
  • all attribute information of the point cloud is divided into at least two attribute information groups, including:
  • the attribute information is divided into corresponding attribute information groups according to the geometric information of the point cloud.
  • the geometric information may be information reflecting the positional relationship of the point cloud position points in space, and may include coordinates and relative distances.
  • the attribute information of each location point can be divided into multiple sets of attribute information groups according to the coordinates of each location point in the point cloud or the relative distance to other location points, wherein the number of divided attribute information groups It can be set by the user or determined by the number of categories of attribute information.
  • the geometric information includes at least one of the following: Morton code sorting, Hilbert code sorting, octree block or layering of point cloud three-dimensional coordinates, point cloud three-dimensional coordinates Chunking or layering of a K-D tree.
  • the statistical range of point cloud attribute information can be changed from all data to partial data, and point cloud attribute information can be sorted according to geometric information first, and then the sorted point cloud attributes can be divided into For each group of N, use the method of the above embodiment to generate its own prediction table for each group of attribute information.
  • the geometric information can be sorted according to Morton code or Hilbert code, then the attribute information is sorted according to the corresponding geometric information, and 512 points are selected as a group, and then a group of point cloud attribute information is counted separately, and the selected occurrence Generate prediction tables for the 32 attribute information with the most number of times, and then select prediction values for these 512 points, generate residuals, and encode the residuals and corresponding index values.
  • geometric information can also be divided into blocks or layers in the form of octree or K-D tree.
  • the attribute information of the block or layered area is grouped into one group, and then a group of point cloud attribute information is counted separately.
  • the prediction table includes at least attribute values, wherein the attribute values include at least one attribute component value.
  • the prediction table may be composed of attribute values, and each attribute value may have one or more attribute components.
  • one attribute value is a color value
  • the color value is a color in RGB mode.
  • the attribute components may be respectively are the color components of different colors in RGB.
  • the prediction table may include an index and a prediction value, where the prediction value may be one component or multiple components.
  • the prediction table can also contain statistical information, such as the number of occurrences of each prediction value, the variance of each prediction value, etc.; an organizational form of the prediction table can be shown in Table 1, where the order of the prediction values is the index, so it is not used in The index value is stored separately in the prediction table. If the index value is not the sequence number of the predicted value arrangement, then the index value needs to be added.
  • numOfPredictor indicates that there are several predicted values
  • numOfPredictorComponent indicates that a predicted value contains several components
  • valueOfPredictor indicates the predicted value
  • the prediction table may be placed in an attribute information header or an attribute slice header (slice header) or an attribute tile header (tile header), or other parameter sets that can indicate attributes of attribute information, as shown in Table 2.
  • withPredict indicates whether to use the predictive coding method, 1 indicates that it is used, and 0 indicates that it is not used; withPredictMap indicates whether the prediction table is included, 1 indicates that it is included, and 0 indicates that it is not included.
  • the prediction table may also adopt other organizational forms, as shown in Table 3.
  • withPredict indicates whether to use the predictive coding method, 1 indicates that it is used, and 0 indicates that it is not used; withPredictMap indicates whether the prediction table is included, 1 indicates that it is included, and 0 indicates that it is not included.
  • Prediction tables can also be in the form of ISO/IEC 14496-12 ISO BMFF.
  • the prediction table can be indicated on the existing Attribute track, and the specific content can be put together with the attribute information.
  • the instruction information in Attribute is implemented as follows:
  • gpcc_type 4 means attribute information
  • predictor_map_present 1 means that there is a prediction table
  • 0 means that there is no prediction table.
  • the prediction table index and the residual exist in the attribute information data can be shown in Table 4.
  • Attr_predict_idx represents the index value
  • attr_predict_residual represents the residual value
  • the number of prediction tables may include at least one.
  • a new table is established for the outliers in the point cloud, where the outliers may mean that the value of the attribute information is very different from all the values in the prediction table, and there may be multiple outliers. It is also possible to combine isolated points and existing prediction tables into one prediction table.
  • a prediction table is established for different components of attribute information, such as color attribute RGB space, red (R), green (G) and blue (B) all have their own prediction tables, point cloud data
  • attribute information such as color attribute RGB space, red (R), green (G) and blue (B) all have their own prediction tables, point cloud data
  • RGB space red (R), green (G) and blue (B) all have their own prediction tables
  • B blue
  • point cloud data The color attribute of each point of is represented using three index values.
  • a prediction table is set in the attribute information header in Embodiment 5, and different attribute fragment headers also have their own prediction tables, which can be the supplement of the attribute information header prediction table, that is, the attribute fragment header.
  • the prediction table in the attribute information header and the prediction table of the attribute information header jointly generate a prediction table, or the prediction table of the attribute fragment header records the difference with the prediction table of the attribute information header, that is, the prediction table in the attribute fragment header needs to be consistent with the attribute
  • the header prediction tables are added to generate the final prediction table used by the attribute slice header.
  • the attribute information includes color attributes
  • the prediction table includes a color palette
  • the attribute component values of the color palette are set with weights.
  • the attribute information is a color attribute, and the color may be in different color spaces, such as luminance chromaticity (YUV), RGB, and the like.
  • the prediction table based on color attributes may be a palette.
  • the prediction table is dominated by the Y component, and the weight of the Y component is higher than that of the UV component when calculating the statistical characteristics of attribute information.
  • the weights of the three components of the RGB color space are the same, then the weights of the three components are the same when calculating the statistical characteristics of attribute information.
  • the prediction table corresponding to the attribute information of the point cloud is determined, including:
  • the prediction table is generated based on a preset neural network, wherein the neural network is pre-trained and generated through attribute information of a threshold number of point clouds, and the attribute information set includes attribute values and predicted values.
  • the threshold number of the attribute information for training the neural network can be a larger value, the larger the threshold number, the higher the accuracy of the neural network, and the threshold number can be set by the user based on experience.
  • the prediction table may be generated by a neural network, and the neural network may be generated by training the attribute information of the point cloud, and the attribute information used for training the neural network may include predicted values and attribute values.
  • a neural network may be used to train and generate the prediction table.
  • the neural network is trained by using the existing point cloud data set to obtain the prediction table with the best performance.
  • Different types of point clouds can be trained in a targeted manner to obtain multiple prediction tables, for example, a prediction value is specially trained for the reflectivity of the car map.
  • the determination of the residual and predicted value index of the attribute information according to the prediction table includes:
  • a prediction value and a prediction value index are selected in the prediction table according to the attribute value of each attribute information.
  • the difference between the predicted value and the attribute value of the attribute information is used as the residual.
  • the prediction value corresponding to the attribute information may be determined in the prediction table, and the difference between the prediction value and the attribute value of the attribute information may be used as a residual.
  • the process of determining the predicted value of the attribute information may be to obtain the corresponding predicted value according to the category or value of the attribute information, and the index number associated with the predicted value stored in the prediction table may be used as the predicted value index.
  • the selection of the predicted value and the predicted value index in the predicted table according to the attribute value of each attribute information includes:
  • the difference and/or weighted difference between the attribute value and each of the attribute values in the prediction table, and the prediction table corresponding to the difference and/or the weighted difference with the smallest absolute value The attribute value in the table is used as the predicted value of the attribute information, and the index of the attribute value in the predicted table is used as the predicted index value index.
  • each attribute value in the prediction table can also have its own weighted weight, and the weighted difference value of each difference can be determined according to the weighted weight, and the weighted difference value with the smallest absolute value can be assigned to the corresponding attribute in the prediction table value as the predictor and the index of that predictor as the predictor index.
  • the prediction table and attribute information may be encoded together in a code stream for transmission, or may be transmitted separately.
  • the prediction table can be sent to the decoder when the link between the encoder and the decoder is established, and the prediction table can also be updated during the interaction process.
  • Fig. 7 is an example diagram of transmission of a prediction table provided by the embodiment of the present application. Referring to Fig. 7, the prediction table can be placed on the server in advance, the encoding end only sends the prediction table number, and the decoding end sends the prediction table number to the server according to the demand. Request to send forecast table.
  • the prediction table may also be stored or transmitted without encoding.
  • the prediction table can be dynamically generated during the decoding process of the decoder, and the prediction table generated during the decoding process of the decoder is exactly the same as the prediction table generated during the encoding process of the encoder. Encode and store or transmit.
  • Fig. 8 is a flow chart of another data processing method provided by the embodiment of the present application.
  • the embodiment of the present application is applicable to the situation where the attribute information of the three-dimensional point cloud is decoded.
  • This method can be implemented by the data processing device provided by the embodiment of the present application. Execution, the device can be implemented by software and/or hardware, see Figure 8, the method provided by the embodiment of the present application includes the following steps:
  • Step 510 receiving the code stream and obtaining the residual and the index of the predicted value in the code stream.
  • the residual and the predicted value index can be parsed from the code stream.
  • Step 520 determine the prediction table corresponding to the residual and the prediction value index.
  • the prediction table is an information table for the prediction value used for decoding, which can be obtained by parsing from the attribute information header or attribute slice header or attribute tile header or other parameter sets that can indicate the characteristics of attribute information, or according to the residual in the code stream and predictor indexes are generated dynamically.
  • the number of prediction tables can be one or more.
  • Step 530 determine the attribute information of the point cloud according to the prediction table, the residual and the index of the predicted value.
  • the corresponding predicted value can be searched in the prediction table according to the predicted value index, and the value of the attribute information of the point cloud can be determined according to the residual value and the predicted value.
  • the residual value and the predicted value can be combined
  • the sum of the values is the attribute value of the attribute information of the point cloud.
  • the attribute information of the point cloud is determined according to the prediction table, residual and predicted value index.
  • the code stream by receiving the code stream and obtaining the residual and predicted value index in the code stream, dynamically generate a prediction table according to the residual and predicted value index, and determine the attribute information of the point cloud according to the prediction table, residual and predicted value index , realizing the fast decoding of point cloud attribute information, which can enhance the transmission efficiency of point cloud.
  • FIG. 9 is an example diagram of a data processing method provided by the embodiment of the present application.
  • the prediction table information is first obtained from the code stream or file, and then the code stream or file Obtain the index information and residual, find the corresponding predicted value in the prediction table according to the index information, and finally calculate the attribute information value from the predicted value and residual.
  • the method provided by the embodiment of the present application includes the following steps:
  • Step S301 Obtain prediction table information.
  • the prediction table information can exist in the attribute information header, in the attribute slice header, in the attribute tile header, or in other parameter sets that can indicate the characteristics of the attribute information. It can exist in the transport layer media description unit. When the prediction table is dynamically generated at the decoder, this step may not be required.
  • Step S302 Obtain an index value and a residual.
  • Step S303 Select the prediction value corresponding to the prediction index from the prediction table.
  • the corresponding prediction value is searched in the prediction table obtained in step S301.
  • Step S304 Calculate the attribute information value from the residual and the predicted value.
  • the residual obtained in step S302 is added to the predicted value obtained in step S303 to generate an attribute information value.
  • Fig. 10 is a schematic structural diagram of a data processing device provided by an embodiment of the present application, which can execute the data processing method provided by any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the method.
  • the device can be implemented by software and/or Implemented by hardware, including: a prediction table module 501 , a residual determination module 502 and a data encoding module 503 .
  • the prediction table module 501 is configured to determine a prediction table corresponding to the attribute information of the point cloud.
  • the residual determining module 502 is configured to determine the residual of the attribute information and the index of the predicted value according to the prediction table.
  • a data encoding module 503, configured to encode the residual and the predicted value index to form a code stream for transmission or storage.
  • the prediction table of the point cloud attribute information is determined by the prediction table module, and the residual determination module processes the attribute information according to the prediction table to obtain the residual and the predicted value index, and obtains the prediction table according to the characteristics of the point cloud attribute information, According to the prediction table, the residual and prediction value index used for attribute information encoding can be determined, which can reduce the encoding cost of point cloud attribute information and improve the encoding performance of attribute information.
  • the prediction table module 501 includes:
  • the distribution feature unit is used to determine the attribute distribution feature of the point cloud according to the value of each attribute information.
  • a table generation unit configured to generate a prediction table according to the point cloud attribute distribution characteristics.
  • the distribution feature unit includes:
  • the number counting subunit is configured to count the number of occurrences of at least one attribute value in various types of attribute information of the point cloud, and use each of the number of occurrences as the attribute distribution feature of the point cloud.
  • the point cloud attribute distribution feature in the distribution feature unit includes at least one of the number of occurrences of the attribute value, the mean value of the attribute value, and the variance of the attribute value.
  • the table generation unit is configured to: sort the attribute values according to the value of each occurrence number from high to low among the various types of attribute information; During sorting, a preset number of attribute values are selected in descending order to fill in the prediction table.
  • the table generating unit is further configured to: sort the attribute values from high to low in various types of attribute information; Selecting the corresponding number of attribute values in the sorting is filled into the prediction table.
  • the prediction table module 501 also includes:
  • an information extraction unit configured to extract current attribute information from the attribute information set of the point cloud
  • an information judging unit configured to determine whether the attribute value of the current attribute information already exists in the prediction table
  • An information processing unit configured to update the count value of the attribute value in the prediction table if it exists; if not, update the attribute value to the prediction if the prediction table is not full surface.
  • the information processing unit updates the attribute value to the prediction table, including at least one of the following:
  • the prediction table module 501 also includes:
  • a grouping unit configured to divide all attribute information of the point cloud into at least two attribute information groups.
  • a table generating unit configured to determine a prediction table corresponding to each attribute information group. Further, on the basis of the above-mentioned application embodiments, the grouping unit is specifically configured to: divide each attribute information into corresponding attribute information groups according to the geometric information of the point cloud.
  • the geometric information includes at least one of the following: Morton code sorting, Hilbert code sorting, octree block or layering of point cloud three-dimensional coordinates, point cloud three-dimensional coordinates Chunking or layering of a K-D tree.
  • the prediction table includes at least attribute values, wherein the attribute values include at least one attribute component value.
  • the attribute information includes color attributes
  • the prediction table includes a color palette
  • the attribute component values of the color palette are set with weights.
  • the prediction table corresponding to the attribute information of the point cloud is determined, including:
  • the prediction table is generated based on a preset neural network, wherein the neural network is pre-trained and generated by a threshold amount of point cloud attribute information, and the attribute information includes attribute values and predicted values.
  • the residual determination module 502 includes:
  • the table difference unit is configured to select a predicted value and a predicted value index in the predicted table according to the attribute value of each attribute information.
  • a residual processing unit configured to use the difference between the predicted value and the attribute value of the attribute information as a residual.
  • the residual processing unit is used to calculate the difference and/or weighted difference between the attribute value and each attribute value in the prediction table, and the difference with the smallest absolute value value and/or the attribute value in the prediction table corresponding to the weighted difference value is used as the predicted value of the attribute information, and the index of the attribute value in the prediction table is used as the predicted value index.
  • Fig. 11 is a schematic structural diagram of a data processing device provided by an embodiment of the present application, which can execute the data processing method provided by any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the method.
  • the device can be implemented by software and/or Hardware implementation, including: code stream receiving module 504 , prediction table module 505 and data decoding module 506 .
  • the code stream receiving module 504 is configured to receive the code stream and obtain the residual and prediction value index in the code stream.
  • a prediction table module 505 configured to determine a prediction table corresponding to the residual and the predicted value index.
  • the data decoding module 506 is configured to determine the attribute information of the point cloud according to the prediction table, the residual and the predicted value index.
  • the code stream receiving module receives the code stream and obtains the residual and predicted value index in the code stream
  • the prediction table module obtains the prediction table according to the residual and the predicted value index
  • the data decoding module obtains the prediction table according to the prediction table, residual and The predicted value index determines the attribute information of the point cloud, realizes the fast decoding of the point cloud attribute information, and can enhance the transmission efficiency of the point cloud.
  • FIG. 12 is an example diagram of a data processing device provided in an embodiment of the present application.
  • the device includes:
  • Forecast table module A01 used to generate a forecast table.
  • Attribute encoding module A02 used to generate residual information and encode residual information and indexes.
  • Transmission module A03 used to transmit the data after attribute encoding, and also encode and transmit the prediction table.
  • Forecast table module A04 used to establish or dynamically generate a forecast table.
  • Attribute decoding module A05 for decoding attribute information, including residual information and indexes.
  • Transmission module A06 used to transmit the compressed data of attribute information, and also decode the prediction table.
  • Figure 13 is a schematic structural diagram of an electronic device provided by an embodiment of the present application, the electronic device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the electronic device can be one or more
  • a processor 60 is taken as an example; the processor 60, memory 61, input device 62 and output device 63 in the electronic device can be connected through a bus or in other ways.
  • the connection through a bus is taken as an example.
  • Memory 61 can be used to store software programs, computer-executable programs and modules, such as modules corresponding to the data processing device in the embodiment of the present application (prediction table module 501, residual determination module 502 and data encoding module 503, or code stream receiving module 504, prediction table module 505 and data decoding module 506).
  • the processor 60 executes various functional applications and data processing of the electronic device by running software programs, instructions and modules stored in the memory 61 , that is, implements the above-mentioned data processing method.
  • the memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required by a function; the data storage area may store data created according to the use of the electronic device, and the like.
  • the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices.
  • the memory 61 may also include a memory that is remotely located relative to the processor 60, and these remote memories may be connected to the electronic device through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • the input device 62 can be used to receive input numbers or character information, and generate key signal input related to user settings and function control of the electronic device.
  • the output device 63 may include a display device such as a display screen.
  • the embodiment of the present application also provides a storage medium containing computer-executable instructions, the computer-executable instructions are used to execute a data processing method when executed by a computer processor, the method comprising:
  • the present application can be realized by means of software and necessary general-purpose hardware, and of course it can also be realized by hardware, but in many cases the former is a better implementation .
  • the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product can be stored in a computer-readable storage medium, such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disc, etc., including several instructions to make a computer device (which can be a personal computer) , server, or network device, etc.) execute the method described in each embodiment of the present application.
  • a computer-readable storage medium such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disc, etc.
  • the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical components. Components cooperate to execute.
  • Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit .
  • a processor such as a central processing unit, digital signal processor, or microprocessor
  • Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
  • computer storage media includes both volatile and nonvolatile media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. permanent, removable and non-removable media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridges, tape, magnetic disk storage or other magnetic storage devices, or can Any other medium used to store desired information and which can be accessed by a computer.
  • communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本申请实施例提供了一种数据处理方法、装置、电子设备和存储介质，其中，该方法包括：确定点云的属性信息对应的预测表(110)；根据所述预测表确定所述属性信息的残差和预测值索引(120)；编码所述残差和所述预测值索引以形成码流发送或存储(130)。

Description

一种数据处理方法、装置、电子设备和存储介质
相关申请的交叉引用
本申请基于申请号为202110662694.5、申请日为2021年06月15日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种数据处理方法、装置、电子设备和存储介质。
背景技术
随着万物互联时代的，多种多样的数据在不同设备间传输。随着三维扫描技术和系统的日趋成熟，描述物体三维结构的点云数据在测绘、汽车驾驶、农业、规划设计、考古与文物保护、医疗和游戏娱乐等方面得到广泛应用。而点云数据具有获取快速且精准的特点，使得点云数据在图像处理技术领域中尤其得到重视。然而经过扫描后生成的点云数据量往往大于几万比特甚至更高，这导致点云数据的存储和传输需要花费大量资源，需要对点云数据进行压缩。
目前常见的点云数据压缩方法可以分为基于视频的点云编码(Video-based Point Cloud Coding，V-PCC)和基于几何的点云编码(Geometry-based Point Cloud Coding，G-PCC)。其中，G-PCC的压缩方法通常为将点云数据转换为几何信息和属性信息等，然后将几何信息和属性信息编码为码流，对于属性信息编码主要分为三类：基于变换的编码方法、基于映射的编码方法以及基于预测的编码方法。基于变换的编码方法是利用重建的几何信息来设计属性变换，以去除属性信息之间的相关性。基于映射的编码方法是基于映射的几何编码采用相同的投影方式，在使用视频编码技术对重着色之后的属性视频进行编码。基于预测的编码方法是利用已有的属性信息对当前属性信息进行预测，降低当前属性信息的编码代价。点云数据的属性信息按照基于预测的编码方法进行编码时，由于三维空间与视频的特征较大，导致属性编码的编码性能较低，目前亟需一种基于三维点云数据的特性进行编码的技术方案。
发明内容
本申请实施例的主要目的在于提出一种数据处理方法、装置、电子设备和存储介质，其旨在提高三维点云数据的编码效率。
本申请实施例提供了一种数据处理方法，该方法包括以下步骤：确定点云的属性信息对应的预测表；根据所述预测表确定所述属性信息的残差和预测值索引；编码所述残差和所述预测值索引以形成码流发送或存储。
本申请实施例还提供了一种数据处理方法，该方法包括以下步骤：接收码流并获取所述码流中的残差和预测值索引；确定所述残差和所述预测值索引对应的预测表；按照所述预测表、所述残差和所述预测值索引确定点云的属性信息。
本申请实施例还提供了一种数据处理装置，该装置包括：预测表模块，用于确定点云的属性信息对应的预测表；残差确定模块，用于根据所述预测表确定所述属性信息的残差和预测值索引；数据编码模块，用于编码所述残差和所述预测值索引以形成码流发送或存储。
本申请实施例还提供了一种数据处理装置，该装置包括：码流接收模块，用于接收码流并获取所述码流中的残差和预测值索引；预测表模块，用于确定所述残差和所述预测值索引对应的预测表；数据解码模块，用于按照所述预测表、所述残差和所述预测值索引确定点云的属性信息。
本申请实施例还提供了一种电子设备，该电子设备包括：一个或多个处理器；存储器，用于存储一个或多个程序；当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如本申请实施例中任一所述的数据处理方法。
本申请实施例还提供了一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行实现如本申请实施例中任一所述的数据处理方法。
本申请实施例，通过确定出点云属性信息的预测表，按照预测表处理属性信息以获取残差和预测值索引，将残差和预测值索引添加到码流发送或存储，根据点云属性信息的特点获取预测表，按照预测表确定残差和预测值索引，可降低点云属性信息编码代价，提高属性信息的编码性能。
附图说明
图1是本申请实施例提供的一种数据处理方法的流程图；
图2是本申请实施例提供的一种数据处理方法的流程图；
图3是本申请实施例提供的一种数据处理方法的流程图；
图4是本申请实施例提供的一种数据处理方法的示例图；
图5是本申请实施例提供的另一种数据处理方法的流程图；
图6是本申请实施例提供的另一种数据处理方法的示例图；
图7是本申请实施例提供的一种预测表传输的示例图；
图8是本申请实施例提供的另一种数据处理方法的流程图；
图9是本申请实施例提供的一种数据处理方法的示例图；
图10是本申请实施例提供的一种数据处理装置的结构示意图；
图11是本申请实施例提供的一种数据处理装置的结构示意图；
图12是本申请实施例提供的一种数据处理装置的示例图；
图13是本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在后续的描述中，使用用于表示元件的诸如“模块”、“部件”或“单元”的后缀仅为了有利于本申请的说明，其本身没有特有的意义。因此，“模块”、“部件”或“单元”可以混合地使用。
图1是本申请实施例提供的一种数据处理方法的流程图，本申请实施例可适用于三维点云的属性信息进行编码的情况，该方法可以由本申请实施例提供的数据处理装置来执行，该装置可以通过软件和/或硬件方式实现，参见图1，本申请实施例提供的方法包括如下步骤：
步骤110、确定点云的属性信息对应的预测表。
其中，点云可以是由空间中一组无规则分布、表达三维物体或场景空间结构和表面属性的离散点集所构成的。点云中的点除了几何坐标还包括了一些附加属性，比如颜色，反射率等。这些附加属性可以被称为属性信息，由于采集点云的方式不同，可以分为根据激光测量原理得到的点云，点云的属性信息可以包括三维坐标(XYZ)和激光反射强度(Reflectance)；根据摄影测量原理得到的点云，点云的属性信息可以包括三维坐标和颜色信息(RGB)；结合激光测量和摄影测量原理得到的点云，点云的属性信息可以包括三维坐标、激光反射强度和颜色信息。预测表可以是在属性信息压缩时使用的信息参照表，预测表中可以包括属性信息的预测值以及预测值索引。
在本申请实施例中，可以提取到点云的属性信息，可以提取属性信息的特征，可以使用特征获取到对应的预测表。示例性的，可以提取点云的所有的属性信息，可以统计属性信息的取值规律，可以按照该取值规律获取与该取值规律对应的预测表，还可以将点云的所有属性信息输入到预先训练的神经网络模型，将神经网络模型输出的预测结果作为预测表。
步骤120、根据预测表确定属性信息的残差和预测值索引。
其中，残差可以是各属性信息与预测表中预测值的差值，残差可以用于属性信息的编码压缩，预测值索引可以标识预测表中各预测值的唯一标识号，预测值索引可以在属性信息编码时使用。
可以将属性信息与预测表中对应的预测值进行比较，可以将预测值与属性信息的取值的差值作为残差，并将该预测值的唯一标识号作为预测值索引。
步骤130、编码残差和预测值索引以形成码流发送或存储。
其中，码流可以是视频或图像数据经过编码压缩后形成的数据，码流可以在发送端和接收端之间发送。
在本申请实施例中，可以将残差和预测值索引进行压缩编码后形成码流，可以将该码流发送或存储。
本申请实施例中，通过确定出点云属性信息的预测表，按照预测表处理属性信息以获取残差和预测值索引，并将残差和预测值索引添加到码流，发送或存储该码流，根据点云属性信息的特点获取预测表，按照预测表确定属性信息编码使用的残差和预测值索引，可降低点云属性信息编码代价，提高属性信息的编码性能。
图2是本申请实施例提供的一种数据处理方法的流程图，本申请实施例是在上述申请实施例基础上的具体化，参见图2，本申请实施例提供的方法包括如下步骤：
步骤210、按照各属性信息的取值确定点云属性分布特征。
其中，点云属性分布特征可以是表示点云中属性信息的取值分布情况的特征，点云属性分布特征可以反映取值的分散情况，点云属性分布特征可以借助神经网络或者统计直方图等方式确定。
在本申请实施例中，可以采集点云所有的属性信息的取值，或者采集部分点云的属性信息的取值，可以按照该取值进行分析确定对应的点云属性分布特征，例如，可以生成各属性信息取值的统计直方图，将统计直方图中的各取值的分布情况作为点云属性分布特征。
步骤220、根据点云属性分布特征生成预测表。
可以按照点云属性分布特征生成预测表，例如，可以根据点云属性分布特征为各属性信息的取值生成对应的权重，可以按照权重将一个或者多个属性信息的取值填充到预测表并生成各取值的预测值索引，又例如，可以将点云属性分布特征输入到预先训练的神经网络模型，可以将神经网络模型输出的预测值构成预测表。
步骤230、根据预测表确定属性信息的残差和预测值索引。
步骤240、编码残差和预测值索引以形成码流发送或存储。
本申请实施例，通过点云的各属性信息的取值确定点云属性分布特征，按照点云属性分布特征生成预测表，并按照预测表处理各属性信息以确定对应的残差和预测值索引，可降低点云属性信息编码代价，提高属性信息的编码性能。
在上述申请实施例的基础上，所述点云属性分布特征包括属性值出现次数、属性值均值、属性值方差中至少之一。
其中，属性值出现次数可以表示属性信息的取值的出现总次数，属性值均值可以是属性信息的取值的均值，属性值方差可以是属性信息的取值的方差。
具体的，可以统计各属性信息的属性值出现次数、属性值均值、属性值方差中的一种或者多种作为点云属性分布特征。
在一个示例性的实施方式中，本实施例的预测表是利用其它统计特性来生成。可以按照属性信息的均值作为预测表生成的统计特性，比如选取均值附近的若干值组成预测表；可以按照属性信息的方差作为预测表生成的统计特性，比如按照属性信息的方差从小到大排序，选取前几位较小方差的属性值组成预测表。可以采用不同的统计特性组合来生成预测表。
图3是本申请实施例提供的一种数据处理方法的流程图，本申请实施例是在上述申请实施例基础上的具体化，参见图3，本申请实施例提供的方法包括如下步骤：
步骤310、在点云的各类属性信息中统计至少一个属性值的出现次数，并将各出现次数作为点云属性分布特征。
在本申请实施例中，可以提取点云的全部或者部分的属性信息的属性值，可以在各属性信息中统计各属性值统计各自出现的出现次数，可以将各属性值的出现次数作为点云属性分布特征。
步骤320、在各类属性信息中，按照各出现次数的取值从高到低对各属性值排序。
可以按照各属性信息中的属性值的出现存储分别从高到低对属性值进行排序，可以理解的是，在存在多类属性信息时，可以存在多个属性值排序。
步骤330、在排序中按照从高到低的顺序选择预设数量的属性值填充到预测表。
在本申请实施例中，预测表可以为一个空表，该预测表可以用于存储属性值，预测表可以存储多类属性信息的属性值，预测表中每类属性信息的属性值的存储上限可以为预设数量，该 预设数量可以由用户设置，在对各属性信息的属性值排序后，可以在各排序中分别选择预设数量的属性值填充到预测表，并为每个填充到预测表的属性值生成唯一的标识号作为预测值索引。
步骤340、根据预测表确定属性信息的残差和预测值索引。
步骤350、编码残差和预测值索引以形成码流发送或存储。
本申请实施例，通过在各属性信息中统计属性值的出现次数，并基于该出现次数按照从高到低的顺序对属性值进行排序，按照顺序在排序中选择阈值数量的属性值填充到预测表，并按照预测表处理属性信息以获取残差和预测值索引，可降低点云属性信息编码代价，提高属性信息的编码性能。
在上述申请实施例的基础上，根据所述点云属性分布特征生成预测表，还可以包括：
在各类所述属性信息中，将各所述属性值从高到低排序；按照各所述属性值对应的出现次数在所述排序中选择对应数量的所述属性值填充到预测表。
可以将属性信息的属性值从高到低进行排序，可以按照各属性值的出现次数选择属性值填充到预测表，例如，出现次数越多的属性值则被填充到预测表中的数量越多。
在一个示例性的实施方式中，图4是本申请实施例提供的一种数据处理方法的示例图，参见图4，本申请实施例先统计点云属性信息不同值出现的次数，再将若干出现不同次数的属性信息组成预测表，接着在预测表中给点云中每一个属性信息寻找预测值，将当前点上属性信息的值减去预测值生成残差，最后，将预测值在预测表中的索引和残差进行编码。本申请实施例的处理过程包括如下步骤：
步骤S101：统计属性信息出现次数
遍历点云的所有属性信息，对不同的属性信息分别计数，获得不同属性信息在此点云中出现的次数。
所述点云是指一组完整表征三维物体或场景空间结构或属性的数据，可以是静态点云数据，也可以是动态点云某一时刻的数据。此数据包括几何信息和属性信息。
当属性信息以整数形式表示时，比如颜色属性的分量值的取值范围是0到255，那么，就是对0到255在点云的属性信息中出现的次数进行统计。
当属性信息以浮点数形式表示时，可以先对浮点数进行取整操作，再进行统计频率操作。其中，取整操作可以采用四舍五入法，也可以采用去尾法。
步骤S102：选取若干次数属性信息生成预测表
根据一定的规则在步骤S101按次数统计生成的不同属性信息中选取若干个值组成预测表。
可以将不同属性信息出现的次数从高到低排序，选取最高次数的前N个值生成预测表。
也可以将属性信息进行数据分段，在不同分段中分别选取M个值组合生成预测表，比如针对取值范围是0到255的颜色属性，在0至16中选取1个值，在17到150中选取14个值，在151到255中选取16个值，这32个值组成预测表。
其中，预测表包含索引号和预测值，还可以包含预测值出现的次数。
步骤S103：在预测表中选取预测值并生成残差
针对点云的每一个属性信息在预测表中选取一个预测值，并相减生成残差。
其中，选取规则可以是最相近的预测值，比如，如果属性信息只有一个分量，那么可以当前属性信息的值分别与预测表中所有预测值相减取绝对值后，选取绝对值数值最小的为预测值。如果属性信息是多个分量，比如颜色属性，那么可以是当前属性信息的所有分量分别与预测表中所有预测值的分量对应相减取绝对值后相加，得到的和数值最小的为预测值。如果属性信息是多个分量，比如颜色属性，那么可以仅选择当前属性信息的一个分量(如亮度分量)与预测表中所有预测值的对应分量相减取绝对值，选取绝对值数值最小的为预测值。如果属性信息是多个分量，比如颜色属性，那么可以先将当前属性信息的分量加权组合后再与预测表中所有预测值的分量加权相减取绝对值，其中加权方式一样，选取绝对值数值最小的为预测值。
在一实施方式中，选取规则还可以是选取其它任意规则。
步骤S104：编码残差和预测值索引
将步骤S103生成的残差和选取的预测值在预测表中的索引编码。
生成的残差和预测值可以与其它信息，比如几何信息，一起编码生成码流或者生成文件， 也可以单独生成码流或者文件。
编码后的数据可以生成文件用于存储，也可以生成码流用于传输。
图5是本申请实施例提供的另一种数据处理方法的流程图，本申请实施例是在上述申请实施例基础上的具体化，参见图5，本申请实施例提供的方法包括如下步骤：
步骤410、在点云的属性信息集中提取当前属性信息。
在本申请实施例中，预测表与属性信息的编码过程可以同时处理，可以在点云的属性信息集中选择一个未被处理的信息作为当前属性信息。
步骤420、确定预测表是否已存在当前属性信息的属性值。
可以在预测表中查找该当前属性信息的属性值是否存在。
步骤430、若存在，则更新预测表中的属性值的计数值。
其中，计数值可以反映预测表中预测值的出现次数的信息，计数值可以经过加权，例如，每当出现一个预测值时，计数值的取值可以增加2。不同属性信息的计数值的加权可以相同也可以不同，当加权不同时，属性信息1的属性值出现1次，则计数值取值加2，属性信息2的属性值出现1次，则计数值取值可以加3。
在本申请实施例中，当预测表中存在当前属性信息的属性值时，可以更新预测表中属性值的计数值进行更新，可以理解的是，可以根据当前属性信息的加权值更新计数值。
步骤440、若不存在，则在预测表未满的情况下将属性值更新到预测表。
在本申请实施例中，预测表中存储预测值的数量存在上限，当预测表存储的预测值的数量达到上限时，则可以认为预测表满，否则预测表未满，在当前属性信息的属性值未在预测表出现且预测表未满时，可以将该预测值添加到预测表中。
步骤450、根据预测表确定属性信息的残差和预测值索引。
步骤460、编码残差和预测值索引以形成码流发送或存储。
在上述申请实施例的基础上，将所述属性值更新到所述预测表，包括以下至少之一：
将阈值时间内出现次数最多的属性值更新到预测表；
根据先入先出原则将属性值更新到预测表；
将预测表中所述计数值最少的已存属性值替换为当前属性值。
其中，阈值时间可以是缓存属性值的时间段，该阈值时间可以由用户设置。
在本申请实施例中，更新属性值到预测表时可以按照以下规则中一种或者多种方式进行，例如，可以将一段时间内出现次数最多的属性值更新到预测表，该一段时间可以由用户设置阈值时间的时间长度的方式确定。还可以按照先入先出原则，将属性值替换预测表中存储时间最长的预测值。还可以使用预测值替换预测表中存储的属性值，该属性值得计数值为预测表中计数值最小的属性值。
在一个示例性的实施方式中，预测表可以不采用遍历点云的所有属性信息后生成的方法，而是随着逐一选取属性信息的同时动态生成，这样可以节省生成预测表的时间。图6是本申请实施例提供的另一种数据处理方法的示例图，参见图6，该数据处理方法可以包括如下步骤：
步骤S201：选取一个属性信息
选取一个点云属性信息，其中已经进行过残差计算的属性信息将不再被选取。
步骤S202：预测表是否有此属性信息值
在预测表中寻找当前属性信息是否存在，这一步骤是为了动态更新预测表。
步骤S203：预测表是否满
预先设定预测表所含有的最大预测值个数，如果当前属性值不在预测表里，再根据预测表当前已经含有的预测个数是否大于最大预测个数，如果大于最大预测值个数，那么预测表为满，否则为不满。
步骤S204：将当前属性信息加入预测表
如果步骤S203判断预测表是不满的，则把当前属性值加入到预测表中，并记录计数。
步骤S205：更新预测表中此属性信息值的计数
如果步骤S202判断预测表里含有当前属性信息值，那么对预测表里的此属性信息值的计数更新。
其中计数更新可以是数量加一，也可以根据当前属性信息的重要性增加不同的数量，比如说重要的属性信息，计数加二。
在一实施方式中，预测表可以采用不同的更新原则，比如，不管预测值出现的次数，采用先入先出的队列原则，仅保存最近出现的最大预测个数的预测值，还可以是结合预测值出现的次数，当满表需要更新的时候，新的属性信息值仅替换表内计数最少的预测值。
步骤S206：选取预测值
在预测表中选取一个预测值，如果步骤S202判断当前属性信息就是预测值，可以直接使用当前属性信息当作预测值，否则，需要在预测表中选择预测值，选取规则与步骤S103中的选取规则一样。
步骤S207：计算残差
与步骤S103中的计算残差方法一样。
步骤S208：是否还有属性信息
判断是否所有的属性信息都已经进行了预测值选取和残差计算。如果所有的属性信息均进行预测值选取和残差计算，则可以进行步骤S209，否则进行步骤S201。
步骤S209：编码残差和索引
和步骤S104一样。也可以在进行完步骤S207后帧对当前属性信息残差和索引进行编码，而不是等到所有的属性信息都遍历完后再进行残差和索引编码。
在上述申请实施例的基础上，确定点云的属性信息对应的预测表，包括：
将所述点云的全部的属性信息划分为到至少两个属性信息组；确定各所述属性信息组对应的预测表。
在本申请实施例中，可以按照点云的属性信息划分为多个属性信息组，每个属性信息组存在各自对应的预测表。例如，可以按照点云的属性信息的种类划分为多个属性信息组，每个属性信息组可以根据各自的属性信息的特征生成对应的预测表。又例如，可以按照点云的属性信息的取值划分为多个属性信息组，取值具有相似方差的属性信息可以划分到相同的属性信息组，每个属性信息组存在各自的预测表。
在上述申请实施例的基础上，将所述点云的全部的属性信息划分为到至少两个属性信息组，包括：
按照点云的几何信息将各所述属性信息划分到对应的所述属性信息组。
其中，几何信息可以是反映点云位置点在空间内位置关系的信息，可以包括坐标和相对距离等。
在本申请实施例中，可以按照各点云中位置点的坐标或者与其他位置点的相对距离，将各位置点的属性信息划分为多组属性信息组，其中，划分的属性信息组的数量可以由用户设置或者由属性信息的类别数量确定。
在上述申请实施例的基础上，所述几何信息包括以下至少之一：莫顿码排序、希尔伯特码排序、点云三维坐标的八叉树分块或分层、点云三维坐标的K-D树的分块或分层。
在一个示例性的实施方式中，点云属性信息的统计范围可以由全部数据变为部分数据，可以将点云属性信息先按照几何信息进行排序，再将排好顺序的点云属性按照顺序分成N个一组，针对每组使用上述实施例的方式生成每一组属性信息自己的预测表。比如，几何信息可以是按照莫顿码或是希尔伯特码排序，那么属性信息根据对应几何信息排序，选取512个点为一组，再分别对一组点云属性信息进行统计，选取出现次数最多的32个属性信息生成预测表，再为这512个点分别选取预测值，生成残差，将残差和对应的索引值进行编码。
再比如，几何信息也可以按照八叉树或者K-D树的形式进行区域分块或分层，分块或者分层区域的属性信息为一组，再分别对一组点云属性信息进行统计，选取出现次数最多的16个属性信息生成预测表，再为这一组点分别选取预测值，生成残差，将残差和对应的索引值进行编码。
在上述申请实施例的基础上，所述预测表至少包括属性值，其中，所述属性值包括至少一个属性分量值。
在本申请实施例中，预测表可以由属性值组成，每个属性值可以存在一个或者多个属性分 量，例如，一个属性值为颜色值，该颜色值为RGB模式的颜色，属性分量可以分别为RGB中不同颜色的颜色分量。
在一个示例性的实施方式中，预测表可以包含索引、预测值，其中预测值可以是一个分量，也可以是多个分量。
预测表还可以包含统计信息，比如每个预测值出现的次数，每个预测值的方差等；预测表的一种组织形式可以如表1所示，其中预测值的顺序就是索引，因此不用在预测表里单独存放索引值。如果索引值不是预测值排列的顺序号，那么需要加入索引值。
表1 预测表的组织形式
Figure PCTCN2022095290-appb-000001
其中，numOfPredictor表示含有几个预测值，numOfPredictorComponent表示一个预测值含有几个分量，valueOfPredictor表示预测值。
在一个示例性的实施方式中，预测表可以放在属性信息头或属性片头(slice header)或属性瓦片头(tile header)，或者其它可指示属性信息特性的参数集中，如表2所示。
表2 预测表在属性信息头中
Figure PCTCN2022095290-appb-000002
其中，withPredict表示是否采用预测编码方式，1表示采用，0表示不采用；withPredictMap表示是否含有预测表，1表示含有，0表示不含有。
在一个示例性的实施方式中，预测表也可以采用其他的组织形式，如表3所示。
表3 预测表在属性信息片头中
Figure PCTCN2022095290-appb-000003
其中，withPredict表示是否采用预测编码方式，1表示采用，0表示不采用；withPredictMap表示是否含有预测表，1表示含有，0表示不含有。
预测表还可以采用ISO/IEC 14496-12 ISO BMFF形式。预测表可以在已有的Attribute轨道指示，具体内容可以与属性信息放在一起。Attribute中的指示信息实现方式如下：
Figure PCTCN2022095290-appb-000004
其中，gpcc_type等于4表示是属性信息；predictor_map_present等于1表示有预测表，等于0表示没有预测表。
在一个实例性是的实施方式中，预测表索引和残差存在于属性信息数据中，它们的引用形式可以表4所示。
表4 预测表索引和残差的引用形式
Figure PCTCN2022095290-appb-000005
其中，attr_predict_idx代表索引值；attr_predict_residual代表残差值。
在上述申请实施例的基础上，所述预测表的数量可以包括至少一个。
例如，在一实施方式中，对于点云中的孤点建立一个新的表格，其中孤点可以是指属性信息的值跟预测表的所有值相差非常大，孤点可以是多个。也可以将孤点与已有预测表合并为一个预测表。
在一实施方式中，对属性信息的不同分量分别建立一个预测表，比如颜色属性RGB空间，红色(R)、绿色(G)和蓝色(B)都有自己的预测表，点云数据里的每个点的颜色属性使用三个索引值表示。
在一实施方式中，实施例五中的属性信息头中设定一个预测表，不同的属性分片头里也都有自己的预测表，它可以是属性信息头预测表的补充，即属性分片头里的预测表与属性信息头的预测表联合生成一个预测表，也可以是属性分片头的预测表记录的是与属性信息头预测表的差值，即属性分片头里的预测表需要与属性信息头预测表相加后生成最终供属性分片头使用的预测表。
在上述申请实施例的基础上，所述属性信息包括颜色属性，则所述预测表包括调色板，所述调色板的属性分量值设置有权重。
在本申请实施例中，属性信息是颜色属性，颜色可以是不同颜色空间，比如亮度色度(YUV)，RGB等。基于颜色属性的预测表可以是调色板(palette)。
例如，YUV颜色空间以Y分量为主，那么预测表以Y分量为主，计算属性信息统计特性的时候Y分量的权重高于UV分量。
又例如，RGB颜色空间三个分量权重相同，那么计算属性信息统计特性的时候三个分量的权重一样。
在上述申请实施例的基础上，确定点云的属性信息对应的预测表，包括：
基于预设的神经网络生成所述预测表，其中，所述神经网络预先经过阈值数量的点云的属性信息训练生成，所述属性信息集包括具有属性值和预测值。
其中，训练神经网络的属性信息的阈值数量可以为一个数值较大的取值，阈值数量越大，则神经网络的准确性越高，阈值数量可以由用户根据经验设定。
在本申请实施例中，预测表可以通过神经网络生成，该神经网络可以通过点云的属性信息训练生成，用于训练神经网络的属性信息可以包括预测值以及属性值。
示例性的，可以利用神经网络对预测表进行训练生成。利用已有的点云数据集对神经网络进行训练，获得性能最优的预测表。可以有针对性的对不同类型的点云进行训练，获得多个预 测表，比如针对车载地图的反射率专门训练一个预测值。
在上述申请实施例的基础上，所述根据所述预测表确定所述属性信息的残差和预测值索引，包括：
按照各属性信息的属性值在预测表内选择预测值以及预测值索引。将预测值与属性信息的属性值之差作为残差。
可以在预测表中确定出属性信息对应的预测值，可以将该预测值与属性信息的属性值的差值作为残差。确定属性信息的预测值可以过程可以是按照属性信息的类别或者取值大小获取对应的预测值，可以将该预测值在预测表中关联存储的索引号作为预测值索引。
在上述申请实施例的基础上，所述按照各所述属性信息的属性值在所述预测表内选择预测值以及预测值索引，包括：
将所述属性值与所述预测表内各所述属性值的差值和/或加权差值，将具有最小绝对值的所述差值和/或所述加权差值对应的所述预测表内的所述属性值作为所述属性信息的预测值，并将所述预测表内所述属性值的索引作为预测索值引。
在本申请实施例中，预测表中一类属性信息可以存在多个属性值，可以分别计算属性信息的属性值与预测表中各属性值的差值，可以将该绝对值最小的差值在预测表中对应的属性值作为预测值，并将该预测值的索引作为预测值索引。可以理解是，预测表中的各属性值还可以存在各自的加权权重，可以根据该加权权重确定出各差值的加权差值，可以将绝对值最小的加权差值在预测表中对应的属性值作为预测值，并将该预测值的索引作为预测值索引。
在上述申请实施例的基础上，预测表和属性信息可以一起被编码在码流中传输，也可以单独传输。预测表可以编码端跟解码端在建立链路的时候发送给解码端，在进行交互过程中还可以更新预测表。图7是本申请实施例提供的一种预测表传输的示例图，参见图7，预测表可以预先放在服务器上，编码端仅发送预测表编号，解码端根据需求向服务器发送预测表编号，请求下发预测表。
预测表也可以不编码存储或者传输。
针对无损压缩，预测表可以在解码端解码过程中动态生成的，而且在解码端解码过程中生成的预测表与编码端编码过程中生成的预测表完全一致，那么，预测表不需要在编码端进行编码并存储或传输。
图8是本申请实施例提供的另一种数据处理方法的流程图，本申请实施例可适用于三维点云的属性信息进行解码的情况，该方法可以由本申请实施例提供的数据处理装置来执行，该装置可以通过软件和/或硬件方式实现，参见图8，本申请实施例提供的方法包括如下步骤：
步骤510、接收码流并获取码流中的残差和预测值索引。
在本申请实施例中，在接收到码流后，可以从码流中解析出残差和预测值索引。
步骤520、确定残差和预测值索引对应的预测表。
预测表是用于解码使用的预测值的信息表，可以从属性信息头或者属性片头或着属性瓦片头或者其它可指示属性信息特性的参数集中解析获得，也可以是根据码流中的残差和预测值索引动态生成。预测表的数量可以为一个或多个。
步骤530、按照预测表、残差和预测值索引确定点云的属性信息。
在本申请实施例中，可以根据预测值索引在预测表中查找对应的预测值，可以根据残差值和预测值确定出点云的属性信息的取值，例如，可以将残差值与预测值的和作为点云的属性信息的属性值。
本申请实施例，通过接收码流并获取码流中的预测表、残差和预测值索引，按照预测表、残差和预测值索引确定出点云的属性信息。
可选的，通过接收码流并获取码流中的残差和预测值索引，按照残差和预测值索引动态生成预测表，按照预测表、残差和预测值索引确定出点云的属性信息，实现了点云属性信息的快速解码，可增强点云的传输效率。
在一个示例性的实施方式中，图9是本申请实施例提供的一种数据处理方法的示例图，参见图9，先从码流或者文件中获取预测表信息，再从码流或或者文件中获取索引信息和残差，根据索引信息在预测表中找到对应的预测值，最后由预测值和残差计算得到属性信息值，本申请 实施例提供的方法包括如下步骤：
步骤S301：获取预测表信息。
从包含点云编码数据的码流或者文件中获取预测表信息。预测表信息可以存在于属性信息头，也可以存在于属性片头(slice header)中，也可以存在于属性瓦片头(tile header)中，也可以存在于其它可指示属性信息特性的参数集中，也可以存在于传输层媒体描述单元中。预测表在解码端动态生成时，可以没有此步骤。
步骤S302：获取索引值和残差。
从包含点云编码数据的码流或者文件中获取预测索引值和残差。
步骤S303：从预测表中选取预测索引对应的预测值。
根据步骤S302中获取的预测索引值，在步骤S301中获取的预测表中查找对应的预测值。
步骤S304：由残差和预测值计算属性信息值。
将步骤S302获取的残差与步骤S303获取的预测值相加生成属性信息值。
图10是本申请实施例提供的一种数据处理装置的结构示意图，可执行本申请任意实施例提供的数据处理方法，具备执行方法相应的功能模块和有益效果，该装置可以由软件和/或硬件实现，包括：预测表模块501、残差确定模块502和数据编码模块503。
预测表模块501，用于确定点云的属性信息对应的预测表。
残差确定模块502，用于根据所述预测表确定所述属性信息的残差和预测值索引。
数据编码模块503，用于编码所述残差和所述预测值索引以形成码流发送或存储。
本申请实施例中，通过预测表模块确定出点云属性信息的预测表，残差确定模块按照预测表处理属性信息以获取残差和预测值索引，根据点云属性信息的特点获取预测表，按照预测表确定属性信息编码使用的残差和预测值索引，可降低点云属性信息编码代价，提高属性信息的编码性能。
在上述申请实施例的基础上，所述预测表模块501包括：
分布特征单元，用于按照各所述属性信息的取值确定点云属性分布特征。
表生成单元，用于根据所述点云属性分布特征生成预测表。
在上述申请实施例的基础上，所述分布特征单元包括：
次数统计子单元，用于在所述点云的各类所述属性信息中统计至少一个属性值的出现次数，并将各所述出现次数作为所述点云属性分布特征。
在上述申请实施例的基础上，分布特征单元中点云属性分布特征包括属性值出现次数、属性值均值、属性值方差中至少之一。
在上述申请实施例的基础上，所述表生成单元用于：在各类所述属性信息中，按照各所述出现次数的取值从高到低对各所述属性值排序；在所述排序中按照从高到低的顺序选择预设数量的所述属性值填充到预测表。
在上述申请实施例的基础上，所述表生成单元还用于：在各类所述属性信息中，将各所述属性值从高到低排序；按照各所述属性值对应的出现次数在所述排序中选择对应数量的所述属性值填充到预测表。
在上述申请实施例的基础上，预测表模块501还包括：
信息提取单元，用于在所述点云的属性信息集中提取当前属性信息；
信息判断单元，用于确定所述预测表是否已存在所述当前属性信息的属性值；
信息处理单元，用于若存在，则更新所述预测表中的所述属性值的计数值；若不存在，则在所述预测表未满的情况下将所述属性值更新到所述预测表。
在上述申请实施例的基础上，信息处理单元中将所述属性值更新到所述预测表，包括以下至少之一：
将阈值时间内出现次数最多的所述属性值更新到所述预测表；
根据先入先出原则将所述属性值更新到所述预测表；
将所述预测表中所述计数值最少的已存属性值替换为当前属性值。
在上述申请实施例的基础上，预测表模块501还包括：
分组单元，用于将所述点云的全部的属性信息划分为到至少两个属性信息组。
表生成单元，用于确定各所述属性信息组对应的预测表。进一步的，在上述申请实施例的基础上，分组单元具体用于：按照点云的几何信息将各所述属性信息划分到对应的所述属性信息组。
在上述申请实施例的基础上，所述几何信息包括以下至少之一：莫顿码排序、希尔伯特码排序、点云三维坐标的八叉树分块或分层、点云三维坐标的K-D树的分块或分层。
在上述申请实施例的基础上，预测表至少包括属性值，其中，所述属性值包括至少一个属性分量值。
在上述申请实施例的基础上，所述属性信息包括颜色属性，则所述预测表包括调色板，所述调色板的属性分量值设置有权重。
在上述申请实施例的基础上，确定点云的属性信息对应的预测表，包括：
基于预设的神经网络生成所述预测表，其中，所述神经网络预先经过阈值数量的点云的属性信息训练生成，所述属性信息包括具有属性值和预测值。
在上述申请实施例的基础上，残差确定模块502包括：
表差值单元，用于按照各所述属性信息的属性值在所述预测表内选择预测值以及预测值索引。
残差处理单元，用于将所述预测值与所述属性信息的属性值之差作为残差。
在上述申请实施例的基础上，残差处理单元用于将所述属性值与所述预测表内各所述属性值的差值和/或加权差值，将具有最小绝对值的所述差值和/或所述加权差值对应的所述预测表内的所述属性值作为所述属性信息的预测值，并将所述预测表内所述属性值的索引作为预测值索引。
图11是本申请实施例提供的一种数据处理装置的结构示意图，可执行本申请任意实施例提供的数据处理方法，具备执行方法相应的功能模块和有益效果，该装置可以由软件和/或硬件实现，包括：码流接收模块504、预测表模块505和数据解码模块506。
码流接收模块504，用于接收码流并获取所述码流中的残差和预测值索引。
预测表模块505，用于确定所述残差和所述预测值索引对应的预测表。
数据解码模块506，用于按照所述预测表、所述残差和所述预测值索引确定点云的属性信息。
本申请实施例，通过码流接收模块接收码流并获取码流中的残差和预测值索引，预测表模块按照残差和预测值索引获取预测表，数据解码模块按照预测表、残差和预测值索引确定出点云的属性信息，实现了点云属性信息的快速解码，可增强点云的传输效率。
示例性的，图12是本申请实施例提供的一种数据处理装置的示例图，参见图12，该装置包括：
预测表模块A01：用于生成预测表。
属性编码模块A02：用于生成残差信息，编码残差信息和索引。
传输模块A03：用于传输属性编码后数据，也可以对预测表进行编码传输。
预测表模块A04：用于建立或者动态生成预测表。
属性解码模块A05：用于解码属性信息，包括残差信息和索引。
传输模块A06：用于传输属性信息压缩数据，也可以对预测表进行解码。
图13是本申请实施例提供的一种电子设备的结构示意图，该电子设备包括处理器60、存储器61、输入装置62和输出装置63；电子设备中处理器60的数量可以是一个或多个，图13中以一个处理器60为例；电子设备中处理器60、存储器61、输入装置62和输出装置63可以通过总线或其他方式连接，图13中以通过总线连接为例。
存储器61作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序以及模块，如本申请实施例中的数据处理装置对应的模块(预测表模块501、残差确定模块502和数据编码模块503，或者，码流接收模块504、预测表模块505和数据解码模块506)。处理器60通过运行存储在存储器61中的软件程序、指令以及模块，从而执行电子设备的各种功能应用以及数据处理，即实现上述的数据处理方法。
存储器61可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序；存储数据区可存储根据电子设备的使用所创建的数据等。此外，存 储器61可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中，存储器61还可包括相对于处理器60远程设置的存储器，这些远程存储器可以通过网络连接至电子设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
输入装置62可用于接收输入的数字或字符信息，以及产生与电子设备的用户设置以及功能控制有关的键信号输入。输出装置63可包括显示屏等显示设备。
本申请实施例还提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行一种数据处理方法，该方法包括：
确定点云的属性信息对应的预测表；
根据所述预测表确定所述属性信息的残差和预测值索引；
编码所述残差和所述预测值索引以形成码流发送或存储。
通过以上关于实施方式的描述，所属领域的技术人员可以清楚地了解到，本申请可借助软件及必需的通用硬件来实现，当然也可以通过硬件实现，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如计算机的软盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、闪存(FLASH)、硬盘或光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
值得注意的是，上述装置的实施例中，所包括的各个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、设备中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。
在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
以上参照附图说明了本申请的若干实施例，并非因此局限本申请的权利范围。本领域技术人员不脱离本申请的范围和实质内所作的任何修改、等同替换和改进，均应在本申请的权利范围之内。

Claims (22)

  1. 一种数据处理方法，所述方法包括：
    确定点云的属性信息对应的预测表；
    根据所述预测表确定所述属性信息的残差和预测值索引；
    编码所述残差和所述预测值索引以形成码流发送或存储。
  2. 根据权利要求1所述的方法，其中，所述确定点云的属性信息对应的预测表，包括：
    按照各所述属性信息的取值确定点云属性分布特征；
    根据所述点云属性分布特征生成预测表。
  3. 根据权利要求2所述的方法，其中，所述按照各所述属性信息的取值确定点云属性分布特征，包括：
    在所述点云的各类所述属性信息中统计至少一个属性值的出现次数，并将各所述出现次数作为所述点云属性分布特征。
  4. 根据权利要求2所述的方法，其中，所述点云属性分布特征包括属性值出现次数、属性值均值、属性值方差中至少之一。
  5. 根据权利要求3所述的方法，其中，所述根据所述点云属性分布特征生成预测表，包括：
    在各类所述属性信息中，按照各所述出现次数的取值从高到低对各所述属性值排序；
    在所述排序中按照从高到低的顺序选择预设数量的所述属性值填充到预测表。
  6. 根据权利要求3所述的方法，其中，所述根据所述点云属性分布特征生成预测表，包括：
    在各类所述属性信息中，将各所述属性值从高到低排序；
    按照各所述属性值对应的出现次数在所述排序中选择对应数量的所述属性值填充到预测表。
  7. 根据权利要求1所述的方法，其中，所述确定点云的属性信息对应的预测表，包括：
    在所述点云的属性信息集中提取当前属性信息；
    确定所述预测表是否已存在所述当前属性信息的属性值；
    若存在，则更新所述预测表中的所述属性值的计数值；
    若不存在，则在所述预测表未满的情况下将所述属性值更新到所述预测表。
  8. 根据权利要求7所述的方法，其中，将所述属性值更新到所述预测表，包括以下至少之一：
    将阈值时间内出现次数最多的所述属性值更新到所述预测表；
    根据先入先出原则将所述属性值更新到所述预测表；
    将所述预测表中所述计数值最少的已存属性值替换为当前属性值。
  9. 根据权利要求1所述的方法，其中，所述确定点云的属性信息对应的预测表，包括：
    将所述点云的全部的属性信息划分为到至少两个属性信息组；
    确定各所述属性信息组对应的预测表。
  10. 根据权利要求9所述的方法，其中，所述将所述点云的全部的属性信息划分为到至少两个属性信息组，包括：
    按照点云的几何信息将各所述属性信息划分到对应的所述属性信息组。
  11. 根据权利要求10所述的方法，其中，所述几何信息包括以下至少之一：莫顿码排序、希尔伯特码排序、点云三维坐标的八叉树分块或分层、点云三维坐标的K-D树的分块或分层。
  12. 根据权利要求1所述的方法，其中，所述预测表至少包括属性值，其中，所述属性值包括至少一个属性分量值。
  13. 根据权利要求1所述的方法，其中，所述预测表的数量包括至少一个。
  14. 根据权利要求1所述的方法，其中，所述属性信息包括颜色属性，则所述预测表包括调色板，所述调色板的属性分量值设置有权重。
  15. 根据权利要求1所述的方法，其中，所述确定点云的属性信息对应的预测表，包括：
    基于预设的神经网络生成所述预测表，其中，所述神经网络预先经过阈值数量的点云的属性信息训练生成，所述属性信息包括具有属性值和预测值。
  16. 根据权利要求1所述的方法，其中，所述根据所述预测表确定所述属性信息的残差和预测值索引，包括：
    按照各所述属性信息的属性值在所述预测表内选择预测值以及预测值索引；
    将所述预测值与所述属性信息的属性值之差作为残差。
  17. 根据权利要求16所述的方法，其中，所述按照各所述属性信息的属性值在所述预测表内选择预测值以及预测值索引，包括：
    将所述属性值与所述预测表内各所述属性值的差值和/或加权差值，将具有最小绝对值的所述差值和/或所述加权差值对应的所述预测表内的所述属性值作为所述属性信息的预测值，并将所述预测表内所述属性值的索引作为预测值索引。
  18. 一种数据处理方法，所述方法包括：
    接收码流并获取所述码流中的残差和预测值索引；
    确定所述残差和所述预测值索引对应的预测表；
    按照所述预测表、所述残差和所述预测值索引确定点云的属性信息。
  19. 一种数据处理装置，所述装置包括：
    预测表模块，用于确定点云的属性信息对应的预测表；
    残差确定模块，用于根据所述预测表确定所述属性信息的残差和预测值索引；
    数据编码模块，用于编码所述残差和所述预测值索引以形成码流发送或存储。
  20. 一种数据处理装置，所述装置包括：
    码流接收模块，用于接收码流并获取所述码流中的残差和预测值索引；
    预测表模块，用于确定所述残差和所述预测值索引对应的预测表；
    数据解码模块，用于按照所述预测表、所述残差和所述预测值索引确定点云的属性信息。
  21. 一种电子设备，所述电子设备包括：
    一个或多个处理器；
    存储器，用于存储一个或多个程序；
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-18中任一所述的数据处理方法。
  22. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行实现如权利要求1-18中任一所述的数据处理方法。
PCT/CN2022/095290 2021-06-15 2022-05-26 一种数据处理方法、装置、电子设备和存储介质 Ceased WO2022262546A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP22824026.3A EP4358516A4 (en) 2021-06-15 2022-05-26 DATA PROCESSING METHOD AND DEVICE, ELECTRONIC DEVICE AND STORAGE MEDIUM
US18/570,629 US20240289993A1 (en) 2021-06-15 2022-05-26 Data processing method and apparatus, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110662694.5A CN115484462A (zh) 2021-06-15 2021-06-15 一种数据处理方法、装置、电子设备和存储介质
CN202110662694.5 2021-06-15

Publications (1)

Publication Number Publication Date
WO2022262546A1 true WO2022262546A1 (zh) 2022-12-22

Family

ID=84418942

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/095290 Ceased WO2022262546A1 (zh) 2021-06-15 2022-05-26 一种数据处理方法、装置、电子设备和存储介质

Country Status (4)

Country Link
US (1) US20240289993A1 (zh)
EP (1) EP4358516A4 (zh)
CN (1) CN115484462A (zh)
WO (1) WO2022262546A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116781908A (zh) * 2022-03-11 2023-09-19 腾讯科技（深圳）有限公司 点云预测处理方法、装置、计算机、存储介质
CN119071499A (zh) * 2023-05-31 2024-12-03 腾讯科技（深圳）有限公司 点云处理方法、装置及计算机设备、存储介质
US20260030222A1 (en) * 2024-07-24 2026-01-29 Zoom Video Communications, Inc. Scalable design of relational database for resource management

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110418135A (zh) * 2019-08-05 2019-11-05 北京大学深圳研究生院 一种基于邻居的权重优化的点云帧内预测方法及设备
CN111145090A (zh) * 2019-11-29 2020-05-12 鹏城实验室 一种点云属性编码方法、解码方法、编码设备及解码设备
CN111405281A (zh) * 2020-03-30 2020-07-10 北京大学深圳研究生院 一种点云属性信息的编码方法、解码方法、存储介质及终端设备
US20210006797A1 (en) * 2019-07-03 2021-01-07 Tencent America LLC Method and apparatus for adaptive point cloud attribute coding
WO2021003726A1 (zh) * 2019-07-10 2021-01-14 深圳市大疆创新科技有限公司 一种数据编码、数据解码方法、设备及存储介质
CN112565734A (zh) * 2020-12-03 2021-03-26 西安电子科技大学 基于混合编码的点云属性编解码方法及装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6034667A (en) * 1992-01-21 2000-03-07 Radius Inc. Method and apparatus for displaying YUV color information on a pseudo-color RGB display
US10212434B2 (en) * 2015-01-30 2019-02-19 Qualcomm Incorporated Palette entries coding in video coding

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210006797A1 (en) * 2019-07-03 2021-01-07 Tencent America LLC Method and apparatus for adaptive point cloud attribute coding
WO2021003726A1 (zh) * 2019-07-10 2021-01-14 深圳市大疆创新科技有限公司 一种数据编码、数据解码方法、设备及存储介质
CN110418135A (zh) * 2019-08-05 2019-11-05 北京大学深圳研究生院 一种基于邻居的权重优化的点云帧内预测方法及设备
CN111145090A (zh) * 2019-11-29 2020-05-12 鹏城实验室 一种点云属性编码方法、解码方法、编码设备及解码设备
CN111405281A (zh) * 2020-03-30 2020-07-10 北京大学深圳研究生院 一种点云属性信息的编码方法、解码方法、存储介质及终端设备
CN112565734A (zh) * 2020-12-03 2021-03-26 西安电子科技大学 基于混合编码的点云属性编解码方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4358516A4 *

Also Published As

Publication number Publication date
EP4358516A4 (en) 2025-05-28
US20240289993A1 (en) 2024-08-29
EP4358516A1 (en) 2024-04-24
CN115484462A (zh) 2022-12-16

Similar Documents

Publication Publication Date Title
US11606551B2 (en) Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2022262546A1 (zh) 一种数据处理方法、装置、电子设备和存储介质
CN111405284B (zh) 一种基于点云密度的属性预测方法及设备
CN104301737B (zh) 目标图像块的解码方法和编码方法以及解码器和编码器
US20260025518A1 (en) Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device
WO2022062369A1 (zh) 点云编解码方法与系统、及点云编码器与点云解码器
EP4258671A1 (en) Point cloud attribute predicting method, encoder, decoder, and storage medium
CN116472556B (zh) 预测方法、编码器、解码器以及存储介质
WO2022133753A1 (zh) 点云编解码方法与系统、及点云编码器与点云解码器
CN115474041B (zh) 点云属性的预测方法、装置及相关设备
WO2022140937A1 (zh) 点云编解码方法与系统、及点云编码器与点云解码器
WO2022141461A1 (zh) 点云编解码方法、编码器、解码器以及计算机存储介质
CN118075464B (zh) 点云属性的预测方法、装置及编解码器
US20240355004A1 (en) Variable rate compression of point cloud geometry
US20260032261A1 (en) Point cloud encoding and decoding methods and decoder
CN115086716A (zh) 点云中邻居点的选择方法、装置及编解码器
WO2023023914A1 (zh) 帧内预测、编解码方法及装置、编解码器、设备、介质
CN110741635A (zh) 编码方法、解码方法、编码设备和解码设备
WO2022217472A1 (zh) 点云编解码方法、编码器、解码器及计算机可读存储介质
CN118890489A (zh) 图像处理方法及系统
CN117615136A (zh) 点云解码方法、点云编码方法、解码器、电子设备以及介质
CN120476590A (zh) 点云编解码方法、装置、设备及存储介质
CN120476596A (zh) 编解码方法、码流、编码器、解码器以及存储介质
CN118279122A (zh) 点云数据处理方法、装置、设备及介质
RU2826546C1 (ru) Способ кодирования и декодирования облака точек, кодер, декодер и компьютерный носитель данных

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22824026

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18570629

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2022824026

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022824026

Country of ref document: EP

Effective date: 20240115