WO2024041238A1 - 一种点云媒体的数据处理方法及相关设备 - Google Patents

一种点云媒体的数据处理方法及相关设备 Download PDF

Info

Publication number
WO2024041238A1
WO2024041238A1 PCT/CN2023/106292 CN2023106292W WO2024041238A1 WO 2024041238 A1 WO2024041238 A1 WO 2024041238A1 CN 2023106292 W CN2023106292 W CN 2023106292W WO 2024041238 A1 WO2024041238 A1 WO 2024041238A1
Authority
WO
WIPO (PCT)
Prior art keywords
attribute
point cloud
parameter set
component
parameter
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/106292
Other languages
English (en)
French (fr)
Inventor
胡颖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to EP23856328.2A priority Critical patent/EP4580177A4/en
Publication of WO2024041238A1 publication Critical patent/WO2024041238A1/zh
Priority to US18/989,557 priority patent/US20250191230A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/001—Model-based coding, e.g. wire frame
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/282—Image signal generators for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/85406—Content authoring involving a specific file format, e.g. MP4 format
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/31—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain

Definitions

  • This application relates to the field of communication technology, specifically to data processing of point cloud media.
  • Embodiments of the present application provide a point cloud media data processing method and related equipment, which can indicate the encapsulation position of the attribute parameter set and obtain the attribute parameter set according to the indicated encapsulation position, which can reduce unnecessary encoding and decoding dependencies and achieve Transmission, decoding and presentation of point cloud media; optimize network bandwidth and utilization of computing resources on the decoding side.
  • embodiments of the present application provide a point cloud media data processing method, which method includes:
  • the media file includes the point cloud code stream of the point cloud media and the attribute parameter set.
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream; the media file also includes the attribute parameters.
  • Set indication information, attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set;
  • the point cloud code stream is decoded to present the point cloud media.
  • embodiments of the present application provide a point cloud media data processing method, which method includes:
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream;
  • the point cloud code stream is encapsulated to obtain a point cloud media media file; the media file also contains attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.
  • embodiments of the present application provide a point cloud media data processing device, which includes:
  • the acquisition unit is used to obtain the media files of the point cloud media.
  • the media files include the point cloud code stream of the point cloud media and the attribute parameter set.
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream; the media file It also includes attribute parameter set indication information, which is used to indicate the encapsulation position of the attribute parameter set;
  • the processing unit is used to decode the point cloud code stream according to the attribute parameter set indication information to present the point cloud media.
  • embodiments of the present application provide a data processing device for point cloud media, which device includes:
  • the processing unit is used to encode the point cloud media and obtain the point cloud code stream and the attribute parameter set.
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream;
  • the processing unit is also used to encapsulate the point cloud code stream to obtain a media file of the point cloud media; the media file also contains attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.
  • embodiments of the present application provide a computer device, which includes:
  • a processor adapted to execute a computer program
  • the computer-readable storage medium stores a computer program.
  • the computer program is executed by the processor, the above-mentioned point cloud media data processing method is implemented.
  • embodiments of the present application provide a computer-readable storage medium that stores a computer program.
  • the computer program is loaded by a processor and executes the above-mentioned point cloud media data processing method.
  • inventions of the present application provide a computer program product.
  • the computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned data processing method of point cloud media.
  • the media file of the point cloud media is obtained.
  • the media file includes the point cloud code stream of the point cloud media and a set of attribute parameters.
  • the set of attribute parameters includes the parameters required for decoding the attribute data in the point cloud code stream.
  • Information; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set.
  • Presenting point cloud media guides the transmission, decoding and presentation of point cloud media, which can reduce unnecessary encoding and decoding dependencies and save transmission and computing resources.
  • Figure 1a is a schematic diagram of a 6DoF provided by an exemplary embodiment of the present application.
  • Figure 1b is a schematic diagram of 3DoF provided by an exemplary embodiment of the present application.
  • Figure 1c is a schematic diagram of 3DoF+ provided by an exemplary embodiment of the present application.
  • Figure 2a is an architectural diagram of a point cloud media data processing system provided by an exemplary embodiment of the present application
  • Figure 2b is a flow chart of data processing of point cloud media provided by an exemplary embodiment of the present application.
  • Figure 3 is a schematic flowchart of a point cloud media data processing method provided by an exemplary embodiment of the present application
  • Figure 4a is a schematic diagram of a parameter set and attribute data divided into data units according to an exemplary embodiment of the present application
  • Figure 4b is a schematic diagram of a parameter set and attribute data divided into data units provided by another exemplary embodiment of the present application.
  • Figure 5 is a schematic flowchart of a point cloud media data processing method provided by another exemplary embodiment of the present application.
  • Figure 6 is a schematic structural diagram of a point cloud media data processing device provided by an exemplary embodiment of the present application.
  • Figure 7 is a schematic structural diagram of a point cloud media data processing device provided by another exemplary embodiment of the present application.
  • Figure 8 is a schematic structural diagram of a computer device provided by another exemplary embodiment of the present application.
  • the term “at least one” refers to one or more, and the term “multiple” refers to two or more; similarly, the term “at least one group” refers to one or more groups, and “multiple groups” " means two or more groups. For example, if a point in a point cloud has multiple sets of attribute data, it means that the point has two or more sets of attribute data.
  • Immersive media refers to media files that can provide immersive media content so that viewers immersed in the media content can obtain visual, auditory and other sensory experiences in the real world.
  • Immersive media can be divided into: 6DoF (Degree of Freedom, degree of freedom) immersive media, 3DoF immersive media, and 3DoF+ immersive media according to the viewer's degree of freedom when consuming media content.
  • 6DoF means that viewers of immersive media can freely translate along the X-axis, Y-axis, and Z-axis. For example, viewers of immersive media can move freely in three-dimensional 360-degree VR content. Similar to 6DoF, there are 3DoF and 3DoF+ production technologies.
  • Figure 1b is a schematic diagram of 3DoF provided by an embodiment of the present application; as shown in Figure 1b, 3DoF means that the viewer of the immersive media is fixed at the center point of a three-dimensional space, and the head of the viewer of the immersive media is along the X-axis, Y-axis and Z-axis rotation to view the picture provided by the media content.
  • Figure 1c is a schematic diagram of 3DoF+ provided by an embodiment of the present application. As shown in Figure 1c, 3DoF+ means that when the virtual scene provided by the immersive media has certain depth information, the head of the viewer of the immersive media can move in a limited space based on 3DoF. Move within the space to watch the images provided by the media content.
  • Point cloud refers to a set of discrete points randomly distributed in space that express the spatial structure and surface properties of a three-dimensional object or scene. Each point in the point cloud includes at least geometric data, which is used to represent the three-dimensional position information of the point. Depending on the application scenario, points in the point cloud may also include one or more sets of attribute data. Each set of attribute data is used to reflect an attribute possessed by the point. The attribute may be, for example, color, material, or other information. Typically, each point in a point cloud has the same number of sets of attribute data.
  • Point cloud can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes, so it is widely used in virtual reality (VR) games, computer-aided design (CAD), geographic information systems (In scenarios such as Geography Information System (GIS), Autonomous Navigation System (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, and three-dimensional reconstruction of biological tissues and organs.
  • VR virtual reality
  • CAD computer-aided design
  • GIS Geography Information System
  • ANS Autonomous Navigation System
  • digital cultural heritage digital cultural heritage
  • free viewpoint broadcasting three-dimensional immersive telepresence
  • three-dimensional reconstruction of biological tissues and organs three-dimensional reconstruction of biological tissues and organs.
  • point clouds can be obtained by collecting real-world visual scenes through a collection device (a set of cameras or a camera device with multiple lenses and sensors). Static real-world three-dimensional objects or scenes can be obtained through 3D laser scanning.
  • point clouds millions of point clouds can be obtained per second; point clouds of dynamic real-world three-dimensional objects or scenes can be obtained through 3D photography, and tens of millions of point clouds can be obtained per second; in addition, in the medical field, magnetic resonance can be used to Imaging (Magnetic Resonance Imaging, MRI), Computed Tomography (Computed Tomography, CT) and electromagnetic positioning information to obtain point clouds of biological tissues and organs.
  • MRI Magnetic Resonance Imaging
  • Computed Tomography Computed Tomography
  • CT Computed Tomography
  • electromagnetic positioning information to obtain point clouds of biological tissues and organs.
  • point clouds can also be directly generated by computers based on virtual three-dimensional objects and scenes.
  • computers can generate point clouds of virtual three-dimensional objects and scenes.
  • Point cloud media is a typical 6DoF immersive media.
  • Point cloud media includes a frame sequence composed of one or more point cloud frames.
  • Each point cloud frame is composed of geometric data and attribute data of one or more points in the point cloud.
  • geometric data can also be called three-dimensional position information.
  • the geometric data of a certain point in the point cloud refers to the spatial coordinates (x, y, z) of the point, which can include the direction of the point in each coordinate axis of the three-dimensional coordinate system.
  • the coordinate values for example, the coordinate value x in the X-axis direction, the coordinate value y in the Y-axis direction, and the coordinate value z in the Z-axis direction.
  • a point in the point cloud can include one or more sets of attribute data.
  • Each set of attribute data is used to reflect an attribute of the point.
  • a point in the point cloud has a set of color attribute data.
  • the color Attribute data is used to reflect the color attribute of the point (such as red, yellow, etc.); for another example, a certain point in the point cloud has a set of reflectivity attribute data, and the reflectance attribute data is used to reflect the laser reflection of the point.
  • Strength attribute When a point in the point cloud has multiple sets of attribute data, the types of the multiple sets of attribute data can be the same or different.
  • a point in the point cloud can have a set of color attribute data and a set of reflectance attribute data. ;
  • a certain point in the point cloud can have two sets of color attribute data, and these two sets of color attribute data are used to reflect the color attributes of the point at different times.
  • each point cloud frame also includes a geometric parameter set corresponding to the geometric data or an attribute parameter set corresponding to the attribute data.
  • the geometric parameter set and the attribute parameter set can form a parameter set.
  • the geometric parameter set may include parameter information required to encode or decode the geometric data in each point cloud frame;
  • the attribute parameter set corresponding to the attribute data may include the parameter information required to encode or decode the attribute data in each point cloud frame.
  • Required parameter information That is to say, the point cloud data contained in a point cloud frame in the point cloud media may include geometric data, a geometric parameter set, one or more sets of attribute data, and one or more attribute parameter sets.
  • a point cloud frame usually contains only one point cloud geometry header (ie Geometry Header) and one point cloud attribute header (ie Sequence Header), the point cloud geometry header includes a set of parameters required to encode or decode the geometric data of a single frame point cloud frame, and the point cloud attribute header includes a set of parameters required to encode or decode the attribute data in a single frame point cloud frame.
  • the point cloud attribute header can contain all parameter information corresponding to one or more sets of attribute data in the point cloud frame; and in MPEG (Moving Picture Expert Group, international video and audio coding standard) point cloud coding technology,
  • the point cloud code stream corresponding to the point cloud media can define a geometric parameter set (GPS, Geometry Parameter Set) and one or more attribute parameter sets (i.e. APS, Attribute Parameter Set).
  • APS can correspond to different types within the point cloud frame. attribute data.
  • attribute parameter sets i.e. APS, Attribute Parameter Set
  • a track refers to a collection of media data in the point cloud media encapsulation process.
  • a track is composed of multiple samples with time series, and one sample corresponds to a point cloud frame of the point cloud media.
  • the packaging methods of point cloud media include single-track method or multi-track method.
  • the so-called single-track method means that all point cloud data of point cloud media are encapsulated into the same track.
  • the media file of point cloud media only contains one track ( That is, a single track obtained by monorail packaging), among the single tracks obtained by monorail method, one
  • one A sample refers to a point cloud frame in point cloud media, and a sample contains all data of the corresponding point cloud frame (including geometric data and attribute data).
  • the so-called multi-track method refers to encapsulating the point cloud data of the point cloud media into multiple different tracks.
  • the media file of the point cloud media can contain multiple tracks.
  • the multi-track method can include a type-based multi-track method; the type-based multi-track method encapsulates one type of data into a component track.
  • the point cloud media contains geometric data, a set of color attribute data and a component track.
  • the geometric data can be encapsulated into the geometry component track
  • the color attribute data can be encapsulated into the color attribute component track
  • the reflectance attribute data can be encapsulated into the reflectance attribute component track; any type-based multi-track method can be obtained
  • a sample only contains part of the data of a point cloud frame in the point cloud media.
  • a sample in the geometry component track contains the geometry data of a point cloud frame in the point cloud media
  • a sample in the color attribute component track Contains a set of color attribute data for a point cloud frame in point cloud media.
  • metadata information can also be included as a media type in the media file of point cloud media in the form of metadata tracks.
  • each component track (such as geometry component track, attribute component track) can include one or more samples.
  • Each sample can correspond to a point cloud frame in the point cloud media, and correspond to the same point cloud frame. Samples should have the same presentation time.
  • the single-track method obtains that in a single track, one sample contains all the data of the corresponding point cloud frame, that is, one sample includes the geometric data, attribute data and parameter set of a point cloud frame.
  • the parameter The set includes a geometric parameter set and an attribute parameter set.
  • the geometric parameter set contains parameter information required for decoding the geometric data of the point cloud frame;
  • the attribute parameter set includes the parameter information required for decoding the attribute data of the point cloud frame.
  • the type-based multi-track method encapsulates one type of data into one track, and the parameter set may be encapsulated in a variety of locations, including but not limited to: 1
  • the parameter set can be encapsulated into the geometric component track corresponding to the geometric data; 2
  • the parameter set can be encapsulated into the attribute component track corresponding to the attribute data; 3
  • the parameter information can be individually encapsulated into the parameter component track as a type of point cloud data.
  • One sample in the parameter component track includes point cloud media. Parameter information of a point cloud frame.
  • point cloud media exists in the track in the form of components after being encapsulated.
  • attribute data in the point cloud media is encapsulated and exists in the track in the form of attribute components
  • geometric data in the point cloud media is encapsulated and exists in the track.
  • Geometry components exist in the track; parameter sets are encapsulated and exist in the track in the form of parameter components.
  • a sample is the encapsulation unit in the media file encapsulation process.
  • a track is composed of many samples.
  • a video track can be composed of many samples, and a sample is usually a video frame.
  • the media file of point cloud media contains one or more tracks, and one sample in the track corresponds to one point cloud frame.
  • the sample can be divided into one or more subsamples.
  • Each subsample can contain one type of point cloud data in the point cloud code stream.
  • the point cloud data in the point cloud code stream can contain the following types: parameter set ( Contains geometric parameter collection and attribute parameter collection), geometric data and attribute data. It can be seen that the sub-sample can contain the parameter set in the point cloud code stream, or the sub-sample can contain the geometric data in the point cloud code stream, or the sub-sample can contain the attribute data in the point cloud code stream.
  • a point cloud frame can be divided into one or more point cloud slices (or point cloud strips). The so-called point cloud slices represent a series of syntaxes of partially or fully encoded data of a point cloud frame.
  • a collection of elements (such as geometric point cloud slices, attribute point cloud slices); at this time, the attribute data and/or geometric data contained in the subsample belongs to the one or more point cloud slices;
  • the type of the subsample corresponding to the point cloud slice can include at least Two types, one is a sub-sample based on the data type carried by the point cloud piece. In this type Under this condition, a subsample only contains one data type and related information carried by the point cloud patch. For example, a subsample only contains geometric data type and geometric data related information. The other is a subsample based on point cloud slices. Under this type, a subsample can contain all the information of a point cloud slice, that is, it includes geometric slice headers and geometric data, attribute slice headers and attribute data.
  • Sample entries are used to indicate metadata information related to all samples in the track.
  • the sample entry of a video track usually contains metadata information related to decoder initialization.
  • Point cloud space block is also called hexahedral space block area within the point cloud frame boundary space area.
  • a point cloud space block is composed of one or more point cloud slices. There is no encoding and decoding dependency between point cloud space blocks. relation.
  • ISOBMFF ISO Based Media File Format, media file format based on ISO standard
  • ISOBMFF is the packaging standard for media files.
  • a typical ISOBMFF file is an MP4 file.
  • DASH Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming based on HTTP
  • DASH is an adaptive bitrate technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP network servers.
  • MPD Media Presentation Description, media presentation description signaling in DASH
  • MPD is used to describe media segment information in media files.
  • Representation refers to a combination of one or more media components in DASH.
  • a video file of a certain resolution can be regarded as a Representation; in this application, a video file of a certain time domain level can be regarded as a Representation.
  • Point cloud encoding refers to the process of encoding the geometric data and attribute data of each point in the point cloud according to the parameter set to obtain the point cloud code stream.
  • point cloud encoding can include two main processes: geometric data encoding and attribute data encoding.
  • the parameter set can include a geometric parameter set and an attribute parameter set. Therefore, point cloud coding technology can be used in the encoding process to encode the geometric data of each point in the point cloud media according to the geometric parameter set in the parameter set to obtain the geometric code stream; and use G-PCC according to the attributes in the parameter set.
  • the parameters encode the attribute data of each point in the point cloud media to obtain the attribute code stream; the geometry code stream and the attribute code stream together constitute the point cloud code stream of the point cloud media.
  • the mainstream point cloud coding technology can be divided into geometric structure-based point cloud coding and projection-based point cloud coding for different types of point clouds.
  • G-PCC Geometry-based Point Cloud Compression
  • point cloud coding based on geometric structures
  • AVS-PCC Point Cloud Coding standard
  • Point cloud decoding refers to the process of decoding the point cloud code stream obtained by point cloud encoding to reconstruct the point cloud; specifically, it refers to the process of reconstructing the point cloud code stream based on the geometry code stream and attribute code stream in the point cloud code stream.
  • the process of geometric information and attribute data of each point After obtaining the point cloud code stream on the decoding side, for the geometry code stream, first obtain the geometric parameter set required to decode the geometric data, and then decode the geometry code stream according to the geometric parameter set to obtain the geometry code stream of each point in the point cloud. Geometric data.
  • For the attribute code stream first obtain the attribute parameter set required to decode the attribute data, and then decode the attribute code stream according to the attribute parameter set to obtain the attribute data of each point in the point cloud; then, the attribute data of each point in the point cloud is obtained.
  • the point reconstructed attribute data corresponds to the reconstructed geometric data in order to reconstruct the point cloud.
  • embodiments of the present application provide a data processing solution based on point cloud media.
  • the general principle of the data processing solution for point cloud media is as follows: On the encoding side, attributes can be added to the media files of point cloud media. Parameter set indication information, and the encapsulation position of the attribute parameter set is indicated through the attribute parameter set indication information. On the decoding side, the encapsulation position of the attribute parameter set required for decoding the attribute data in the point cloud code stream can be determined based on the attribute parameter set indication information, and then the attribute parameter set can be obtained from the encapsulation position, and the attribute parameter set can be processed according to the attribute parameter set. The attribute data in the point cloud code stream is decoded to present the point cloud media.
  • the track when the point cloud code stream is encapsulated in a single track, the track may include samples, and the samples may include subsamples.
  • a set of attribute parameters may be identified in the subsample, and the set of attribute parameters and the attributes in the point cloud media may be identified. relationship between data; in this implementation, the attribute parameter set indication information in the media file is used to indicate that the encapsulation position of the attribute parameter set is a subsample, and the attribute parameter set can also be used to indicate that the attribute parameter set is related to the point cloud code Relationships between attribute data in streams.
  • the attribute parameter set indication information used to indicate the encapsulation position of the attribute parameter set may include any one or more of the following: (1) Attribute parameters The set is encapsulated as a separate track to obtain a parameter component track, and the attribute parameter set indication information is used to indicate that the encapsulation position of the attribute parameter set is the parameter component track. (2) Since the attribute component track must depend on the geometry component track, even if a certain attribute parameter set is referenced by different attribute components, when the attribute parameter set is located in the geometry component track, no additional decoding dependencies will be introduced. Under this premise, the attribute parameter set can be directly encapsulated into the geometric component track.
  • the attribute parameter set indication information is used to indicate that the encapsulation position of the attribute parameter set is the geometric component track.
  • the attribute parameter set can be copied multiple times, and the attribute parameter set is encapsulated in each attribute component track.
  • the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in each attribute component track. middle.
  • the index of the attribute parameter set can be encapsulated in each attribute component track.
  • the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in each attribute component track respectively, and can further indicate that any attribute component
  • the index encapsulated in the track is used to find the attribute parameter set, or to find the data unit formed after the attribute parameter set is reorganized.
  • the above data processing solution has the following beneficial effects: through the attribute parameter set indication information contained in the media file of the point cloud media, the encapsulation position of the attribute parameter set can be indicated, thereby guiding the transmission, decoding and presentation of the point cloud media; at the same time,
  • the codec independence between various components of point cloud media can be effectively utilized to solve the unnecessary codec dependency problem caused by encapsulating different point cloud data into different file tracks in order to support partial transmission and partial decoding, such as Geometric data is encapsulated into a geometric component track; color attribute data and reflectivity attribute data are encapsulated into an attribute component track respectively.
  • the color attribute data and reflectivity attribute data share an attribute parameter set, regardless of whether this attribute parameter set is encapsulated In which attribute component track, the encoding and decoding relationship will be generated; thus saving the use of transmission resources and computing resources.
  • the point cloud media data processing solution provided by the embodiments of the present application can also be combined with the Internet of Vehicles technology.
  • the point cloud media data processing solution can collect buildings, traffic signs, etc. in the environment, and construct a point cloud map in the vehicle for positioning, or use the point cloud map to achieve automatic navigation.
  • the data processing system 20 of point cloud media may include a content production device 201 and a media processing device 202.
  • the content production device 201 is located on the encoding side of the point cloud media; the content production device 201 may be a terminal device. device or server.
  • the media processing device 202 is located on the decoding side of the point cloud media.
  • the media processing device 202 may be a terminal device or a server.
  • a communication connection may be established between the content production device 201 and the media processing device 202.
  • the terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, a smart TV, etc., but is not limited thereto.
  • the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, and middleware. Cloud servers for basic cloud computing services such as software services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and big data and artificial intelligence platforms.
  • CDN Content Delivery Network, content distribution network
  • the specific process by which the content production device 201 and the media processing device 202 perform data processing of the point cloud media is as follows:
  • the content production device 201 mainly includes the following data processing processes: (1) the acquisition process of the point cloud media; (2) The process of encoding and file encapsulation of point cloud media.
  • the media processing device 201 mainly includes the following data processing processes: (3) the process of file decapsulation and decoding of point cloud media; (4) the rendering process of point cloud media.
  • the transmission process of point cloud media involved between the content production device 201 and the media processing device 202 can be performed based on various transmission protocols (or transmission signaling).
  • the transmission protocols here can include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming media transmission) protocol, HLS (HTTP Live Streaming, dynamic code rate adaptive transmission) protocol, SMTP (Smart Media Transport Protocol, intelligent media transmission protocol), TCP (Transmission Control Protocol) , Transmission Control Protocol), etc.
  • the content production device 201 can obtain point cloud media, and the point cloud media can be obtained through scene capture or device generation.
  • Scene capture point cloud media refers to collecting real-world visual scenes through the capture device associated with the content production device 201 to obtain point cloud media; wherein, the capture device is used to provide point cloud media acquisition services for the content production device 201, and the capture device may include But it is not limited to any of the following: camera equipment, sensing equipment, scanning equipment; where the camera equipment can include ordinary cameras, stereo cameras, light field cameras, etc.
  • Sensing devices may include laser devices, radar devices, etc. Scanning equipment may include three-dimensional laser scanning equipment, etc.
  • the capture device associated with the content production device 201 may refer to a hardware component provided in the content production device 201.
  • the capture device may be a camera, a sensor, etc. of a terminal.
  • the capture device associated with the content production device may also refer to a capture device associated with the content production device 201.
  • the connected hardware device is, for example, a camera connected to the content production device 201.
  • the device generating point cloud media means that the content production device 201 generates point cloud media based on virtual objects (such as virtual three-dimensional objects and virtual three-dimensional scenes obtained through three-dimensional modeling).
  • the content production device 201 can encode the acquired geometric data in the point cloud media based on the geometric parameter set to obtain a geometric code stream; the content production device 201 can also encode the acquired attribute data in the point cloud media based on the attribute parameter set. Process to get the attribute code stream.
  • the geometric code stream and the attribute code stream form the point cloud code stream of the point cloud media; in addition, the point cloud code stream can contain an attribute parameter set and a geometric parameter set.
  • the point cloud code stream can be encapsulated to obtain a point cloud media media file; further, the media file can also contain attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the attribute parameters.
  • the encapsulated location of the collection can be encapsulated to obtain a point cloud media media file; further, the media file can also contain attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the attribute parameters.
  • the single track may include one or Multiple samples, each sample can contain all the data of a point cloud frame in the point cloud media, including geometric data, geometric parameter sets, attribute data and attribute parameter sets; each sample is divided into one or more sub-samples.
  • the attribute parameter set can be encapsulated in the sub-sample, and the attribute parameter set indication information can be used to indicate that the attribute parameter set is encapsulated in the sub-sample.
  • a sub-sample can be included in the point cloud code stream.
  • a set of parameters, or a subsample can contain attribute data in the point cloud code stream, or a subsample can contain geometric data in the point cloud code stream.
  • any track can include one or more samples, and each sample can contain a type of point cloud frame in the point cloud media.
  • Data for example, a sample in track 1 (geometry component track) can contain geometric data in the corresponding point cloud frame, and a sample in track 2 (color component track) can contain a set of color attribute data in the corresponding point cloud frame;
  • a sample in Track 3 (Parametric Component Track) can contain a collection of attribute parameters in a point cloud frame.
  • the attribute parameter set indication information can be used to indicate that the attribute parameter set is independently packaged in the parameter component track.
  • the multiple tracks include a geometric component track and one or more attribute component tracks.
  • the attribute parameter set can be directly is encapsulated into the geometric component track, and the attribute parameter set indication information can be used to indicate that the attribute parameter set is encapsulated in the geometric component track.
  • the content production device 201 can transfer the media file to the media processing device 202, so that the media processing device 202 can learn the encapsulation position of the attribute parameter set according to the attribute parameter set indication information, and based on the attribute parameter set pair
  • the point cloud code stream decodes the attribute data.
  • the transmission signaling may contain flow indication information, which is used to indicate the parameter component.
  • the data stream corresponding to the track is transmitted to the media processing device 202 through transmission signaling.
  • the media processing device 202 can obtain the media file of the point cloud media and the corresponding media presentation description information through the content production device 201.
  • the media files and media presentation description information of the point cloud media are transmitted from the content production device 201 to the media processing device 202 through transmission signaling (such as DASH, SMT).
  • the file decapsulation process of the media processing device 202 is opposite to the file encapsulation process of the media processing device 202.
  • the media processing device 202 decapsulates the media file resources according to the file format requirements of the point cloud media to obtain the point cloud code stream.
  • the decoding process of the media processing device 202 is opposite to the encoding process of the content production device 201.
  • the media processing device 202 decodes the point cloud code stream and restores the point cloud media.
  • the media processing device 202 can obtain the attribute parameter set indication information from the media file, and can determine the encapsulation position where the attribute parameter set is located based on the attribute parameter indication information, and obtain the attribute parameter set from the encapsulation position, Then the attribute data in the media file is decoded according to the attribute parameter set, and finally the point cloud media can be restored.
  • the attribute parameter indication information can be used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream.
  • the relationship can be that the attribute parameter set only takes effect on a certain set of attribute data in the point cloud code stream.
  • the attribute parameter set can be effective only for a certain kind of attribute data in the point cloud code stream, or the attribute parameter set can be effective for all attribute data in the point cloud code stream.
  • effectiveness means effectiveness, that is, the attribute data in the point cloud code stream can be decoded according to the effective attribute parameter set.
  • the point cloud code stream can include multiple sets of attribute data.
  • multiple sets of attribute data include: a set of yellow attribute data, a set of red attribute data, and a set of reflectivity attribute data.
  • the set of attribute parameters is only for point clouds.
  • Validation of a certain set of attribute data in the code stream can mean that it is only valid for a set of yellow attribute data; and that the set of attribute parameters is only valid for a certain type of attribute data in the point cloud code stream can mean that it is valid for color attribute data. Since yellow Both attribute data and red attribute data belong to color attribute data, so the attribute parameter set only takes effect on a certain type of attribute data in the point cloud code stream, which can include taking effect on both yellow attribute data and red attribute data.
  • the media processing device 202 renders the decoded point cloud media according to the metadata related to rendering and window in the media presentation description information, obtains the point cloud frame of the point cloud media, and presents the point cloud media according to the presentation time of the point cloud frame.
  • the content production device side first samples the real-world visual scene A through the collection device to obtain the point cloud source data B of the point cloud media corresponding to the real-world visual scene.
  • the cloud source data B is a frame sequence composed of a large number of point cloud frames; then the obtained point cloud media is encoded according to the parameter set (i.e., the geometric parameter set and the attribute parameter set) to obtain the point cloud code stream E (including the encoded geometry code stream and attribute code stream), optionally, the point cloud code stream E may also include a parameter set; then, the point cloud code stream E is encapsulated to obtain the media file corresponding to the point cloud media.
  • the content production device 201 According to the specific media container file format, one or more encoded code streams are synthesized into a media file F for file playback, or an initialization segment and a sequence of media segments (FS) for streaming transmission; where, the media container
  • the file format may refer to the ISO basic media file format specified in the International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) 14496-12.
  • the media file may also contain parameter set indication information, and the parameter set indication information is used to indicate the encapsulation position of the parameter set.
  • the content production device also encapsulates the metadata into a media file or a sequence of initialization/media fragments, and transmits the sequence of initialization/media fragments through transmission signaling (such as a dynamic adaptive streaming media transmission interface) to media processing device 202.
  • transmission signaling such as a dynamic adaptive streaming media transmission interface
  • the media processing device side first receive the media file sent by the content production device 201.
  • the media file may include: a media file F′ for file playback, or an initialization segment and a sequence of media segments Fs′ for streaming; Then the media file is decapsulated to obtain the point cloud code stream E'; then the parameter set is obtained according to the encapsulation position indicated by the parameter set indication information contained in the media file, and the point cloud code stream is decoded according to the parameter set (that is, according to the parameter set Decode the attribute data in the point cloud code stream using the attribute parameter set in the parameter set and decode the geometric data in the point cloud code stream according to the geometric parameter set in the parameter set) to obtain the point cloud media D'; in the specific implementation , the media processing device determines the media files or media fragment sequences required to present the point cloud media based on the viewing position/viewing direction of the current object; and decodes the media files or media fragment sequences required to present the point cloud media, Get the point cloud media you need for rendering.
  • the decoded point cloud media is rendered to obtain the point cloud frame A′ of the point cloud media, and is displayed on the head-mounted display carried by the media processing device according to the presentation time of the point cloud frame. or any other display device to render point cloud media on the screen.
  • the viewing position/viewing direction of the current object is determined by the head following and possibly the visual following function.
  • the audio decoder can also be used to decode and optimize the audio in the viewing (window) direction of the current object. Among them, during the window-based transmission process, the current viewing position and viewing direction are also passed to the strategy module for determining the track to be received.
  • Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize data calculation, storage, processing, and sharing.
  • the media file of the point cloud media is obtained.
  • the media file includes the point cloud code stream of the point cloud media and a set of attribute parameters.
  • the set of attribute parameters includes the parameters required for decoding the attribute data in the point cloud code stream.
  • Information; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set.
  • Presenting point cloud media guides the transmission, decoding and presentation of point cloud media, which can reduce unnecessary encoding and decoding dependencies and save transmission and computing resources.
  • FIG. 3 is a schematic flowchart of a point cloud media data processing method provided by an embodiment of the present application. This method can be executed by the above-mentioned media processing device.
  • the point cloud media data processing method described in this embodiment can include the following S301-S302:
  • the media file includes the point cloud code stream of the point cloud media and the attribute parameter set.
  • the attribute parameter set includes the parameter information required for decoding the attribute data in the point cloud code stream; the media file also includes Attribute parameter set indication information, the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.
  • the attribute parameter set indication information can also be used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream, that is, the attribute parameter set indication information can indicate the attribute parameter set and the attribute parameter set.
  • the attribute parameter set indication information indicates that the attribute parameter set is effective for a specific type of attribute data (such as color) in the point cloud code stream; for another example, the attribute parameter set indication information indicates the attribute parameter set. It is effective for a specific set of attribute data (such as a set of yellow attribute data) in the point cloud code stream; for another example, the attribute parameter set indication information indicates that the attribute parameter set is effective for all attribute data in the point cloud code stream.
  • the media file is obtained by encapsulating the point cloud code stream of the point cloud media.
  • the encapsulation position of the attribute parameter set has the following implementation methods (1)-(2):
  • the set of attribute parameters can be encapsulated in the sub-sample information data box.
  • the media file When using a single-track method to encapsulate the point cloud code stream of point cloud media into the same track, the media file includes a track, which can contain one or more samples.
  • One sample corresponds to one point cloud frame in the point cloud media; one The sample can be divided into one or more sub-samples, and each sub-sample can include any type of point cloud data in the point cloud code stream.
  • the sub-sample can contain parameter sets, or the sub-sample can contain attribute data or geometry.
  • the media file may also include a sub-sample information data box, and the attribute parameter set may be encapsulated in the sub-sample information data box.
  • the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in the sub-sample information data box.
  • the attribute parameter set indication information can be represented as a sub-sample information data box, and the sub-sample information data box can contain at least one of the following fields: payload type field, reference attribute information flag field, reference attribute type flag field, reference attribute identification field and reference attribute type fields.
  • the subsample information data box can be used when encapsulating the point cloud code stream; when a point cloud frame can be When divided into one or more point cloud slices, the attribute data and/or geometric data contained in the subsample belongs to the one or more point cloud slices; the subsample information data box may contain a flag field of the subsample information data.
  • a subsample can be defined according to the value of the flag field (flag) of the subsample information data.
  • the flag field specifies the type of subsample in the subsample information data box: if the flag field is the first preset value (such as 0), Then the subsample refers to a subsample based on the data type carried by the point cloud slice.
  • a subsample only contains one data type and related data.
  • a subsample only contains geometric data type and geometric data.
  • the flag field is the second preset value (such as 1), then the subsample refers to the subsample based on the point cloud slice.
  • a subsample only contains all relevant data of a point cloud slice, that is, it includes the geometric slice header and geometric data. , attribute header and attribute data.
  • the flag field can hold other flag values.
  • codec_specific_parameters (codec specific specifications) field definition of the sub-sample information data box can be as shown in Table 1:
  • PayloadType This payload type field is used to indicate the type of point cloud data contained in the subsample; if the payload type field is the first value (such as 2), it indicates that the subsample contains point clouds. The type of data is a set of attribute parameters; if the payload type field is the second value (such as 3), it indicates that the type of point cloud data contained in the subsample is a set of geometric parameters; if the payload type field is the third value If the payload type field is the fourth value (such as 1), it indicates that the type of point cloud data contained in the subsample is geometry. data. It should be understood that the first value, the second value, the third value, and the fourth value can be set arbitrarily according to requirements. For example, the first value can be 0, the second value can be 1, and the third value can be 0. The value may be 2, and the fourth value may be 3, which is not limited in the embodiment of the present application.
  • Attribute presentation flag field (attribute_present_flag): This attribute presentation flag field can be used to indicate whether the subsample contains color attribute data and/or reflectance attribute data. For definitions, please refer to AVS (China National Audio and Video Coding Standard AVS, Audio Video Coding Standard )-PCC. If the value of the attribute presentation flag field is the first setting value (such as 0), it indicates whether the subsample contains the color attribute; if the value of the attribute presentation flag field is the second setting value (such as 1), it indicates whether the subsample contains the color attribute. Contains reflectivity properties.
  • Point cloud slice data field (slice_data): This field indicates whether the subsample contains point cloud slice data. If the value of the point cloud patch data field is the first set value (such as 1), it indicates that the subsample contains point cloud patch geometry and/or attribute type data; if the value of the point cloud patch data field is the second set value (such as 0), indicates that the subsample does not contain point cloud parameter information.
  • Point cloud slice identification field (slice_id): This field indicates the identity of the point cloud slice corresponding to the data contained in the subsample.
  • Reference attribute information flag field (ref_attr_info_flag): This reference attribute information flag field is used to indicate the effective range of the current attribute parameter set for attribute data; if the reference attribute information flag field is the first preset value (such as 1), it indicates the current attribute parameter The set is only effective for a specific group (i.e. a certain group) or a specific type (i.e. a certain category) of attribute data in the current sample; if the reference attribute information flag field is the second default value (such as 0), then Indicates that the current attribute parameter set takes effect on all attribute data in the current sample;
  • Reference attribute type flag field (ref_attr_type_flag): This reference attribute type flag field is used to indicate the type of attribute data that the current attribute parameter set takes effect on; if the reference attribute type flag field is the first value (such as 1), it indicates the current attribute The parameter set is only effective for a specific type of attribute data in the current sample; if the reference attribute type flag field is the second value (0), it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample.
  • Reference attribute identification field (ref_attr_id): This field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set, which can be understood as the identifier of the attribute data that is valid for the current attribute parameter set.
  • Reference attribute type field (ref_attr_type): This field is used to indicate the attribute data corresponding to the current attribute parameter set.
  • the current attribute parameter set refers to the attribute parameter set being used
  • the current sample refers to the sample being decoded
  • the encapsulation position of the attribute parameter set can be as follows a-d:
  • the attribute parameter collection is independently encapsulated in the parameter component track.
  • the embodiment of the present application can define parameter component tracks, and the point cloud code stream is encapsulated into the media file using a type-based multi-track method.
  • the media file can Including parameter component track, this attribute parameter collection is independently encapsulated in the parameter component track.
  • the sample entry type of the parameter component track can be 'apcp' or 'gpp1'; included in: sample description data box (SampleDescriptionBox); mandatory: no; quantity: one or more.
  • the attribute parameter set indication information may be used to indicate that the attribute parameter set is independently packaged into the parameter component track.
  • the parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains parameter information required for decoding the attribute data of the corresponding point cloud frame.
  • the media file includes one or more parameter component tracks; in a feasible implementation, when the media file only includes one parameter component track, the attribute parameter set in the point cloud code stream is encapsulated into the Parameter component track. It can be understood that in this implementation, the geometric parameter set in the point cloud code stream can also be encapsulated in the parameter component track, that is, both the geometric parameter set and the attribute parameter set of the point cloud code stream can be encapsulated. in the parameter component track. In another feasible implementation, when the media file includes multiple parameter component tracks, the geometric parameter set in the point cloud code stream can be encapsulated into a separate parameter component track, and the attributes in the point cloud code stream Parameter collections can be encapsulated into a separate parameter component track.
  • the geometric parameter set in the point cloud code stream can be encapsulated into a single parameter component track, and the attributes in the point cloud code stream
  • the parameter set can be encapsulated into multiple parameter component tracks based on type.
  • the attribute parameter set includes a color attribute parameter set and a reflectance attribute parameter set.
  • the color attribute parameter set can be encapsulated into A separate Parametric Component track, and a collection of reflectance property parameters can be packaged into another separate Parametric Component track.
  • the parameter component track may contain a track data box (TrackBox), and the track data box may contain a track reference type data box (TrackReferenceTypeBox).
  • the reference type (reference_type) of the track reference type data box is 'appr'.
  • the track reference type data box is used to indicate that the parameter component track is associated with one or more attribute component tracks (ie, point cloud data tracks) corresponding to the attribute parameter set in the parameter component track.
  • the track reference type data box is used to reference or associate one or more attribute component tracks corresponding to the attribute parameter set in the parameter component track. For example, if the attribute parameter set in the parameter component track includes color parameters and reflectance parameters, then the track reference type data box is used to reference or associate the color attribute component track and reflection corresponding to the attribute parameter set in the parameter component track. Emissivity attribute component track.
  • the parameter component track when the media file complies with the first encoding standard (such as the AVS encoding standard), the parameter component track includes a component information data box (AVSPCCComponentInfoBox), and the component information data box is set at the header sample entry ( AvsPccParaHeaderSampleEntry); the component information data box contains at least one of the following fields: component type field, header type field, attribute header quantity field, reference attribute type flag field, reference attribute identification field and reference attribute type field; the component information data box
  • the syntax can be found in Table 2:
  • Component type field (avs_pcc_type): This component type field is used to indicate the component type in the track; when the component type field is set to the target preset value (such as 5), it indicates that the component type in the parameter component track is a parameter collection. Among them, the corresponding relationship between the value of the component type field and the component type can be shown in Table 3:
  • Attribute type field This attribute type field is used to indicate the type of attribute components contained in the track. If the attribute type field is the first value (such as 0), it indicates that the track only contains color attribute data; if If the attribute type field is the second value (such as 1), it indicates that only reflectance attribute data is included; if the attribute type field is the third value (such as 2), it indicates that both color attribute data and reflectance attribute data are included.
  • Header type field This header type field is used to indicate the type of parameter set contained in the parameter component track; if the header type field is the first setting value (such as 0), it indicates the type of parameter set contained in the parameter component track.
  • the parameter collection type is an attribute parameter collection; if the header type field is the second setting value (such as 1), it indicates that the parameter collection type contained in the parameter component track is a geometric parameter collection; if the header type field is the third setting value A fixed value (such as 2) indicates that the parameter collection type contained in the parameter component track is a geometric parameter collection and an attribute parameter collection.
  • the value of the header type field can only be the first set value and the second set value.
  • Attribute header number field (attr_header_num): used to indicate the number of attribute parameter sets contained in the sample in the parameter component track.
  • the number of attribute parameter sets can be 1, 2, etc.; among them, the attribute parameters included in the sample
  • the order of the sets is consistent with the identifier of the attribute parameter set or the order of the attribute parameter set types indicated in the component information data box; for example, the order of the attribute parameter sets contained in the sample is attribute parameter set 1, attribute parameter set 2, then the The order of the identifiers of the attribute parameter sets indicated in the component information data should also be the identifier of the attribute parameter set 1 and the identifier of the attribute parameter set 2.
  • Reference attribute type field (ref_attr_type_flag): This reference attribute type field is used to indicate the type of attribute data that the current attribute parameter set takes effect on; if the reference attribute type flag field is the first preset value (such as 0), it indicates the current attribute parameter set It is only effective for a specific type of attribute data in the current sample; if the reference attribute type flag field is the second preset value (such as 1), it means that the current attribute parameter set is effective for a specific set of attribute data in the current sample.
  • Reference attribute identification field (ref_attr_id): The reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set.
  • Reference attribute type (ref_attr_type): This reference attribute category is used to indicate the type of attribute data corresponding to the current attribute parameter set;
  • the current attribute parameter set refers to the attribute parameter set being used
  • the current sample refers to the sample being decoded
  • the media file may comply with a second encoding standard (such as the MPEG encoding standard);
  • the track contains the component information data box (GPCCComponentInfoBox) and the configuration data box (GPCCConfigurationBox); among them, the syntax of the configuration data box can be shown in Table 4:
  • Configuration version field (configurationVersion): This configuration version field is used to configure record version information; if the version information of the configuration version field in the corresponding track cannot be recognized by the player/decoder, the player/decoder should not try to access the corresponding track. to decode.
  • Profile field This profile field is used to indicate the profile information of the code stream corresponding to the current track.
  • Profile compatibility field (profile_compatibility_flags): This profile compatibility field is used to indicate the profile compatibility information of the code stream corresponding to the current track.
  • Setup unit number field (num_of_setup_unit_arrays): This setup unit number field is used to indicate the number of parameter sets included in the current decoder configuration record. Each parameter set corresponds to a type of parameter set.
  • Setup unit type field (setup_unit_type): The setup unit type field is used to indicate the parameter set type corresponding to the current parameter set group.
  • Parameter set indication field (array_completeness): This parameter set indication field is used to indicate that the current type of parameter set contained in the corresponding point cloud code stream is indicated in the decoder configuration record.
  • the parameter set indication field is the first value (such as 1), it indicates that all parameter sets of the current type contained in the corresponding code stream are indicated in the decoder configuration record.
  • the parameter set indication field is the first value (such as 0), it means that part of the current type of parameter set contained in the corresponding code stream is indicated in the decoder configuration record, and part is indicated in the code stream.
  • Setup unit number field (num_of_setup_units): This setup unit number field is used to indicate the number of parameter sets of the current type. Among them, the setup unit (setupUnit) contains data units of SPS (sequence parameter set), GPS (geometric parameter set), APS (attribute parameter set) or point cloud block information.
  • SPS sequence parameter set
  • GPS geometric parameter set
  • APS attribute parameter set
  • point cloud block information point cloud block information
  • the parameter set type corresponding to the setup unit contained in the configuration data box is the same as the parameter set type indicated by the component information data box;
  • the setup unit (setupUnit) may include a sequence parameter set (SPS), a geometric parameter set (i.e. GPS), attribute parameter set (i.e. APS) or point cloud block information data unit.
  • SPS sequence parameter set
  • a geometric parameter set i.e. GPS
  • attribute parameter set i.e. APS
  • point cloud block information data unit point cloud block information data unit.
  • the above component information data box contains at least one of the following fields: component type field, parameter setting type quantity field, and parameter setting type field.
  • the syntax of the component information data box is shown in Table 5:
  • the meanings of the fields contained in the component information data box are as follows:
  • Component type field (gpcc_type): This field is used to indicate the type of the component in the track; if the component type field is the target setting value (such as 5), it indicates that the type of the component in the parameter component track is a parameter set, where the component
  • the corresponding relationship between the value of the type field and the component type can be shown in Table 6:
  • Parameter set type number field (num_para_set_type): This field is used to indicate the number of parameter sets contained in the parameter component track. For example, the number of parameter sets contained in the parameter component track can be 1, 2, etc.
  • the media file includes multiple parameter component tracks, for example, the geometric parameter set is individually encapsulated into one parameter component track, and the attribute parameter set is individually encapsulated into another parameter component track. At this time, each parameter component track only Contains a type of parameter collection, then you can set the type quantity field without parameters.
  • Parameter setting type field (parameter_set_type): This parameter setting type field is used to indicate the type of parameter set contained in the parameter component track; if the parameter setting type field is the first preset value (such as 0), it means that the parameter component track contains The type of the parameter collection is an attribute parameter collection; if the parameter setting type field is the second preset value (such as 1), it means that the type of the parameter collection contained in the parameter component track is a geometric parameter collection; if the parameter setting type field is the second preset value (such as 1) Three default values (such as 2) indicate that the type of parameter set contained in the parameter component track is a sequence parameter set.
  • Attribute index field (attr_index): This attribute index field is used to indicate the sequence number of the attribute parameter indicated in the SPS (ie, attribute parameter set).
  • Attribute type presentation field (attr_type_present_flag), this attribute type presentation field is used to indicate whether the attribute data type information is indicated in the component information data box; if the attribute type presentation field is the first value (such as 1), it indicates the component information data box The attribute data type information is indicated in; if the attribute type presentation field is the second value (such as 0), it means that the attribute data type information is indicated in the component information data box.
  • the relationship between four bytes and attribute data types is shown in Table 7:
  • Attribute type field (attr_type): This attribute type field is used to indicate the type of attribute component. Its value refers to the definition in the ISO/IEC 23090-9 [GPCC] standard.
  • Attribute name field (attr_name): This attribute name field is used to indicate intuitively interpretable (human-readable) attribute component type information.
  • the attribute parameter set is encapsulated in the geometric parameter component track.
  • the above media file can include a geometry component track and at least one attribute component track; among them, the track defined by ISO/IEC14496-12 [ISOBMFF] needs to be used between the geometry component track and the attribute component track in AVS PCC Reference tools for relational indexing.
  • ISO/IEC14496-12 ISO/IEC14496-12
  • the geometry component track can include one or more samples.
  • the decoder configuration data box contains SPS (Sequence Parameter Set), GPS, APS and other parameter collections, and the decoder configuration data box should not be included in the attribute component track.
  • the set of attribute parameters should be included in the corresponding sample of the geometry component track.
  • the attribute component track should not contain any type of attribute parameter collection, only the corresponding type of attribute data.
  • the modified fields are as follows: Component type field (gpcc_type): This component type field is used to indicate the type of component in the track; the value of this component type field is shown in Table 8 below. :
  • the attribute parameter set is encapsulated in the geometric component track.
  • the geometric component track can include a component information data box.
  • the component information data box includes a component type field, and the component type field is For indicating the type of the component in the geometric component track, if the component type field is the first setting value (such as 2), it indicates that the type of the component in the geometric component track is a set of geometric data and parameters, and the parameter set can include attribute parameters. Collections and geometric parameter collections.
  • the component type field included in the component information data box is the second setting value (such as 4), indicating that the types of components in the attribute component track are all attribute data and do not include parameter sets.
  • the attribute parameter set can be encapsulated into each attribute component track.
  • the media file may include one or more property component tracks, and the property parameter set may be copied in each property component track, so that each property component track contains a corresponding property parameter set.
  • the attribute parameter set is encapsulated in each attribute track respectively; the attribute parameter set indication information can be used to indicate that the attribute parameter set is encapsulated in each attribute track respectively.
  • the attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set. That is to say, when the number of attribute component tracks is N, N copies of the attribute parameter set can be directly copied, and one copy of the attribute parameter set is placed in each attribute component track.
  • the index of the attribute parameter collection can be encapsulated into each attribute component track.
  • the attribute parameter set includes an index, and the attribute parameter set can be repeatedly indexed in each attribute component track, that is, the index of the attribute parameter set is set in each attribute component track, and the attribute parameter set indication information is used to indicate the index of the attribute parameter set.
  • the point cloud code stream can be reorganized so that samples of different types of attribute component tracks and corresponding attribute parameter sets are indexed into the same data unit.
  • the point cloud code stream includes geometric parameter set (GPS), geometry data (Geometry), attribute parameter set (APS), attribute data 1 (Attribute1), attribute data 2 (Attribute2); for point cloud code
  • the geometric parameter set and the samples in the geometric component track to which the geometric data belong can be indexed to the same data unit 41; the attribute parameter set and the samples in the attribute component track to which the attribute data 1 belongs can be indexed to the same data unit 41.
  • Data unit 42 index the attribute parameter set and the sample in the attribute component track to which attribute data 2 belongs to the same data unit 43.
  • attribute data can be The samples in the attribute component track to which attribute data 1 belongs and the samples in the attribute component track to which attribute data 2 belongs are indexed to the same data unit 45; in this data unit 45, the index and attribute encapsulated in the attribute component track to which attribute data 1 belongs The indexes encapsulated in the attribute component track to which data 2 belongs can all be indexed to the data unit 45 .
  • obtaining the media file of the point cloud media may include: obtaining the transmission signaling of the point cloud media, and then obtaining the media file of the point cloud media according to the transmission signaling.
  • the transmission signaling when transmitting media files through transmission signaling, due to the streaming transmission method for media file transmission, when attribute data needs to be decoded, the data stream corresponding to the parameter component track needs to be requested to obtain the corresponding attribute parameter set to achieve decoding.
  • Point cloud media corresponding media file transfer guidance.
  • the transmission signaling also contains flow indication information, and the flow indication information is used to indicate the data corresponding to the parameter component track. flow.
  • the data stream may be a Representation
  • the transmission signaling may be DASH signaling
  • the stream indication information may refer to a component descriptor (GPCCComponent descriptor), which is set in the data stream corresponding to the parameter component track.
  • the component descriptor includes a component type element and a parameter type element.
  • the component type element is used to indicate the type of component in the parameter component track; if the component type element (component@component_type) is a target character (such as 'prms') , then the type of component in the parameter component track is parameter component.
  • the parameter type element is used to indicate the type of parameter set contained in the parameter component track.
  • Table 9 the attributes and elements of the component descriptor As shown in Table 9:
  • M Mandatory mandatory field
  • CM Conditional Mandatory condition mandatory
  • O Optional optional field.
  • S302 Decode the point cloud code stream according to the attribute parameter set indication information to present the point cloud media.
  • decoding the point cloud code stream to present point cloud media according to the attribute parameter set indication information may include: obtaining the attribute parameter set according to the encapsulation position indicated by the attribute parameter set indication information; and then The attribute data in the point cloud code stream is decoded using the attribute parameter set to present the point cloud media.
  • the encapsulation position indicated by the attribute parameter set indication information may be decoded first to obtain the attribute parameter set.
  • the encapsulation position indicated by the attribute parameter set indication information is the parameter component track, and the parameter component track needs to be decoded to obtain the attribute parameter set. Then, determine the attribute data that needs to be decoded according to the requirements, and use the attribute parameter set to decode the attribute data. In this way, the relevant point cloud media can be partially decoded, saving decoding resources.
  • the media file of the point cloud media is obtained.
  • the media file includes the point cloud code stream of the point cloud media and a set of attribute parameters.
  • the set of attribute parameters includes the parameters required for decoding the attribute data in the point cloud code stream.
  • Information; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set.
  • Presenting point cloud media guides the transmission, decoding and presentation of point cloud media, which can reduce unnecessary encoding and decoding dependencies and save transmission and computing resources.
  • Figure 5 is a schematic flowchart of a point cloud media data processing method provided by an embodiment of the present application.
  • the data processing method of point cloud media can be executed by a content production device.
  • the data processing method of the point cloud media may include the following S501-S503:
  • S502. Encode the point cloud media to obtain a point cloud code stream and an attribute parameter set.
  • the attribute parameter set contains parameter information required for decoding attribute data in the point cloud code stream.
  • attribute parameter indication information can be added to the media file, and the attribute parameter set indication information can be used to indicate Indicates the encapsulation position of the attribute parameter set; in addition, the attribute parameter set indication information can also be used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream, such as the relationship between the attribute parameter set and a certain point cloud code stream.
  • a type of attribute data or a certain group takes effect.
  • Encapsulating the point cloud code stream and obtaining the media file of the point cloud media can include the following implementation methods:
  • the media file includes a track.
  • the point cloud code stream is encapsulated in the track.
  • the track contains one or more samples.
  • One sample corresponds to one point cloud frame in the point cloud media; one sample is divided into one or more subsamples. ;
  • Each subsample contains one type of point cloud data;
  • the media file also includes a subsample information data box, and the attribute parameter set is encapsulated in the subsample information data box.
  • content production equipment can use a single-track method to encapsulate the point cloud code stream into a track.
  • the track contains one or more samples, and one sample corresponds to a point cloud frame in the point cloud media; then each sample Divide it into one or more sub-samples; when the single-track method is used to encapsulate the point cloud code stream of the point cloud media to the same track, the sub-sample information data box can be used when encapsulating the point cloud code stream, and the point cloud code stream can be
  • the attribute parameter set in is set in the sub-sample information data box to form a media file of the point cloud media; wherein, the attribute parameter set indication information is used to indicate that the attribute parameter set is set in the sub-sample information data box.
  • the sub-sample information data box may contain a flag field of the sub-sample information data.
  • the attribute data and/or geometric data contained in the sub-sample belongs to the one or more point cloud slices; according to the flag field (flag) of the sub-sample information data ) defines a subsample, and the flag field specifies the type of subsample in the subsample information data box: if the flag field is the first preset value (such as 0), then the subsample is based on the point cloud piece.
  • Subsamples of data types At this time, a subsample only contains one data type and related data.
  • a subsample only contains geometric data types and geometric data. If the flag field is the second preset value (such as 1), then the subsample refers to the subsample based on the point cloud slice. At this time, a subsample only contains all relevant data of a point cloud slice, that is, it includes the geometric slice header and geometric data. , attribute header and attribute data.
  • the sub-sample information data box may contain at least one of the following fields: payload type field, reference attribute information flag field, reference attribute type flag field, reference attribute identification field and reference attribute type field.
  • the payload type field is used to indicate the type of point cloud data contained in the sub-sample package;
  • the reference attribute information flag field is used to indicate the effective range of the current attribute parameter set for attribute data;
  • the reference attribute type flag field is used to indicate the current attribute parameter set pair
  • the reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set;
  • the reference attribute type field is used to indicate the type of attribute data corresponding to the current attribute parameter set;
  • setting the attribute parameter set in the point cloud code stream in the sub-sample information data box may include: if the type of point cloud data contained in the sub-sample is an attribute parameter set, then setting the payload field to the first value (such as 2); if the type of point cloud data contained in the sub-sample is a geometric parameter set, set the payload field to the second value (such as 3); if the current attribute parameter set is only for a specific group in the current sample (i.e. A certain group) or a specific type (i.e.
  • the reference attribute information flag field is set to the first default value (such as 1); if the current attribute parameter set is valid for all the attributes in the current sample If the attribute data is valid, set the reference attribute information flag field to the second preset value (such as 0); if the current attribute parameter set is only valid for a specific type of attribute data in the current sample, set the reference attribute type flag field to The first value (such as 1); if the current attribute parameter set is only effective for a specific set of attribute data in the current sample, the reference attribute type flag field will be the second value (such as 0).
  • the current attribute parameter set refers to the current The set of attribute parameters used.
  • the current sample refers to the sample being encoded.
  • the point cloud code stream is encapsulated into the media file using a multi-track method; the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track; the parameter component track contains one or more samples, each The sample corresponds to a point cloud frame of the point cloud media, and each sample contains parameter information required for decoding the attribute data of the corresponding point cloud frame.
  • This attribute parameter set indication information is used to indicate that the attribute parameter set is independently packaged into the parameter component track.
  • the media file includes one or more parameter component tracks; in a feasible implementation, when the media file only includes one parameter component track, the attribute parameter set in the point cloud code stream is encapsulated into the parameter component track. ; It can be understood that in this implementation, the geometric parameter set in the point cloud code stream can also be encapsulated in the parameter component track, that is, both the geometric parameter set and the attribute parameter set of the point cloud code stream can be Encapsulated in this parameter component track. In another feasible implementation, when the media file includes multiple parameter component tracks, the geometric parameter set in the point cloud code stream can be encapsulated into a separate parameter component track, and the attributes in the point cloud code stream Parameter collections can also be encapsulated into a separate parameter component track.
  • the geometric parameter set in the point cloud code stream can be encapsulated into a single parameter component track, and the attributes in the point cloud code stream Parameter collections can be packaged into multiple parameter component tracks based on type.
  • the attribute parameter set in the point cloud code stream can be encapsulated into the parameter component track in a type-based multi-track manner, and the geometric data in the point cloud code stream can be encapsulated into the geometry component track, and the point cloud code stream can be encapsulated into the geometric component track.
  • the attribute data is encapsulated into the attribute component track, and finally forms a media file of the point cloud media; the media file can include a parameter component track, an attribute component track, and a geometry component track.
  • the parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains parameter information required for decoding the attribute data of the corresponding point cloud frame.
  • the attribute component track can contain one or more samples.
  • Each sample corresponds to a point cloud frame of the point cloud media. Each sample contains the attribute data of the corresponding point cloud frame.
  • the geometry component track can contain one or more samples. Each sample corresponds to a point cloud frame of point cloud media, and each sample contains the geometric data of the corresponding point cloud frame.
  • the parameter component track includes a track data box; the track data box contains a track reference type data box, and the track reference type data box is used to indicate one or more attribute components corresponding to the parameter component track and the attribute parameter set in the parameter component track. track associated.
  • the parameter component track can contain a component information data box.
  • the component information data box contains at least one of the following fields: component type field, header type field, and attribute header quantity field. , reference attribute type flag field, reference attribute identification field and reference attribute type field.
  • the component type field is used to indicate the component type in the parameter component track;
  • the header type field is used to indicate the parameter collection type contained in the parameter component track;
  • the attribute header quantity field is used to indicate the attribute parameters contained in the sample in the parameter component track
  • the reference attribute type field is used to indicate the current attribute parameter set pair
  • the reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set;
  • the reference attribute type is used to indicate the type of attribute data corresponding to the current attribute parameter set;
  • the current attribute parameter set refers to the The set of attribute parameters used, the current sample refers to the sample being encoded.
  • multi-track encapsulation of the attribute parameter collection in the point cloud code stream into the parameter component track may include: if the component type in the parameter component track is a parameter collection, set the component type field to the target preset value; if The parameter collection type contained in the parameter component track is an attribute parameter collection, then the header type field is set to the first setting value; if the parameter If the parameter set type contained in the parameter component track is a geometric parameter set, set the header type field to the second setting value; if the parameter set type contained in the parameter component track is a geometry parameter set and an attribute parameter set, set the header type field to the second setting value.
  • the Department Type field is set to the third set value.
  • the reference attribute type flag field is set to the first preset value (such as 1); if the current attribute parameter set is effective for a specific group of attribute data in the current sample When the attribute data takes effect, the reference attribute type flag field is set to the second preset value (such as 0).
  • the parameter component track contains the component information data box and the configuration data box; the parameter set type corresponding to the setting unit contained in the configuration data box is the same as the parameter indicated by the component information data box.
  • the set types are the same; the setup unit (setupUnit) can contain data units of sequence parameter set (SPS), geometric parameter set (i.e. GPS), attribute parameter set (i.e. APS) or point cloud block information.
  • the component information data box contains at least one of the following fields: component type field, parameter setting type quantity field, and parameter setting type field.
  • the component type field is used to indicate the type of the component in the parameter component track;
  • the parameter setting type quantity field is used to indicate the number of parameter sets contained in the parameter component track;
  • the parameter setting type field is used to indicate the parameter set contained in the parameter component track type.
  • encapsulating the attribute parameter set in the point cloud code stream into the parameter component track may include: if the type of the component in the parameter component track is a parameter set, then setting the component type field to the target setting value; if the parameter component If the type of the parameter collection contained in the track is an attribute parameter collection, then the parameter setting type field is set to the first preset value (such as 0); if the type of the parameter collection contained in the parameter component track is a geometric parameter collection, then the parameter setting type field is set to the first preset value (such as 0). The type field is set to the second preset value (for example, 1); if the type of the parameter set contained in the parameter component track is a sequence parameter set, the parameter set type field is set to the third preset value (for example, 2).
  • the media file includes a geometric component track, and the attribute parameter set is encapsulated in the geometric component track.
  • the attribute parameter set can be encapsulated into the geometric parameter component track.
  • the attribute parameter set and geometric data in the point cloud code stream are encapsulated into the geometric component track, and the attribute data in the point cloud code stream are encapsulated into the attribute component track to form a media file of the point cloud media.
  • the media file can include geometry component tracks and attribute component tracks.
  • the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in the geometric component track.
  • the geometric component track includes one or more samples, each sample corresponds to a point cloud frame, and each sample contains parameter information required for decoding attribute data of a corresponding point cloud frame.
  • the geometric component track includes a component information data box, and the component information data box includes a component type field.
  • the component type field is used to indicate the type of component in the geometric component track; if the component type in the geometric component track is geometry Data and parameter collection, you can set the component type field to the first setting value (such as 2).
  • the media file includes one or more attribute component tracks, and the attribute parameter set is encapsulated in each attribute component track; among them, the attribute parameter set encapsulated in each attribute component track is passed by the same attribute parameter set. Copied.
  • a multi-track method can be used to encapsulate the attribute data in the point cloud code stream into one or more attribute component tracks, copy the attribute parameter set, and then encapsulate a copy of the attribute parameters in each attribute component track.
  • the media file includes one or more attribute component tracks, the attribute parameter set includes an index, and the attribute parameter set The index of is encapsulated in each property component track.
  • a multi-track method is used to encapsulate the attribute data in the point cloud code stream into one or more attribute component tracks, and encapsulate the index of the attribute parameter set in the point cloud code stream into each attribute component track to form a point cloud code stream.
  • the data unit formed after the attribute parameter set is reorganized refers to reorganizing the point cloud code stream so that the samples of different types of attribute component tracks and the corresponding attribute parameter sets are indexed into the same data unit.
  • the media file of the point cloud media is sent through transmission signaling. If the attribute parameter set is independently encapsulated into the parameter component track, then the transmission signaling contains flow indication information. The stream indication information is used to indicate the data stream corresponding to the parameter component track.
  • the transmission signaling can be DASH signaling;
  • the above-mentioned flow indication information can refer to a component descriptor, and the component descriptor is set in the data stream corresponding to the parameter component track;
  • the component descriptor includes a composition type element and a parameter type element, consisting of The type element is used to indicate the type of the component in the parameter component track; if the type of the component in the parameter component track is a parameter collection component, the component type element in the component descriptor is set to the target character.
  • the parameter type element is used to indicate the type of parameter collection contained in the parameter component track.
  • the point cloud media is obtained, and the point cloud media is encoded to obtain the point cloud code stream and the attribute parameter set.
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream.
  • the point cloud code stream is encapsulated to obtain a point cloud media media file; the media file also contains attribute parameter set indication information, which is used to indicate the encapsulation position of the attribute parameter set; add the attribute parameter set indication in the media file
  • the information can help the media processing device directly obtain the attribute parameter set according to the attribute parameter set indication information, thereby guiding the transmission, decoding and presentation of point cloud media and reducing certain codec dependencies.
  • Content production equipment can collect point cloud media and encode the attribute data in the point cloud media according to the attribute parameter set to obtain the attribute code stream; encode the geometric data in the point cloud media according to the geometric parameter set to obtain the geometric code Stream, the attribute code stream and the geometry code stream are combined to generate a point cloud code stream, which also includes a set of attribute parameters; then, when encapsulating the file of the point cloud code stream, the point cloud code stream is multi-tracked.
  • the attribute parameter set in the cloud code stream is encapsulated into the parameter component track, the attribute data in the point cloud code stream is encapsulated into the attribute component track, and the geometric data in the point cloud code stream is encapsulated into the geometry component track, forming a point cloud media Media file; at this time, the media file may contain attribute parameter set indication information, and the attribute parameter set indication information is used to indicate that the attribute parameter set is encapsulated in the parameter component track.
  • the information of each track after packaging is as follows:
  • the component information data box includes the component type field gpcc_type.
  • the data box includes a component type field gpcc_type and a parameter setting type field parameter_set_type.
  • the set of attribute parameters in track (tranck4) is associated with track2 and track3.
  • the content production device can use transmission signaling to transmit the media file to the media processing device.
  • the transmission signaling contains flow indication information.
  • Track1 ⁇ Track4 correspond to Representation1 ⁇ Representation4 respectively.
  • the media processing device can request the corresponding media file according to the stream indication information contained in the transmission signaling. At this time, as long as the media processing device needs to request attribute data, it will definitely request the data stream corresponding to the parameter component track, that is, Representation4.
  • the media processing device can decapsulate and decode the media file, obtain the attribute parameter set from the parameter component track track4 according to the attribute parameter set indication information, and obtain the attribute parameter set according to the attribute parameter set indication information. Collects point cloud media related to partial decoding to maximize saving of decoding resources. If the media processing file only needs to decode the color attribute, track4+track2 can be decoded. Specifically, the parameter component track track4 can be decoded first to obtain the attribute parameter set, and then the attribute component track corresponding to the color attribute data is decoded according to the attribute parameter set to obtain the color attribute data in the point cloud media.
  • the media processing device can render and process the color attribute data in the point cloud media and present the point cloud media.
  • the content production equipment can collect point cloud media and encode the attribute data in the point cloud media according to the attribute parameter set to obtain the attribute code stream; and encode the geometric data in the point cloud media according to the geometric parameter set to obtain the geometry Code stream, the attribute code stream and the geometry code stream are combined to generate a point cloud code stream, which also includes a set of attribute parameters; then, when encapsulating the file of the point cloud code stream, a multi-track method is used to The attribute parameter set and geometric data in the point cloud code stream are encapsulated into the geometric component track, and the attribute data in the point cloud code stream are encapsulated into the attribute component track to form a media file of the point cloud media; at this time, the media file can contain attributes Parameter set indication information. This attribute parameter set indication information is used to instruct the attribute parameter set to be encapsulated into the geometry component track; after encapsulation, the information of each track is as follows:
  • the track contains the component information data box GPCCComponentInfoBox.
  • the component information data box includes the component type field gpcc_type.
  • the parameter collection contains a set of attribute parameters.
  • the content production device can transfer media files to the media processing device.
  • the media processing device can obtain the attribute parameter set from the geometry component track track1 according to the attribute parameter set indication information, and partially decode the relevant point cloud media based on the attribute parameter set to maximize Save decoding resources. If the media processing device only needs to decode color attribute data, it will decode track1+track2. Specifically, the geometry component track track1 can be decoded first to obtain the attribute parameter set, and then the attribute component track truck2 corresponding to the color attribute data can be decoded according to the attribute parameter set to obtain the color attribute data in the point cloud media.
  • the media processing device can render and process the color attribute data in the point cloud media and present the point cloud media.
  • Content production equipment can collect point cloud media; then, encode the attribute data in the point cloud media according to the attribute parameter set to obtain the attribute code stream, and encode the geometric data in the point cloud media according to the geometric parameter set to obtain Geometric code stream, the attribute code stream and the geometric code stream are combined to generate a point cloud code stream, which also contains a set of attribute parameters; then, when encapsulating the file of the point cloud code stream, a multi-track method is used Encapsulate the color attribute data in the point cloud code stream into the attribute component track (truck2) corresponding to the color attribute data, and encapsulate the reflectivity attribute data in the point cloud code stream into the attribute component track (truck3) corresponding to the reflectance attribute data, and Encapsulate the geometric data in the point cloud code stream into the geometric component track, and encapsulate the attribute parameter set into the attribute component track corresponding to the color attribute data and the attribute component track corresponding to the reflectivity attribute data to form a media file of the point cloud media ; At this time, the media file may contain attribute parameter set indication information.
  • the component information data box includes the component type field gpcc_type.
  • the attribute component track contains the attribute Parameter collection.
  • the content production device transmits the media files to the media processing device.
  • the media processing device can partially decode the relevant point cloud media according to the requirements. If you only need to decode the color attribute data, you can directly decode track2. Specifically, the attribute component track corresponding to the color attribute data can be directly decoded according to the attribute parameter set indication information to obtain the color attribute data in the point cloud media.
  • the media processing device can render and process the color attribute data in the point cloud media and present the point cloud media.
  • the media file of the point cloud media is obtained.
  • the media file includes the point cloud code stream of the point cloud media and a set of attribute parameters.
  • the set of attribute parameters includes the parameters required for decoding the attribute data in the point cloud code stream.
  • Information; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set.
  • Presenting point cloud media guides the transmission, decoding and presentation of point cloud media, which can reduce unnecessary encoding and decoding dependencies and save transmission and computing resources.
  • Figure 6 is a schematic structural diagram of a point cloud media data processing device provided by an embodiment of the present application.
  • the data processing device of the point cloud media can be provided in the computer equipment provided by an embodiment of the present application.
  • the computer equipment It may be the media processing device mentioned in the above method embodiment.
  • the point cloud media data processing device shown in Figure 6 can be a computer program (including program code) running in a computer device, and the point cloud media data processing device can be used to execute the method embodiment shown in Figure 3 some or all of the steps.
  • the data processing device of the point cloud media may include the following units:
  • the acquisition unit 601 is used to acquire the media file of the point cloud media.
  • the media file includes the point cloud code stream of the point cloud media and the attribute parameter set.
  • the attribute parameter set includes the parameter information required for decoding the attribute data in the point cloud code stream; media
  • the file also includes attribute parameter set indication information, which is used to indicate the encapsulation position of the attribute parameter set;
  • the processing unit 602 is configured to decode the point cloud code stream according to the attribute parameter set indication information to present the point cloud media.
  • the media file includes a track, the track contains one or more samples, one sample corresponds to one point cloud frame in the point cloud media; one sample is divided into one or more sub-samples, each sub-sample contains a Type of point cloud data;
  • the media file also includes a sub-sample information data box, and the attribute parameter set is encapsulated in the sub-sample information data box.
  • the attribute parameter set indication information is represented as a subsample information data box, and the attribute parameter set indication information is also used to indicate the relationship between the attribute parameter set and the attribute data in the point cloud code stream;
  • the subsample data box includes at least one of the following fields: payload type field, reference attribute information flag field, reference attribute type flag field, reference attribute identification field and reference attribute type field;
  • the payload type field is used to indicate the type of point cloud data contained in the subsample; if the payload type field is the first value, it indicates that the type of point cloud data contained in the subsample is a set of attribute parameters; if the payload type field is the second value, indicating that the type of point cloud data contained in the subsample is a geometric parameter set, and the geometric parameter set contains parameter information required for geometric data decoding;
  • the reference attribute information flag field is used to indicate the effective range of the current attribute parameter set for attribute data; if the reference attribute information flag field is the first preset value, it indicates that the current attribute parameter set is only effective for a specific group or a specific type in the current sample. The attribute data of the type is valid; if the reference attribute information flag field is the second preset value, it indicates that the current attribute parameter set is valid for all attribute data in the current sample;
  • the reference attribute type flag field is used to indicate the type of attribute data that the current attribute parameter set is effective for; if the reference attribute type flag field is the first value, it indicates that the current attribute parameter set is only effective for a specific type of attribute data in the current sample; If the reference attribute type flag field is the second value, it indicates that the current attribute parameter set is only effective for a specific set of attribute data in the current sample;
  • the reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set;
  • the reference attribute type field is used to indicate the type of attribute data corresponding to the current attribute parameter set;
  • the current attribute parameter set refers to the attribute parameter set being used
  • the current sample refers to the sample being decoded
  • the point cloud code stream is encapsulated in a media file in a multi-track manner;
  • the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track;
  • the parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains parameter information required for decoding the attribute data of the corresponding point cloud frame.
  • the media file includes one or more parameter component tracks; when the media file only includes one parameter component track, the set of attribute parameters in the point cloud code stream is encapsulated into the parameter component track; when the media file includes multiple When a parameter component track is used, the attribute parameter set in the point cloud code stream is encapsulated into a single parameter component track or the attribute parameter set in the point cloud code stream is encapsulated into multiple parameter component tracks based on the type.
  • the parameter component track includes a track data box
  • the track data box contains a track reference type data box, and the track reference type data box is used to indicate that the parameter component track is associated with one or more property component tracks corresponding to the property parameter set in the parameter component track.
  • the media file follows the first encoding standard;
  • the parameter component track contains a component information data box, and the component information data box is set in the header sample entry of the parameter component track;
  • the component information data box contains at least one of the following fields: component type Field, header type field, attribute header quantity field, reference attribute type flag field, reference attribute identification field and reference attribute type field;
  • the component type field is used to indicate the component type in the parameter component track; when the component type field is set to the target preset value, it indicates that the component type in the parameter component track is a parameter collection;
  • the header type field is used to indicate the type of parameter set contained in the parameter component track; if the header type field is the first setting value, it indicates that the type of parameter set contained in the parameter component track is an attribute parameter set; if the header type field for The second set value indicates that the parameter set type contained in the parameter component track is a geometric parameter set; if the header type field is the third set value, it indicates that the parameter set type contained in the parameter component track is a geometric parameter set and Attribute parameter collection;
  • the attribute header quantity field is used to indicate the number of attribute parameter sets contained in the sample in the parameter component track; where the order of the attribute parameter sets contained in the sample is consistent with the identifier or attribute parameter set of the attribute parameter set indicated in the component information data box.
  • the order of types is consistent;
  • the reference attribute type field is used to indicate the type of attribute data that the current attribute parameter set takes effect on; if the reference attribute type flag field is the first preset value, it means that the current attribute parameter set only takes effect on a specific type of attribute data in the current sample. ; If the reference attribute type flag field is the second preset value, it means that the current attribute parameter set is effective for a specific set of attribute data in the current sample;
  • the reference attribute identification field is used to indicate the identifier of the attribute data corresponding to the current attribute parameter set
  • the reference attribute type field is used to indicate the type of attribute data corresponding to the current attribute parameter set
  • the current attribute parameter set refers to the attribute parameter set being used
  • the current sample refers to the sample being decoded
  • the media file follows the second encoding standard;
  • the parameter component track contains a component information data box and a configuration data box;
  • the parameter set type corresponding to the setting unit contained in the configuration data box is the same as the parameter set type indicated by the component information data box. same;
  • the component information data box contains at least one of the following fields: component type field, parameter setting type quantity field, and parameter setting type field;
  • the component type field is used to indicate the type of the component in the parameter component track; if the component type field is the target setting value, it indicates that the type of the component in the parameter component track is a parameter collection;
  • the parameter setting type quantity field is used to indicate the number of parameter sets contained in the parameter component track
  • the parameter setting type field is used to indicate the type of the parameter set contained in the parameter component track; if the parameter setting type field is the first preset value, it means that the type of the parameter set contained in the parameter component track is an attribute parameter set; if the parameter setting If the type field is the second preset value, it means that the type of the parameter set contained in the parameter component track is a geometric parameter set; if the parameter setting type field is the third preset value, it means that the type of the parameter set contained in the parameter component track is is a set of sequence parameters.
  • the media file includes a geometry component track, and a set of attribute parameters is encapsulated in the geometry component track.
  • the geometric component track includes a component information data box, and the attribute parameter set indication information is represented as a component information data box;
  • the component information data box includes a component type field.
  • the component type field is used to indicate the type of the component in the geometric component track; if the component type field is the first setting value, it indicates that the type of the component in the geometric component track is geometric data and parameters.
  • Set, parameter set includes attribute parameter set and geometric parameter set.
  • the media file includes one or more property component tracks, and a set of property parameters is encapsulated in each property component track;
  • the attribute parameter set encapsulated in each attribute component track is copied from the same attribute parameter set. owned.
  • the media file includes one or more attribute component tracks, the attribute parameter set contains an index, and the index of the attribute parameter set is encapsulated in each attribute component track;
  • the index encapsulated in any attribute component track is used to find the attribute parameter collection, or is used to find the data unit formed after the attribute parameter set is reorganized.
  • point cloud media is transmitted using streaming transmission; when acquiring the media files of the point cloud media, the acquisition unit 601 may be specifically used to:
  • the transmission signaling contains flow indication information.
  • the flow indication information is used to indicate the data flow corresponding to the parameter component track;
  • the transmission signaling is dynamic adaptive flow signaling
  • the flow indication information refers to a component descriptor.
  • the component descriptor is set in the data stream corresponding to the parameter component track;
  • the component descriptor includes a composition type element and a parameter type.
  • the composition type element is used to indicate the type of the component in the parameter component track;
  • the parameter type element is used to indicate the type of the parameter set contained in the parameter component track;
  • the type of the component in the parameter component track is the parameter collection component.
  • the processing unit 602 when decoding the point cloud code stream to present the point cloud media according to the attribute parameter set indication information, the processing unit 602 may be specifically used to:
  • the attribute data in the point cloud code stream is decoded using the parameter information in the attribute parameter set to present the point cloud media.
  • the media file of the point cloud media is obtained.
  • the media file includes the point cloud code stream of the point cloud media and a set of attribute parameters.
  • the set of attribute parameters includes the parameters required for decoding the attribute data in the point cloud code stream.
  • Information; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set.
  • Presenting point cloud media guides the transmission, decoding and presentation of point cloud media, which can reduce unnecessary encoding and decoding dependencies and save transmission and computing resources.
  • Figure 7 is a schematic structural diagram of a point cloud media data processing device provided by an embodiment of the present application.
  • the data processing device of the point cloud media can be provided in the computer equipment provided by an embodiment of the present application.
  • the computer equipment It may be the content production device mentioned in the above method embodiment.
  • the data processing device of point cloud media shown in Figure 7 can be a computer program (including program code) running in a computer device, and the data processing device of point cloud media can be used to execute the method embodiment shown in Figure 5 some or all of the steps.
  • the data processing device of the point cloud media may include the following units:
  • Acquisition unit 701 used to acquire point cloud media
  • the processing unit 702 is used to encode the point cloud media to obtain the point cloud code stream and the attribute parameter set.
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream;
  • the processing unit 702 is also used to encapsulate the point cloud code stream to obtain a media file of the point cloud media; the media file also contains attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.
  • the media file includes a track
  • the point cloud code stream is encapsulated in the track
  • the track contains one or more samples, one sample corresponds to one point cloud frame in the point cloud media; one sample is divided into one or Multiple subsamples, each subsample contains one type of point cloud data;
  • the media file also includes a sub-sample information data box, and the attribute parameter set is encapsulated in the sub-sample information data box.
  • the point cloud code stream is encapsulated into a media file using a multi-track method;
  • the media file includes a parameter component track, and the attribute parameter set is independently encapsulated in the parameter component track;
  • the parameter component track contains one or more samples, each sample corresponds to a point cloud frame of the point cloud media, and each sample contains parameter information required for decoding the attribute data of the corresponding point cloud frame.
  • the media file includes a geometry component track, and a set of attribute parameters is encapsulated in the geometry component track.
  • the media file includes one or more attribute component tracks, and a set of attribute parameters is encapsulated in each attribute component track;
  • the attribute parameter set encapsulated in each attribute component track is obtained by copying the same attribute parameter set.
  • the media file includes one or more attribute component tracks, the attribute parameter set contains an index, and the index of the attribute parameter set is encapsulated in each attribute component track;
  • the index encapsulated in any attribute component track is used to find the attribute parameter collection, or is used to find the data unit formed after the attribute parameter set is reorganized.
  • processing unit 702 is also used to:
  • the media files of the point cloud media are sent through transmission signaling.
  • the transmission signaling contains flow indication information.
  • the flow indication information is used to indicate the data flow corresponding to the parameter component track.
  • the point cloud media is obtained, and the point cloud media is encoded to obtain the point cloud code stream and the attribute parameter set.
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream.
  • the point cloud code stream is encapsulated to obtain a point cloud media media file; the media file also contains attribute parameter set indication information, which is used to indicate the encapsulation position of the attribute parameter set; add the attribute parameter set indication in the media file
  • the information can help the media processing device directly obtain the attribute parameter set according to the attribute parameter set indication information, thereby guiding the transmission, decoding and presentation of point cloud media and reducing certain codec dependencies.
  • the embodiment of the present application also provides a schematic structural diagram of a computer device.
  • the structural schematic diagram of the computer device can be seen in Figure 8; the computer device may be the above-mentioned media processing device or content production device; the computer device may include: processing 801, input device 802, output device 803 and memory 804.
  • the above-mentioned processor 801, input device 802, output device 803 and memory 804 are connected through a bus.
  • the memory 804 is used to store computer programs.
  • the computer programs include program instructions.
  • the processor 801 is used to execute the program instructions stored in the memory 804.
  • the processor 801 performs the following operations by running the executable program code in the memory 804:
  • the media file includes the point cloud code stream of the point cloud media and the attribute parameter set.
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream; the media file also includes the attribute parameters.
  • Set indication information, attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set;
  • the point cloud code stream is decoded to present the point cloud media.
  • the media file of the point cloud media is obtained.
  • the media file includes the point cloud code stream of the point cloud media and a set of attribute parameters.
  • the set of attribute parameters includes the parameters required for decoding the attribute data in the point cloud code stream.
  • Information; the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set; according to the attribute parameter set indication information, the encapsulation position of the attribute parameter set can be directly known, and the point cloud code stream can be decoded based on the attribute parameter set.
  • Presenting point cloud media guides the transmission, decoding and presentation of point cloud media, which can reduce unnecessary encoding and decoding dependencies and save transmission and computing resources.
  • the processor 801 performs the following operations by running the executable program code in the memory 804:
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream;
  • the point cloud code stream is encapsulated to obtain a point cloud media media file; the media file also contains attribute parameter set indication information, and the attribute parameter set indication information is used to indicate the encapsulation position of the attribute parameter set.
  • the point cloud media is obtained, and the point cloud media is encoded to obtain the point cloud code stream and the attribute parameter set.
  • the attribute parameter set contains the parameter information required for decoding the attribute data in the point cloud code stream.
  • the point cloud code stream is encapsulated to obtain a point cloud media media file; the media file also contains attribute parameter set indication information, which is used to indicate the encapsulation position of the attribute parameter set; add the attribute parameter set indication in the media file
  • the information can help the media processing device directly obtain the attribute parameter set according to the attribute parameter set indication information, thereby guiding the transmission, decoding and presentation of point cloud media and reducing certain codec dependencies.
  • embodiments of the present application also provide a computer-readable storage medium, and a computer program is stored in the computer-readable storage medium, and the computer program includes program instructions.
  • the processor executes the above program instructions
  • the methods in the embodiments corresponding to FIG. 3 and FIG. 5 can be executed, and therefore, they will not be described again here.
  • program instructions may be deployed on one computer device, or executed on multiple computer devices located at one location, or on multiple computer devices distributed across multiple locations and interconnected by a communications network.
  • a computer program product includes a computer program, and the computer program is stored in a computer-readable storage medium.
  • the processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device can perform the methods in the embodiments corresponding to FIG. 3 and FIG. 5. Therefore, the details will not be discussed here. Elaborate.
  • the program can be stored in a computer-readable storage medium.
  • the program can be stored in a computer-readable storage medium.
  • the process may include the processes of the embodiments of each of the above methods.
  • the storage medium can be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请实施例提供了一种点云媒体的数据处理方法及相关设备，其中方法包括：获取点云媒体的媒体文件，所述媒体文件包括所述点云媒体的点云码流以及属性参数集合，所述属性参数集合包含所述点云码流中的属性数据解码所需的参数信息；所述媒体文件中还包括属性参数集合指示信息，所述属性参数集合指示信息用于指示所述属性参数集合的封装位置；按照所述属性参数集合指示信息，对所述点云码流进行解码处理以呈现点云媒体。本申请实施例可以指示属性参数集合的封装位置，能够减少不必要的编解码依赖关系，实现点云媒体的传输、解码及呈现；优化网络带宽及解码侧计算资源的利用。

Description

一种点云媒体的数据处理方法及相关设备
本申请要求于2022年08月22日提交中国专利局、申请号为202211007891.4、申请名称为“一种点云媒体的数据处理方法及相关设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及通信技术领域，具体涉及点云媒体的数据处理。
背景技术
随着点云技术的不断发展，点云媒体的压缩编码成为重要的研究问题。从目前的点云媒体的压缩编码技术来看，在点云码流对属性参数定义中，允许一个属性参数集合中包含多组属性数据解码所需的参数信息。但是，目前点云媒体的压缩编码技术还存在一些问题，例如在支持部分传输或部分解码过程中，易造成不必要的编解码依赖关系以及解码侧资源浪费等问题。因此，如何改进属性编码技术成为点云编码技术领域的热门问题。
发明内容
本申请实施例提供了一种点云媒体的数据处理方法及相关设备，可以指示属性参数集合的封装位置，根据所指示的封装位置获取属性参数集合，能够减少不必要的编解码依赖关系，实现点云媒体的传输、解码及呈现；优化网络带宽及解码侧计算资源的利用。
一方面，本申请实施例提供一种点云媒体的数据处理方法，该方法包括：
获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；媒体文件中还包括属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置；
按照属性参数集合指示信息，对点云码流进行解码处理以呈现点云媒体。
一方面，本申请实施例提供一种点云媒体的数据处理方法，该方法包括：
获取点云媒体；
对点云媒体进行编码处理，得到点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；
对点云码流进行封装，得到点云媒体的媒体文件；媒体文件还包含属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置。
一方面，本申请实施例提供了一种点云媒体的数据处理装置，该装置包括：
获取单元，用于获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；媒体文件中还包括属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置；
处理单元，用于按照属性参数集合指示信息，对点云码流进行解码处理以呈现点云媒体。
一方面，本申请实施例提供一种点云媒体的数据处理装置，该装置包括：
获取单元，用于获取点云媒体；
处理单元，用于对点云媒体进行编码处理，得到点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；
处理单元，还用于对点云码流进行封装，得到点云媒体的媒体文件；媒体文件还包含属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置。
一方面，本申请实施例提供一种计算机设备，该计算机设备包括：
处理器，适用于执行计算机程序；
计算机可读存储介质，计算机可读存储介质中存储有计算机程序，计算机程序被处理器执行时，实现如上述点云媒体的数据处理方法。
一方面，本申请实施例提供一种计算机可读存储介质，该计算机可读存储介质存储有计算机程序，该计算机程序被处理器加载并执行如上述点云媒体的数据处理方法。
一方面，本申请实施例提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备执行上述点云媒体的数据处理方法。
在本申请实施例中，获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流及属性参数集合，该属性参数集合中包含点云码流中的属性数据解码所需的参数信息；属性参数集合指示信息用于指示属性参数集合的封装位置；按照属性参数集合指示信息，可以直接获知属性参数集合的封装位置，并可基于该属性参数集合对点云码流进行解码处理以呈现点云媒体，指导了点云媒体的传输、解码和呈现，可减少不必要的编解码依赖，节省传输资源和计算资源。
附图说明
图1a是本申请一个示例性实施例提供的一种6DoF的示意图；
图1b是本申请一个示例性实施例提供的一种3DoF的示意图；
图1c是本申请一个示例性实施例提供的一种3DoF+的示意图；
图2a是本申请一个示例性实施例提供的一种点云媒体的数据处理系统的架构图；
图2b是本申请一个示例性实施例提供的一种点云媒体的数据处理的流程图；
图3是本申请一个示例性实施例提供的一种点云媒体的数据处理方法的流程示意图；
图4a是本申请一个示例性实施例提供的一种参数集合和属性数据划分为数据单元的示意图；
图4b是本申请另一个示例性实施例提供的一种参数集合和属性数据划分为数据单元的示意图；
图5是本申请另一个示例性实施例提供的一种点云媒体的数据处理方法的流程示意图；
图6是本申请一个示例性实施例提供的一种点云媒体的数据处理装置的结构示意图；
图7是本申请另一个示例性实施例提供的一种点云媒体的数据处理装置的结构示意图；
图8是本申请另一个示例性实施例提供的一种计算机设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实 施例，都属于本申请保护的范围。
本申请中术语“第一”“第二”等字样用于对作用和功能基本相同的相同项或相似项进行区分，应理解，“第一”、“第二”、“第n”之间不具有逻辑或时序上的依赖关系，也不对数量和执行顺序进行限定。
本申请中术语“至少一个”是指一个或多个，“多个”的含义是指两个或两个以上；同理，术语“至少一组”是指一组或多组，“多组”的含义是指两组或两组以上，例如：点云中的某个点具备多组属性数据是指该点具备两组或两组以上的属性数据。
下面对本申请中涉及的其他技术术语进行介绍：
一、沉浸媒体
沉浸媒体是指能够提供沉浸式的媒体内容，使沉浸于该媒体内容中的观看者能够获得现实世界中视觉、听觉等感官体验的媒体文件。沉浸式媒体按照观看者在消费媒体内容时的自由度，可以分为：6DoF(Degree of Freedom，自由度)沉浸媒体，3DoF沉浸媒体，3DoF+沉浸媒体。其中，如图1a所示，6DoF是指沉浸媒体的观看者可以沿着X轴、Y轴、Z轴自由平移，例如，沉浸媒体的观看者可以在三维的360度VR内容中自由的走动。与6DoF相类似的，还有3DoF和3DoF+制作技术。图1b为本申请实施例提供的一种3DoF的示意图；如图1b所示，3DoF是指沉浸媒体的观看者在一个三维空间的中心点固定，沉浸媒体的观看者头部沿着X轴、Y轴和Z轴旋转来观看媒体内容提供的画面。图1c为本申请实施例提供的一种3DoF+的示意图，如图1c所示，3DoF+是指当沉浸媒体提供的虚拟场景具有一定的深度信息，沉浸媒体的观看者头部可以基于3DoF在一个有限的空间内移动来观看媒体内容提供的画面。
二、点云
点云是指空间中一组无规则分布的、表达三维物体或场景的空间结构及表面属性的离散点集。点云中的每个点至少包括几何数据，该几何数据用于表示点的三维位置信息。根据应用场景的不同，点云中的点还可以包括一组或多组属性数据，每一组属性数据用于反映点所具备的一种属性，该属性例如可以是色彩、材质或其他信息。通常，点云中的每个点都具有相同组数的属性数据。
点云可以灵活方便地表达三维物体或场景的空间结构及表面属性，因而应用广泛，可以应用于虚拟现实(Virtual Reality，VR)游戏、计算机辅助设计(Computer Aided Design,CAD)、地理信息系统(Geography Information System，GIS)、自动导航系统(Autonomous Navigation System，ANS)、数字文化遗产、自由视点广播、三维沉浸远程呈现、生物组织器官三维重建等场景中。
点云的获取主要有以下途径：计算机生成、三维(3-Dimension，3D)激光扫描、3D摄影测量等。具体来说，点云可以是通过采集设备(一组摄像机或具有多个镜头和传感器的摄像机设备)对现实世界的视觉场景进行采集得到的，通过3D激光扫描可以获得静态现实世界三维物体或场景的点云，每秒可以获取百万级点云；通过3D摄影可以获得动态现实世界三维物体或场景的点云，每秒可以获取千万级点云；此外，在医学领域，可以通过磁共振成像(Magnetic Resonance Imaging，MRI)、电子计算机断层扫描(Computed Tomography， CT)、电磁定位信息获得生物组织器官的点云。又如，点云还可以由计算机根据虚拟三维物体及场景直接生成，如计算机可以生成虚拟三维物体及场景的点云。伴随着大规模的点云数据不断积累，点云数据的高效存储、传输、发布、共享和标准化，成为点云应用的关键。
三、点云媒体
点云媒体是一种典型的6DoF沉浸媒体。点云媒体包括由一帧或多帧点云帧组成的帧序列，每帧点云帧由点云中的一个或多个点具备的几何数据和属性数据共同组成。其中，几何数据也可以称为三维位置信息，点云中某个点的几何数据是指该点的空间坐标(x，y，z)，可以包括该点在三维坐标系统的各个坐标轴方向上的坐标值，例如，X轴方向上的坐标值x，Y轴方向上的坐标值y和Z轴方向上的坐标值z。点云中的某个点可以包括一组或多组属性数据，每一组属性数据用于反映点所具备的一种属性，例如，点云中某个点具备一组颜色属性数据，该颜色属性数据用于反映该点的颜色属性(如红色、黄色等等)；再如，点云中的某个点具备一组反射率属性数据，该反射率属性数据用于反映该点的激光反射强度属性。当点云中某个点具备多组属性数据时，该多组属性数据的类型可以相同，也可以不同，例如，点云中某个点可以具有一组颜色属性数据和一组反射率属性数据；再如，点云中某个点可以具有两组颜色属性数据，这两组颜色属性数据分别用于反映该点在不同时刻下的颜色属性。
特别的，每帧点云帧中还包括几何数据对应的几何参数集合或属性数据对应的属性参数集合，几何参数集合和属性参数集合可以组成参数集合。其中，几何参数集合可以包括对每帧点云帧中的几何数据进行编码或解码所需要的参数信息；属性数据对应的属性参数集合可以包括对每帧点云帧中的属性数据编码或解码所需要的参数信息。也就是说，点云媒体中的一帧点云帧所包含的点云数据可包括几何数据、几何参数集合、一组或多组属性数据、一个或多个属性参数集合。
其中，在AVS(Audio Video Coding Standard，中国国家音视频编码标准)点云编码技术中，一个点云帧内通常仅包含一个点云几何头(即Geometry Header)和一个点云属性头(即Sequence Header)，该点云几何头包括对单帧点云帧的几何数据编码或解码所需要的参数集合，点云属性头包括对单帧点云帧中的属性数据编码或解码所需要的参数集合，即在点云属性头中可以包含对应于点云帧中一组或多组属性数据的所有参数信息；而在MPEG(Moving Picture Expert Group，国际视音频编解码标准)点云编码技术中，点云媒体对应的点云码流中可以定义一个几何参数集合(GPS，Geometry Parameter Set)和一个或多个属性参数集合(即APS，Attribute Parameter Set)，一个APS可以对应点云帧内不同种类的属性数据。在本申请实施例中，对于属性头和APS不作任何区分，均描述为属性参数集合。
四、轨道(Track)
轨道是指点云媒体封装过程中的媒体数据集合，一个轨道中由多个具备时序的样本组成，一个样本对应点云媒体的一个点云帧。点云媒体的封装方式包括单轨方式或多轨方式，所谓单轨方式是指将点云媒体的所有点云数据均封装至同一个轨道中，此时，点云媒体的媒体文件仅包含一个轨道(即单轨封装得到的单一轨道)，单轨方式得到的单一轨道中，一 个样本即是指点云媒体中的一个点云帧，一个样本包含对应点云帧的所有数据(包括几何数据和属性数据)。
所谓多轨方式是指将点云媒体的点云数据封装至多个不同的轨道中，此时，点云媒体的媒体文件可以包含多个轨道。进一步，多轨方式可以包括基于类型的多轨方式；基于类型的多轨方式是将一种类型的数据封装至一个组件轨道中，例如，点云媒体包含几何数据、一组颜色属性数据和一组反射率属性数据，那么可以将几何数据封装至几何组件轨道，将颜色属性数据封装至颜色属性组件轨道，将反射率属性数据封装至反射率属性组件轨道；基于类型的多轨方式得到的任一组件轨道中，一个样本仅包含点云媒体的一个点云帧的部分数据，例如几何组件轨道中的一个样本包含点云媒体中一个点云帧的几何数据；颜色属性组件轨道中的一个样本包含点云媒体中一个点云帧的一组颜色属性数据。特别地，元数据信息也可以作为一种媒体类型，以元数据轨道包含于点云媒体的媒体文件中。其中，每个组件轨道(如几何组件轨道、属性组件轨道)中均可以包括一个或多个样本，每个样本均可以对应点云媒体中的一个点云帧，且对应同一个点云帧的样本应具备相同的呈现时间。
此外，当点云媒体中包含参数集合时，单轨方式得到单一轨道中，一个样本包含对应点云帧的所有数据，即一个样本包括一个点云帧的几何数据、属性数据和参数集合，该参数集合包括几何参数集合和属性参数集合，几何参数集合中包含点云帧的几何数据解码所需的参数信息；属性参数集合包括点云帧的属性数据解码所需的参数信息。而基于类型的多轨方式将一种类型的数据封装至一个轨道中，参数集合的封装位置则可能有多种，包括但不限于：①参数集合可以被封装到几何数据对应的几何组件轨道；②参数集合可以被封装至属性数据对应的属性组件轨道中；③参数信息可以作为一种类型的点云数据，被单独封装到参数组件轨道中，该参数组件轨道中的一个样本包括点云媒体中一个点云帧的参数信息。
应理解的是，点云媒体被封装后以组件形式存在于轨道中，例如：点云媒体中的属性数据被封装后以属性组件形式存在于轨道中，点云媒体中几何数据被封装后以几何组件形式存在于轨道中；参数集合被封装后以参数组件形式存在于轨道中。
五、样本(Sample)以及子样本(SubSample)
样本是媒体文件封装过程中的封装单位，一个轨道由很多个样本组成，例如：一个视频轨道可以由很多个样本组成，一个样本通常为一个视频帧。在本申请实施例中，点云媒体的媒体文件包含一个或多个轨道，轨道中的一个样本对应一个点云帧。
样本可以被划分为一个或多个子样本，每个子样本中可以包含点云码流中的一种类型的点云数据，点云码流中的点云数据可以包含以下几种类型：参数集合(包含几何参数集合和属性参数集合)、几何数据和属性数据。可见，子样本可以包含点云码流中的参数集合，或者子样本可以包含点云码流中的几何数据，或者子样本可以包含点云码流中的属性数据。在一种实现中，一个点云帧可被划分为一个或多个点云片(或称点云条)，所谓点云片代表一个点云帧被部分或全部编码后的数据的一系列语法元素(例如几何点云片、属性点云片)集合；此时子样本包含的属性数据和/或几何数据属于该一个或多个点云片；对应点云片的子样本的类型可以包括至少两种，一种是基于点云片承载的数据类型的子样本，在此类型 下，一个子样本仅包含点云片承载的一种数据类型以及相关信息，如一个子样本只包含几何数据类型以及几何数据相关信息。另一种是基于点云片的子样本，在此类型下，一个子样本可以包含一个点云片所有信息，即包含几何片头和几何数据，属性片头和属性数据。
六、样本入口(Sample Entry)
样本入口用于指示轨道中所有样本相关的元数据信息。比如在视频轨道的样本入口中，通常会包含解码器初始化相关的元数据信息。
七、点云空间分块(Tile)：
点云空间分块又称为点云帧边界空间区域内的六面体空间分块区域，一个点云空间分块由一个或多个点云片组成，点云空间分块之间不存在编解码依赖关系。
八、ISOBMFF(ISO Based Media File Format，基于ISO标准的媒体文件格式)
ISOBMFF是媒体文件的封装标准，较为典型的ISOBMFF文件即为MP4文件。
九、DASH(Dynamic Adaptive Streaming over HTTP，基于HTTP的动态自适应流)DASH是一种自适应比特率技术，使高质量流媒体可以通过传统的HTTP网络服务器在互联网传递。
十、MPD(Media Presentation Description，DASH中的媒体演示描述信令):
MPD用于描述媒体文件中的媒体片段信息。
十一、表示(Representation)：
Representation是指DASH中一个或多个媒体成分的组合，比如某种分辨率的视频文件可以看作一个Representation；在本申请中，某种时域层级的视频文件可以看作一个Representation。
十二、点云编码(Point Cloud Compression，PCC)
点云编码是指根据参数集合对点云中各点的几何数据和属性数据进行编码，得到点云码流的过程。其中，点云编码可以包括几何数据编码和属性数据编码两个主要过程，参数集合可以包括几何参数集合和属性参数集合。因此在编码过程中可以采用点云编码技术，根据参数集合中的几何参数集合对点云媒体中各点的几何数据进行编码处理，得到几何码流；并采用G-PCC根据参数集合中的属性参数对点云媒体中各点的属性数据进行编码，得到属性码流；几何码流、属性码流共同组成点云媒体的点云码流。目前，主流的点云编码技术，针对点云的不同类型，可以分为基于几何结构的点云编码以及基于投影的点云编码，在此以MPEG中的G-PCC(Geometry-based Point Cloud Compression，基于几何结构的点云编码)，以及AVS中的点云编码标准AVS-PCC。
十五、点云解码
点云解码是指对点云编码得到的点云码流进行解码，以重建点云的过程；详细地说，是指基于点云码流中的几何码流和属性码流，重建点云中各点的几何信息和属性数据的过程。在解码侧获得点云码流之后，对于几何码流，首先获取对几何数据进行解码所需要的几何参数集合，然后根据该几何参数集合对几何码流进行解码处理，得到点云中各点的几何数据。而对于属性码流，首先获取对属性数据进行解码所需要的属性参数集合，然后根据属性参数集合对属性码流进行解码处理，得到点云中各点的属性数据；然后，将点云中各点重建的属性数据，按顺序与重建的几何数据一一对应，以重建点云。
基于上述相关描述，本申请实施例提供了一种基于点云媒体的数据处理方案，该点云媒体的数据处理方案的大致原理如下：在编码侧，可以在点云媒体的媒体文件中添加属性参数集合指示信息，并通过该属性参数集合指示信息来指示属性参数集合的封装位置。在解码侧，可根据属性参数集合指示信息确定点云码流中的属性数据解码所需的属性参数集合的封装位置，然后可以从封装位置中获取该属性参数集合，并根据属性参数集合可以对该点云码流中的属性数据进行解码，从而呈现该点云媒体。
在一个实施例中，当对点云码流进行单轨封装时，该轨道可以包括样本，该样本包括子样本，可以在子样本中标识属性参数集合以及该属性参数集合与点云媒体中的属性数据之间的关系；在此实现中，媒体文件中的属性参数集合指示信息用于指示属性参数集合的封装位置为子样本，并且该属性参数集合还可用于指示该属性参数集合与点云码流中的属性数据之间的关系。
在另一个实施例中，当对点云码流进行多轨封装时，该属性参数集合指示信息用于指示属性参数集合的封装位置可以包括以下任一种或多种：(1)将属性参数集合作为单独的轨道进行封装，得到参数组件轨道，该属性参数集合指示信息用于指示属性参数集合的封装位置为参数组件轨道。(2)由于属性组件轨道一定依赖于几何组件轨道，因此，即使某一个属性参数集合被不同的属性组件引用，当该属性参数集合位于几何组件轨道时，也不会引入额外的解码依赖关系，在此前提下，可以将属性参数集合直接封装到几何组件轨道，此时属性参数集合指示信息用于指示属性参数集合的封装位置为几何组件轨道。(3)可以将属性参数集合复制多份，在每个属性组件轨道中均封装该属性参数集合，此时，该属性参数集合指示信息用于指示属性参数集合分别被封装在每个属性组件轨道中。(4)可以将属性参数集合的索引封装在每个属性组件轨道中，属性参数集合指示信息用于指示属性参数集合分别被封装在每个属性组件轨道中，并可进一步指示，任一个属性组件轨道中封装的索引用于查找属性参数集合，或用于查找属性参数集合被重新组织后形成的数据单元。
通过上述该数据处理方案有如下有益效果：通过点云媒体的媒体文件中包含的属性参数集合指示信息，可以指示属性参数集合的封装位置，从而指导点云媒体的传输、解码及呈现；同时，可以有效利用点云媒体各个组件之间的编解码独立性，解决为了支持部分传输和部分解码，将不同的点云数据封装为不同的文件轨道所带来的不必要编解码依赖关系问题，例如几何数据封装为一个几何组件轨道；颜色属性数据、反射率属性数据分别封装为一个属性组件轨道，此时，如果颜色属性数据和反射率属性数据共用一个属性参数集合，不管将这个属性参数集合封装在哪一个属性组件轨道，均会产生编解码以来关系；从而节省传输资源和计算资源的使用。
本申请实施例所提供的点云媒体的数据处理方案还可以与车联网技术相结合。具体的，该点云媒体的数据处理方案可以采集环境中的建筑物、交通标志物等等，并在车辆中构建点云地图用于定位，或者利用该点云地图实现自动导航。
基于上述描述，下面结合图2a对适于实现本申请实施例提供的点云媒体的数据处理系统进行介绍。如图2a所示，点云媒体的数据处理系统20中可以包括内容制作设备201和媒体处理设备202，内容制作设备201位于点云媒体的编码侧；该内容制作设备201可以是终端设 备，也可以是服务器。媒体处理设备202位于点云媒体的解码侧，该媒体处理设备202可以是终端设备，也可以是服务器。内容制作设备201和媒体处理设备202之间可以建立通信连接。其中，终端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表、车载终端、智能电视等，但并不局限于此。服务器可以是独立的物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、CDN(Content Delivery Network，内容分发网络)、以及大数据和人工智能平台等基础云计算服务的云服务器。
在一个实施例中，内容制作设备201和媒体处理设备202执行该点云媒体的数据处理的具体流程如下：针对内容制作设备201主要包括以下数据处理过程：(1)点云媒体的获取过程；(2)点云媒体的编码及文件封装的过程。针对媒体处理设备201主要包括以下数据处理过程：(3)点云媒体的文件解封装及解码的过程；(4)点云媒体的渲染过程。
另外，内容制作设备201与媒体处理设备202之间涉及点云媒体的传输过程，该传输过程可以基于各种传输协议(或者传输信令)来进行，此处的传输协议可包括但不限于：DASH(Dynamic Adaptive Streaming over HTTP，动态自适应流媒体传输)协议、HLS(HTTP Live Streaming，动态码率自适应传输)协议、SMTP(Smart Media Transport Protocol，智能媒体传输协议)、TCP(Transmission Control Protocol，传输控制协议)等。
下面对点云媒体的数据处理过程进行详细描述：
(1)点云媒体的获取过程。
内容制作设备201可以获取点云媒体，点云媒体可以通过场景捕获或设备生成两种方式获取得到。场景捕获点云媒体是指通过内容制作设备201关联的捕获设备采集真实世界的视觉场景得到点云媒体；其中，捕获设备用于为内容制作设备201提供点云媒体的获取服务，捕获设备可以包括但不限于以下任一种：摄像设备、传感设备、扫描设备；其中，摄像设备可以包括普通摄像头、立体摄像头、以及光场摄像头等。传感设备可以包括激光设备、雷达设备等。扫描设备可以包括三维激光扫描设备等。内容制作设备201关联的捕获设备可以是指设置于内容制作设备201中的硬件组件，例如捕获设备是终端的摄像头、传感器等，内容制作设备关联的捕获设备也可以是指与内容制作设备201相连接的硬件装置，例如与内容制作设备201相连接的摄像头等。设备生成点云媒体是指内容制作设备201根据虚拟对象(例如通过三维建模得到的虚拟三维物体及虚拟三维场景)生成点云媒体。
(2)点云媒体的编码及文件封装的过程。
内容制作设备201可以基于几何参数集合对获取的点云媒体中的几何数据进行编码处理，得到几何码流；内容制作设备201还可以基于属性参数集合对获取的点云媒体中的属性数据进行编码处理，得到属性码流。几何码流和属性码流形成点云媒体的点云码流；另外，在该点云码流中可以包含属性参数集合和几何参数集合。在得到点云码流之后，可对点云码流进行封装，得到点云媒体的媒体文件；进一步，媒体文件中还可包含属性参数集合指示信息，该属性参数集合指示信息用于指示属性参数集合的封装位置。
当采用单轨方式对点云码流进行封装得到单一轨道时，该单一轨道中可以包括一个或 多个样本，每个样本可以包含点云媒体中的一个点云帧所有的数据，即包括几何数据、几何参数集合、属性数据和属性参数集合；每个样本被划分一个或多个子样本。在一种实施方式中，属性参数集合可以封装在子样本中，属性参数集合指示信息则可以用于指示属性参数集合被封装在子样本中，此时，一个子样本可以包含点云码流中的参数集合，或者一个子样本可以包含点云码流中的属性数据，或者，一个子样本可以包含点云码流中的几何数据。
当采用基于类型的多轨方式对点云码流进行封装得到多个轨道时，任一轨道中可以包括一个或多个样本，每个样本可以包含点云媒体中的一个点云帧的一类数据，如轨道1(几何组件轨道)中的一个样本可以包含相应点云帧中的几何数据，轨道2(颜色组件轨道)中的一个样本可以包含相应点云帧中的一组颜色属性数据；轨道3(参数组件轨道)中的一个样本可以包含点云帧中的属性参数集合。此时，该属性参数集合指示信息可以用于指示属性参数集合被独立封装在参数组件轨道中。另外，当采用基于类型的多轨方式对点云码流进行封装得到多个轨道，该多个轨道中包括一个几何组件轨道以及一个或多个属性组件轨道，此时，该属性参数集合可以直接被封装到几何组件轨道中，该属性参数集合指示信息可以用于指示属性参数集合封装在几何组件轨道中。
在得到媒体文件之后，内容制作设备201可以将该媒体文件传输给媒体处理设备202，使得在媒体处理设备202中可以根据属性参数集合指示信息获知属性参数集合的封装位置，并基于属性参数集合对点云码流对属性数据进行解码处理。
可选地，当点云媒体采用流化传输方式进行传输，且该属性参数集合被独立封装在参数组件轨道时，在传输信令中可以包含流指示信息，该流指示信息用于指示参数组件轨道对应的数据流，并通过传输信令向媒体处理设备202传输上述媒体文件。
(3)点云媒体的文件解封装及解码的过程；
媒体处理设备202可以通过内容制作设备201获得点云媒体的媒体文件和相应的媒体呈现描述信息。点云媒体的媒体文件和媒体呈现描述信息通过传输信令(如DASH、SMT)由内容制作设备201传输给媒体处理设备202。媒体处理设备202的文件解封装的过程与媒体处理设备202的文件封装过程是相逆的，媒体处理设备202按照点云媒体的文件格式要求对媒体文件资源进行解封装，得到点云码流。媒体处理设备202的解码过程与内容制作设备201的编码过程是相逆的，媒体处理设备202对点云码流进行解码，还原出点云媒体。
其中，在解码过程中，媒体处理设备202可以从媒体文件中获取属性参数集合指示信息，并根据该属性参数指示信息可以确定属性参数集合所在的封装位置，并从该封装位置获取属性参数集合，然后根据该属性参数集合对媒体文件中的属性数据进行解码，最终可以还原出点云媒体。
可选地，该属性参数指示信息可以用于指示属性参数集合和点云码流中属性数据之间的关系，该关系可以是属性参数集合只对点云码流中某一组属性数据生效，或者可以是属性参数集合只对点云码流中的某一种属性数据生效，或者可以是属性参数集合对点云码流中的所有属性数据均生效。所谓生效是指有效力，即根据生效的属性参数集合能够对点云码流中的属性数据进行解码。
其中，点云码流中可包括多组属性数据，例如：多组属性数据包括：一组黄色属性数据、一组红色属性数据、一组反射率属性数据；那么，属性参数集合只对点云码流中某一组属性数据生效可以是指仅对一组黄色属性数据生效；而属性参数集合只对点云码流中某一类属性素数据生效可以是指对颜色属性数据生效，由于黄色属性数据和红色属性数据均属于颜色属性数据，那么属性参数集合只对点云码流中某一类属性素数据生效可以包括对黄色属性数据和红色属性数据均生效。
(4)点云媒体的渲染过程。
媒体处理设备202根据媒体呈现描述信息中与渲染、视窗相关的元数据对解码得到的点云媒体进行渲染，得到点云媒体的点云帧，并根据点云帧的呈现时间呈现点云媒体。
在一个实施例中，请参见图2b，内容制作设备端：首先通过采集设备对真实世界的视觉场景A进行采样，得到与真实世界的视觉场景对应的点云媒体的点云源数据B，点云源数据B是由大量点云帧组成的帧序列；然后根据参数集合(即几何参数集合和属性参数集合)对获取的点云媒体进行编码处理，得到点云码流E(包括编码的几何码流和属性码流)，可选地，该点云码流E中还可以包括参数集合；接着，对点云码流E进行封装得到点云媒体对应的媒体文件，具体地，内容制作设备201根据特定媒体容器文件格式，将一个或多个编码的码流合成为用于文件回放的媒体文件F，或用于流式传输的初始化片段和媒体片段的序列(FS)；其中，媒体容器文件格式可以是指在国际标准化组织(International Organization for Standardization，ISO)/国际电工委员会(International Electrotechnical Commission，IEC)14496-12中规定的ISO基本媒体文件格式。该媒体文件中可以还包含参数集合指示信息，参数集合指示信息用于指示参数集合的封装位置。在一种实施方式中，内容制作设备还将元数据封装到媒体文件或初始化/媒体片段的序列中，并通过传输信令(如动态自适应流媒体传输接口)将初始化/媒体片段的序列传送给媒体处理设备202。
在媒体处理设备端：首先接收内容制作设备201发送的媒体文件，该媒体文件可以包括：用于文件回放的媒体文件F′，或用于流式传输的初始化片段和媒体片段的序列Fs′；然后对媒体文件进行解封装处理，得到点云码流E′；接着根据媒体文件包含的参数集合指示信息所指示的封装位置获取参数集合，并根据参数集合解码点云码流(即根据参数集合中的属性参数集合对点云码流中的属性数据进行解码处理以及根据参数集合中的几何参数集合对点云码流中的几何数据进行解码处理)，得到点云媒体D′；在具体实现中，媒体处理设备基于当前对象的观看位置/观看方向确定呈现点云媒体所需的媒体文件，或者媒体片段序列；并对呈现点云媒体所需的媒体文件，或者媒体片段序列进行解码处理，得到呈现所需的点云媒体。最后基于当前对象的观看(视窗)方向，对解码后的点云媒体进行渲染，得到点云媒体的点云帧A′，并按照点云帧的呈现时间在媒体处理设备携带的头戴式显示器或任何其他显示设备的屏幕上呈现点云媒体。需要说明的是，当前对象的观看位置/观看方向由头部跟随以及可能还有视觉跟随功能确定。除了通过渲染器用来渲染当前对象的观看位置/观看方向的点云媒体外，还可以通过音频解码器来对当前对象的观看(视窗)方向的音频进行解码优化。其中，在基于视窗的传输过程中，当前的观看位置和观看方向也被传递给策略模块，用于确定要接收的轨道。
可以理解的是，本申请涉及点云媒体的数据处理技术可以依托于云技术进行实现；例如，将云服务器作为内容制作设备。云技术(Cloud technology)是指在广域网或局域网内将硬件、软件、网络等系列资源统一起来，实现数据的计算、储存、处理和共享的一种托管技术。
在本申请实施例中，获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流及属性参数集合，该属性参数集合中包含点云码流中的属性数据解码所需的参数信息；属性参数集合指示信息用于指示属性参数集合的封装位置；按照属性参数集合指示信息，可以直接获知属性参数集合的封装位置，并可基于该属性参数集合对点云码流进行解码处理以呈现点云媒体，指导了点云媒体的传输、解码和呈现，可减少不必要的编解码依赖，节省传输资源和计算资源。
接下来对本申请实施例涉及到点云媒体的数据处理方法进行相关阐述。请参阅图3，图3为本申请实施例提供的一种点云媒体的数据处理方法的流程示意图。该方法可由上述媒体处理设备执行，本实施例中所描述的点云媒体的数据处理方法，可以包括以下S301-S302：
S301、获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；该媒体文件还包括属性参数集合指示信息，该属性参数集合指示信息用于指示属性参数集合的封装位置。
在一个实施例中，该属性参数集合指示信息还可用于指示属性参数集合与点云码流中的属性数据之间的关系，即该属性参数集合指示信息可以指示属性参数集合以及该属性参数集合对应的属性数据，例如，该属性参数集合指示信息指示属性参数集合对点云码流中的一种特定类型的属性数据(如颜色)生效；又例如，该属性参数集合指示信息指示属性参数集合对点云码流中的特定一组属性数据(如一组黄色属性数据)生效；再如，该属性参数集合指示信息指示属性参数集合对点云码流中的所有属性数据均生效。
其中，媒体文件是对点云媒体的点云码流进行封装处理得到的。在本申请实施例中，从单轨方式和多轨方式两种方式出发，属性参数集合的封装位置存在如下几种实现方式(1)-(2)：
(1)当采用单轨方式将点云媒体的点云码流封装至同一轨道时，该属性参数集合可以封装在子样本信息数据盒中。
在采用单轨方式将点云媒体的点云码流封装至同一轨道时，媒体文件中包括轨道，该轨道中可以包含一个或多个样本，一个样本对应点云媒体中的一个点云帧；一个样本可以被划分为一个或多个子样本，每个子样本中可以包括点云码流中的任一类型的点云数据，如子样本中可以包含参数集合，或者子样本中可以包含属性数据或者几何数据，该媒体文件还可以包括子样本信息数据盒，属性参数集合可以封装在子样本信息数据盒中，此时，属性参数集合指示信息用于指示属性参数集合封装于子样本信息数据盒中。其中，属性参数集合指示信息可以表示为子样本信息数据盒，该子样本信息数据盒可以包含以下至少一个字段：有效载荷类型字段、参考属性信息标志字段、参考属性类型标志字段、参考属性标识字段和参考属性类型字段。
具体的，在对点云码流进行封装时可以使用子样本信息数据盒；当一个点云帧可被划 分为一个或多个点云片时，子样本包含的属性数据和/或几何数据属于该一个或多个点云片；在该子样本信息数据盒中可以包含子样本信息数据的标志字段。根据该子样本信息数据的标志字段(flag)的值可以定义子样本，该标志字段指定该子样本信息数据盒中的子样本的类型：若标志字段为第一预设值(如0)，那么该子样本是指基于点云片承载的数据类型的子样本，此时，一个子样本只包含一种数据类型及相关数据，如一个子样本只包含几何数据类型以及几何数据。若标志字段为第二预设值(如1)，那么该子样本是指基于点云片的子样本，此时一个子样本仅包含一个点云片所有相关数据，即包含几何片头和几何数据，属性片头和属性数据。当然，标志字段可以保留其他标志值。
此时，子样本信息数据盒的codec_specific_parameters(编码译码器具体规范)字段定义可以如表1：
表1

其中，子样本信息数据盒包含的各个字段含义如下：
有效载荷类型字段(payloadType)：该有效载荷类型字段用于指示子样本中包含点云数据的类型；若该有效载荷类型字段为第一取值(如2)，则指示子样本中包含点云数据的类型为属性参数集合；若该有效载荷类型字段为第二取值(如3)，则指示子样本中包含点云数据的类型为几何参数集合；若该有效载荷类型字段为第三取值(如0)，则指示子样本中包含点云数据的类型为属性数据；若该有效载荷类型字段为第四取值(如1)，则指示子样本中包含点云数据的类型为几何数据。应理解的是，第一取值、第二取值、第三取值、第四取值可以根据需求任意设置，如第一取值可以为0，第二取值可以为1，第三取值可以为2，第四取值可以为3，本申请实施例对此不作限定。
属性呈现标志字段(attribute_present_flag)：该属性呈现标志字段可以用于指示子样本中是否包含颜色属性数据和/或反射率属性数据，定义可以参见AVS(中国国家音视频编码标准AVS，Audio Video Coding Standard)-PCC。若属性呈现标志字段的值为第一设定值(如0)，则指示子样本中是否包含颜色属性；若属性呈现标志字段的值为第二设定值(如1)指示子样本中是否包含反射率属性。
点云片数据字段(slice_data)：该字段指示子样本中是否包含点云片的数据。若点云片数据字段的值为第一设定值(如1)，则指示子样本中包含点云片几何和/或属性类型数据；若点云片数据字段的值为第二设定值(如0)，则指示子样本中不包含点云的参数信息。
点云片标识字段(slice_id)：该字段指示子样本中包含的数据对应的点云片的标识。
参考属性信息标志字段(ref_attr_info_flag)：该参考属性信息标志字段用于指示当前属性参数集合对属性数据生效范围；若参考属性信息标志字段为第一预设值(如1)，则指示当前属性参数集合仅对当前样本中的特定一组(即某一组)或一种特定类型(即某一类)的属性数据生效；若参考属性信息标志字段为第二预设值(如0)，则指示当前属性参数集合对当前样本中的所有属性数据生效；
参考属性类型标志字段(ref_attr_type_flag)：该参考属性类型标志字段用于指示所述当前属性参数集合对属性数据生效的类型；若参考属性类型标志字段为第一数值(如1)，则指示当前属性参数集合仅对当前样本中一种特定类型的属性数据生效；若参考属性类型标志字段为第二数值(0)，则指示当前属性参数集合仅对当前样本中特定一组属性数据生效。
参考属性标识字段(ref_attr_id)：该字段用于指示当前属性参数集合对应的属性数据的标识符，即可理解为当前属性参数集合生效的属性数据的标识符。
参考属性类型字段(ref_attr_type)：该字段用于指示当前属性参数集合对应的属性数据 的类型；即可理解为当前属性参数集合生效的属性数据的类型；例如，参考属性类型字段可以用于指示当前属性参数集合对应的属性数据的类型为颜色；又例如，参考属性类型字段可以用于指示当前属性参数集合对应的属性数据的类型为反射率。
保留字段(reserved)：该保留字段的值可以为0。
其中，当前属性参数集合是指正在被使用的属性参数集合，当前样本是指正在被解码的样本。
(2)当多轨方式将点云媒体的点云码流封装至多个轨道时，该属性参数集合的封装位置可以有如下a-d几种情况：
a、属性参数集合被独立封装在参数组件轨道中。
当点云码流采用多轨方式被封装至媒体文件时，本申请实施例可以定义参数组件轨道，点云码流采用基于类型的多轨方式被封装至媒体文件，此时该媒体文件中可以包括参数组件轨道，该属性参数集合被独立封装在参数组件轨道中。其中，该参数组件轨道的样本入口类型可以为‘apcp’或‘gpp1’；包含于：样本描述数据盒(SampleDescriptionBox)；强制性：否；数量：一个或多个。
此时，属性参数集合指示信息可以用于指示属性参数集合被独立封装至参数组件轨道中。在参数组件轨道中包含一个或多个样本，每个样本对应点云媒体的一个点云帧，每个样本均包含对应点云帧的属性数据解码所需的参数信息。
在一个实施例中，媒体文件包括一个或多个参数组件轨道；一种可行的实施方式中，当媒体文件仅包括一个参数组件轨道时，该点云码流中的属性参数集合被封装至该参数组件轨道中。可以理解的是，在此实施方式下，点云码流中的几何参数集合也可一并被封装在该参数组件轨道中，即点云码流的几何参数集合和属性参数集合均可被封装在该参数组件轨道中。在另一种可行的实施方式中，当媒体文件包括多个参数组件轨道时，点云码流中的几何参数集合可被封装至一个单独的参数组件轨道中，而点云码流中的属性参数集合可被封装至另一个单独的参数组件轨道中。在再一种可行的实施方式中，当媒体文件包括多个参数组件轨道时，点云码流中的几何参数集合可被封装至一个单独的参数组件轨道中，而点云码流中的属性参数集合可基于类型被封装至多个参数组件轨道中，例如，属性参数集合包括颜色属性参数集合和反射率属性参数集合，当媒体文件包括多个参数组件轨道时，颜色属性参数集合可被封装至一个单独的参数组件轨道，反射率属性参数集合可被封装至另一个单独的参数组件轨道。
在多轨方式下，点云码流中的属性参数集合被封装为参数组件轨道。该参数组件轨道可以包含轨道数据盒(TrackBox)，该轨道数据盒可以包含一个轨道参考类型数据盒(TrackReferenceTypeBox)，该轨道参考类型数据盒的参考类型(reference_type)为'appr'。该轨道参考类型数据盒用于指示该参数组件轨道与该参数组件轨道中的属性参数集合对应的一个或多个属性组件轨道(即点云数据轨道)相关联。应理解为：该轨道参考类型数据盒用于参考或者关联参数组件轨道中的属性参数集合对应的一个或多个属性组件轨道。例如，如果参数组件轨道中的属性参数集合包括颜色参数和反射率参数，那么该轨道参考类型数据盒用于参考或关联该参数组件轨道中的属性参数集合对应的颜色属性组件轨道和反 射率属性组件轨道。
在一个实施例中，当媒体文件遵循第一编码标准(例如AVS编码标准)时，该参数组件轨道包含组件信息数据盒(AVSPCCComponentInfoBox)，该组件信息数据盒设置于参数组件轨道的头样本入口(AvsPccParaHeaderSampleEntry)中；该组件信息数据盒包含以下至少一个字段：组件类型字段、头部类型字段、属性头数量字段、参考属性类型标志字段、参考属性标识字段和参考属性类型字段；该组件信息数据盒的语法可以参见表2：
表2
其中，该组件信息数据盒包含的各个字段含义如下：
组件类型字段(avs_pcc_type)：该组件类型字段用于指示轨道中的组件类型；当组件类型字段被设置为目标预设值(如5)时，指示参数组件轨道中的组件类型为参数集合。其中，组件类型字段的取值与组件类型之间的对应关系可如表3所示：
表3
属性类型字段(attr_type)：该属性类型字段用于指示轨道中包含的属性组件的类型，若该属性类型字段为第一取值(如0)，则指示该轨道中仅包含颜色属性数据；若属性类型字段为第二取值(如1)，则指示仅包含反射率属性数据；该属性类型字段为第三取值(如2)，则指示包含颜色属性数据和反射率属性数据。
头部类型字段(header_type)：该头部类型字段用于指示参数组件轨道中包含的参数集合类型；若头部类型字段为第一设定值(如0)，则指示参数组件轨道中包含的参数集合类型为属性参数集合；若头部类型字段为第二设定值(如1)，则指示参数组件轨道中包含的参数集合类型为几何参数集合；若该头部类型字段为第三设定值(如2)，则指示参数组件轨道中包含参数集合类型为几何参数集合和属性参数集合。其中，当点云码流中的属性参数集合被封装至多个参数组件轨道时，如几何参数集合被封装到一个单独的参数组件轨道，属性参数集合被封装到一个单独的参数组件轨道，此时，头部类型字段的取值只能为第一设定值和第二设定值。
属性头数量字段(attr_header_num)：用于指示参数组件轨道中的样本包含的属性参数集合的数量，例如，该属性参数集合的数量可以为1个、2个等；其中，样本中包含的属性参数集合的顺序与组件信息数据盒中指示的属性参数集合的标识符或属性参数集合类型的顺序一致；例如，样本中包含的属性参数集合的顺序为属性参数集合1、属性参数集合2，那么该组件信息数据中指示的属性参数集合的标识符的顺序应该也为属性参数集合1的标识符、属性参数集合2的标识符。
参考属性类型字段(ref_attr_type_flag)：该参考属性类型字段用于指示当前属性参数集合对属性数据生效的类型；若参考属性类型标志字段为第一预设值(如0)，则表示当前属性参数集合仅对当前样本中一种特定类型的属性数据生效；若参考属性类型标志字段为第二预设值(如1)，则表示当前属性参数集合对当前样本中特定一组属性数据生效。
参考属性标识字段(ref_attr_id)：该参考属性标识字段用于指示当前属性参数集合对应的属性数据的标识符。
参考属性类型(ref_attr_type)：该参考属性类别用于指示当前属性参数集合对应的属性数据的类型；
其中，当前属性参数集合是指正在被使用的属性参数集合，当前样本是指正在被解码的样本。
在另一个实施例中，媒体文件可以遵循第二编码标准(如MPEG编码标准)；参数组件 轨道包含组件信息数据盒(GPCCComponentInfoBox)和配置数据盒(GPCCConfigurationBox)；其中，配置数据盒的语法可以如表4所示：
表4
其中，配置数据盒包括的各个字段的含义如下：
配置版本字段(configurationVersion)：该配置版本字段用于配置记录版本信息；若相应轨道中该配置版本字段的版本信息无法被播放器/解码器识别，则播放器/解码器不应尝试对相应轨道进行解码。
档次字段(profile_idc)：该档次字段用于指示当前轨道对应的码流的档次信息。
档次兼容字段(profile_compatibility_flags)：该档次兼容字段用于指示当前轨道对应的码流的档次兼容性信息。
设置单元数量字段(num_of_setup_unit_arrays)：该设置单元数量字段用于指示当前解码器配置记录中包含的参数集组个数，每个参数集组对应一种类型的参数集合。
设置单元类型字段(setup_unit_type)：该设置单元类型字段用于指示当前参数集组对应的参数集合类型。
参数集合指示字段(array_completeness)：该参数集合指示字段用于指示相应点云码流包含的当前类型的参数集合在解码器配置记录中指示情况。当参数集合指示字段为第一取值(如1)时，指示相应码流包含的当前类型的参数集全部在解码器配置记录中指示。当参数集合指示字段为第一取值(如0)时，表示相应码流包含的当前类型的参数集合一部分在解码器配置记录中指示，一部分在码流中指示。
设置单元数量字段(num_of_setup_units)：该设置单元数量字段用于指示当前类型的参数集合的个数。其中，设置单元(setupUnit)包含SPS(序列参数集合)、GPS(几何参数集合)、APS(属性参数集合)或者点云分块信息的数据单元。
在一个实施例中，配置数据盒中包含的设置单元对应的参数集合类型与组件信息数据盒指示的参数集合类型相同；设置单元(setupUnit)可以包含序列参数集合(SPS)、几何参数集合(即GPS)、属性参数集合(即APS)或者点云分块信息的数据单元。应理解为：当设置单元对应的参数集合类型为属性参数集合时，该组件信息数据盒指示的参数集合类型也为属性参数集合；当该配置数据盒中包含的设置单元对应的参数集合类型为几何参数集合，该组件信息数据盒指示的参数集合类型也为几何参数集合。
上述组件信息数据盒包含以下至少一个字段：组件类型字段、参数设置类型数量字段、参数设置类型字段。组件信息数据盒的语法如表5所示：
表5

其中，组件信息数据盒包含的各个字段含义如下：
组件类型字段(gpcc_type)：该字段用于指示轨道中的组件的类型；若组件类型字段为目标设定值(如5)，则指示参数组件轨道中的组件的类型为参数集合，其中，组件类型字段的取值与组件类型之间的对应关系可如表6所示：
表6
参数设置类型数量字段(num_para_set_type)：该字段用于指示参数组件轨道包含的参数集合的数量。如参数组件轨道包含的参数集合的数量可以为1个、2个等。可选地，当媒体文件包括多个参数组件轨道时，如几何参数集合被单独封装到一个参数组件轨道，属性参数集合被单独封装到另一个参数组件轨道，此时，每个参数组件轨道只包含一种类型的参数集合，那么可以不需要参数设置类型数量字段。
参数设置类型字段(parameter_set_type)：该参数设置类型字段用于指示参数组件轨道中包含的参数集合的类型；若参数设置类型字段为第一预设值(如0)，则表示参数组件轨道中包含的参数集合的类型为属性参数集合；若参数设置类型字段为第二预设值(如1)，则表示参数组件轨道中包含的参数集合的类型为几何参数集合；若参数设置类型字段为第三预设值(如2)，则表示参数组件轨道中包含的参数集合的类型为序列参数集合。
属性索引字段(attr_index)：该属性索引字段用于指示在SPS(即属性参数集合)中指示的属性参数的序号。
属性类型呈现字段(attr_type_present_flag)，该属性类型呈现字段用于指示组件信息数据盒中是否指示了属性数据类型信息；若属性类型呈现字段为第一取值(如1)，则指示组件信息数据盒中指示了属性数据类型信息；若属性类型呈现字段为第二取值(如0)，则表示组件信息数据盒中为指示属性数据类型信息。其中，四字节与属性数据类型之间的关系如表7所示：
表7属性标签四字节(attribute_label_four_bytes)

属性类型字段(attr_type)：该属性类型字段用于指示属性成分的类型，其取值参考ISO/IEC 23090-9[GPCC]标准中的定义。
属性名称字段(attr_name)：该属性名称字段用于指示可直观解读(human-readable)的属性成分类型信息。
b、属性参数集合被封装在几何参数组件轨道中。
在采用多轨方式对点云码流进行封装至多个轨道时，除了将属性参数集合封装至独立的参数组件轨道外，还可以将属性参数集合全部封装在媒体文件包含的几何组件轨道中。因为属性组件轨道一定依赖于几何组件轨道，所以即使某一个属性参数集合被不同的属性组件引用，当该属性参数集合位于几何组件轨道时，也不会引入额外的解码依赖关系。
在多轨方式下，上述媒体文件可以包括一个几何组件轨道和至少一个属性组件轨道；其中，AVS PCC中的几何组件轨道和属性组件轨道之间需要使用ISO/IEC14496-12[ISOBMFF]定义的轨道参考工具以进行关联索引。
其中，几何组件轨道可以包括一个或多个样本，在几何组件轨道的样本入口中，存在一个解码器配置数据盒，该解码器配置数据盒包含SPS(序列参数集合，Sequence Parameter Set)、GPS、APS等参数集合，而属性组件轨道中不应该包含解码器配置数据盒。
当点云码流中包含属性参数集合时，该属性参数集合应包含于几何组件轨道的相应样本中。而属性组件轨道中不应包含任何类型的属性参数集合，仅包含相应类型的属性数据。基于此，对组件数据信息盒相关字段进行修改，该修改字段如下：组件类型字段(gpcc_type)：该组件类型字段用于指示轨道中组件的类型；该组件类型字段的取值如下表8所示：
表8
当媒体文件中包括几何组件轨道和属性组件轨道时，该属性参数集合被封装在几何组件轨道中，该几何组件轨道可以包括组件信息数据盒，组件信息数据盒包括组件类型字段，组件类型字段用于指示几何组件轨道中的组件的类型，若组件类型字段为第一设定值(如2)，则指示几何组件轨道中的组件的类型为几何数据和参数集合，该参数集合可以包含属性参数集合和几何参数集合。而在属性组件轨道中，组件信息数据盒所包括的组件类型字段为第二设定值(如4)，指示属性组件轨道中的组件的类型均为属性数据，不包含参数集合。
c、属性参数集合可以封装到每个属性组件轨道中。
媒体文件可以包括一个或多个属性组件轨道，可以在每个属性组件轨道中复制属性参数集合，使得每个属性组件轨道中均包含相应的属性参数集合。此时，该属性参数集合分别被封装在每个属性轨道中；该属性参数集合指示信息可以用于指示该属性参数集合分别被封装在每个属性轨道中。其中，每个属性组件轨道中封装的属性参数集合，可是由同一个属性参数集合经过复制得到的。也就是说，当属性组件轨道的数量为N个时，可直接将属性参数集合复制N份，并在每个属性组件轨道中放置一份属性参数集合。
d、属性参数集合的索引可以封装到每个属性组件轨道中。
属性参数集合包括索引，可以在每个属性组件轨道中重复索引该属性参数集合，也即将属性参数集合的索引设置在每个属性组件轨道中，属性参数集合指示信息用于指示属性参数集合的索引被封装在每个属性组件轨道中。其中，通过任一个属性组件轨道中封装的索引均能够查找到属性参数集合，或能够查找到属性参数集合被重新组织后形成的数据单元。
在一个实施例中，可对点云码流进行重新组织后将不同类型的属性组件轨道的样本和对应的属性参数集合索引至同一个数据单元。如图4a所示，点云码流中包括几何参数集合(GPS)、几何数据(Geometry)、属性参数集合(APS)、属性数据1(Attribute1)、属性数据2(Attribute2)；对点云码流进行重新组织后，可以将几何参数集合和几何数据所属的几何组件轨道中的样本索引至同一个数据单元41；将属性参数集合和属性数据1所属的属性组件轨道中的样本索引至同一个数据单元42；将属性参数集合和属性数据2所属的属性组件轨道中的样本索引至同一个数据单元43。又例如，如图4b所示，对点云码流进行重新组织后，可以将几何参数集合和几何数据所属的几何组件轨道中的样本引至同一个数据单元44，将属性参数集合、属性数据1所属的属性组件轨道中的样本以及属性数据2所属的属性组件轨道中的样本索引至同一个数据单元45；在该数据单元45中，属性数据1所属的属性组件轨道中封装的索引和属性数据2所属的属性组件轨道中封装的索引均可以索引到数据单元45。
在一个实施例中，当点云媒体采用流化传输方式进行传输；获取点云媒体的媒体文件可以包括：获取点云媒体的传输信令，然后根据传输信令获取点云媒体的媒体文件。当通过传输信令传输媒体文件时，由于流化传输方式进行媒体文件传输，因此需要对属性数据进行解码时，需要请求该参数组件轨道对应的数据流，以获取相应的属性参数集合，实现对点云媒体对应媒体文件的传输指导。需要说明的是，在流化传输方式下，如果属性参数集合被独立封装至参数组件轨道中，那么该传输信令中还包含流指示信息，该流指示信息用于指示参数组件轨道对应的数据流。
其中，该数据流可以是表示(Representation)，传输信令可以是DASH信令，该流指示信息可以是指组件描述子(GPCCComponent descriptor)，该组件描述子被设置于参数组件轨道对应的数据流；该组件描述子包括组成类型元素及参数类型元素，组成类型元素用于指示在所述参数组件轨道中的组件的类型；若组成类型元素(component@component_type)为目标字符(如’prms’)，则在参数组件轨道中的组件的类型为参数组件。参数类型元素用于指示所述参数组件轨道中包含的参数集合的类型。其中，该组件描述子的属性和元素 可如表9所示：
表9

其中，M：Mandatory强制字段；CM：Conditional Mandatory条件强制；O：Optional可选字段。
S302、按照属性参数集合指示信息，对点云码流进行解码处理以呈现点云媒体。
在一个实施例中，按照属性参数集合指示信息，对所述点云码流进行解码处理以呈现点云媒体，可以包括：根据属性参数集合指示信息所指示的封装位置，获取属性参数集合；然后利用属性参数集合对点云码流中的属性数据进行解码处理，以呈现点云媒体。具体的，可对属性参数集合指示信息所指示的封装位置先进行解码处理，得到属性参数集合。例如，属性参数集合指示信息所指示的封装位置为参数组件轨道，需要对参数组件轨道进行解码，得到属性参数集合。然后，根据需求确定需要解码的属性数据，并利用属性参数集合对该属性数据进行解码处理，通过这种方式，可以部分解码相关的点云媒体，节省了解码资源。
在本申请实施例中，获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流及属性参数集合，该属性参数集合中包含点云码流中的属性数据解码所需的参数信息；属性参数集合指示信息用于指示属性参数集合的封装位置；按照属性参数集合指示信息，可以直接获知属性参数集合的封装位置，并可基于该属性参数集合对点云码流进行解码处理以呈现点云媒体，指导了点云媒体的传输、解码和呈现，可减少不必要的编解码依赖，节省传输资源和计算资源。
请参见图5，图5是本申请实施例提供的一种点云媒体的数据处理方法的流程示意图。该点云媒体的数据处理方法可由内容制作设备来执行。该点云媒体的数据处理方法，可以包括以下S501-S503：
S501、获取点云媒体。
S502、对点云媒体进行编码处理，得到点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息。
其中，对点云媒体进行编码处理的具体实现方式具体可参见前述相应部分的描述，在此不再赘述。
S503、对点云码流进行封装处理，得到点云媒体的媒体文件；该媒体文件还包含属性参数集合指示信息。
其中，可在媒体文件中添加属性参数指示信息，该属性参数集合指示信息可以用于指 示属性参数集合的封装位置；此外，该属性参数集合指示信息还可以用于指示属性参数集合与点云码流中的属性数据之间的关系，如属性参数集合对点云码流中的某一类属性数据或某一组生效。
对点云码流进行封装，得到点云媒体的媒体文件可以包括如下几种实现方式：
(1)媒体文件包括轨道，点云码流被封装在轨道中，轨道中包含一个或多个样本，一个样本对应点云媒体中的一个点云帧；一个样本被划分为一个或多个子样本；每个子样本包含一种类型的点云数据；媒体文件还包括子样本信息数据盒，属性参数集合封装于子样本信息数据盒中。
作为一种实现方式，内容制作设备可采用单轨方式将点云码流封装至轨道中，轨道中包含一个或多个样本，一个样本对应点云媒体中的一个点云帧；然后将每个样本划分为一个或多个子样本；当采用单轨方式将点云媒体的点云码流封装至同一轨道时，在对点云码流进行封装时可以使用子样本信息数据盒，可将点云码流中的属性参数集合设置于子样本信息数据盒中，形成点云媒体的媒体文件；其中，属性参数集合指示信息用于指示属性参数集合设置于子样本信息数据盒中。
其中，在该子样本信息数据盒中可以包含子样本信息数据的标志字段。当一个点云帧可被划分为一个或多个点云片时，子样本包含的属性数据和/或几何数据属于该一个或多个点云片；根据该子样本信息数据的标志字段(flag)的值定义子样本，该标志字段指定该子样本信息数据盒中的子样本的类型：若标志字段为第一预设值(如0)，那么该子样本是指基于点云片承载的数据类型的子样本，此时，一个子样本只包含一种数据类型及相关数据，如一个子样本只包含几何数据类型以及几何数据。若标志字段为第二预设值(如1)，那么该子样本是指基于点云片的子样本，此时一个子样本仅包含一个点云片所有相关数据，即包含几何片头和几何数据，属性片头和属性数据。
其中，该子样本信息数据盒可以包含以下至少一个字段：有效载荷类型字段、参考属性信息标志字段、参考属性类型标志字段、参考属性标识字段和参考属性类型字段。该有效载荷类型字段用于指示子样本包中含点云数据的类型；参考属性信息标志字段用于指示当前属性参数集合对属性数据生效范围；参考属性类型标志字段用于指示当前属性参数集合对属性数据生效的类型；参考属性标识字段用于指示当前属性参数集合对应的属性数据的标识符；参考属性类型字段用于指示当前属性参数集合对应的属性数据的类型；
此时，将点云码流中的属性参数集合设置于子样本信息数据盒中可以包括：若子样本中包含点云数据的类型为属性参数集合，则将有效载荷字段设置为第一取值(如2)；若子样本中包含点云数据的类型为几何参数集合，则将有效载荷字段设置为第二取值(如3)；若当前属性参数集合仅对当前样本中的特定一组(即某一组)或一种特定类型(即某一类)的属性数据生效，则将参考属性信息标志字段设置为第一预设值(如1)；若当前属性参数集合对当前样本中的所有属性数据生效，则将参考属性信息标志字段设置为第二预设值(如0)；若当前属性参数集合仅对当前样本中一种特定类型的属性数据生效，则将参考属性类型标志字段为第一数值(如1)；若当前属性参数集合仅对当前样本中的特定一组属性数据生效，则将参考属性类型标志字段为第二数值(如0)。其中，当前属性参数集合是指正在 被使用的属性参数集合，当前样本是指正在被编码的样本。
(2)点云码流采用多轨方式被封装至媒体文件中；媒体文件包括参数组件轨道，属性参数集合被独立封装在参数组件轨道中；参数组件轨道中包含一个或多个样本，每个样本对应点云媒体的一个点云帧，每个样本包含对应点云帧的属性数据解码所需的参数信息。该属性参数集合指示信息用于指示属性参数集合被独立封装至参数组件轨道
其中，媒体文件包括一个或多个参数组件轨道；在一种可行的实施方式中，当媒体文件仅包括一个参数组件轨道时，点云码流中的属性参数集合被封装至该参数组件轨道中；可以理解的是，在此实施方式下，点云码流中的几何参数集合也可一并被封装在该参数组件轨道中，即点云码流的几何参数集合和属性参数集合均可被封装在该参数组件轨道中。在另一种可行的实施方式中，当媒体文件包括多个参数组件轨道时，点云码流中的几何参数集合可被封装至一个单独的参数组件轨道中，而点云码流中的属性参数集合也可被封装至一个单独的参数组件轨道中。在再一种可行的实施方式中，当媒体文件包括多个参数组件轨道时，点云码流中的几何参数集合可被封装至一个单独的参数组件轨道中，而点云码流中的属性参数集合可基于类型被封装至多个参数组件轨道中。
具体的，可以基于类型的多轨方式将点云码流中的属性参数集合封装至参数组件轨道中，并将点云码流中的几何数据封装至几何组件轨道，以及将点云码流中的属性数据封装至属性组件轨道，最终形成点云媒体的媒体文件；该媒体文件可以包括参数组件轨道、属性组件轨道和几何组件轨道。其中，参数组件轨道中包含一个或多个样本，每个样本对应点云媒体的一个点云帧，每个样本包含对应点云帧的属性数据解码所需的参数信息。属性组件轨道中可以包含一个或多个样本，每个样本对应点云媒体的一个点云帧，每个样本包含对应的点云帧的属性数据，几何组件轨道中可以包括一个或多个样本，每个样本对应点云媒体的一个点云帧，每个样本包含对应的点云帧的几何数据。参数组件轨道中包括轨道数据盒；轨道数据盒中包含轨道参考类型数据盒，轨道参考类型数据盒用于指示参数组件轨道与所述参数组件轨道中的属性参数集合对应的一个或多个属性组件轨道相关联。
a、当媒体文件遵循第一编码标准(如AVS)时，该参数组件轨道可以包含组件信息数据盒，组件信息数据盒包含以下至少一个字段：组件类型字段、头部类型字段、属性头数量字段、参考属性类型标志字段、参考属性标识字段和参考属性类型字段。其中，组件类型字段用于指示参数组件轨道中的组件类型；头部类型字段用于指示参数组件轨道中包含的参数集合类型；属性头数量字段用于指示参数组件轨道中的样本包含的属性参数集合的数量，其中，样本中包含的属性参数集合的顺序与组件信息数据盒中指示的属性参数集合的标识符或属性参数集合类型的顺序一致；参考属性类型字段用于指示当前属性参数集合对属性数据生效的类型；参考属性标识字段用于指示当前属性参数集合对应的属性数据的标识符；参考属性类型用于指示当前属性参数集合对应的属性数据的类型；当前属性参数集合是指正在被使用的属性参数集合，当前样本是指正在被编码的样本。
此时，将点云码流中的属性参数集合进行多轨封装至参数组件轨道中可以包括：若参数组件轨道中的组件类型为参数集合，则将组件类型字段设置为目标预设值；若参数组件轨道中包含的参数集合类型为属性参数集合，则将头部类型字段设置为第一设定值；若参 数组件轨道中包含的参数集合类型为几何参数集合，则将头部类型字段设置为第二设定值；若参数组件轨道中包含的参数集合类型为几何参数集合和属性参数集合，则将头部类型字段设置为第三设定值。若当前属性参数集合仅对当前样本中一种特定类型的属性数据生效，则将参考属性类型标志字段设置为第一预设值(如1)；若当前属性参数集合对当前样本中特定一组属性数据生效，则将参考属性类型标志字段设置为第二预设值(如0)。
b、当媒体文件遵循第二编码标准(如MPEG)时，参数组件轨道包含组件信息数据盒和配置数据盒；配置数据盒中包含的设置单元对应的参数集合类型与组件信息数据盒指示的参数集合类型相同；设置单元(setupUnit)可以包含序列参数集合(SPS)、几何参数集合(即GPS)、属性参数集合(即APS)或者点云分块信息的数据单元。该组件信息数据盒包含以下至少一个字段：组件类型字段、参数设置类型数量字段、参数设置类型字段。其中，组件类型字段用于指示参数组件轨道中的组件的类型；参数设置类型数量字段用于指示参数组件轨道包含的参数集合的数量；参数设置类型字段用于指示参数组件轨道中包含的参数集合的类型。
此时，将点云码流中的属性参数集合封装至参数组件轨道中可以包括：若参数组件轨道中的组件的类型为参数集合，则将组件类型字段设置为目标设定值；若参数组件轨道中包含的参数集合的类型为属性参数集合，则将参数设置类型字段为第一预设值(如0)；若参数组件轨道中包含的参数集合的类型为几何参数集合，则将参数设置类型字段设置为第二预设值(如1)；若参数组件轨道中包含的参数集合的类型为序列参数集合，则将参数设置类型字段为第三预设值(如2)。
(3)媒体文件包括几何组件轨道，属性参数集合被封装在几何组件轨道中。
在采用多轨方式将点云码流封装至多个轨道时，可将属性参数集合封装至几何参数组件轨道。具体的，将点云码流中的属性参数集合和几何数据封装至几何组件轨道，并将点云码流中的属性数据封装至属性组件轨道，形成点云媒体的媒体文件。此时，该媒体文件中可以包括几何组件轨道和属性组件轨道。其中，属性参数集合指示信息用于指示属性参数集合被封装在几何组件轨道中。该几何组件轨道包括一个或多个样本，每个样本对应一个点云帧，每个样本包含一个对应点云帧的属性数据解码所需的参数信息。
在一个实施例中，几何组件轨道包括组件信息数据盒，该组件信息数据盒包括组件类型字段，组件类型字段用于指示几何组件轨道中的组件的类型；若几何组件轨道中的组件类型为几何数据和参数集合，则可以将组件类型字段设置为第一设定值(如2)。
(4)媒体文件包括一个或多个属性组件轨道，该属性参数集合被封装在每个属性组件轨道中；其中，每个属性组件轨道中封装的属性参数集合，是由同一个属性参数集合经过复制得到的。
具体的，可采用多轨方式将点云码流中的属性数据进行封装至一个或多个属性组件轨道，并对属性参数集合进行复制，然后在每个属性组件轨道中均封装一份属性参数集合，形成点云媒体的媒体文件；其中，属性参数集合指示信息用于指示属性参数集合分别被封装在每个属性组件轨道中。
(5)媒体文件包括一个或多个属性组件轨道，属性参数集合包含索引，属性参数集合 的索引被封装在每个属性组件轨道中。
具体的，采用多轨方式将点云码流中属性数据进行封装至一个或多个属性组件轨道，并将点云码流中的属性参数集合的索引封装至每个属性组件轨道中，形成点云媒体的媒体文件；其中，通过任一个属性组件轨道中封装的索引均能够查找到所述属性参数集合，或能够查找到属性参数集合被重新组织后形成的数据单元。属性参数集合被重新组织后形成的数据单元是指对点云码流进行重新组织后将不同类型的属性组件轨道的样本和对应的属性参数集合索引至同一个数据单元。
在一个实施例中，在流化传输方式下，通过传输信令发送点云媒体的媒体文件，如果属性参数集合被独立封装至参数组件轨道中，那么该传输信令中包含流指示信息，该流指示信息用于指示参数组件轨道对应的数据流。其中，传输信令可以为DASH信令；上述该流指示信息可以是指组件描述子，组件描述子被设置于参数组件轨道对应的数据流；组件描述子包括组成类型元素及参数类型元素，组成类型元素用于指示在参数组件轨道中的组件的类型；若在参数组件轨道中的组件的类型为参数集合组件，则将组件描述子中的组成类型元素设置为目标字符。参数类型元素用于指示参数组件轨道中包含的参数集合的类型。
在本申请实施例中，获取点云媒体，对点云媒体进行编码处理，得到点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息，对点云码流进行封装，得到点云媒体的媒体文件；媒体文件还包含属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置；在媒体文件中添加该属性参数集合指示信息可以利于媒体处理设备根据该属性参数集合指示信息直接获取属性参数集合，从而指导点云媒体的传输、解码和呈现，减少一定的编解码依赖关系。
下面通过三个完整的例子对本申请提供的点云媒体的数据处理方法进行详细说明：
实施例一：
1.内容制作设备可以采集点云媒体，并根据属性参数集合对点云媒体中的属性数据进行编码，得到属性码流；根据几何参数集合对点云媒体中的几何数据进行编码，得到几何码流，该属性码流和几何码流组合后生成点云码流，在该点云码流中还包括属性参数集合；然后，在对点云码流进行文件封装时，采用多轨方式将点云码流中的属性参数集合封装至参数组件轨道，将点云码流中的属性数据封装至属性组件轨道，以及将点云码流中的几何数据封装至几何组件轨道，形成点云媒体的媒体文件；此时，在媒体文件可以包含属性参数集合指示信息，该属性参数集合指示信息用于指示属性参数集合被封装在参数组件轨道中。其中，封装之后各轨道信息如下：
Track1:几何组件轨道{GPCCComponentInfoBox：gpcc_type＝2}，其中，几何组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type，gpcc_type＝2指示该几何组件轨道中的组件类型为几何数据。
Track2：属性组件轨道-颜色{GPCCComponentInfoBox：gpcc_type＝4,attr_type＝0}；其中，属性组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type和属性类型字段attr_type，gpcc_type＝4指示该几何组件轨道中的组件类型为属性数据；attr_type＝0指示该属性成分的类型为颜色。
Track3：属性组件轨道-反射率{GPCCComponentInfoBox：gpcc_type＝4,attr_type＝1}；其中，属性组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type和属性类型字段attr_type，gpcc_type＝4指示该属性组件轨道中的组件类型为属性数据；attr_type＝1指示该属性成分的类型为反射率。
Track4：参数组件轨道{GPCCComponentInfoBox：gpcc_type＝5,parameter_set_type＝0}；{TrackReferenceTypeBox：reference_type＝'appr'，track_id＝(2,3)}；其中，参数组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type、参数设置类型字段parameter_set_type，gpcc_type＝5指示该参数组件轨道中的组件类型为属性参数集合；该参数组件轨道还包含轨道参考类型数据盒TrackReferenceTypeBox，该轨道参考类型数据盒的参考类型reference_type为'appr'；该轨道参考类型数据盒用于将参数组件轨道与该参数组件轨道中的属性参数集合对应的属性组件轨道相关联，track_id＝(2,3)指示参数组件轨道(tranck4)中的属性参数集合与轨道2和轨道3相关联。
2.内容制作设备可以采用传输信令将媒体文件传输给媒体处理设备，该传输信令中包含流指示信息，该流指示信息用于指示参数组件轨道对应的数据流component@component_type＝’prms’。组成类型元素component_type＝’prms’，用于指示该参数组件轨道中的组件的类型为参数集合组件。
假设Track1～Track4分别对应Representation1～Representation4
3.媒体处理设备可以根据传输信令包含的流指示信息请求相应的媒体文件，此时，只要媒体处理设备需要请求属性数据，则一定会请求参数组件轨道对应的数据流，即Representation4。
4.在获取到参数组件轨道对应的数据流之后，媒体处理设备可以对媒体文件进行解封装和解码，可以根据属性参数集合指示信息从参数组件轨道track4中获取属性参数集合，并根据该属性参数集合部分解码相关的点云媒体，最大化节省解码资源。如媒体处理文件只需要解码颜色属性时，则可以解码track4+track2。具体的，可以先对参数组件轨道track4进行解码以获取属性参数集合，然后根据该属性参数集合解码颜色属性数据对应的属性组件轨道，得到点云媒体中的颜色属性数据。
5、媒体处理设备可以对点云媒体中的颜色属性数据进行渲染处理，呈现该点云媒体。
实施例二：
1.内容制作设备可以采集点云媒体，并根据属性参数集合对点云媒体中的属性数据进行编码，得到属性码流；并根据几何参数集合对点云媒体中的几何数据进行编码，得到几何码流，该属性码流和几何码流组合后生成点云码流，在该点云码流中还包括属性参数集合；然后，在对点云码流进行文件封装时，采用多轨方式将点云码流中的属性参数集合和几何数据封装至几何组件轨道，将点云码流中的属性数据封装至属性组件轨道，形成点云媒体的媒体文件；此时，在媒体文件可以包含属性参数集合指示信息，该属性参数集合指示信息用于指示将属性参数集合封装至几何组件轨道中；封装之后各轨道信息如下：
Track1:几何组件轨道{GPCCComponentInfoBox：gpcc_type＝2}，其中，几何组件轨 道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type，gpcc_type＝2指示该几何组件轨道中的组件类型为几何数据和参数集合，该参数集合包含属性参数集合。
Track2：属性组件轨道-颜色{GPCCComponentInfoBox：gpcc_type＝4,attr_type＝0}；其中，属性组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type和属性类型字段attr_type，gpcc_type＝4指示该几何组件轨道中的组件类型为属性数据；attr_type＝0指示该属性成分的类型为颜色。
Track3：属性组件轨道-反射率{GPCCComponentInfoBox：gpcc_type＝4,attr_type＝1}；其中，属性组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type和属性类型字段attr_type，gpcc_type＝4指示该属性组件轨道中的组件类型为属性数据；attr_type＝1指示该属性成分的类型为反射率。
2.内容制作设备可以将媒体文件传输给媒体处理设备。
3.媒体处理设备可以在媒体文件进行解封装和解码时，可以根据属性参数集合指示信息从几何组件轨道track1中获取属性参数集合，并根据该属性参数集合部分解码相关的点云媒体，最大化节省解码资源。如媒体处理设备只需要解码颜色属性数据时，则解码track1+track2。具体的，可以先对几何组件轨道track1进行解码以获取属性参数集合，然后根据该属性参数集合解码颜色属性数据对应的属性组件轨道truck2，得到点云媒体中的颜色属性数据。
5、媒体处理设备可以对点云媒体中的颜色属性数据进行渲染处理，呈现该点云媒体。
实施例三：
1.内容制作设备可以采集点云媒体；然后，根据属性参数集合对点云媒体中的属性数据进行编码，得到属性码流，并根据几何参数集合对点云媒体中的几何数据进行编码，得到几何码流，该属性码流和几何码流组合后生成点云码流，在该点云码流中还包含属性参数集合；然后，在对点云码流进行文件封装时，采用多轨方式将点云码流中的颜色属性数据封装至颜色属性数据对应的属性组件轨道(truck2)以及点云码流中的反射率属性数据封装至反射率属性数据对应的属性组件轨道(truck3)，以及将点云码流中的几何数据封装至几何组件轨道，并将该属性参数集合分别封装至颜色属性数据对应的属性组件轨道和反射率属性数据对应的属性组件轨道，形成点云媒体的媒体文件；此时，在媒体文件可以包含属性参数集合指示信息，该属性参数集合指示信息用于指示将属性参数集合封装到每个属性组件轨道；然后各轨道信息如下：
Track1:几何组件轨道{GPCCComponentInfoBox：gpcc_type＝2}，其中，几何组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type，gpcc_type＝2指示该几何组件轨道中的组件类型为几何数据。
Track2：属性组件轨道-颜色{GPCCComponentInfoBox：gpcc_type＝4,attr_type＝0}；其中，属性组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type和属性类型字段attr_type，gpcc_type＝4指示该几何组件轨道中的组件类型为属性数据；attr_type＝0指示该属性成分的类型为颜色。该属性组件轨道中包含属性 参数集合。
Track3：属性组件轨道-反射率{GPCCComponentInfoBox：gpcc_type＝4,attr_type＝1}；其中，属性组件轨道包含组件信息数据盒GPCCComponentInfoBox，该组件信息数据盒中包括组件类型字段gpcc_type和属性类型字段attr_type，gpcc_type＝4指示该属性组件轨道中的组件类型为属性数据；attr_type＝1指示该属性成分的类型为反射率；该属性组件轨道中包含属性参数集合。
2.内容制作设备将媒体文件传输给媒体处理设备。
3.媒体处理设备可以根据需求部分解码相关的点云媒体，如只需要解码颜色属性数据，可以直接解码track2。具体的，可以根据该属性参数集合指示信息直接解码颜色属性数据对应的属性组件轨道，得到点云媒体中的颜色属性数据。
4、媒体处理设备可以对点云媒体中的颜色属性数据进行渲染处理，呈现该点云媒体。
应理解是，上述3个实施例为本申请示例性给出的方式，根据实际情况可以根据属性参数集合的封装位置进行灵活选择使用或组合使用。本申请对此不作限定。
在本申请实施例中，获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流及属性参数集合，该属性参数集合中包含点云码流中的属性数据解码所需的参数信息；属性参数集合指示信息用于指示属性参数集合的封装位置；按照属性参数集合指示信息，可以直接获知属性参数集合的封装位置，并可基于该属性参数集合对点云码流进行解码处理以呈现点云媒体，指导了点云媒体的传输、解码和呈现，可减少不必要的编解码依赖，节省传输资源和计算资源。
请参见图6，图6是本申请实施例提供的一种点云媒体的数据处理装置的结构示意图，该点云媒体的数据处理装置可以设置于本申请实施例提供的计算机设备中，计算机设备可以是上述方法实施例中提及的媒体处理设备。图6所示的点云媒体的数据处理装置可以是运行于计算机设备中的一个计算机程序(包括程序代码)，该点云媒体的数据处理装置可以用于执行图3所示的方法实施例中的部分或全部步骤。请参见图6，该点云媒体的数据处理装置可以包括如下单元：
获取单元601，用于获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；媒体文件中还包括属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置；
处理单元602，用于按照属性参数集合指示信息，对点云码流进行解码处理以呈现点云媒体。
在一个实施例中，媒体文件包括轨道，轨道中包含一个或多个样本，一个样本对应点云媒体中的一个点云帧；一个样本被划分为一个或多个子样本，每个子样本包含一种类型的点云数据；
媒体文件还包括子样本信息数据盒，属性参数集合封装于子样本信息数据盒中。
在一个实施例中，属性参数集合指示信息表示为子样本信息数据盒，属性参数集合指示信息还用于指示属性参数集合与点云码流中的属性数据之间的关系；
子样本数据盒包括以下至少一个字段：有效载荷类型字段、参考属性信息标志字段、参考属性类型标志字段、参考属性标识字段和参考属性类型字段；
有效载荷类型字段用于指示子样本中包含的点云数据类型；若有效载荷类型字段为第一取值，则指示子样本中包含的点云数据的类型为属性参数集合；若有效载荷类型字段为第二取值，则指示子样本中包含的点云数据的类型为几何参数集合，几何参数集合包含几何数据解码所需的参数信息；
参考属性信息标志字段用于指示当前属性参数集合对属性数据生效范围；若参考属性信息标志字段为第一预设值，则指示当前属性参数集合仅对当前样本中的特定一组或一种特定类型的属性数据生效；若参考属性信息标志字段为第二预设值，则指示当前属性参数集合对当前样本中的所有属性数据生效；
参考属性类型标志字段用于指示当前属性参数集合对属性数据生效的类型；若参考属性类型标志字段为第一数值，则指示当前属性参数集合仅对当前样本中一种特定类型的属性数据生效；若参考属性类型标志字段为第二数值，则指示当前属性参数集合仅对当前样本中特定一组属性数据生效；
参考属性标识字段用于指示当前属性参数集合对应的属性数据的标识符；参考属性类型字段用于指示当前属性参数集合对应的属性数据的类型；
其中，当前属性参数集合是指正在被使用的属性参数集合，当前样本是指正在被解码的样本。
在一个实施例中，点云码流采用多轨方式被封装在媒体文件中；媒体文件包括参数组件轨道，属性参数集合被独立封装在参数组件轨道中；
参数组件轨道中包含一个或多个样本，每个样本对应点云媒体的一个点云帧，每个样本包含对应点云帧的属性数据解码所需的参数信息。
在一个实施例中，媒体文件包括一个或多个参数组件轨道；当媒体文件仅包括一个参数组件轨道时，点云码流中的属性参数集合被封装至参数组件轨道中；当媒体文件包括多个参数组件轨道时，点云码流中的属性参数集合被封装至一个单独的参数组件轨道中或点云码流中的属性参数集合基于类型被封装至多个参数组件轨道中。
在一个实施例中，参数组件轨道中包括轨道数据盒；
轨道数据盒中包含轨道参考类型数据盒，轨道参考类型数据盒用于指示参数组件轨道与参数组件轨道中的属性参数集合对应的一个或多个属性组件轨道相关联。
在一个实施例中，媒体文件遵循第一编码标准；参数组件轨道包含组件信息数据盒，组件信息数据盒设置于参数组件轨道的头样本入口中；组件信息数据盒包含以下至少一个字段：组件类型字段、头部类型字段、属性头数量字段、参考属性类型标志字段、参考属性标识字段和参考属性类型字段；
组件类型字段用于指示参数组件轨道中的组件类型；当组件类型字段被设置为目标预设值时，指示参数组件轨道中的组件类型为参数集合；
头部类型字段用于指示参数组件轨道中包含的参数集合类型；若头部类型字段为第一设定值，则指示参数组件轨道中包含的参数集合类型为属性参数集合；若头部类型字段为 第二设定值，则指示参数组件轨道中包含的参数集合类型为几何参数集合；若头部类型字段为第三设定值，则指示参数组件轨道中包含的参数集合类型为几何参数集合和属性参数集合；
属性头数量字段用于指示参数组件轨道中的样本包含的属性参数集合的数量；其中，样本中包含的属性参数集合的顺序与组件信息数据盒中指示的属性参数集合的标识符或属性参数集合类型的顺序一致；
参考属性类型字段用于指示当前属性参数集合对属性数据生效的类型；若参考属性类型标志字段为第一预设值，则表示当前属性参数集合仅对当前样本中一种特定类型的属性数据生效；若参考属性类型标志字段为第二预设值，则表示当前属性参数集合对当前样本中特定一组属性数据生效；
参考属性标识字段用于指示当前属性参数集合对应的属性数据的标识符；
参考属性类型字段用于指示当前属性参数集合对应的属性数据的类型；
其中，当前属性参数集合是指正在被使用的属性参数集合，当前样本是指正在被解码的样本。
在一个实施例中，媒体文件遵循第二编码标准；参数组件轨道包含组件信息数据盒和配置数据盒；配置数据盒中包含的设置单元对应的参数集合类型与组件信息数据盒指示的参数集合类型相同；
组件信息数据盒包含以下至少一个字段：组件类型字段、参数设置类型数量字段、参数设置类型字段；
组件类型字段用于指示参数组件轨道中的组件的类型；若组件类型字段为目标设定值，则指示参数组件轨道中的组件的类型为参数集合；
参数设置类型数量字段用于指示参数组件轨道包含的参数集合的数量；
参数设置类型字段用于指示参数组件轨道中包含的参数集合的类型；若参数设置类型字段为第一预设值，则表示参数组件轨道中包含的参数集合的类型为属性参数集合；若参数设置类型字段为第二预设值，则表示参数组件轨道中包含的参数集合的类型为几何参数集合；若参数设置类型字段为第三预设值，则表示参数组件轨道中包含的参数集合的类型为序列参数集合。
在一个实施例中，媒体文件包括几何组件轨道，属性参数集合被封装在几何组件轨道中。
在一个实施例中，几何组件轨道包括组件信息数据盒，属性参数集合指示信息表示为组件信息数据盒；
组件信息数据盒包括组件类型字段，组件类型字段用于指示几何组件轨道中的组件的类型；若组件类型字段为第一设定值，则指示几何组件轨道中的组件的类型为几何数据和参数集合，参数集合包含属性参数集合和几何参数集合。
在一个实施例中，媒体文件包括一个或多个属性组件轨道，属性参数集合被封装在每个属性组件轨道中；
其中，每个属性组件轨道中封装的属性参数集合，是由同一个属性参数集合经过复制 得到的。
在一个实施例中，媒体文件包括一个或多个属性组件轨道，属性参数集合包含索引，属性参数集合的索引被封装在每个属性组件轨道中；
其中，任一个属性组件轨道中封装的索引用于查找属性参数集合，或用于查找属性参数集合被重新组织后形成的数据单元。
在一个实施例中，点云媒体采用流化传输方式进行传输；获取单元601在获取点云媒体的媒体文件时，可具体用于：
获取点云媒体的传输信令，传输信令中包含流指示信息，流指示信息用于指示参数组件轨道对应的数据流；
根据传输信令获取点云媒体的媒体文件。
在一个实施例中，传输信令为动态自适应流信令，流指示信息是指组件描述子，组件描述子被设置于参数组件轨道对应的数据流；组件描述子包括组成类型元素及参数类型元素，组成类型元素用于指示在参数组件轨道中的组件的类型；参数类型元素用于指示参数组件轨道中包含的参数集合的类型；
若组成类型元素为目标字符，则在参数组件轨道中的组件的类型为参数集合组件。
在一个实施例中，处理单元602在按照属性参数集合指示信息，对点云码流进行解码处理以呈现点云媒体时，可具体用于：
根据属性参数集合指示信息所指示的封装位置，获取属性参数集合；
利用属性参数集合中的参数信息对点云码流中的属性数据进行解码处理，以呈现点云媒体。
在本申请实施例中，获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流及属性参数集合，该属性参数集合中包含点云码流中的属性数据解码所需的参数信息；属性参数集合指示信息用于指示属性参数集合的封装位置；按照属性参数集合指示信息，可以直接获知属性参数集合的封装位置，并可基于该属性参数集合对点云码流进行解码处理以呈现点云媒体，指导了点云媒体的传输、解码和呈现，可减少不必要的编解码依赖，节省传输资源和计算资源。
请参见图7，图7是本申请实施例提供的一种点云媒体的数据处理装置的结构示意图，该点云媒体的数据处理装置可以设置于本申请实施例提供的计算机设备中，计算机设备可以是上述方法实施例中提及的内容制作设备。图7所示的点云媒体的数据处理装置可以是运行于计算机设备中的一个计算机程序(包括程序代码)，该点云媒体的数据处理装置可以用于执行图5所示的方法实施例中的部分或全部步骤。请参见图7，该点云媒体的数据处理装置可以包括如下单元：
获取单元701，用于获取点云媒体；
处理单元702，用于对点云媒体进行编码处理，得到点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；
处理单元702，还用于对点云码流进行封装，得到点云媒体的媒体文件；媒体文件还包含属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置。
在一个实施例中，媒体文件包括轨道，点云码流被封装在轨道中，轨道中包含一个或多个样本，一个样本对应点云媒体中的一个点云帧；一个样本被划分为一个或多个子样本，每个子样本包含一种类型的点云数据；
媒体文件还包括子样本信息数据盒，属性参数集合封装于子样本信息数据盒中。
在一个实施例中，点云码流采用多轨方式被封装至媒体文件中；媒体文件包括参数组件轨道，属性参数集合被独立封装在参数组件轨道中；
参数组件轨道中包含一个或多个样本，每个样本对应点云媒体的一个点云帧，每个样本包含对应点云帧的属性数据解码所需的参数信息。
在一个实施例中，媒体文件包括几何组件轨道，属性参数集合被封装在几何组件轨道中。
在一个实施例中，媒体文件包括一个或多个属性组件轨道，属性参数集合被封装在每个属性组件轨道中；
其中，每个属性组件轨道中封装的属性参数集合，是由同一个属性参数集合经过复制得到的。
在一个实施例中，媒体文件包括一个或多个属性组件轨道，属性参数集合包含索引，属性参数集合的索引被封装在每个属性组件轨道中；
其中，任一个属性组件轨道中封装的索引用于查找属性参数集合，或用于查找属性参数集合被重新组织后形成的数据单元。
在一个实施例中，处理单元702，还用于：
通过传输信令发送点云媒体的媒体文件，传输信令中包含流指示信息，流指示信息用于指示参数组件轨道对应的数据流。
在本申请实施例中，获取点云媒体，对点云媒体进行编码处理，得到点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息，对点云码流进行封装，得到点云媒体的媒体文件；媒体文件还包含属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置；在媒体文件中添加该属性参数集合指示信息可以利于媒体处理设备根据该属性参数集合指示信息直接获取属性参数集合，从而指导点云媒体的传输、解码和呈现，减少一定的编解码依赖关系。
进一步地，本申请实施例还提供了一种计算机设备的结构示意图，该计算机设备的结构示意图可参见图8；该计算机设备可以是上述媒体处理设备或者内容制作设备；该计算机设备可以包括：处理器801、输入设备802，输出设备803和存储器804。上述处理器801、输入设备802、输出设备803和存储器804通过总线连接。存储器804用于存储计算机程序，计算机程序包括程序指令，处理器801用于执行存储器804存储的程序指令。
当计算机设备为上述媒体处理设备时，在本申请实施例中，处理器801通过运行存储器804中的可执行程序代码，执行如下操作：
获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；媒体文件中还包括属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置；
按照属性参数集合指示信息，对点云码流进行解码处理以呈现点云媒体。
在本申请实施例中，获取点云媒体的媒体文件，媒体文件包括点云媒体的点云码流及属性参数集合，该属性参数集合中包含点云码流中的属性数据解码所需的参数信息；属性参数集合指示信息用于指示属性参数集合的封装位置；按照属性参数集合指示信息，可以直接获知属性参数集合的封装位置，并可基于该属性参数集合对点云码流进行解码处理以呈现点云媒体，指导了点云媒体的传输、解码和呈现，可减少不必要的编解码依赖，节省传输资源和计算资源。
可选地，当计算机设备为上述内容制作设备时，在本申请实施例中，处理器801通过运行存储器804中的可执行程序代码，执行如下操作：
获取点云媒体；
对点云媒体进行编码处理，得到点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息；
对点云码流进行封装，得到点云媒体的媒体文件；媒体文件还包含属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置。
在本申请实施例中，获取点云媒体，对点云媒体进行编码处理，得到点云码流以及属性参数集合，属性参数集合包含点云码流中的属性数据解码所需的参数信息，对点云码流进行封装，得到点云媒体的媒体文件；媒体文件还包含属性参数集合指示信息，属性参数集合指示信息用于指示属性参数集合的封装位置；在媒体文件中添加该属性参数集合指示信息可以利于媒体处理设备根据该属性参数集合指示信息直接获取属性参数集合，从而指导点云媒体的传输、解码和呈现，减少一定的编解码依赖关系。
此外，这里需要指出的是：本申请实施例还提供了一种计算机可读存储介质，且计算机可读存储介质中存储有计算机程序，且该计算机程序包括程序指令，当处理器执行上述程序指令时，能够执行前文图3和图5所对应实施例中的方法，因此，这里将不再进行赘述。对于本申请所涉及的计算机可读存储介质实施例中未披露的技术细节，请参照本申请方法实施例的描述。作为示例，程序指令可以被部署在一个计算机设备上，或者在位于一个地点的多个计算机设备上执行，又或者，在分布在多个地点且通过通信网络互连的多个计算机设备上执行。
根据本申请的一个方面，提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机程序，处理器执行该计算机程序，使得该计算机设备可以执行前文图3和图5所对应实施例中的方法，因此，这里将不再进行赘述。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。
以上所揭露的仅为本申请一种较佳实施例而已，当然不能以此来限定本申请之权利范 围，本领域普通技术人员可以理解实现上述实施例的全部或部分流程，并依本申请权利要求所作的等同变化，仍属于发明所涵盖的范围。

Claims (27)

  1. 一种点云媒体的数据处理方法，所述方法由媒体处理设备执行，所述方法包括：
    获取点云媒体的媒体文件，所述媒体文件包括所述点云媒体的点云码流以及属性参数集合，所述属性参数集合包含所述点云码流中的属性数据解码所需的参数信息；所述媒体文件中还包括属性参数集合指示信息，所述属性参数集合指示信息用于指示所述属性参数集合的封装位置；
    按照所述属性参数集合指示信息，对所述点云码流进行解码处理以呈现点云媒体。
  2. 如权利要求1所述的方法，所述媒体文件包括轨道，所述轨道中包含一个或多个样本，一个样本对应所述点云媒体中的一个点云帧；一个样本被划分为一个或多个子样本，每个子样本包含一种类型的点云数据；
    所述媒体文件还包括子样本信息数据盒，所述属性参数集合封装于所述子样本信息数据盒中。
  3. 如权利要求2所述的方法，所述属性参数集合指示信息表示为所述子样本信息数据盒，所述属性参数集合指示信息还用于指示所述属性参数集合与所述点云码流中的属性数据之间的关系；
    所述子样本数据盒包括以下至少一个字段：有效载荷类型字段、参考属性信息标志字段、参考属性类型标志字段、参考属性标识字段和参考属性类型字段；
    所述有效载荷类型字段用于指示所述子样本中包含的点云数据类型；若所述有效载荷类型字段为第一取值，则指示所述子样本中包含的点云数据的类型为属性参数集合；若所述有效载荷类型字段为第二取值，则指示所述子样本中包含的点云数据的类型为几何参数集合，所述几何参数集合包含几何数据解码所需的参数信息；
    所述参考属性信息标志字段用于指示当前属性参数集合对属性数据生效范围；若所述参考属性信息标志字段为第一预设值，则指示所述当前属性参数集合仅对当前样本中的特定一组或一种特定类型的属性数据生效；若所述参考属性信息标志字段为第二预设值，则指示所述当前属性参数集合对当前样本中的所有属性数据生效；
    所述参考属性类型标志字段用于指示所述当前属性参数集合对属性数据生效的类型；若所述参考属性类型标志字段为第一数值，则指示所述当前属性参数集合仅对当前样本中一种特定类型的属性数据生效；若所述参考属性类型标志字段为第二数值，则指示当前属性参数集合仅对当前样本中特定一组属性数据生效；
    所述参考属性标识字段用于指示所述当前属性参数集合对应的属性数据的标识符；所述参考属性类型字段用于指示所述当前属性参数集合对应的属性数据的类型；
    其中，所述当前属性参数集合是指正在被使用的属性参数集合，所述当前样本是指正在被解码的样本。
  4. 如权利要求1所述的方法，所述点云码流采用多轨方式被封装在所述媒体文件中；所述媒体文件包括参数组件轨道，所述属性参数集合被独立封装在所述参数组件轨道中；
    所述参数组件轨道中包含一个或多个样本，每个样本对应所述点云媒体的一个点云帧，所述每个样本包含对应点云帧的属性数据解码所需的参数信息。
  5. 如权利要求4所述的方法，所述媒体文件包括一个或多个参数组件轨道；当所述媒体文件仅包括一个参数组件轨道时，所述点云码流中的属性参数集合被封装至所述参数组件轨道中；
    当所述媒体文件包括多个参数组件轨道时，所述点云码流中的属性参数集合被封装至一个单独的参数组件轨道中或所述点云码流中的属性参数集合基于类型被封装至多个参数组件轨道中。
  6. 如权利要求4所述的方法，所述参数组件轨道中包括轨道数据盒；
    所述轨道数据盒中包含轨道参考类型数据盒，所述轨道参考类型数据盒用于指示所述参数组件轨道与所述参数组件轨道中的属性参数集合对应的一个或多个属性组件轨道相关联。
  7. 如权利要求4所述的方法，所述参数组件轨道包含组件信息数据盒，所述组件信息数据盒设置于所述参数组件轨道的头样本入口中；所述组件信息数据盒包含以下至少一个字段：组件类型字段、头部类型字段、属性头数量字段、参考属性类型标志字段、参考属性标识字段和参考属性类型字段；
    所述组件类型字段用于指示所述参数组件轨道中的组件类型；当所述组件类型字段被设置为目标预设值时，指示所述参数组件轨道中的组件类型为参数集合；
    所述头部类型字段用于指示所述参数组件轨道中包含的参数集合类型；若所述头部类型字段为第一设定值，则指示所述参数组件轨道中包含的参数集合类型为属性参数集合；若所述头部类型字段为第二设定值，则指示所述参数组件轨道中包含的参数集合类型为几何参数集合；若所述头部类型字段为第三设定值，则指示所述参数组件轨道中包含的参数集合类型为几何参数集合和属性参数集合；
    所述属性头数量字段用于指示所述参数组件轨道中的样本包含的属性参数集合的数量；其中，所述样本中包含的属性参数集合的顺序与所述组件信息数据盒中指示的属性参数集合的标识符或属性参数集合类型的顺序一致；
    所述参考属性类型字段用于指示当前属性参数集合对属性数据生效的类型；若所述参考属性类型标志字段为第一预设值，则表示当前属性参数集合仅对当前样本中一种特定类型的属性数据生效；若所述参考属性类型标志字段为第二预设值，则表示所述当前属性参数集合对当前样本中特定一组属性数据生效；
    所述参考属性标识字段用于指示所述当前属性参数集合对应的属性数据的标识符；
    所述参考属性类型字段用于指示所述当前属性参数集合对应的属性数据的类型；
    其中，所述当前属性参数集合是指正在被使用的属性参数集合，所述当前样本是指正在被解码的样本。
  8. 如权利要求4所述的方法，所述参数组件轨道包含组件信息数据盒和配置数据盒；所述配置数据盒中包含的设置单元对应的参数集合类型与所述组件信息数据盒指示的参数集合类型相同；
    所述组件信息数据盒包含以下至少一个字段：组件类型字段、参数设置类型数量字段、参数设置类型字段；
    所述组件类型字段用于指示所述参数组件轨道中的组件的类型；若所述组件类型字段为目标设定值，则指示所述参数组件轨道中的组件的类型为参数集合；
    所述参数设置类型数量字段用于指示所述参数组件轨道包含的参数集合的数量；
    所述参数设置类型字段用于指示所述参数组件轨道中包含的参数集合的类型；若所述参数设置类型字段为第一预设值，则表示所述参数组件轨道中包含的参数集合的类型为属性参数集合；若所述参数设置类型字段为第二预设值，则表示所述参数组件轨道中包含的参数集合的类型为几何参数集合；若所述参数设置类型字段为第三预设值，则表示所述参数组件轨道中包含的参数集合的类型为序列参数集合。
  9. 如权利要求1所述的方法，所述媒体文件包括几何组件轨道，所述属性参数集合被封装在所述几何组件轨道中。
  10. 如权利要求9所述的方法，所述几何组件轨道包括组件信息数据盒，所述属性参数集合指示信息表示为所述组件信息数据盒；
    所述组件信息数据盒包括组件类型字段，所述组件类型字段用于指示所述几何组件轨道中的组件的类型；若所述组件类型字段为第一设定值，则指示所述几何组件轨道中的组件的类型为几何数据和参数集合，所述参数集合包含属性参数集合和几何参数集合。
  11. 如权利要求1所述的方法，所述媒体文件包括一个或多个属性组件轨道，所述属性参数集合被封装在每个属性组件轨道中；
    其中，所述每个属性组件轨道中封装的属性参数集合，是由同一个属性参数集合经过复制得到的。
  12. 如权利要求1所述的方法，所述媒体文件包括一个或多个属性组件轨道，所述属性参数集合包含索引，所述属性参数集合的索引被封装在每个属性组件轨道中；
    其中，任一个属性组件轨道中封装的索引用于查找所述属性参数集合，或用于查找所述属性参数集合被重新组织后形成的数据单元。
  13. 如权利要求4-7任一项所述的方法，所述点云媒体采用流化传输方式进行传输；所述获取点云媒体的媒体文件，包括：
    获取所述点云媒体的传输信令，所述传输信令中包含流指示信息，所述流指示信息用于指示所述参数组件轨道对应的数据流；
    根据所述传输信令获取所述点云媒体的媒体文件。
  14. 如权利要求13所述的方法，所述传输信令为动态自适应流信令，所述流指示信息是指组件描述子，所述组件描述子被设置于所述参数组件轨道对应的数据流；所述组件描述子包括组成类型元素及参数类型元素，所述组成类型元素用于指示在所述参数组件轨道中的组件的类型；所述参数类型元素用于指示所述参数组件轨道中包含的参数集合的类型；
    若所述组成类型元素为目标字符，则在所述参数组件轨道中的组件的类型为参数集合组件。
  15. 如权利要求1所述的方法，所述按照所述属性参数集合指示信息，对所述点云码流进行解码处理以呈现点云媒体，包括：
    根据所述属性参数集合指示信息所指示的封装位置，获取所述属性参数集合；
    利用所述属性参数集合中的参数信息对所述点云码流中的属性数据进行解码处理，以呈现所述点云媒体。
  16. 一种点云媒体的数据处理方法，所述方法由内容制作设备执行，所述方法包括：
    获取点云媒体；
    对所述点云媒体进行编码处理，得到点云码流以及属性参数集合，所述属性参数集合包含所述点云码流中的属性数据解码所需的参数信息；
    对所述点云码流进行封装，得到所述点云媒体的媒体文件；所述媒体文件还包含属性参数集合指示信息，所述属性参数集合指示信息用于指示所述属性参数集合的封装位置。
  17. 如权利要求16所述的方法，所述媒体文件包括轨道，所述点云码流被封装在所述轨道中，所述轨道中包含一个或多个样本，一个样本对应所述点云媒体中的一个点云帧；一个样本被划分为一个或多个子样本，每个子样本包含一种类型的点云数据；
    所述媒体文件还包括子样本信息数据盒，所述属性参数集合封装于所述子样本信息数据盒中。
  18. 如权利要求16所述的方法，所述点云码流采用多轨方式被封装至所述媒体文件中；所述媒体文件包括参数组件轨道，所述属性参数集合被独立封装在所述参数组件轨道中；
    所述参数组件轨道中包含一个或多个样本，每个样本对应所述点云媒体的一个点云帧，所述每个样本包含对应点云帧的属性数据解码所需的参数信息。
  19. 如权利要求16所述的方法，所述媒体文件包括几何组件轨道，所述属性参数集合被封装在所述几何组件轨道中。
  20. 如权利要求16所述的方法，所述媒体文件包括一个或多个属性组件轨道，所述属性参数集合被封装在每个属性组件轨道中；
    其中，所述每个属性组件轨道中封装的属性参数集合，是由同一个属性参数集合经过复制得到的。
  21. 如权利要求16所述的方法，所述媒体文件包括一个或多个属性组件轨道，所述属性参数集合包含索引，所述属性参数集合的索引被封装在每个属性组件轨道中；
    其中，通过任一个属性组件轨道中封装的索引均能够查找到所述属性参数集合，或能够查找到所述属性参数集合被重新组织后形成的数据单元。
  22. 如权利要求16所述的方法，所述方法还包括：
    通过传输信令发送所述点云媒体的媒体文件，所述传输信令中包含流指示信息，所述流指示信息用于指示所述参数组件轨道对应的数据流。
  23. 一种点云媒体的数据处理装置，包括：
    获取单元，用于获取点云媒体的媒体文件，所述媒体文件包括所述点云媒体的点云码流以及属性参数集合，所述属性参数集合包含所述点云码流中的属性数据解码所需的参数信息；所述媒体文件中还包括属性参数集合指示信息，所述属性参数集合指示信息用于指示所述属性参数集合的封装位置；
    处理单元，用于按照所述属性参数集合指示信息，对所述点云码流进行解码处理以呈现点云媒体。
  24. 一种点云媒体的数据处理装置，包括：
    获取单元，用于获取点云媒体；
    处理单元，用于对所述点云媒体进行编码处理，得到点云码流以及属性参数集合，所述属性参数集合包含所述点云码流中的属性数据解码所需的参数信息；
    所述处理单元，还用于对所述点云码流进行封装，得到所述点云媒体的媒体文件；所述媒体文件还包含属性参数集合指示信息，所述属性参数集合指示信息用于指示所述属性参数集合的封装位置。
  25. 一种计算机设备，包括：
    处理器，适用于执行计算机程序；
    计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序被所述处理器执行时，执行如权利要求1-22任一项所述的点云媒体的数据处理方法。
  26. 一种计算机可读存储介质，所述计算机存储介质存储有计算机程序，所述计算机程序被处理器执行时，执行如权利要求1-22任一项所述的点云媒体的数据处理方法。
  27. 一种包括计算机程序的计算机程序产品，当其在计算机上运行时，使得所述计算机执行权利要求1-22任一项所述的点云媒体的数据处理方法。
PCT/CN2023/106292 2022-08-22 2023-07-07 一种点云媒体的数据处理方法及相关设备 Ceased WO2024041238A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP23856328.2A EP4580177A4 (en) 2022-08-22 2023-07-07 MULTIMEDIA DATA PROCESSING METHOD IN POINT CLOUD AND ASSOCIATED DEVICE
US18/989,557 US20250191230A1 (en) 2022-08-22 2024-12-20 Data processing method for point cloud media, and related device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211007891.4A CN115396646B (zh) 2022-08-22 2022-08-22 一种点云媒体的数据处理方法及相关设备
CN202211007891.4 2022-08-22

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/989,557 Continuation US20250191230A1 (en) 2022-08-22 2024-12-20 Data processing method for point cloud media, and related device

Publications (1)

Publication Number Publication Date
WO2024041238A1 true WO2024041238A1 (zh) 2024-02-29

Family

ID=84120834

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/106292 Ceased WO2024041238A1 (zh) 2022-08-22 2023-07-07 一种点云媒体的数据处理方法及相关设备

Country Status (4)

Country Link
US (1) US20250191230A1 (zh)
EP (1) EP4580177A4 (zh)
CN (2) CN115396646B (zh)
WO (1) WO2024041238A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115396646B (zh) * 2022-08-22 2024-04-26 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法及相关设备
CN118334198A (zh) * 2023-01-10 2024-07-12 海信视像科技股份有限公司 一种场景描述文件的解析方法及装置
CN119071571A (zh) * 2023-05-31 2024-12-03 腾讯科技（深圳）有限公司 点云文件的处理方法、装置、介质及电子设备
CN116743722A (zh) * 2023-06-28 2023-09-12 腾讯科技（深圳）有限公司 点云媒体的数据处理方法、装置、存储介质及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112804256A (zh) * 2021-02-09 2021-05-14 腾讯科技（深圳）有限公司 多媒体文件中轨道数据的处理方法、装置、介质及设备
US20210320962A1 (en) * 2020-04-11 2021-10-14 Lg Electronics Inc. Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method
CN113766271A (zh) * 2020-06-04 2021-12-07 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法
WO2022148650A1 (en) * 2021-01-06 2022-07-14 Canon Kabushiki Kaisha Method, device, and computer program for encapsulating timed media content data in a single track of encapsulated media content data
CN115396646A (zh) * 2022-08-22 2022-11-25 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法及相关设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130170561A1 (en) * 2011-07-05 2013-07-04 Nokia Corporation Method and apparatus for video coding and decoding
CN110876083B (zh) * 2018-08-29 2021-09-21 浙江大学 指定参考图像的方法及装置及处理参考图像请求的方法及装置
WO2020070379A1 (en) * 2018-10-03 2020-04-09 Nokia Technologies Oy Method and apparatus for storage and signaling of compressed point clouds
US11122102B2 (en) * 2019-07-03 2021-09-14 Lg Electronics Inc. Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus and point cloud data reception method
CN114073086B (zh) * 2019-07-04 2024-12-17 Lg电子株式会社 点云数据处理设备和方法
WO2021210867A1 (ko) * 2020-04-12 2021-10-21 엘지전자 주식회사 포인트 클라우드 데이터 송신 장치, 포인트 클라우드 데이터 송신 방법, 포인트 클라우드 데이터 수신 장치 및 포인트 클라우드 데이터 수신 방법
EP4131961A4 (en) * 2020-04-13 2023-09-13 LG Electronics, Inc. DEVICE FOR TRANSMITTING POINT CLOUD DATA, METHOD FOR TRANSMITTING POINT CLOUD DATA, DEVICE FOR RECEIVING POINT CLOUD DATA AND METHOD FOR RECEIVING POINT CLOUD DATA
MX2023000131A (es) * 2020-06-22 2023-04-05 Interdigital Patent Holdings Inc Transmisión adaptativa de nubes de puntos basados en geometría.
CN114697668B (zh) * 2022-04-22 2023-06-30 腾讯科技（深圳）有限公司 点云媒体的编解码方法及相关产品

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210320962A1 (en) * 2020-04-11 2021-10-14 Lg Electronics Inc. Point cloud data transmission device, point cloud data transmission method, point cloud data reception device, and point cloud data reception method
CN113766271A (zh) * 2020-06-04 2021-12-07 腾讯科技（深圳）有限公司 一种沉浸媒体的数据处理方法
WO2022148650A1 (en) * 2021-01-06 2022-07-14 Canon Kabushiki Kaisha Method, device, and computer program for encapsulating timed media content data in a single track of encapsulated media content data
CN112804256A (zh) * 2021-02-09 2021-05-14 腾讯科技（深圳）有限公司 多媒体文件中轨道数据的处理方法、装置、介质及设备
CN115396646A (zh) * 2022-08-22 2022-11-25 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法及相关设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4580177A4 *

Also Published As

Publication number Publication date
CN115396646B (zh) 2024-04-26
EP4580177A4 (en) 2025-12-10
EP4580177A1 (en) 2025-07-02
CN118612397B (zh) 2025-09-16
US20250191230A1 (en) 2025-06-12
CN115396646A (zh) 2022-11-25
CN118612397A (zh) 2024-09-06

Similar Documents

Publication Publication Date Title
EP4576768A1 (en) Data processing method for point cloud media, and related device
JP6984841B2 (ja) イメージ処理方法、端末およびサーバ
WO2024041238A1 (zh) 一种点云媒体的数据处理方法及相关设备
CN114697668B (zh) 点云媒体的编解码方法及相关产品
US20230421810A1 (en) Encapsulation and decapsulation methods and apparatuses for point cloud media file, and storage medium
WO2024060793A1 (zh) 一种点云媒体的数据处理方法及相关设备
WO2024041239A1 (zh) 一种沉浸媒体的数据处理方法、装置、设备、存储介质及程序产品
WO2023169004A1 (zh) 点云媒体的数据处理方法、装置、设备及介质
US20250373832A1 (en) Point cloud file processing
US20260032270A1 (en) Media data processing
WO2025039852A1 (zh) 点云文件封装与解封装方法、装置、设备及存储介质
WO2024183506A1 (zh) 沉浸媒体的数据处理方法、装置、计算机设备、存储介质及程序产品
WO2023226504A1 (zh) 一种媒体数据处理方法、装置、设备以及可读存储介质
WO2023169003A1 (zh) 点云媒体的解码方法、点云媒体的编码方法及装置
CN115086635B (zh) 多视角视频的处理方法、装置、设备及存储介质
CN115061984A (zh) 点云媒体的数据处理方法、装置、设备、存储介质
JP2023550752A (ja) 非時系列ポイントクラウドメディアの処理方法、装置、機器、及びコンピュータプログラム
CN116781674B (zh) 一种沉浸媒体的数据处理方法、装置、设备及存储介质
HK40099879A (zh) 点云文件封装与解封装方法、装置、设备及存储介质
HK40097789A (zh) 媒体数据处理方法、装置、设备及存储介质
CN115426502A (zh) 点云媒体的数据处理方法、装置、设备及存储介质
HK40086888A (zh) 一种沉浸媒体的数据处理方法及相关设备
HK40086888B (zh) 一种沉浸媒体的数据处理方法及相关设备
HK40067090B (zh) 点云文件传输方法、应用方法、装置、设备及存储介质
HK40083114B (zh) 点云数据处理方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23856328

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023856328

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2023856328

Country of ref document: EP

Effective date: 20250324

WWP Wipo information: published in national office

Ref document number: 2023856328

Country of ref document: EP