WO2024255912A1 - Encoding method, decoding method, bitstream, encoder, decoder, medium and program product - Google Patents

Encoding method, decoding method, bitstream, encoder, decoder, medium and program product Download PDF

Info

Publication number: WO2024255912A1
Authority: WO; WIPO (PCT)
Prior art keywords: mesh; lod; current; reconstructed; determining
Prior art date: 2023-06-15
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.): Ceased

Application number

PCT/CN2024/099684

Other languages

English (en)

French (fr)

Inventor

Vladyslav ZAKHARCHENKO

Yue Yu

Haoping Yu

Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)

Guangdong Oppo Mobile Telecommunications Corp Ltd

Original Assignee

Guangdong Oppo Mobile Telecommunications Corp Ltd

Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)

2023-06-15

Filing date

2024-06-17

Publication date

2024-12-19

2024-06-17 Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd

2024-06-17 Priority to CN202480027912.XA priority Critical patent/CN121058043A/zh

2024-12-19 Publication of WO2024255912A1 publication Critical patent/WO2024255912A1/en

2025-12-15 Anticipated expiration legal-status Critical

Status Ceased legal-status Critical Current

Links

Classifications

- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/001—Model-based coding, e.g. wire frame
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/004—Predictors, e.g. intraframe, interframe coding
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

the displacement coefficients close to the vertices in the base layer can be removed from the displacement list in the encoding stage, and the displacement coefficients close to the vertices in the base layer is inferred to be equal to 0 in the decoding stage. Therefore, in the subsequent process of recursively applying the displacement coefficients to the previously reconstructed Levels of Details (LODs) , the redundancy problem of the displacement coefficients at the higher levels can be improved, the encoded bits of the displacement coefficients can be reduced, and the encoding and decoding efficiency can be improved.
LODs Levels of Details
FIG. 2 is a schematic diagram of a generation process of displacement components
FIG. 3 is a decomposed schematic diagram of displacement components in a local coordinate system
FIG. 5 is a schematic diagram of a mesh subdivision
FIG. 6 is a schematic diagram of a mesh subdivision result with 2 LODs and one-dimensional displacements
FIG. 7 is a schematic diagram of an encoding process of a parameterized mesh
FIG. 8 is a schematic diagram of geometry information in a mesh frame
FIG. 9 is a surface diagram of a mesh consisting of four vertices and three faces
FIG. 10 is a schematic diagram of a data structure of a mesh consisting of four vertices and three faces;
FIG. 11 is a schematic diagram of a data structure of a parameterized mesh having an attribute texture map
FIG. 12 is a schematic diagram of a mesh with attribute mapping characteristics consisting of four vertices and three triangular faces;
FIG. 13A is a schematic diagram of a manifold mesh
FIG. 13B is a schematic diagram of a non-manifold mesh
FIG. 14 is a mapping diagram of displacement components
FIG. 15B is a schematic diagram of displacement component packing of another two-dimensional image
FIG. 16 is a first flowchart of a decoding method provided by an embodiment of the present disclosure.
FIG. 28 is a fourth flowchart of an encoding method provided in an embodiment of the present disclosure.
Animated mesh - is a dynamic mesh with constant connectivity.
Connectivity -a set of vertex indices describing how to connect the mesh vertices to create a 3D surface. (Geometry and all the attributes share the same unique connectivity information) .
Geometry -a set of vertex 3D (x, y, z) coordinates describing positions associated with the mesh vertices.
the (x, y, z) coordinates representing the positions should have finite precision and dynamic range.
Mapping -a description of how to map the mesh surface to 2D regions of the plane. Such mapping is described by a set of UV parametric/texture [mapping] coordinates associated with the mesh vertices together with the connectivity information.
a static or dynamic mesh is input to a preprocessing module for decimating geometry information to generate a base mesh and displacement components.
the decimated base mesh is encoded by a general-purpose mesh encoder (such as an "edgebreaker" ) , and the displacement components are packaged into a 2-dimensional image, and then the displacement information is encoded by a video encoder such as HEVC, and the resulting encoded bits are written into a bitstream.
the subdivision process includes the following steps.
step 1 an edge is defined using two neighboring points PB0 and PB1 in a reconstructed base mesh.
step 2 normal to the edge is calculated using a face that contains point PB0 and PB1.
step 4 displacement d1_1 is applied to point PS1_1 along the normal defined in step 2.
step 5 two edges: PB0 PS1_1 and PS1_1 PB1 are created. ( (a) and (b) in FIG. 4)
the normal is always calculated with reference to the reconstructed base mesh.
the displacements are calculated with reference to the subdivision edges generated in step 5, rather than the reconstructed base edges obtained in step 1.
step S501 an original mesh is decimated to obtain a base mesh.
step S502 the base mesh is quantized.
step S503 the quantized base mesh is encoded.
step S504 the base network is decoded from a bitstream to obtain a reconstructed base mesh.
step S505 0 is initialized, and the reconstructed base mesh is subdivided to obtain a subdivided mesh.
step S506 If the result of the determination in step S506 is NO, that is, n ⁇ L-1 is not established, the subdivision is completed, and then steps S508 and S509 are performed. That is to say, after the subdivision is completed, the displacement coefficients are obtained by calculating the subdivided mesh and the original mesh, and the displacement coefficients are encoded into the bitstream.
n represents the number of iteration subdivisions, and n is an integer indexed from 0, that is, n is an integer greater than or equal to 0.
L denotes the number of LODs, and L is an integer indexed from 1.
FIG. 6 provides a mesh subdivision result of a three-dimensional content with 2 LODs and 1-dimensional displacements.
the black solid line (composed of vertices PB_1, PB_2 and PB_3) represents the base mesh
the black dashed line represents the subdivided mesh
the first thick solid line is the LOD1 applied to the displacements
the second thick solid line is the LOD2 applied to the displacements.
the displacements are generated using hierarchical subdivision process and represent the difference between the original mesh topology and previously reconstructed subdivided LOD.
the first iteration of the displacements is using base mesh as input.
the displacements are processed by a hierarchical wavelet transform (or other transform) that recursively applies LODs to the reconstructed base mesh.
the wavelet coefficients are then quantized, packed into a 2-dimensional image/video, and can be compressed by using a traditional image/video encoder.
the reconstructed version of the wavelet coefficients is obtained by applying image unpacking and inverse quantization to the reconstructed wavelet coefficient image/video generated during the image/video decoding process.
Reconstructed displacements are then computed by applying the inverse wavelet transform to the reconstructed wavelet coefficients.
Wavelet coefficients are calculated in floating-point format and may be positive and negative. In some systems, to compose a 2-dimensional image, the coefficients are first converted to positive and mapped to a given bit-depth.
c’ (i) 2 ⁇ [bit_depth-1] + [c (i) *2 ⁇ bit_depth] / [c_max -c_min] ,
FIG. 8 illustrates an example of geometry information in a mesh frame, specifically, an example of a data structure of a mesh, each vertex of which has attributes.
FIG. 11 An example of a data structure with a parameterized mesh with an attribute texture map is shown in FIG. 11.
FIG. 12 shows an example of a surface, represented by a mesh with attribute mapping characteristics (FIG. 11) that consists of four vertices and three faces.
Each vertex in space is described by its X, Y, Z position coordinates.
(U, V) denotes attribute coordinates in the 2-dimensional texture vertex map.
Each face is defined by three pairs of vertex indices and texture vertex coordinates that form a triangle in 3-dimensional space and a triangle in the 2-dimensional texture map.
the orientation of a face is determined using the right-hand coordinate system.
a face consists of three vertices that belong to three edges, and the three vertex indices describe each face.
Manifold mesh is a mesh where one edge belongs to two different faces at most, as shown in FIG. 13A.
Non-manifold mesh is a mesh with an edge that belongs to more than two faces, as shown in FIG. 13B.
the transformed displacement components are mapped from a one-dimensional array to a 2-dimensional image, as shown in FIG. 14.
Each unit vector component is associated with a different color plane.
the Normal unit vector is mapped to Y-plane; the Tangent unit vector is mapped to U-plane; the BiTangent unit vector is mapped to V-plane.
YUV444 color mapping is used for encoding.
FIGS. 15A and 15B an example of Video component for displacement coefficients 8x8 packing block is provided.
FIG. 15A shows forward packing
FIG. 15B shows backward packing.
the displacement coefficients are applied to the previously reconstructed LODs.
the displacements coefficients are often redundant at the higher LODs, in some instances, which usually results in a large bit overhead and reduces the efficiency of encoding and decoding.
an embodiment of the present disclosure provides an encoding method and decoding method, by adding a first syntax element which indicates that the current mesh adopts a nonlinear subdivision mode, and when it is determined that the current mesh adopts a nonlinear subdivision mode, the reconstructed mesh of the current LOD in the current mesh is subdivided to determine a subdivided mesh of the current LOD.
the displacement coefficients of first points (points coinciding with or neighboring a base point) of the current LOD is directly set to 0. That is, the displacements neighboring the base point (e.g., PB_1, PB_2, and PB_3) are inferred to be equal to 0.
a reconstructed mesh of the next LOD in the current mesh is determined.
the non-linear subdivision method is adopted in the current mesh, the redundancy problem of the displacement coefficients in the higher levels can be further improved, the encoded bits of the displacement coefficients may be reduced, and the encoding and decoding efficiency can be improved.
displacement coefficients that are close to a vertex in a base layer may be dropped from the displacement list at the encoder stage, and the displacement coefficients that are close to the vertex in the base layer are inferred to be equal to zero in the decoder stage. Therefore, in the subsequent process of recursively applying the displacement coefficients to the previously reconstructed LOD, the redundancy problem of the displacement coefficients at the higher levels can be improved, the encoded bits of the displacement coefficients can be reduced, and the encoding and decoding efficiency can be improved.
FIG. 16 is a flowchart of a decoding method provided in an embodiment of the present disclosure. As shown in FIG. 16, the method may include the following steps.
step 101 the bitstream is decoded and a value of the first syntax element is determined.
the decoding method may refer to a mesh subdivision method, in particular, a recursive subdivision method for dynamic mesh decoding, which can improve the encoding and decoding efficiency.
VDMC Video-based Dynamic Mesh Coding
V3C Video-based Visual Volumetric Video-based Coding
the subdivision method shares functionality across all LODs and is defined by two syntax elements asps_vdmc_ext_subdivision_method and asps_vdmc_ext_subdivision_iteration_count in the Atlas sequence parameter set VDMC extension RBSP syntax structure.
Table 1 provides a schematic syntax structure of the Atlas sequence parameter set VDMC extension.
asps_vdmc_ext_subdivision_method indicates the identifier of the method to subdivide the meshes associated with the current atlas sequence parameter set
asps_vdmc_ext_subdivision_iteration_count indicates the number of subdivision iterations of the mesh
asps_vdmc_ext_disposition_coordinate_system indicates the coordinate system identifier of the mesh
asps_vdmc_ext_transform_method indicates the wavelet transform identifier of the mesh.
Table 2 describes a list of supported subdivision methods corresponding to asps_vdmc_ext_subdivision_method.
the value of asps_vdmc_ext_subdivision_iteration_count is inferred to be equal to 0 when it does not exist.
Table 3 describes a list of supported coordinate systems corresponding to asps_vdmc_ext_disposition_coordination_system.
Table 4 describes a list of correspondences between the supported wavelet transform methods and asps_vdmc_ext_transform_method.
the first syntax element herein may be represented by asps_vdmc_ext_subdivision_method for indicating a subdivision method of the mesh, which may be, for example, midpoint subdivision, Loop subdivision, or the like.
An embodiment of the present disclosure provides a new adaptive subdivision method in which edges that are neighboring the base points at higher LODs shall be always equal to 0.
the non-linear subdivision may create same subdivision as that in the related arts.
the displacements neighboring base point e.g., PB_1, PB_2, and PB_3
the displacements are calculated based on the subdivision vertices in the previously reconstructed LOD, so that the displacements can be recursively applied to the previously reconstructed LOD. Therefore, the adaptive subdivision method here is called "Recursive subdivision method" . Taking midpoint subdivision as an example, this recursive subdivision method can be called midpoint recursive subdivision method.
a new asps_vdmc_ext_subdivision_method equal to 2 and 3 are added,
the new asps_vdmc_ext_subdivision_method would create same subdivision as that in the related arts.
the displacements neighboring base point e.g., PB_1, PB_2, and PB_3 are inferred to be equal to zero, thereby saving the coded bits of the displacements.
step 102 when the first syntax element indicates that the current mesh adopts a non-linear subdivision mode, the reconstructed mesh of the current LOD in the current mesh is determined, and the reconstructed mesh of the current LOD is subdivided to determine the subdivided mesh of the current LOD.
the first syntax element indicates whether the current mesh adopts a non-linear subdivision mode.
the value of the first syntax element is the first value
when the value of the first syntax element is a second value it is determined that the first syntax element indicates that the current mesh adopts a midpoint subdivision mode
when the value of the first syntax element is a third value it is determined that the first syntax element indicates that the current mesh adopts a recursive subdivision mode
the value of the first syntax element is a fourth value, it is determined that the first syntax element indicates that the current mesh adopts a non-linear subdivision mode.
the first value, the second value, the third value, and the fourth value are different from each other.
the first value may be set to 0, the second value may be set to 1, the third value may be set to 2, and the fourth value may be set to 3. That is, according to the different values of the first syntax element, it is possible to determine whether the current mesh adopts s a non-linear subdivision method.
the value of the first syntax element if the value of the first syntax element is 0, it can be determined that the current mesh does not adopt any subdivision, such as midpoint subdivision, recursive subdivision, nonlinear subdivision, and the like. If the value of the first syntax element is 1, it can be determined that the current mesh adopts midpoint subdivision. If the value of the first syntax element is 2, it can be determined that the current mesh is subdivided recursively. If the value of the first syntax element is 3, it can be determined that the current mesh adopts a non-linear subdivision.
the reconstructed mesh of the current mesh may be obtained by recursive reconstruction of at least one LOD. Specifically, after obtaining the reconstructed mesh of the current LOD in the current mesh, the reconstructed mesh of the current LOD may be subdivided to determine the subdivided mesh of the current LOD. Then, a reconstructed mesh of the next LOD of the current LOD may be recursively obtained according to the subdivided mesh, and preset displacement coefficients or displacement coefficients in the bitstream.
a reconstructed mesh of the first LOD of the current mesh is firstly determined.
the method may include: decoding the bitstream to determine a reconstruction base mesh; subdividing the reconstructed base mesh to determine an initial subdivided mesh; decoding the bitstream to determine displacement coefficients of the initial subdivided mesh; determining a reconstructed mesh of the first LOD in the current mesh according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh, .
the base mesh may also be referred to as a "decimated mesh" .
a reconstructed base mesh is determined by decoding the base mesh bitstream.
the base mesh bitstream may be decoded by a mesh decoder (such as EdgeBreaker) to obtain a reconstructed base mesh.
the reconstructed mesh of the first LOD is subdivided to determine a subdivided mesh of the second LOD
the bitstream is decoded to determine displacement coefficients of the subdivided mesh of the seond LOD
a reconstructed mesh of a next LOD of the current mesh is determined according to the subdivided mesh of the second LOD and the displacement coefficients of the subdivided mesh of the second LOD
subdivision operation is continued, until the reconstructed mesh of the L-th LOD in the current mesh is determined.
the value of L is correlated with the number of subdivision iterations of the current mesh.
the displacement coefficient refers to the difference between the vertex coordinate information of the original mesh and the vertex coordinate information of the initial subdivided mesh.
the corresponding displacement operation is performed on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficient, and the reconstructed mesh of the first LOD, i.e., the reconstructed mesh of LOD1, may be obtained.
the reconstructed mesh is subdivided based on LOD1 to obtain a subdivided mesh on LOD1, and a reconstructed mesh of LOD2 may be determined according to the subdivided mesh on LOD1and corresponding displacement coefficients, and so on until the reconstruction is completed.
step 103 when the current LOD is the i-th LOD, the displacement coefficients of the first points of the current LOD are determined to be 0.
i is an integer greater than 2.
the first points include vertices of the reconstructed base mesh corresponding to the current mesh.
the displacement coefficients may include displacement components in one or more directions.
the displacement coefficients include displacement components in three directions, namely, the normal direction, the tangent direction, and the bitangent direction.
the displacement coefficients of the first LOD are calculated relative to the subdivision vertices of the base mesh
the displacement coefficients of the subsequent LODs are calculated relative to the subdivision vertices of the reconstructed LOD of a previous LOD, so that the recursive subdivision of the displacement coefficients can be realized.
the displacement coefficients of the first points of the current LOD may be determined to be 0.
the first points of the current LOD include vertices of the reconstructed base mesh corresponding to the current mesh, that is, the first points may be understood as a points coinciding with or neighboring a base point (e.g., PB_1, PB_2, PB_3) .
the displacement coefficients of the first points of the current LOD may be directly set to 0, and subsequent mesh reconstruction may be carried out using the displacement coefficients.
step 104 a reconstructed mesh of the next LOD in the current mesh is determined based on the subdivided mesh of the current LOD and the displacement coefficients of the first points of the current LOD.
the reconstructed mesh of the next LOD in the current mesh is determined. Specifically, according to the displacement coefficients of the first points, displacements are calculated based on the vertex coordinate information of the subdivided mesh of the current LOD, and then the reconstructed mesh of the next LOD in the current mesh is obtained.
the displacement coefficients are recursively applied to the previously reconstructed LOD, specifically, the decoded displacement coefficient is added to each vertex of the subdivided mesh of the current LOD in order, so that the reconstructed mesh of the next LOD may be obtained.
the displacement coefficients of the first points of the current LOD is set to 0, that is, the displacements of the points (first points) overlapping or neighboring the base point are set to 0.
the displacement calculation for the first point is actually carried out on the subdivided mesh of the current LOD and 0.
the method may further include: after determining the reconstructed mesh of the next LOD in the current mesh, taking the reconstructed mesh of the next LOD as the reconstructed mesh of the current LOD, returning to performing the step of subdividing the reconstructed mesh of the current LOD, and determining the subdivided mesh of the current LOD until the reconstructed mesh of the L-th LOD in the current mesh is determined.
the value of L is correlated with the number of subdivision iterations of the current mesh.
the number of subdivision iterations of the current mesh may be indicated by a second syntax element in the bitstream.
the method may include: decoding a bitstream to determine a value of a second syntax element; determining the number of subdivision iterations of the current mesh according to the value of the second syntax element.
the second syntax element may be represented by asps_vmc_ext_subdivision_iteration_count. That is, the number of subdivision iterations for the current mesh may be a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.
L represents the number of LODs corresponding to the current mesh, and the value of L may be determined by the number of subdivision iterations of the current mesh, that is, the value of L is equal to the value indicated by asps_vmc_ext_subdivision_iteration_count.
FIG. 17 is a second flowchart of a decoding method provided in an embodiment of the present disclosure. As shown in FIG. 17, after step 102, the method may include the following steps.
step 105 the bitstream is decoded to determine the displacement coefficients of the second points of the current LOD if the current LOD is the i-th LOD.
the second points do not include the vertices of the reconstructed base mesh corresponding to the current mesh.
the second points are points of the reconstructed base mesh corresponding to the current mesh except the first points.
the displacement coefficients of the first points of the current LOD may be determined to be 0, and the displacement coefficients of the second points may be obtained by decoding the bitstream.
the bitstream may be a displacement bitstream.
specific decoding methods for the displacement bitstream such as video decoding, entropy decoding and so on. That is, in an embodiment of the present disclosure, the displacement information may be obtained by decoding using a video decoder, or the displacement information may be obtained by decoding with an entropy decoder, without any limitation herein.
the second points of the current LOD do not include vertices of the reconstructed base mesh corresponding to the current mesh, i.e., the second points may be understood as point that are not close to the base point (e.g., PB_1, PB_2, PB_3) .
the displacement coefficients of the first points of the current LOD may be set to 0, and at the same time, the displacement coefficients of the second points of the current LOD may be determined by decoding the bitstream.
a reconstructed mesh of the next LOD in the current mesh is determined based on the subdivided mesh of the current LOD and the displacement coefficients of the second points of the current LOD.
the reconstructed mesh of the next LOD in the current mesh is determined, specifically, according to the displacement coefficients of the second points, the corresponding displacement operation is performed on the vertex coordinate information of the subdivided mesh of the current LOD, and the reconstructed mesh of the next LOD in the current mesh is obtained.
the displacement coefficients is recursively applied to the previously reconstructed LOD, specifically, the decoded displacement coefficients are added to respective vertices of the subdivided mesh of the current LOD in order, so that the reconstructed mesh of the next LOD can be obtained.
FIG. 18 is a third flowchart of a decoding method provided in an embodiment of the present disclosure. As shown in FIG. 18, after step 102, the method may include the following steps.
step 107 when the current LOD is a second LOD, the bitstream is decoded to determine the displacement coefficient of the current LOD.
the displacement coefficients of the current LOD may be obtained by decoding the bitstream.
the bitstream may be a displacement bitstream.
specific decoding methods for the displacement bitstream such as video decoding, entropy decoding and so on. That is, in an embodiment of the present disclosure, the displacement information may be obtained by decoding using a video decoder, or the displacement information may be obtained by decoding using an entropy decoder, without any limitation herein.
step 108 a reconstructed mesh of the next LOD in the current mesh is determined based on the subdivided mesh of the current LOD and the displacement coefficients of the current LOD.
the reconstructed mesh of the next LOD in the current mesh is determined, specifically, according to the displacement coefficients, the corresponding displacement operation is performed on the vertex coordinate information of the subdivided mesh of the current LOD, and the reconstructed mesh of the next LOD in the current mesh is obtained.
the displacement coefficients are recursively applied to the previously reconstructed LOD, specifically, the decoded displacement coefficient is added to each vertex of the subdivided mesh of the current LOD in order, so that the reconstructed mesh of the next LOD can be obtained.
the method may further include: after determining the reconstructed mesh of the next LOD in the current mesh, taking the reconstructed mesh of the next LOD as the reconstructed mesh of the current LOD, returning to performing the step of subdividing the reconstructed mesh of the current LOD, and determining the subdivided mesh of the current LOD until the reconstructed mesh of the L-th LOD in the current mesh is determined.
the value of L is correlated with the number of subdivision iterations of the current mesh.
the number of subdivision iterations of the current mesh may be indicated by a second syntax element in the bitstream.
the method may include: decoding a bitstream to determine a value of a second syntax element; and determining the number of subdivision iterations of the current mesh according to the value of the second syntax element, .
the second syntax element may be represented by asps_vmc_ext_subdivision_iteration_count. That is, the number of subdivision iterations for the current mesh may be a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.
L represents the number of LODs corresponding to the current mesh, and the value of L may be determined by the number of subdivision iterations of the current mesh, that is, the value of L is equal to the value indicated by asps_vmc_ext_subdivision_iteration_count.
FIG. 19 is a fourth flowchart of a decoding method provided in an embodiment of the present disclosure. As shown in FIG. 19, after step 101, the method may include the following steps.
step 109 when the first syntax element indicates that the current mesh adopts a recursive subdivision mode, a reconstructed mesh of the current LOD in the current mesh is determined, and the reconstructed mesh of the current LOD is subdivided to determine a subdivided mesh of the current LOD.
the first syntax element may also indicates whether the current mesh adopts recursive subdivision.
the value of the first syntax element is the first value, it is determined that the first syntax element indicates that the current mesh does not adopt the subdivision mode; when the value of the first syntax element is a second value, it is determined that the first syntax element indicates that the current mesh adopts a midpoint subdivision mode; when the value of the first syntax element is a third value, it is determined that the first syntax element indicates that the current mesh adopts a recursive subdivision mode; and when the value of the first syntax element is a fourth value, it is determined that the first syntax element indicates that the current mesh adopts a non-linear subdivision mode.
the first value, the second value, the third value, and the fourth value are different from each other.
the first value may be set to 0, the second value may be set to 1, the third value may be set to 2, and the fourth value may be set to 3. That is, according to the different values of the first syntax element, it is possible to determine whether the current mesh adopts a recursive subdivision method.
the value of the first syntax element if the value of the first syntax element is 0, it may be determined that the current mesh does not adopt any subdivision, such as midpoint subdivision, recursive subdivision, nonlinear subdivision, and the like; if the value of the first syntax element is 1, it may be determined that the current mesh adopts midpoint subdivision; if the value of the first syntax element is 2, it may be determined that the current mesh is subdivided recursively; if the value of the first syntax element is 3, it may be determined that the current mesh adopts a non-linear subdivision.
the reconstructed mesh of the current mesh may be obtained by recursive reconstruction of at least one LOD. Specifically, after obtaining the reconstructed mesh of the current LOD in the current mesh, the reconstructed mesh of the current LOD may be subdivided to determine the subdivided mesh of the current LOD. Then, the reconstructed mesh of the next LOD of the current LOD may be recursively obtained according to the subdivided mesh and the displacement coefficients in the bitstream, .
a reconstructed mesh of the first LOD of the current mesh is firstly determined.
the method may include: decoding the bitstream to determine a reconstructed base mesh; subdividing the reconstructed base mesh to determine an initial subdivided mesh; decoding the bitstream to determine the displacement coefficients of the initial subdivided mesh; and determining a reconstructed mesh of the first LOD in the current mesh according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh.
the base mesh may also be referred to as a "decimated mesh" .
a reconstructed base mesh is determined by decoding the base mesh bitstream.
the base mesh bitstream may be decoded by a mesh decoder (such as EdgeBreaker) to obtain a reconstructed base mesh.
the displacement coefficient refers to the difference between the vertex coordinate information of the original mesh and the vertex coordinate information of the initial subdivided mesh. In this way, after the displacement coefficients of the initial subdivided mesh are decoded, the corresponding displacement operation is performed on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficient, and the reconstructed mesh of the first LOD, that is, the reconstructed mesh of LOD1, may be obtained.
the reconstructed mesh is subdivided based on LOD1 to obtain a subdivided mesh on LOD1, and then a reconstructed mesh of LOD2 is determined according to the reconstructed mesh of LOD1 and the displacement coefficients, and so on, until the reconstruction is completed.
step 110 the bitstream is decoded to determine the displacement coefficient of the current LOD.
the displacement coefficients may include displacement components in one or more directions.
the displacement coefficients include displacement components in three directions, namely, the normal direction, the tangent direction, and the bitangent direction.
the displacement coefficients of the first LOD are calculated relative to the subdivision vertices of the base mesh
the displacement coefficients of the subsequent LODs are calculated by reconstructing the subdivision vertices of LOD of relative to the previous LOD, so that the recursive subdivision of the displacement coefficients can be realized.
the displacement coefficients of the current LOD are obtained by decoding the bitstream.
the bitstream may be a displacement bitstream.
specific decoding methods for the displacement bitstream such as video decoding, entropy decoding and so on. That is, in an embodiment of the present disclosure, the displacement information may be obtained by decoding using a video decoder, or the displacement information may be obtained by decoding using an entropy decoder, without any limitation herein.
the method of determining the displacement coefficients of the current LOD involved in any one of steps 105, 107, and 110 may specifically include decoding the bitstream to determine the decoded displacement coefficients of the current LOD.
the decoded displacement coefficients of the current LOD are inverse quantized according to the quantization parameters of the current LOD, to determine the inverse quantized displacement coefficients of the current LOD.
Inverse wavelet transform is performed on the inverse quantized displacement coefficients of the current LOD to determine the displacement coefficients of the current LOD.
the decoded displacement coefficients of the current LOD may be preprocessed to determine the displacement coefficients of the current LOD.
the preprocessing may include inverse quantization, inverse wavelet transform and other processing.
the decoded displacement coefficients of the current LOD are preprocessed to determine the displacement coefficients of the current LOD, specifically, quantization parameters of the current LOD is determined; the decoded displacement coefficients of the current LOD are inverse quantized according to the quantization parameters of the current LOD to determine the inverse quantized displacement coefficients of the current LOD; inverse wavelet transform is performed on the inverse quantized displacement coefficients of the current LOD to determine the displacement coefficients of the current LOD.
inverse quantization is an inverse process of quantization, and is used for converting quantized fixed points into floating points
inverse wavelet transform is the inverse process of wavelet transform, which is used to restore the signal in the wavelet domain to the original time domain, so that the displacement coefficients of the current LOD can be obtained.
the Quantizer Parameter reflects the spatial detail compression situation.
the smaller the value the finer the quantization, the higher the image quality, and the longer the bitstream. If the QP is small, most of the details will be retained; QP increases, some details are lost, bit rate is reduced, but image distortion is enhanced and quality is degenerated.
QP is the serial number of the quantization step Qstep, and when the value of QP is 0, it means that the quantization is the finest. on the contrary, when the value of QP is 51, it means that the quantization is rougher.
the method may include decoding the bitstream, determining the quantization parameters of the current LOD.
the method may include: determining quantization parameters of a previous LOD of the current LOD; decoding the bitstream to determine a quantization parameter increment of the current LOD.
the quantization parameters of the current LOD are determined according to the quantization parameters of the previous LOD and the quantization parameter increment.
the encoder side may directly write the quantization parameters into the bitstream, so that the decoder side may obtain the corresponding quantization parameters through decoding; or the encoder side may write the quantization parameter increment into the bitstream, and then the decoder side may obtain the corresponding quantization parameters according to the quantization parameter increment obtained by decoding and the quantization parameter of the previous LOD.
the quantization parameters of the current LOD may be represented by QP(i)
the quantization parameters of a previous LOD may be represented by QP (i-1)
the quantization parameter increment may be represented by ⁇ QP
QP (i) QP (i-1) + ⁇ QP
the corresponding quantization parameter QP (1) may be directly written into the bitstream.
a reference quantization parameters may also be set at the encoder side and the decoder side, and then the quantization parameter increment between the quantization parameter of each LOD and the reference quantization parameter is written into the bitstream.
the decoder side may determine the quantization parameter of the current LOD according to the quantization parameter increment of the current LOD and the reference quantization parameter, so that the signaling overhead of decoding the quantization parameter can also be saved.
decoding the bitstream to determine decoded displacement coefficients for the current LOD may include: decoding the bitstream to determine a two-dimensional image; extracting displacement coefficients from the two-dimensional image according to the preset packing mode to obtain the decoded displacement coefficients of the current LOD.
the decoded displacement coefficients of the current LOD may be determined by decoding the displacement bitstream.
the preset packing mode includes a forward packing mode or a inverse packing mode.
the decoder side and the encoder side may set the same packing mode, or the packing mode may be indicated according to a third syntax element in the bitstream.
the method may include: decoding a bitstream to determine a value of a third syntax element; and determining the preset packing mode according to the value of the third syntax element.
the third syntax element may be represented by dmsps_packing_order. That is, the preset packing mode may be the packing mode indicated by dmsps_packing_order in the bitstream.
the displacement bitstream may be decoded by a displacement decoder. If the encoder side compresses the displacements by video encoding, the decoder side decodes the displacements using corresponding video decoder, and restores it from the two-dimensional image in a corresponding order according to the preset packing mode. Then, inverse quantization is performed on the displacements, inverse wavelet transform and other operations are performed to restore the same displacement coefficients as the encoder side.
the decoder may directly decode the displacement coefficients using entropy decoding, and then perform subsequent operations such as inverse quantization and inverse wavelet transform to obtain the displacement coefficients of the current LOD.
a reconstructed mesh of the next LOD in the current mesh is determined according to the subdivided mesh of the current LOD and the displacement coefficients of the current LOD.
the reconstructed mesh of the next LOD in the current mesh is determined, speciffically, a reconstructed mesh of the next LOD in the current mesh is obtained by performing a corresponding displacement operation on the vertex coordinate information of the subdivided mesh of the current LOD according to the displacement coefficients.
the displacement coefficients are recursively applied to the previously reconstructed LOD. Specifically, the decoded displacement coefficients are added to respective vertices of the subdivided mesh of the current LOD in order, so that the reconstructed mesh of the next LOD can be obtained.
the method may further include: after determining the reconstructed mesh of the next LOD in the current mesh, taking the reconstructed mesh of the next LOD as the reconstructed mesh of the current LOD, returning to performing the step of subdividing the reconstructed mesh of the current LOD, and determining the subdivided mesh of the current LOD until the reconstructed mesh of the L-th LOD in the current mesh is determined.
the value of L is correlated with the number of subdivision iterations of the current mesh.
the number of subdivision iterations of the current mesh may be indicated by a second syntax element in the bitstream.
the method may include decoding a bitstream to determine a value of a second syntax element. According to the value of the second syntax element, the number of subdivision iterations of the current mesh is determined.
the second syntax element may be represented by asps_vmc_ext_subdivision_iteration_count. That is, the number of subdivision iterations for the current mesh may be a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.
L represents the number of LODs corresponding to the current mesh, and the value of L may be determined by the number of subdivision iterations of the current mesh, that is, the value of L is equal to the value indicated by asps_vmc_ext_subdivision_iteration_count.
the base mesh is subdivided first, and an initial subdivided mesh can be obtained.
a corresponding displacement operation is carried out on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficients, and the reconstructed mesh of the first LOD, that is, the reconstructed mesh of LOD1, may be obtained.
the reconstructed mesh is subdivided based on LOD1 to obtain a subdivided mesh on LOD1.
the reconstructed mesh of LOD2 may be determined according to the subdivided mesh on LOD1 and the displacement coefficients of LOD1, and so on, until the reconstructed mesh of LODL is obtained, indicating that the current mesh reconstruction is completed.
a black solid line (composed of vertices PB_1, PB_2, and PB_3) represents a base mesh
a black dashed line represents a subdivided mesh
the first thick solid line is LOD1 onto which the displacements are applied
the second thick solid line is LOD2 onto which the displacements are applied.
the detailed flow of recursive subdivision is shown in FIG. 21, and the detailed flow may include the following steps.
step S1901 firstly, an original mesh is decimated to obtain a base mesh.
step S1902 the base mesh is quantized.
step S1903 the quantized base mesh is encoded.
step S1904 the bitstream is parsed to decode the base mesh from the bitstream to obtain a reconstructed base mesh.
step S1906 displacement coefficients are calculated from the subdivided mesh and the original mesh.
step S1907 the interim mesh is updated according to the displacement coefficients and the subdivided mesh, and the displacement coefficients are saved to the memory.
step S1909 it is determined whether n is less than L-1.
the base mesh is subdivided first, and an initial subdivided mesh can be obtained.
a corresponding displacement operation is carried out on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficients, and the reconstructed mesh of the first LOD, that is, the reconstructed mesh of LOD1, can be obtained.
the reconstructed mesh is subdivided based on LOD1 to obtain a subdivided mesh on LOD1.
the reconstructed mesh of LOD2 may be determined according to the subdivided mesh on LOD1 and the displacement coefficients of LOD1.
the preset value may be for example 2
the displacement coefficients of neighboring vertices (first points) of the base mesh in LOD i may be set to 0, while the displacement coefficients of other points (second points) may still be decoded.
the next LOD of reconstructed meshes may be determined, and so on, until a reconstructed mesh of LOD L is obtained, indicating that the current mesh reconstruction is completed.
a black solid line (composed of vertices PB_1, PB_2, and PB_3) represents a base mesh
a black dashed line represents a subdivided mesh
a first thick solid line is LOD1 onto which the displacements are applied
a second thick solid line is LOD2 onto which the displacements are applied.
the preset value may be for example 2, the embodiments are not limited thereto. In various embodiments, the preset value may be 2, 3, 4, ..., or any other integer value.
the detailed flow of non-linear subdivision is shown in FIG. 23, and the detailed flow may include the following steps.
step S1901 an original mesh is decimated to obtain the base mesh.
step S1902 the base mesh is quantized.
step S1903 the quantized base mesh is encoded.
step S1904 the bitstream is parsed to decode the base mesh from the bitstream to obtain a reconstructed base mesh.
step S1906 the displacement coefficient is calculated from the subdivided mesh and the original mesh.
step S1911 the displacement coefficients of adjacent foundation mesh vertices are deleted in order to de-redundancy.
step S1907 the interim mesh is updated according to the displacement coefficient and the subdivided mesh.
step S1908 the other displacement coefficients are saved to the memory.
the method may include the following steps.
step 201 when the first syntax element indicates that the current mesh adopts the midpoint subdivision mode, the bitstream is decoded to determine the reconstructed base mesh.
step 202 the reconstruction base mesh is subdivided to determine a subdivided mesh for at least one LOD of the current mesh.
step 203 the bitstream is decoded to determine a displacement coefficient for at least one LOD in the current mesh.
a reconstructed mesh of at least one LOD in the current mesh is determined based on a subdivided mesh of at least one LOD in the current mesh and displacement coefficients of at least one LOD.
the number of LODs of at least one LOD is correlated with the number of subdivision iterations of the current mesh.
L denotes the number of LODs of at least one LOD, that is, the number of LODs corresponding to the current mesh, which may be equal to a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.
the displacement coefficient of each LOD is calculated by the displacement of the vertex coordinate information of the subdivided mesh and the vertex coordinate information of the original mesh, instead of being recursively obtained based on the previously reconstructed LOD.
An embodiment of the present disclosure provides a decoding method, in particular, a recursive subdivision method of a dynamic mesh.
the displacement coefficient close to the vertex in the base layer is inferred to be equal to 0 in the decoding stage, so that in the subsequent process of recursively applying the displacement coefficient to the previously reconstructed LOD, the redundancy problem of the displacement coefficient at the higher levels can be improved, the encoded bits of the displacement coefficient can be saved, and the encoding and decoding efficiency can be improved.
FIG. 25 is a flowchart of an encoding method provided in an embodiment of the present disclosure. As shown in FIG. 25, the method may include the following steps.
step 301 when the non-linear subdivision mode is adopted in the current mesh, the value of the first syntax element is determined, and the first syntax element is written into the bitstream, the reconstructed mesh of the current LOD in the current mesh is determined, and the reconstructed mesh of the current LOD is subdivided, and the subdivided mesh of the current LOD is determined.
the encoding method may be a mesh subdivision method, in particular, a recursive subdivision method for dynamic mesh encoding, which can improve the encoding and decoding efficiency.
VDMC Video-based Dynamic Mesh Coding
V3C Video-based Visual Volumetric Video-based Coding
the subdivision method shares functionality across all LODs and is defined by two syntax elements asps_vdmc_ext_subdivision_method and asps_vdmc_ext_subdivision_iteration_count in the Atlas sequence parameter set VDMC extension RBSP syntax structure.
Embodiments of the present disclosure propose a new adaptive subdivision scheme in which the edge neighboring the base point is always equal to 0 at higher LODs.
the current mesh adopts a non-linear subdivision mode by setting a first syntax element.
the first syntax element herein may be represented by asps_vdmc_ext_subdivision_method for indicating the subdivision method of the mesh. For example, as shown in Table 2, if the value of the first syntax element is 0, it means that the current mesh does not adopt the subdivision mode; and if the value of the first syntax element is 1, it means that the current mesh adopts the midpoint subdivision method.
the non-linear subdivision may be to create the same subdivision as in the related arts, but the displacement neighboring the base point (e.g., PB_1, PB_2, and PB_3) is inferred to be equal to 0.
the displacements are calculated based on the subdivision vertices in the previously reconstructed LOD, so that the displacements can be recursively used for the previously reconstructed LOD.
the adaptive subdivision method here is called "Recursive subdivision method" . Taking midpoint subdivision as an example, this recursive subdivision method may be called midpoint recursive subdivision method.
a new asps_vdmc_ext_subdivision_method equal to 2 and 3 are added, the new asps_vdmc_ext_subdivision_method would create same subdivision as that in the related arts.
the displacements neighboring base point e.g., PB_1, PB_2, and PB_3 are inferred to be equal to zero, thereby saving the coded bits of the displacements.
the first syntax element indicates whether the current mesh adopts a non-linear subdivision mode.
the method may include determining a value of a first syntax element. The value of the first syntax element is encoded, and the encoded bits are written into the bitstream.
the value of the first syntax element is a fourth value.
the current mesh when the current mesh does not adopt the subdivision mode, it is determined that the value of the first syntax element is a first value.
the current mesh adopts the midpoint subdivision mode it is determined that the value of the first syntax element is a second value.
the current mesh adopts the recursive subdivision mode it is determined that the value of the first syntax element is a third value.
the value of the first syntax element is a fourth value.
the first value, the second value, the third value, and the fourth value are different from each other.
the first value may be set to 0, the second value may be set to 1, the third value may be set to 2, and the fourth value may be set to 3. That is, according to the different values of the first syntax element, it is possible to determine whether the current mesh adopts a non-linear subdivision method.
the value of the first syntax element if the value of the first syntax element is 0, it may be determined that the current mesh does not adopt any subdivision, such as midpoint subdivision, recursive subdivision, or nonlinear subdivision. If the value of the first syntax element is 1, it may be determined that the current mesh adopts midpoint subdivision. If the value of the first syntax element is 2, it may be determined that the current mesh is subdivided recursively. If the value of the first syntax element is 3, it may be determined that the current mesh adopts a non-linear subdivision.
the reconstructed mesh of the current mesh may be obtained by recursive reconstruction of at least one LOD. Specifically, after obtaining the reconstructed mesh of the current LOD in the current mesh, the reconstructed mesh of the current LOD may be subdivided to determine the subdivided mesh of the current LOD. Then, a reconstructed mesh of the next LOD of the current LOD may be recursively obtained according to the subdivided mesh, and preset displacement coefficients or the displacement coefficients in the bitstream.
a reconstructed mesh of the first LOD of the current mesh is first determined.
the method may include: determining a base mesh of a current mesh; encoding and decoding the base mesh to determine a reconstructed base mesh; subdividing the reconstructed base mesh to determine the initial subdivided mesh; determining displacement coefficients of the initial subdivided mesh; and determining the reconstructed mesh of the first LOD in the current mesh is determined according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh.
the base mesh may also be referred to as a "decimated mesh" .
determining the base mesh of the current mesh may include determining an original mesh of the current mesh; and obtaining the base mesh by downsampling the original mesh.
the current input mesh may be referred to as the original mesh.
the base mesh may be obtained by downsampling the input mesh, that is, a simplifying operation of the input mesh may be realized.
the method may also include encoding the base mesh and writing the resulting encoded bits into the bitstream.
the bitstream herein may refer to the base mesh bitstream.
the base mesh may be written into the base mesh bitstream by a mesh encoder (e.g. EdgeBreaker, etc. ) .
a mesh encoder e.g. EdgeBreaker, etc.
the method may include: performing a displacement calculation on the vertex coordinate information of the initial subdivided mesh and the vertex coordinate information of the original mesh to determine the displacement coefficients of the initial subdivided mesh.
the reconstructed mesh of the first LOD is subdivided to determine a subdivided mesh of the second LOD
the bitstream is decoded to determine displacement coefficients of the subdivided mesh of the seond LOD
a reconstructed mesh of a next LOD of the current mesh is determined according to the subdivided mesh of the second LOD and the displacement coefficients of the subdivided mesh of the second LOD
subdivision operation is continued, until the reconstructed mesh of the L-th LOD in the current mesh is determined.
the value of L is correlated with the number of subdivision iterations of the current mesh.
the displacement coefficient refers to the difference between the vertex coordinate information of the original mesh and the vertex coordinate information of the initial subdivided mesh.
the corresponding displacement operation is performed on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficients, to obtain a reconstructed mesh of the first LOD, that is, a reconstructed mesh of LOD1.
the reconstructed mesh is subdivided based on LOD1 to obtain the subdivided mesh on LOD1, and then the reconstructed mesh of LOD2 is determined according to the displacement coefficients and the subdivided mesh, and so on until the reconstruction is completed.
step 302 when the current LOD is the i-th LOD, the displacement coefficients of the first points of the current LOD are determined to be 0.
i is an integer greater than 2.
the first points include vertices of the reconstructed base mesh corresponding to the current mesh.
the displacement coefficients may include displacement components in one or more directions, and as shown in FIG. 3, the displacement coefficients include displacement components in three directions, namely, the normal direction, the tangent direction, and the bitangent direction.
the displacement coefficients of the first LOD are calculated relative to the subdivision vertices of the base mesh
the displacement coefficients of the subsequent LODs are calculated relative to the subdivision vertices of the reconstructed LOD of the previous LOD, so that the recursive subdivision of the displacement coefficients can be realized.
the displacement coefficients of the first points of the current LOD can be determined to be 0.
the first points of the current LOD include vertices of the reconstructed base mesh corresponding to the current mesh, that is, the first point may be understood as points coinciding with or neighboring the base point (e.g., PB_1, PB_2, PB_3) .
the displacement coefficients of the first points of the current LOD can be directly set to 0, and subsequent mesh reconstruction can be carried out using the displacement coefficients.
a reconstructed mesh of the next LOD in the current mesh is determined based on the subdivided mesh of the current LOD and the displacement coefficients of the first points of the current LOD.
the reconstructed mesh of the next LOD in the current mesh is determined, which may include: a corresponding displacement operation is performed on the vertex coordinate information of the subdivided mesh of the current LOD according to the displacement coefficients of the first point, , to obtain a reconstructed mesh of the next LOD in the current mesh.
the displacement coefficients are recursively applied to the previously reconstructed LOD, in particular, a corresponding displacement coefficient is added to each vertex of the subdivided mesh of the current LOD in order, so that the reconstructed mesh of the next LOD can be obtained.
the displacement coefficients of the first points of the current LOD are set to 0, that is, the displacements of the points (first points) overlapping or neighboring the base point are set to 0.
the displacement calculation for the first point is actually carried out on the subdivided mesh of the current LOD and 0.
the method may further include: after determining the reconstructed mesh of the next LOD in the current mesh, taking the reconstructed mesh of the next LOD as the reconstructed mesh of the current LOD, returning to performing the step of subdividing the reconstructed mesh of the current LOD, and determining the subdivided mesh of the current LOD until the reconstructed mesh of the L-th LOD in the current mesh is determined.
the value of L is correlated with the number of subdivision iterations of the current mesh.
the number of subdivision iterations of the current mesh may be indicated by a second syntax element in the bitstream.
the method may include decoding a bitstream to determine a value of a second syntax element; and determining the number of subdivision iterations of the current mesh according to the value of the second syntax element.
the second syntax element may be represented by asps_vmc_ext_subdivision_iteration_count. That is, the number of subdivision iterations for the current mesh may be a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.
L represents the number of LODs corresponding to the current mesh, and the value of L may be determined by the number of subdivision iterations of the current mesh, that is, the value of L is equal to the value indicated by asps_vmc_ext_subdivision_iteration_count.
FIG. 26 is a second flowchart of an encoding method provided in embodiments of the present disclosure. As shown in FIG. 26, after step 301, the method may include the following steps.
step 304 when the current LOD is the i-th LOD, the displacement coefficients of the second points of the current LOD are determined, and the displacement coefficients of the second points are written into the bitstream.
the second points do not include the vertices of the reconstructed base mesh corresponding to the current mesh.
the second points are points of the reconstructed base mesh corresponding to the current mesh except the first points.
the displacement coefficients of the first points of the current LOD may be determined to be 0, and the displacement coefficients of the second points may be obtained by the corresponding position displacement operation, and then the displacement coefficients of the second points may be written into the bitstream.
the bitstream may be a displacement bitstream.
specific coding methods for the displacement bitstream such as video coding, entropy coding and so on. That is, in embodiments of the present disclosure, the displacement information may be written into the displacement bitstream using a video encoder, or the displacement information may be written into the displacement bitstream using an entropy encoder, without any limitation herein.
determining the displacement coefficients of the second points of the current LOD may include calculating the displacement of the vertex coordinate information of the subdivided mesh of the current LOD and the vertex coordinate information of the original mesh to determine the displacement coefficients of the second points of the current LOD.
the method may further include: preprocessing the displacement coefficients of the second points of the current LOD to determine the quantized displacement coefficients of the second points of the current LOD.
the quantized displacement coefficients of the second point of the current LOD are encoded, and the obtained encoded bits are written into the bitstream.
the second points of the current LOD do not include vertices of the reconstructed base mesh corresponding to the current mesh.
the second points are points of the reconstructed base mesh corresponding to the current mesh except the first points. That is, the second points may be understood as points that are not close to the base point (e.g., PB_1, PB_2, PB_3) .
the displacement coefficients of the first points of the current LOD may be set to 0, and at the same time, the displacement coefficients of the second points of the current LOD may be determined by calculation, and the displacement coefficients of the second points may be encoded.
a reconstructed mesh of the next LOD in the current mesh is determined according to the subdivided mesh of the current LOD and the displacement coefficients of the second points of the current LOD.
the reconstructed mesh of the next LOD in the current mesh is determined according to the subdivided mesh of the current LOD and the displacement coefficients of the second points of the current LOD, specifically, a corresponding displacement operation is performed on the vertex coordinate information of the subdivided mesh of the current LOD according to the displacement coefficients of the second points to obtain a reconstructed mesh of the next LOD in the current mesh.
the displacement coefficients are recursively applied to the previously reconstructed LOD, in particular, the displacement coefficients are added to respective vertices of the subdivided mesh of the current LOD in order, so that the reconstructed mesh of the next LOD can be obtained.
FIG. 27 is a third flowchart of an encoding method provided in embodiments of the present disclosure. As shown in FIG. 27, after step 301, the method may include the following steps.
step 306 when the current LOD is a second LOD, the displacement coefficients of the current LOD are determined and the displacement coefficients of the current LOD are written into the bitstream.
the displacement coefficients of the current LOD may be determined by a displacement operation, and the displacement coefficients of the current LOD may be written into a bitstream.
the bitstream may be a displacement bitstream.
the displacement information may be written into the displacement bitstream using a video encoder, or the displacement information may be written into the displacement bitstream using an entropy encoder, without any limitation herein.
determining the displacement coefficients of the current LOD may include: calculating the displacements of the vertex coordinate information of the subdivided mesh of the current LOD and the vertex coordinate information of the original mesh to determine the displacement coefficients of the current LOD.
the method may further include: preprocessing the displacement coefficients of the current LOD to determine the quantized displacement coefficients of the current LOD; and encoding the quantized displacement coefficients of the current LOD and writing the encoded bits into a bitstream.
a reconstructed mesh of the next LOD in the current mesh is determined based on the subdivided mesh of the current LOD and the displacement coefficients of the current LOD.
the reconstructed mesh of the next LOD in the current mesh is determined according to the subdivided mesh of the current LOD and the displacement coefficients of the current LOD, specifically, a corresponding displacement operation is performed on the vertex coordinate information of the subdivided mesh of the current LOD according to the displacement coefficients, to obtain a reconstructed mesh of the next LOD in the current mesh.
the displacement coefficients are recursively applied to the previously reconstructed LOD, specifically, the decoded displacement coefficients are added to respective vertices of the subdivided mesh of the current LOD in order, so that the reconstructed mesh of the next LOD can be obtained.
the method may further include: after determining the reconstructed mesh of the next LOD in the current mesh, taking the reconstructed mesh of the next LOD as the reconstructed mesh of the current LOD, returning to performing the step of subdividing the reconstructed mesh of the current LOD, and determining the subdivided mesh of the current LOD until the reconstructed mesh of the L-th LOD in the current mesh is determined.
the value of L is correlated with the number of subdivision iterations of the current mesh.
the number of subdivision iterations of the current mesh may be indicated by a second syntax element in the bitstream.
the method may include decoding a bitstream to determine a value of a second syntax element. According to the value of the second syntax element, the number of subdivision iterations of the current mesh is determined.
the second syntax element may be represented by asps_vmc_ext_subdivision_iteration_count. That is, the number of subdivision iterations for the current mesh may be a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.
L represents the number of LODs corresponding to the current mesh, and the value of L may be determined by the number of subdivision iterations of the current mesh, that is, the value of L is equal to the value indicated by asps_vmc_ext_subdivision_iteration_count.
FIG. 28 is a fourth flowchart of an encoding method provided in an embodiment of the present disclosure. As shown in FIG. 28, the method may include the following steps.
step 308 when the current mesh adopts a recursive subdivision mode, the value of the first syntax element is determined, and the first syntax element is written into the bitstream, a reconstructed mesh of the current LOD in the current mesh is determined, and the reconstructed mesh of the current LOD is subdivided to determine a subdivided mesh of the current LOD.
the current mesh adopts a recursive subdivision mode by setting a first syntax element.
the first syntax element herein may be represented by asps_vdmc_ext_subdivision_method for indicating the subdivision method of the mesh. For example, as shown in Table 2, if the value of the first syntax element is 0, it means that the current mesh does not adopt the subdivision mode. If the value of the first syntax element is 1, it means that the current mesh adopts the midpoint subdivision method.
the recursive subdivision may create the same subdivision as that in the related arts, but the displacement neighboring the base point (e.g., PB_1, PB_2, and PB_3) is inferred to be equal to 0.
the displacement is calculated based on the subdivision vertex in the previously reconstructed LOD, so that the displacement can be recursively used for the previously reconstructed LOD. Therefore, the adaptive subdivision method here is called "Recursive subdivision method" . Taking midpoint subdivision as an example, the recursive subdivision method may be called midpoint recursive subdivision method.
a new asps_vdmc_ext_subdivision_method equal to 2 and 3 are added, the new asps_vdmc_ext_subdivision_method would create same subdivision as that in the related arts.
the displacements neighboring base point e.g., PB_1, PB_2, and PB_3 are inferred to be equal to zero, thereby saving the coded bits of the displacements.
the first syntax element indicates whether the current mesh adopts a recursive subdivision mode.
the method may include: determining a value of a first syntax element; and encoding the value of the first syntax element, and writing encoded bits into the bitstream.
the current mesh adopts a recursive subdivision method, it may be determined that the value of the first syntax element is a third value.
the current mesh when the current mesh does not adopt the subdivision mode, it is determined that the value of the first syntax element is a first value; when the current mesh adopts the midpoint subdivision mode, it is determined that the value of the first syntax element is a second value; when the current mesh adopts the recursive subdivision mode, it is determined that the value of the first syntax element is a third value; and when the current mesh adopts a non-linear subdivision mode, it is determined that the value of the first syntax element is a fourth value.
the first value, the second value, the third value, and the fourth value are different from each other.
the first value may be set to 0, the second value may be set to 1, the third value may be set to 2, and the fourth value may be set to 3. That is, according to the different values of the first syntax element, it is possible to determine whether the current mesh adopts a non-linear subdivision method.
the value of the first syntax element if the value of the first syntax element is 0, it may be determined that the current mesh does not adopt any subdivision, such as midpoint subdivision, recursive subdivision, nonlinear subdivision, and the like, if the value of the first syntax element is 1, it may be determined that the current mesh adopts midpoint subdivision; if the value of the first syntax element is 2, it may be determined that the current mesh is subdivided recursively; if the value of the first syntax element is 3, it may be determined that the current mesh adopts a non-linear subdivision.
the reconstructed mesh of the current mesh may be obtained by recursive reconstruction of at least one LOD. Specifically, after obtaining the reconstructed mesh of the current LOD in the current mesh, the reconstructed mesh of the current LOD may be subdivided to determine the subdivided mesh of the current LOD. Then, a reconstructed mesh of the next LOD of the current LOD may be recursively obtained according to the subdivided mesh, and preset displacement coefficients or displacement coefficients in the bitstream.
a reconstructed mesh of the first LOD of the current mesh is first determined.
the method may include: determining a base mesh of a current mesh; encoding and decoding the base mesh to determine a reconstructed base mesh; subdividing the reconstructed base mesh to determine the initial subdivided mesh; determining displacement coefficients of the initial subdivided mesh; and determining the reconstructed mesh of the first LOD in the current mesh is determined according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh.
the base mesh may also be referred to as a "decimated mesh" .
determining the base mesh of the current mesh may include determining an original mesh of the current mesh; and obtaining the base mesh by downsampling the original mesh.
the current input mesh may be referred to as the original mesh.
the base mesh may be obtained by downsampling the input mesh, that is, a simplifying operation of the input mesh may be realized.
the method may also include encoding the base mesh and writing the resulting encoded bits into the bitstream.
the bitstream herein may refer to the base mesh bitstream.
the base mesh may be written into the base mesh bitstream by a mesh encoder (e.g. EdgeBreaker, etc. ) .
a mesh encoder e.g. EdgeBreaker, etc.
the method may include: performing a displacement calculation on the vertex coordinate information of the initial subdivided mesh and the vertex coordinate information of the original mesh to determine the displacement coefficients of the initial subdivided mesh.
the displacement coefficient refers to the difference between the vertex coordinate information of the original mesh and the vertex coordinate information of the initial subdivided mesh.
the corresponding displacement operation is performed on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficient, and the reconstructed mesh of the first LOD, that is, the reconstructed mesh of LOD1, can be obtained.
the reconstructed mesh is subdivided based on LOD1 to obtain a subdivided mesh on LOD1, and the reconstructed mesh of LOD2 is determined according to the subdivided mesh and corresponding displacement coefficients, and so on, until the reconstruction is completed.
step 309 the displacement coefficients of the current LOD are determined, and the displacement coefficients of the current LOD are written into a bitstream.
the displacement coefficients may include displacement components in one or more directions, and as shown in FIG. 3, the displacement coefficients include displacement components in three directions, namely, the normal direction, the tangent direction, and the bitangent direction.
the displacement coefficients of the first LOD are calculated relative to the subdivision vertices of the base mesh
the displacement coefficients of the subsequent LODs are calculated relative to the subdivision vertices of the reconstructed LOD of the previous LOD, so that the recursive subdivision based on the displacement coefficients can be realized.
the method for determining the displacement coefficients of the current LOD involved in any one of steps 304, 306, and 309 may specifically include: calculating the displacements of the vertex coordinate information of the subdivided mesh of the current LOD and the vertex coordinate information of the original mesh, and determining the displacement coefficients of the current LOD.
the method may further include: preprocessing the displacement coefficients of the current LOD to determine the quantized displacement coefficients of the current LOD; and encoding the quantized displacement coefficients of the current LOD and writing the encoded bits into the bitstream.
a geometry displacement vector may be calculated for each vertex of the subdivided mesh so that the shape of the subdivided mesh is as close as possible to the shape of the original mesh.
These geometry displacement vectors are the displacement coefficients herein. Specifically, there is a difference in the geometry information between the vertices of the subdivided mesh and the vertices of the original mesh, which is the displacement coefficient.
the quantized displacement coefficients of the current LOD are encoded into a bitstream.
the bitstream may be a displacement bitstream.
specific coding methods for the displacement bitstream such as video coding, entropy coding and so on. That is, in an embodiment of the present disclosure, the displacement information may be written into the displacement bitstream using video encoder, or the displacement information may be written into the displacement bitstream using an entropy encoder, without any limitation herein.
the displacement coefficients of the current LOD may be preprocessed.
the preprocessing can include wavelet transform, quantization and other processing.
preprocessing the displacement coefficients of the current LOD to determine the quantized displacement coefficients of the current LOD may include: performing a wavelet transform on the displacement coefficients of the current LOD to determine the wavelet transform coefficients of the current LOD; and quantizing the wavelet transform coefficients of the current LOD to determine quantized displacement coefficients of the current LOD.
the wavelet transform converts the displacement coefficients of the current LOD into signals in the wavelet domain.
the wavelet transform coefficients are calculated in floating-point format.
Quantization is to convert a floating-point number into a fixed-point number of preset accuracy.
the preset accuracy may be an accuracy indicated in an encoded bitstream at the level of patch, picture, or sequence.
quantizing the wavelet transform coefficients of the current LOD to determine the quantized displacement coefficients of the current LOD may include: determining quantization parameters of the current LOD; and quantizing the wavelet transform coefficients of the current LOD according to the quantization parameters of the current LOD, to determine the quantized displacement coefficients of the current LOD.
the Quantizer Parameter reflects the spatial detail compression situation.
the smaller the value the finer the quantization, the higher the image quality, and the longer the bitstream. If the QP is small, most of the details will be retained. If QP increases, some details are lost, bit rate will be reduced, but image distortion is enhanced and quality is degenerated.
QP is the serial number of a quantization step Qstep. When the value of QP is 0, it means that the quantization is the finest. On the contrary, when the value of QP is 51, it means that the quantization is rougher.
determining quantization parameters of the current LOD may include: determining a plurality of candidate quantization parameters for the current LOD; calculating the quantization and coding cost of the wavelet transform coefficients of the current LOD based on the plurality of candidate quantization parameters; and determining the cost results of each of the plurality of candidate quantization parameters; determining the minimum cost result the respective cost results of the plurality of candidate quantization parameters; and determining the candidate quantization parameter corresponding to the minimum cost result as the quantization parameter of the current LOD.
the cost calculation herein may include at least one of: Rate-Distortion Optimization (RDO) Mean Square Error (MSE) , Sum of Squared Difference (SSD) , Sum of Absolute Difference (SAD) , Sum of Absolute Transformed Difference (SATD) , Peak Signal to Noise Ratio (PSNR) , and the like.
RDO Rate-Distortion Optimization
MSE Mean Square Error
SSD Sum of Squared Difference
SAD Sum of Absolute Difference
SATD Sum of Absolute Transformed Difference
PSNR Peak Signal to Noise Ratio
quantization parameters may be represented by vmc_transform_lifting_quantization_parameters [ltpIndex] [i] [j] .
the wavelet transform coefficients may be converted into fixed-point representations according to the accuracy indicated in the coded bitstream at patch level, picture level, or sequence level according to the quantization parameter vmc_transform_lifting_quantification_parameters [ltpIndex] [i] [j] of LoD j and the corresponding coordinates i in the bitstream.
i represents component information (x, y, z for canonical coordinate systems, n, t, bt for local coordinate systems) of displacement coefficients;
j denotes LOD;
ltpIndex represents the applied level;
0 represents the sequence level, 1 represents the image level, and 2 represents the patch level.
the method may include encoding the quantization parameters of the current LOD and writing the resulting encoded bits into a bitstream.
the method may include: determining the quantization parameter of the previous LOD of the current LOD; determining the quantization parameter increment of the current LOD according to the quantization parameter of the previous LOD and the quantization parameter of the current LOD; and encoding the quantization parameter increment of the current LOD, and writing the encoded bits into a bitstream.
the encoder side may directly write the quantization parameters into the bitstream, so that the decoder side may obtain the corresponding quantization parameters by decoding.
the encoder side may write the quantization parameter increment into the bitstream, and then the decoder side may obtain the corresponding quantization parameter according to the decoded quantization parameter increment and the quantization parameter of the previous LOD.
the quantization parameter increment of the current LOD may be determined by subtracting the quantization parameter of the current LOD from the quantization parameter of the previous LOD.
the quantization parameter of the current LOD may be represented by QP (i)
the quantization parameter of the previous LOD may be represented by QP (i-1)
the quantization parameter increment may be represented by ⁇ QP
⁇ QP QP (i) -QP (i-1)
the corresponding quantization parameter QP (1) may be directly written into the bitstream.
a reference quantization parameter may be set at the encoder side and the decoder side, and then the quantization parameter increment between the quantization parameter of each LOD and the reference quantization parameter is written into the bitstream.
the decoder side may determine the quantization parameter of the current LOD according to the quantization parameter increment of the current LOD and the reference quantization parameter, so that the coded bits of the quantization parameter in the bitstream can also be saved.
the method may further include: determining quantized displacement coefficients of at least one LOD in the current mesh, wherein the at least one LOD includes the current LOD; packing quantized displacement coefficients of at least one LOD into a two-dimensional image according to a preset packing mode; encoding the two-dimensional image, and writing the encoded bits into a bitstream.
the preset packing mode includes a forward packing mode or an inverse packing mode.
the determination of the preset packing mode may be that the decoder side and the encoder side set the same packing mode, or may be indicated according to a third syntax element in the bitstream.
the method may further include: determining a value of a third syntax element according to a preset packing mode; and encoding the value of the third syntax element, and writing the encoded bits into a bitstream.
the third syntax element may be represented by dmsps_packing_order. That is, the preset packing mode may be the packing mode indicated by dmsps_packing_order in the bitstream.
the decoder side decodes the displacements by a corresponding video decoder, and recovers the displacements from the two-dimensional image according to a corresponding order according to the preset packing mode. Then, inverse quantization is performed on the displacements, inverse wavelet transform and other operations are performed to restore the same displacement coefficients as the encoder side. Otherwise, if the encoder side uses entropy coding for the displacement coefficients, the decoder side may directly decode the displacement coefficients using entropy decoding, and then perform subsequent operations such as inverse quantization and inverse wavelet transform to obtain the displacement coefficients of the current LOD.
FIG. 29A is a schematic diagram of a packing mode of displacement coefficients in a two-dimensional image provided by an embodiment of the present disclosure
FIG. 29B is a schematic diagram of another packing mode of displacement coefficients in a two-dimensional image provided by an embodiment of the present disclosure
FIG. 29A is an exemplary forward packing mode, or "continuous-packing"
FIG. 29B is an exemplary inverse packing.
the quantized displacement coefficients are scanned along a three-dimensional spatial scanning pattern (e.g., Morton, Hilbert, or along other space filling curve) within each LOD, forming three one-dimensional arrays per each component (see FIGS. 29A and 29B) .
the quantized displacement coefficients are converted into a two-dimensional image according to LOD and selected packing mode indicated by the syntax element dmsps_packing_order.
the unoccupied symbols in the Coding Tree Unit (CTU) may be padded using one of the padding methods (e.g., zero-padding) .
a reconstructed mesh of the next LOD in the current mesh is determined based on the subdivided mesh of the current LOD and the displacement coefficients of the current LOD.
the reconstructed mesh of the next LOD in the current mesh is determined according to the subdivided mesh of the current LOD and the displacement coefficients of the current LOD, specifically, the corresponding displacement operation is performed on the vertex coordinate information of the subdivided mesh of the current LOD according to the displacement coefficients, to obtain a reconstructed mesh of the next LOD in the current mesh.
the displacement coefficients are recursively applied to the previously reconstructed LOD. Specifically, the decoded displacement coefficient is added to each vertex of the subdivided mesh of the current LOD in order, so that the reconstructed mesh of the next LOD can be obtained.
the method may further include: after determining the reconstructed mesh of the next LOD in the current mesh, taking the reconstructed mesh of the next LOD as the reconstructed mesh of the current LOD, returning to performing the step of subdividing the reconstructed mesh of the current LOD, and determining the subdivided mesh of the current LOD until the reconstructed mesh of the L-th LOD in the current mesh is determined.
the value of L is correlated with the number of subdivision iterations of the current mesh.
the number of subdivision iterations of the current mesh may be indicated by a second syntax element in the bitstream.
the method may include: determining the number of subdivision iterations for a current mesh; determining the value of the second syntax element according to the number of subdivision iterations of the current mesh; and encoding the value of the second syntax element, and writing the encoded bits into a bitstream.
the second syntax element may be represented by asps_vmc_ext_subdivision_iteration_count. That is, the number of subdivision iterations for the current mesh may be a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.
L represents the number of LODs corresponding to the current mesh, and the value of L may be determined by the number of subdivision iterations of the current mesh, that is, the value of L is equal to the value indicated by asps_vmc_ext_subdivision_iteration_count.
the base mesh is subdivided first, and an initial subdivided mesh may be obtained.
the corresponding displacement operation is carried out on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficients, and the reconstructed mesh of the first LOD, that is, the reconstructed mesh of LOD1, may be obtained.
the reconstructed mesh is subdivided based on LOD1 to obtain a subdivided mesh on LOD1.
a reconstructed mesh of LOD2 may be determined according to the subdivided mesh on LOD1 and the displacement coefficients of LOD1, and so on, until a reconstructed mesh of LODL is obtained, indicating that the current mesh reconstruction is completed.
a black solid line (composed of vertices PB_1, PB_2, and PB_3) represents a base mesh
a black dashed line represents a subdivided mesh
the first thick solid line is LOD1 onto which the displacements are applied
the second thick solid line is LOD2 onto which the displacements are applied.
the detailed flow of recursive subdivision is shown in FIG. 21, and the detailed flow may include the following steps.
step S1901 the original mesh is decimated to obtain the base mesh.
step S1902 the base mesh is quantized.
step S1903 the quantized base mesh is encoded.
step S1904 a bitstream is parsed to decode the base mesh from the bitstream to obtain a reconstructed base mesh.
step S1906 the displacement coefficients are calculated from the subdivided mesh and the original mesh.
step S1907 the interim mesh is updated according to the displacement coefficients and the subdivided mesh.
step S1908 the displacement coefficients are saved to the memory.
step S1909 it is determined whether n is less than L-1.
the base mesh is subdivided first, and an initial subdivided mesh may be obtained.
a corresponding displacement operation is carried out on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficients, and the reconstructed mesh of the first LOD, that is, the reconstructed mesh of LOD1, may be obtained.
the reconstructed mesh is subdivided based on LOD1 to obtain a subdivided mesh on LOD1.
a reconstructed mesh of LOD2 may be determined according to the subdivided mesh on LOD1 and the displacement coefficients of LOD1.
the preset value may be for example 2
the displacement coefficients of neighboring vertices (first points) of the base mesh in LOD i may be set to 0, while the displacement coefficients of other points (second points) may still be decoded.
the next LOD of reconstructed meshes may be determined, and so on, until the reconstructed mesh of LOD L is obtained, indicating that the current mesh reconstruction is completed.
a black solid line (composed of vertices PB_1, PB_2, and PB_3) represents a base mesh
a black dashed line represents a subdivided mesh
a first thick solid line is LOD1 onto which the displacements are applied
a second thick solid line is LOD2 onto which the displacements are applied.
the preset value may be for example 2, the embodiments are not limited thereto. In various embodiments, the preset value may be 2, 3, 4, ..., or any other integer value.
the detailed flow of non-linear subdivision is shown in FIG. 23, and the detailed flow may include the following steps.
step S1901 the original mesh is decimated to obtain the base mesh.
step S1902 the base mesh is quantized.
step S1903 the quantized base mesh is encoded.
step S1904 the bitstream is parsed to decode the base mesh from the bitstream to obtain a reconstructed base mesh.
step S1906 displacement coefficients are calculated from the subdivided mesh and the original mesh.
step S1911 the displacement coefficients of adjacent foundation mesh vertices are deleted in order to remove redundancy.
step S1907 the interim mesh is updated according to the displacement coefficients and the subdivided mesh.
step S1908 the other displacement coefficients are saved to the memory.
the current mesh when the current mesh does not adopt recursive subdivision, the current mesh can use midpoint subdivision.
the method may include: determining a reconstructed base mesh when the current mesh adopts a midpoint subdivision mode; subdividing the reconstructed base mesh to determine a subdivided mesh of at least one LOD in the current mesh; determining displacement coefficients of at least one LOD in the current mesh; and determining a reconstructed mesh of at least one LOD in the current mesh according to the subdivided mesh of at least one LOD in the current mesh and the displacement coefficient of at least one LOD.
the number of at least one LOD is correlated with the number of subdivision iterations of the current mesh.
L denotes the number of LODs of at least one LOD, that is, the number of LODs corresponding to the current mesh, which may be equal to a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.
the displacement coefficients of each LOD are calculated by the displacement of the vertex coordinate information of the subdivided mesh and the vertex coordinate information of the original mesh, instead of being recursively obtained based on the previously reconstructed LOD.
the information to be encoded includes at least one of type of information including: a base mesh of the current mesh, displacement coefficients of at least one LOD of the current mesh, quantization parameters of at least one LOD of the current mesh, a quantization parameter increment of at least one LOD of the current mesh, a value of a first syntax element, a value of a second syntax element, and a value of a third syntax element.
An embodiment of the present disclosure provides an encoding method, in particular a recursive subdivision method of a dynamic mesh.
the displacement coefficient close to the vertex in the base layer can be removed from the displacement list in the coding stage, so that in the subsequent process of recursively applying the displacement coefficient to the previously reconstructed LOD, the redundancy problem of the displacement coefficient at the higher levels can be improved, the coded bits of the displacement coefficient can be saved, and the encoding and decoding efficiency can be improved.
displacement coefficients are applied to a previously reconstructed LOD. In some cases, displacement coefficients are often redundant at higher LODs.
the subdivision method shares functionality across all LODs and is defined by the two syntax elements asps_VDMC_ext_subdivision_method and asps_VDMC_ext_subdivision_iteration_count in the Atlas sequence parameter set VDMC extension RBSP syntax structure, as shown in Table 1 above.
Asps_vdmc_ext_subdivision_method indicates the subdivision method identifier of the mesh associated with the current atlas sequence parameter set. Table 2 above describes the list of supported subdivision methods and their relationship to asps_vdmc_ext_subdivision_method.
Asps_vdmc_ext_subdivision_iteration_count indicates the number of subdivision iterations for the mesh. When asps_vdmc_ext_subdivision_iteration_count is not present, the value of asps_vdmc_ext_subdivision_iteration_count is inferred to be equal to 0.
the proposed solution of the embodiments of the present disclosure introduces an efficient and flexible method and apparatus for efficient coding of the geometry component of the mesh representation of the volumetric content by removing redundant displacements in cases such as base mesh edge and inheriting value from previous LOD.
an embodiment of the present disclosure propose a new adaptive subdivision method whereas edges that are adjacent to the base points at higher LODs shall be always equal to 0.
One way of implementing this is by adding a new asps_vdmc_ext_subdivision_method equal to 2 and 3. That would create the same subdivision as that in the related arts.
the displacements neighboring base points e.g., PB_1, PB_2, and PB_3 is inferred to be equal to zero, as shown in FIGS. 20, 21, 22 and 23.
a method of efficiently encoding geometry displacement coefficients in a mesh by an encoder side includes a subdivision process with recursive subdivision updates, where the displacement coefficients are calculated relative to subdivision vertices in a previously reconstructed LOD.
Stage 1 Mesh segmentation is a step that creates segments or blocks of mesh content representing individual objects/regions of interest/volumetric tiles, semantic blocks, etc.
the number of LOD subdivisions is defined by asps_vmc_ext_subdivision_iteration_count.
Stage 2 Mesh decimation creates a base mesh, and the base mesh is coded with an undefined static mesh encoder.
the base mesh is decoded and recursively subdivided to the number of LOD.
Wavelet transform coefficients are converted to a fix-point representation with a precision indicated in the coded bitstream at either patch, picture, or sequence level (ltpIndex) depending on the quantization parameter vmc_transform_lifting_quantization_parameters [ltpIndex ] [i ] [j ] for a corresponding coordinate I and LOD j as signaled in the bitstream.
the quantized wavelet coefficients are scanned along a 3d space scanning pattern (e.g. Morton, Hilbert, or along other space filling curve) within each LoD, forming three 1-dimensional arrays per each component (FIGS. 29A and 29B) .
a 3d space scanning pattern e.g. Morton, Hilbert, or along other space filling curve
the coefficients are converted to a two-dimensional image according to LoD and selected packing order indicated by a flag dmsps_packing_order.
the unoccupied symbols in CTU are padded using one of the padding methods (e.g. zero-padding) .
the decoding process is the inverse of the encoding process and includes the following stages.
the base mesh is decoded from geometry bitstream and recursively subdivided to the LOD defined by the encoder.
Stage 2 A coded bitstream for geometry displacements is obtained and decoded with a codec responding to the dmsps_mesh_codec_id decoder.
Stage 4 The dequantized disposition wavelet coefficients are processed with an inverse wavelet transform.
the reconstructed mesh of the current LOD in the current mesh is subdivided, and the subdivided mesh of the current LOD is determined, the displacement coefficients of the first points (points coinciding with or neighboring the base point) of the current LOD are directly set to 0, that is, the displacements neighboring the base point (e.g., PB_1, PB_2, and PB_3) is inferred to be equal to 0, the reconstructed mesh of the next LOD in the current mesh is determined according to the subdivided mesh of the current LOD and the displacement coefficient of the current LOD.
the displacement coefficient close to the vertex in the base layer can be removed from the displacement list in the encoding stage, and the displacement coefficient close to the vertex in the base layer is inferred to be equal to 0 in the decoding stage. Therefore, in the subsequent process of recursively applying the displacement coefficient to the previously reconstructed LOD, the redundancy problem of the displacement coefficient at the higher LODs can be improved, the coded bits of the displacement coefficient can be saved, and the encoding and decoding efficiency can be improved.
FIG. 30 is a schematic diagram of a structure of an encoder provided in an embodiment of the present disclosure.
the encoder 300 may include a first determination unit 3001.
the first determination unit 3001 is configured to: determine a value of a first syntax element when the current mesh adopts a non-linear subdivision mode; write the first syntax element into a bitstream; determine a reconstructed mesh of a current LOD in the current mesh; subdivide the reconstructed mesh of the current LOD to determine a subdivided mesh of the current LOD; determine displacement coefficients of the first points of the current LOD as 0 when the current LOD is an i-th LOD, and determine a reconstructed mesh of a next LOD in the current mesh based on a subdivided mesh of the current LOD and displacement coefficients of the current LOD.
i is an integer greater than 2
the first points include vertices of a reconstructed base mesh corresponding to the current mesh.
the "unit" may be a partial circuit, a partial processor, a partial program or software, etc., and of course may be a module, or may be non-modular.
the components in this embodiment may be integrated in one processing unit, each unit may exist physically individually, or two or more units may be integrated in one unit.
the integrated unit can be realized in the form of hardware or in the form of software function modules.
the integrated units may be stored in a computer-readable storage medium if implemented in the form of software functional modules and not sold or used as stand-alone products.
the technical solution of the present embodiment may be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions to cause a computer device (may be a personal computer, server, network device, etc. ) or a processor (processor) to perform all or part of the steps of the method described in the present embodiment.
the aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a Read Only Memory (ROM) , a Random Access Memory (RAM) , a magnetic disk, or an optical disk.
embodiments of the present disclosure provide a computer-readable storage medium, applied to an encoder 300, that stores a computer program that, when executed by a first processor, implements the method described in any of the preceding embodiments.
FIG. 31 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present disclosure.
the encoder 300 may include a first communication interface 3101, a first memory 3102, and a first processor 3103.
the components are coupled together via a first bus system 3104.
the first bus system 3104 is used to implement connection communication between these components.
the first bus system 3104 includes a data bus, a power bus, a control bus, and a status signal bus.
the various buses are labeled in FIG. 31 as the first bus system 3104.
the first communication interface 3101 is configured to receive and transmit signals in the process of transmitting and receiving information with other external network elements.
the first memory 3102 is configured to store computer programs capable of running on the first processor 3103.
the first processor 3103 is configured to: when running the computer program, perform operations of: when a current mesh adopts a nonlinear subdivision mode, determining a value of a first syntax element, writing the first syntax element into a bitstream, determining a reconstructed mesh of a current LOD in the current mesh, and subdividing the reconstructed mesh of the current LOD to determine a subdivided mesh of the current LOD; when the current LOD is an i-th LOD, determining displacement coefficients of first points of the current LOD as 0, wherein i is an integer greater than 2, and the first points include vertices of a reconstructed base mesh corresponding to the current mesh; and determining a reconstructed mesh of a next LOD in the current mesh according to the subdivided mesh of the current LOD and the displacement coefficients of the first points of the current LOD.
the first memory 3102 in an embodiment of the present disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.
the non-volatile memory may be a Read-Only Memory (ROM) , a Programmable ROM (PROM) , an Erasable PROM (EPROM) , an Electrically Erasable EPROM (EEPROM) , or a Flash memory.
the volatile memory may be a Random Access Memory (RAM) , which serves as an external cache.
RAM Random Memory Bus RAM
SRAM Static RAM
DRAM Dynamic RAM
SDRAM Synchronous DRAM
DDRSDRAM Double Data Rate Synchronous Dynamic RAM
ESDRAM Enhanced Synchronous Dynamic RAM
SLDRAM Synchlink DRAM
DRRAM Direct Memory Bus RAM
the first memory 3102 of the systems and methods described in the present disclosure is intended to include, but is not limited to, these and any other suitable types of memory.
the first processor 3103 may be an integrated circuit chip having signal processing capability. In implementation, various steps of the method described above may be completed by integrated logic circuits of hardware in the first processor 3103 or by instructions in the form of software.
the first processor 3103 may be a general purpose processor, a Digital Signal Processor (DSP) , an Application Specific Integrated Circuit (ASIC) , an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.
DSP Digital Signal Processor
ASIC Application Specific Integrated Circuit
FPGA off-the-shelf programmable gate array
the disclosed methods, steps, and logical block diagrams in an embodiment of the present disclosure, may be implemented or executed.
the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
the steps of the method disclosed in combination with the embodiments of the present disclosure can be directly reflected as the completion of execution by a hardware decoding processor, or the completion of execution by combining hardware and software modules in the decoding processor.
Software modules can be located in memory media mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, etc.
the storage medium is located in the first memory 3102, and the first processor 3103 reads the information in the first memory 3102, and completes the steps of the above method in combination with its hardware.
the processing unit may be implemented in one or more Application Specific Integrated Circuits (ASICs) , Digital Signal Processors (DSPDs) , Digital Signal Processing Devices (DSPDs) , Programmable Logic Devices (PLDs) , Field-Programmable Gate Arrays (FPGAs) , general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing functions described in the present disclosure, or combinations thereof.
ASICs Application Specific Integrated Circuits
DSPDs Digital Signal Processors
DSPDs Digital Signal Processing Devices
PLDs Programmable Logic Devices
FPGAs Field-Programmable Gate Arrays
the techniques described in the present disclosure may be implemented by modules (e.g., processes, functions, etc. ) that perform the functions described in the present disclosure.
Software code may be stored in memory and executed by the processor.
the memory may be implemented in the processor or outside the processor.
the first processor 3103 is also configured to execute the method described in any of the preceding embodiments when running the computer program.
the present embodiment provides an encoder.
the displacement coefficients of the first points are set to 0, that is, the displacements neighboring the base point (e.g., PB_1, PB_2, and PB_3) are inferred to be equal to 0, and are recursively applied to a previously reconstructed LOD, so that the redundancy problem of the displacement coefficients at higher LODs can be improved, the coded bits of the displacement coefficients can be saved, and the encoding and decoding efficiency can be improved.
FIG. 32 is a schematic diagram of a component structure of a decoder provided in an embodiment of the present disclosure.
the decoder 320 may include a second determination unit 3201.
the second determining unit 3201 is configured to: decode the bitstream and determine a value of a first syntax element; determine a reconstructed mesh of a current LOD in the current mesh when the first syntax element indicates that the current mesh adopts a non-linear subdivision mode; subdivide the reconstructed mesh of the current LOD to determine a subdivided mesh of the current LOD; determine displacement coefficients of the first points of the current LOD as 0 when the current LOD is an i-th LOD; ; determine a reconstructed mesh of a next LOD in the current mesh based on the subdivided mesh of the current LOD and the displacement coefficients of the first points of the current LOD.
i is an integer greater than 2
the first points include vertices of a reconstructed base mesh corresponding to the current mesh.
the "unit" may be a partial circuit, a partial processor, a partial program or software, etc., and of course may be a module, or may be non-modular.
the components in this embodiment may be integrated in one processing unit, various units may exist physically individually, or two or more units may be integrated in one unit.
the integrated unit can be realized in the form of hardware or in the form of software function modules.
the integrated units may be stored in a computer-readable storage medium if implemented in the form of software functional modules and not sold or used as stand-alone products. Based on this understanding, the present embodiment provides a computer-readable storage medium, applied to a decoder 320, that stores a computer program that, when executed by a second processor, implements the method described in any of the preceding embodiments.
FIG. 33 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present disclosure.
the decoder 320 may include a second communication interface 3301, a second memory 3302, and a second processor 3303.
the components are coupled together via a second bus system 3304.
the second bus system 3304 is used to implement connection communication between these components.
the second bus system 3304 includes a power bus, a control bus, and a status signal bus in addition to a data bus.
the various buses are designated in FIG. 33 as the second bus system 3304.
the second communication interface 3301 is configured to receive and transmit signals in the process of transmitting and receiving information with other external network elements.
the second processor 3303 is configured to perform operations of when running the computer program: decoding a bitstream to determine a value of a first syntax element; when the value of first syntax element indicates that a current mesh adopts a non-linear subdivision mode, determining a reconstructed mesh of a current LOD in the current mesh, and subdividing the reconstructed mesh of the current LOD to determine a subdivided mesh of the current LOD; determining displacement coefficients of first points of the current LOD as 0 when the current LOD is an i-th LOD, wherein i is an integer greater than 2, the first points include vertices of a reconstructed base mesh corresponding to the current mesh; and determining a reconstructed mesh of a next LOD in the current mesh according to the subdivided mesh of the current LOD and the displacement coefficients of the first points of the current LOD.
the second processor 3303 is further configured to execute the method described in any of the preceding embodiments when running the computer program.
the present embodiment provides a decoder.
the displacement coefficients of first points are set to 0 when it is determined that the current mesh adopts a non-linear subdivision method, that is, the displacements neighboring the base point (e.g. PB_1, PB_2, and PB_3) are inferred to be equal to 0, and are recursively applied to the previously reconstructed LOD, so that the redundancy problem of the displacement coefficients at higher LODs can be improved, the coded bits of the displacement coefficient can be saved, and the encoding and decoding efficiency can be improved.
the displacements neighboring the base point e.g. PB_1, PB_2, and PB_3
FIG. 34 is a schematic diagram of an integral structure of a codec system provided in an embodiment of the present disclosure.
the codec system 340 may include an encoder 3401 and a decoder 3402.
encoder 3401 may be an encoder of any of the preceding embodiments
decoder 3402 may be a decoder of any of the preceding embodiments.
the terms “include” , “include” or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device including a set of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent in such a process, method, article or device.
an element qualified by the phrase “includes a... " does not exclude the existence of another identical element in the process, method, article or device in which it is included.
embodiments of the present disclosure also provide a computer program product including a computer program or instruction.
the computer program product may be applied to the terminal device in an embodiment of the present disclosure, and the computer program or instruction causes the computer to execute the corresponding flow implemented by the terminal device in the various methods of the embodiment of the present disclosure, which will not be repeated here for the sake of brevity.
embodiments of the present disclosure also provide a computer program.
the computer program can be applied to the terminal device in an embodiment of the present disclosure, and when the computer program is run on the computer, the computer is caused to execute the corresponding flow implemented by the terminal device in the various methods of the embodiment of the present disclosure. For the sake of brevity, it will not be repeated here.
an encoding and decoding method a bitstream, an encoder, a decoder, a medium and a product.
the bitstream is decoded to determine a value of a first syntax element; when the first syntax element indicates that the current mesh adopts a nonlinear subdivision mode, reconstructed mesh of the current LOD in the current mesh is determined, the reconstructed mesh of the current LOD is subdivided to determine a subdivided mesh of the current LOD; the displacement coefficients of the first points of the current LOD are determined as 0 when the current LOD is the i-th LOD.
i is an integer greater than 2 and the first points include vertices of the reconstructed base mesh corresponding to the current mesh.
a reconstructed mesh of the next LOD in the current mesh is determined.
a value of a first syntax element is determined, the first syntax element is written into a bitstream, a reconstructed mesh of the current LOD in the current mesh is determined, and the reconstructed mesh of the current LOD is subdivided to determine a subdivided mesh of the current LOD.
the displacement coefficients of the first points of the current LOD are determined as 0 when the current LOD is the i-th LOD.
i is an integer greater than 2 and the first points include vertices of the reconstructed base mesh corresponding to the current mesh.
a reconstructed mesh of the next LOD in the current mesh is determined.
the displacement coefficient neighboring the vertex in the base layer can be removed from the displacement list in the encoding stage, and the displacement coefficient neighboring the vertex in the base layer is inferred to be equal to 0 in the decoding stage. Therefore, in the subsequent process of recursively applying the displacement coefficients to the previously reconstructed LOD, the redundancy problem of the displacement coefficients at the higher LODs can be improved, the coded bits of the displacement coefficient can be saved, and the encoding and decoding efficiency can be improved.

Landscapes

Engineering & Computer Science (AREA)
Multimedia (AREA)
Physics & Mathematics (AREA)
General Physics & Mathematics (AREA)
Theoretical Computer Science (AREA)
Signal Processing (AREA)
Compression Or Coding Systems Of Tv Signals (AREA)

PCT/CN2024/099684 2023-06-15 2024-06-17 Encoding method, decoding method, bitstream, encoder, decoder, medium and program product Ceased WO2024255912A1 (en)

Priority Applications (1)

Application Number	Priority Date	Filing Date	Title
CN202480027912.XA CN121058043A (zh)	2023-06-15	2024-06-17	编码方法、解码方法、码流、编码器、解码器、介质及程序产品

Applications Claiming Priority (2)

Application Number	Priority Date	Filing Date	Title
US202363521327P	2023-06-15	2023-06-15
US63/521,327		2023-06-15

Publications (1)

Publication Number	Publication Date
WO2024255912A1 true WO2024255912A1 (en)	2024-12-19

Family

ID=93851365

Family Applications (1)

Application Number	Title	Priority Date	Filing Date
PCT/CN2024/099684 Ceased WO2024255912A1 (en)	2023-06-15	2024-06-17	Encoding method, decoding method, bitstream, encoder, decoder, medium and program product

Country Status (2)

Country	Link
CN (1)	CN121058043A (zh)
WO (1)	WO2024255912A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
WO2008019262A2 (en) *	2006-08-03	2008-02-14	Qualcomm Incorporated	Mesh-based video compression with domain transformation
CN106408620A (zh) *	2016-09-08	2017-02-15	成都希盟泰克科技发展有限公司	基于压缩感知的三维网格模型数据处理方法
US20210090301A1 (en) *	2019-09-24	2021-03-25	Apple Inc.	Three-Dimensional Mesh Compression Using a Video Encoder
US20210287431A1 (en) *	2020-03-15	2021-09-16	Intel Corporation	Apparatus and method for displaced mesh compression
CN114331816A (zh) *	2021-12-27	2022-04-12	北京达佳互联信息技术有限公司	图像处理方法、装置、电子设备及存储介质

2024
- 2024-06-17 CN CN202480027912.XA patent/CN121058043A/zh active Pending
- 2024-06-17 WO PCT/CN2024/099684 patent/WO2024255912A1/en not_active Ceased

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
WO2008019262A2 (en) *	2006-08-03	2008-02-14	Qualcomm Incorporated	Mesh-based video compression with domain transformation
CN106408620A (zh) *	2016-09-08	2017-02-15	成都希盟泰克科技发展有限公司	基于压缩感知的三维网格模型数据处理方法
US20210090301A1 (en) *	2019-09-24	2021-03-25	Apple Inc.	Three-Dimensional Mesh Compression Using a Video Encoder
US20210287431A1 (en) *	2020-03-15	2021-09-16	Intel Corporation	Apparatus and method for displaced mesh compression
CN114331816A (zh) *	2021-12-27	2022-04-12	北京达佳互联信息技术有限公司	图像处理方法、装置、电子设备及存储介质

Also Published As

Publication number	Publication date
CN121058043A (zh)	2025-12-02

Legal Events

Date

Code

Title

Description

2025-01-29

121

Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24822857

Country of ref document: EP

Kind code of ref document: A1

2026-01-16

NENP

Non-entry into the national phase

Ref country code: DE

Publication	Publication Date	Title
US12217465B2 (en)	2025-02-04	Method and apparatus for point cloud coding
KR20180087348A (ko)	2018-08-01	점군을 압축하는 방법
CN120130073A (zh)	2025-06-10	网格压缩纹理坐标信令和解码
JP7537033B2 (ja)	2024-08-20	メッシュ展開のための方法、装置、およびコンピュータプログラム
WO2024163690A2 (en)	2024-08-08	Visual volumetric video-based coding method, encoder and decoder
WO2024074122A1 (en)	2024-04-11	Method, apparatus, and medium for point cloud coding
WO2024010919A1 (en)	2024-01-11	System and method for geometry point cloud coding
JP2024512921A (ja)	2024-03-21	メッシュ圧縮のためのパッチ時間的位置合わせの符号化
WO2024255912A1 (en)	2024-12-19	Encoding method, decoding method, bitstream, encoder, decoder, medium and program product
CN121359444A (zh)	2026-01-16	用于视频处理的方法、装置和介质
WO2024255475A1 (en)	2024-12-19	Coding and decoding methods, bitstream, encoder, decoder and storage medium
WO2024039703A1 (en)	2024-02-22	Dynamic mesh geometry refinement
WO2024148573A1 (zh)	2024-07-18	编解码方法、编码器、解码器以及存储介质
WO2025067513A1 (en)	2025-04-03	Encoding and decoding methods, encoder, decoder and storage medium
WO2025081769A1 (en)	2025-04-24	Coding and decoding methods, bitstream, encoder, decoder and storage medium
US20240282011A1 (en)	2024-08-22	Fix-point implementation of mesh codec
WO2024213067A1 (en)	2024-10-17	Decoding method, encoding method, bitstream, decoder, encoder and storage medium
WO2024074121A9 (en)	2025-05-22	Method, apparatus, and medium for point cloud coding
US20250234036A1 (en)	2025-07-17	V-dmc displacement wavelet coefficient inter prediction with fixed-point quantization
US20260017831A1 (en)	2026-01-15	System and method for geometry point cloud coding
CN121444131A (zh)	2026-01-30	用于视频处理的方法、装置和介质
WO2025151992A1 (zh)	2025-07-24	编解码方法、码流、编码器、解码器以及存储介质
WO2025000342A1 (zh)	2025-01-02	编解码方法、编码器、解码器以及存储介质
CN121986360A (zh)	2026-05-05	用于点云编解码的方法、装置和介质
WO2025152005A1 (zh)	2025-07-24	编解码方法及装置、编解码器、码流、设备、存储介质