WO2024255475A1

WO2024255475A1 - Coding and decoding methods, bitstream, encoder, decoder and storage medium

Info

Publication number: WO2024255475A1
Application number: PCT/CN2024/090846
Authority: WO
Inventors: Vladyslav ZAKHARCHENKO; Yue Yu; Haoping Yu
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2023-06-15
Filing date: 2024-04-30
Publication date: 2024-12-19
Anticipated expiration: 2025-12-15
Also published as: CN121079962A

Abstract

Disclosed are coding and decoding methods, a bitstream, an encoder, a decoder and a storage medium. The method includes: a bitstream is decoded to determine a value of a first syntax element; when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, the reconstructed mesh is subdivided, and a subdivided mesh of the current level is determined; displacement coefficients of the current level are determined by decoding the bitstream; and a reconstructed mesh of a next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level. In this way, the coded bits of the displacement coefficients can be saved, and the coding and decoding efficiency can be improved.

Description

CODING AND DECODING METHODS, BITSTREAM, ENCODER, DECODER AND STORAGE MEDIUM

CROSS-REFERENCE OF RELATED APPLICATION

This disclosure is based on and claims priority to U.S. Patent Application No. 63/521,328 filed on June 15, 2023, the disclosure of which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The disclosure relates to the technical field of video coding and decoding, and in particular to coding and decoding methods, a bitstream, an encoder, a decoder and a storage medium.

BACKGROUND

In a standard reference software of Dynamic Mesh Coding (DMC) provided by Moving Picture Experts Group (MPEG) , when encoding and decoding geometry information of a mesh, firstly, an original mesh is preprocessed to obtain a base mesh encoded using genetic coding methods (e.g., “edgebreaker” ) . Then, the base mesh is subdivided to obtain a subdivided mesh, and displacement coefficients are determined according to the difference between the subdivided mesh and the approximation of the original mesh. Finally, the displacement coefficients are packed into a two-dimensional (2D) image and encoded with lossless video coding methods such as HEVC.

However, in the process of encoding and decoding the geometry information of the mesh, there is redundancy in encoding the displacement coefficients due to incomplete consideration of the subdivision, which reduces the encoding and decoding efficiency.

SUMMARY

The disclosure provides coding and decoding methods, a bitstream, an encoder, a decoder and a storage medium, which may reduce coded bits of displacement coefficients and further improve coding and decoding efficiency.

The technical solutions of the disclosure may be implemented as follows.

In a first aspect, embodiments of the disclosure provide a decoding method applied to a decoder. The method includes the following operations.

A bitstream is decoded to determine a value of a first syntax element.

When the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, and the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level.

Displacement coefficients of the current level are determined by decoding the bitstream.

According to the subdivided mesh of the current level and the displacement coefficients of the current level, a reconstructed mesh of a next level in the current mesh is determined.

In a second aspect, the embodiments of the disclosure provide an encoding method applied to an encoder. The method includes the following operations.

When a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, the reconstructed mesh is subdivided, and a subdivided mesh of the current level is determined.

Displacement coefficients of the current level are determined.

In a third aspect, the embodiments of the disclosure provide a bitstream. The bitstream is generated by encoding according to information to be encoded. The information to be encoded includes at least one of: a base mesh of a current mesh, displacement coefficients of at least one level in the current mesh, a quantization parameter of at least one level in the current mesh, a quantization parameter increment of at least one level in the current mesh, a value of a first syntax element, a value of a second syntax element or a value of a third syntax element.

The value of the first syntax element indicates whether a recursive subdivision mode is applied to the current mesh, the value of the second syntax element indicates a number of subdivision iterations for the current mesh, and the value of the third syntax element indicates a preset packing mode.

In a fourth aspect, the embodiments of the disclosure provide an encoder. The encoder includes a first subdivision unit, a first determination unit and a first reconstruction unit.

The first subdivision unit is configured to determine, when a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, subdivide the reconstructed mesh, and determine a subdivided mesh of the current level.

The first determination unit configured to determine displacement coefficients of the current level.

The first reconstruction unit is configured to determine a reconstructed mesh of a next level in the current mesh according to the subdivided mesh of the current level and the displacement coefficients of the current level.

In a fifth aspect, the embodiments of the disclosure provide an encoder. The encoder includes a first processor and a first memory for storing a computer program executable by the first processor. The first processor, when executing the computer program, implements the method according to the second aspect.

In a sixth aspect, the embodiments of the disclosure provide a decoder. The decoder includes a decoding unit, a second subdivision unit and a second reconstruction unit.

The decoding unit is configured to decode a bitstream to determine a value of a first syntax element.

The second subdivision unit is configured to determine, when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, subdivide the reconstructed mesh, and determine a subdivided mesh of the current level.

The decoding unit is further configured to decode the bitstream to determine displacement coefficients of the current level.

The second reconstruction unit is configured to determine a reconstructed mesh of a next level in the current mesh according to the subdivided mesh of the current level and the displacement coefficients of the current level.

In a seventh aspect, the embodiments of the disclosure provide a decoder. The decoder includes a second processor and a second memory for storing a computer program executable by the second processor. The second processor, when executing the computer program, implements the method according to the first aspect.

In an eighth aspect, the embodiments of the disclosure provide a computer-readable storage medium having stored thereon a computer program. The computer program, when executed by at least one processor, implements the method as described in the first aspect or the method as described in the second aspect.

In a ninth aspect, the embodiments of the disclosure provide a computer program product including a computer program or instructions. The computer program or instructions, when executed by at least one processor, implement the method as described in the first aspect or the method as described in the second aspect.

The embodiments of the disclosure provide coding and decoding methods, a bitstream, an encoder, a decoder and a storage medium. At the encoding end, when a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, and the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level; displacement coefficients of the current level are determined; and a reconstructed mesh of a next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level. At the decoding end, a bitstream is decoded to determine a value of a first syntax element; when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, and the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level; displacement coefficients of the current level are determined by decoding the bitstream; and a reconstructed mesh of a next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level. In this way, by adding a first syntax element for indicating that recursive subdivision is applied to the current mesh, the displacements are applied recursively to the previously reconstructed level of details (LoD) when determining that recursive subdivision is applied to the current mesh, thereby removing redundant displacements at a higher level, reducing coded bits for the displacement coefficients, and further improving the coding and decoding efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic process diagram of geometry coding.

FIG. 2 is a schematic diagram of displacement component generation process.

FIG. 3 is a schematic diagram of displacement component decomposition in local coordinate system.

FIG. 4 is a schematic diagram of recursive subdivision using displacement in a direction of a normal to surface.

FIG. 5 is a schematic diagram of mesh subdivision.

FIG. 6 is a schematic diagram of mesh subdivision results with two levels of details (LoDs) and one-dimensional displacements.

FIG. 7 is a schematic diagram of parametrized mesh coding process.

FIG. 8 is a schematic diagram of geometry information in one mesh frame.

FIG. 9 is a schematic surface diagram of a mesh composed of four vertices and three faces.

FIG. 10 is a schematic diagram of data structure of a mesh composed of four vertices and three faces.

FIG. 11 is a schematic diagram of data structure of a parameterized mesh with attribute texture maps.

FIG. 12 is a schematic diagram of a mesh composed of four vertices and three triangular faces with attribute mapping characteristics.

FIG. 13A is a schematic diagram of a manifold mesh.

FIG. 13B is a schematic diagram of a non-manifold mesh.

FIG. 14 is a schematic mapping diagram of a displacement component.

FIG. 15A is a schematic diagram of displacement component packing of a two-dimensional image.

FIG. 15B is a schematic diagram of another displacement component packing of a two-dimensional image.

FIG. 16 is a first schematic flowchart of a decoding method according to an embodiment of the disclosure.

FIG. 17 is a second schematic flowchart of a decoding method according to an embodiment of the disclosure.

FIG. 18 is a schematic diagram of recursive mesh subdivision results according to an embodiment of the disclosure.

FIG. 19 is a detailed flowchart of recursive subdivision according to an embodiment of the disclosure.

FIG. 20 is a third schematic flowchart of a decoding method according to an embodiment of the disclosure.

FIG. 21 is a schematic flowchart of a coding method according to an embodiment of the disclosure.

FIG. 22A is a schematic diagram of a packing mode for displacement coefficients in a two-dimensional image according to an embodiment of the disclosure.

FIG. 22B is a schematic diagram of another packing mode of displacement coefficients in a two-dimensional image according to an embodiment of the disclosure.

FIG. 23 is a schematic structure diagram of an encoder according to an embodiment of the disclosure.

FIG. 24 is a schematic diagram of hardware structure of an encoder according to an embodiment of the disclosure.

FIG. 25 is a schematic structure diagram of a decoder according to an embodiment of the disclosure.

FIG. 26 is a schematic diagram of hardware structure of a decoder according to an embodiment of the disclosure.

FIG. 27 is a schematic structure diagram of a codec system according to an embodiment of the disclosure.

DETAILED DESCRIPTION

In order to make the characteristics and technical contents of the embodiments of the disclosure understood in more detail, the implementation of the embodiments of the disclosure will be described below in combination with the drawings in detail. The appended drawings are only for description as references and not intended to limit the embodiments of the disclosure.

Unless otherwise defined, all technological and scientific terms used in the disclosure have meanings the same as those usually understood by those skilled in the art of the disclosure. Terms used in the disclosure are only adopted to describe the embodiments of the disclosure and not intended to limit the disclosure.

"Some embodiments" involved in the following descriptions describes a subset of all possible embodiments. However, it is to be understood that "some embodiments" may be the same subset or different subsets of all the possible embodiments, and may be combined without conflicts.

Term "first/second/third" involved in the following descriptions is only for distinguishing similar objects, and does not represent a specific sequence of the objects. It is to be understood that "first/second/third" may be interchanged to specific sequences or orders if allowed to implement the embodiments of the disclosure described herein in sequences except the illustrated or described ones.

In embodiments of the disclosure, bitstreams in different data formats may be allowed to be decoded and generated in the same video scene, which may at least include image format, Point Cloud format and Mesh format. In this way, real-time immersive video interaction services can be provided for multiple data formats (e.g., mesh, point cloud, image, etc. ) with different sources.

In embodiments of the disclosure, a data format-based approach may allow independent processing at the bitstream-level data format. That is, like tiles or slices in video encoding, different data formats in the scene may be encoded independently, so independent encoding and decoding may be performed based on the data format.

Generally, three-dimensional (3D) animation content adopts key frame-based representation, that is, each frame is a static mesh. Static meshes at different times have the same topology structure and different geometry structures. However, a 3D dynamic mesh based on key frame representation has a large amount of data, so how to store, transmit and render a 3D dynamic mesh effectively has become a problem for the development of 3D dynamic mesh. In addition, it is necessary to support spatial scalability of the mesh for different user terminals (e.g., computers, notebooks, portable devices, mobile phones) , and different network bandwidths (wideband, narrowband, wireless) need to support the quality scalability of the mesh. Therefore, 3D dynamic mesh compression is a key problem. Herein, "one frame" may be understood as an image/picture. Exemplarily, a key frame may be understood as a key image in 3D animation.

Before the embodiments of the disclosure are further described in detail, key terms and related techniques involved in the embodiments of the disclosure are introduced.

(1) Key terms

Nouns and terms involved in the embodiments of the disclosure are suitable for the following explanations.

Mesh -is a collection of vertices, edges, and faces that defines the shape/topology of a polyhedral object. The faces usually consist of triangles (triangle mesh) .

Base mesh -is a mesh with fewer vertexes but preserves similarity to the original surface.

Dynamic mesh -is a mesh with at least one of the five components (Connectivity, Geometry, Mapping, Vertex Attribute, and Attribute Map) varying in time.

Animated mesh -is a dynamic mesh with constant connectivity.

Parametrized mesh -is a mesh with the topology defined as the Mapping component.

Connectivity -a set of vertex indices describing how to connect the mesh vertices to create a 3D surface. (Geometry and all the attributes share the same unique connectivity information) .

Geometry -a set of vertex 3D (x, y, z) coordinates describing positions associated with the mesh vertices. The (x, y, z) coordinates representing the positions should have finite precision and dynamic range.

Mapping -a description of how to map the mesh surface to 2D regions of the plane. Such mapping is described by a set of UV parametric/texture [mapping] coordinates associated with the mesh vertices together with the connectivity information.

Vertex attribute -a scalar of vector attribute values associated with the mesh vertices.

Attribute Map -attributes associated with the mesh surface and stored as 2D images/videos. The mapping between the videos (i.e., parametric space) and the surface is defined by the mapping information.

Vertex -a position (usually in 3D space) along with other information such as color, normal vector, and texture coordinates.

Edge -a connection between two vertices.

Face -a closed set of edges in which a triangle face has three edges defined by three vertices. Orientation of the face is determined using a “right-hand” coordinate system.

Surface -a collection of faces that separates the three-dimensional object from the environment.

bpp (bits per point) -an amount of information in terms of bits used to describe one point in the mesh.

Displacements -the difference between the original mesh geometry and the mesh geometry reconstructed due to the base mesh subdivision process.

LoD (Level of details) -scalable representation of mesh reconstruction, each level of detail contains enough information to reconstruct mesh to an indicated precision or spatial resolution. Each following level of details is a refinement on top of the plurality of previously reconstructed mesh.

(2) Related technology

Current algorithms apply a two-stage coding process to encode geometry information. First, the geometry information is decimated to create a base mesh encoded using generic geometry coding methods, e.g., “edgebreaker. ” Then the base mesh is hierarchically subdivided, and the difference between the subdivided point and the approximation of the original mesh is stored as the geometry displacements component. The displacement components are packed into a 2D image and encoded with lossless video coding methods such as High Efficiency Video Coding (HEVC) . A diagram of the two-stage geometry coding process is illustrated in FIG. 1.

In FIG. 1, a static or dynamic mesh is input into a preprocessing module for decimating geometry information to generate a base mesh and displacement components. Then, the decimated base mesh is encoded by a generic mesh encoder (such as "edgebreaker" ) , and the displacement components are packed into a 2D image. Then, the displacement information is encoded by a video encoder such as HEVC, and the obtained coded bits are written into a bitstream.

FIG. 2 depicts a displacement generation process for one face in a base mesh with one refinement step. PB1, PB2, and PB3 denote base mesh points, PS1, PS2, and PS3 represent subdivided points, and PSD1, PSD2, and PSD3 represent displaced subdivided points. Subdivided point PS1 is calculated as a mid-point between the PB1 and PB2 points. The process can be recursively repeated.

In FIG. 3, each vector of PS1 and PSD1 is described by three components in normal, tangent and bitangent directions. The three components are further processed with wavelet transform and the corresponding transform coefficients can be mapped to color planes (e.g., Y, U, and V components in the YUV 444 color-space) , or sequentially mapped to a single color plane (e.g. Y plane component in the YUV 420, or YUV 400 color-space) , or packed using interleaving method between color planes (e.g., Y, U, and V components in the YUV 420 color space) .

The subdivision process is recursively repeated until a desired point density is achieved. FIG. 4 depicts one example of recursive subdivision in 2D space using displacement in a direction of a normal to surface. As illustrated in FIG. 4, the iterative subdivision process includes: (a) , (c) and (e) are three-level subdivision processes for convex continuous surfaces, and (b) , (d) and (f) are three-level subdivision processes for oscillating surfaces.

As an example, there are three LoD refinements, and the detailed subdivision process is performed as follows.

Step 1: define an edge using two neighboring points PB0 and PB1 in a reconstructed base mesh.

Step 2: calculate normal to the edge using a face that contains point PB0 and PB1.

Step 3: subdivide reconstructed base mesh at point PS1_1.

Step 4: apply displacement d1_1 to point PS1_1 along the normal defined in step 2.

Step 5: create two edges: PB0 PS1_1 and PS1_1 PB1, as illustrated in (a) and (b) of FIG. 4.

Step 6: apply step 3 -5 to each new edge from step 5 until desired LoD is generated, as illustrated in (c) and (e) of FIG. 4 and in (d) and (f) of FIG. 4.

It should be noted that the normal is always calculated with reference to the reconstructed base mesh. In addition, the displacement is calculated with reference to the subdivision edges generated in step 5, rather than with reference to the reconstructed base edge obtained in step 1.

Further, FIG. 5 illustrates a block chart for the subdivision process, which may include the following.

S501: Mesh decimation.

S502: Base mesh quantization.

S503: Coding Base mesh.

S504: Decoding Base mesh.

S505: Mesh subdivision.

S506: Whether n < L-1?

S507: n=n+1.

S508: Calculate displacement coefficients.

S509: Save to memory.

In FIG. 5, an original mesh is decimated to obtain a base mesh. Then the base mesh is quantized and encoded, and a reconstructed base mesh is obtained by decoding the bitstream. Initializing n=0, the reconstructed base mesh is subdivided. If the determination result is yes, i.e., n < L-1, then n=n+1, and S505 to S506 are continued to be executed. If the determination result is no, i.e., n < L-1 is not true, then the subdivision is completed, and then S508 and S509 are executed. That is, after subdivision is completed, displacement calculation is performed on the obtained subdivided mesh and the original mesh to obtain displacement coefficients, and the displacement coefficients are encoded into the bitstream.

It should be noted that n represents the number of iterative subdivisions, and n is an integer indexed from 0; L represents the number of LoDs, and L is an integer indexed from 1.

Exemplarily, FIG. 6 illustrates mesh subdivision results for three-dimensional content with two LoDs and one-dimensional displacements. The black solid line (composed of vertices PB_1, PB_2 and PB_3) represents the base mesh, the black dotted line represents the subdivided mesh, the first bold solid line is LoD1 (i.e. LoD_1) with applied displacements, and the second bold solid line is LoD2 (i.e. LoD_2) with applied displacements.

Further, FIG. 7 illustrates a schematic diagram of parametrized mesh coding process. The coding process specifically includes the following.

The base mesh frame is quantized and encoded using a static mesh encoder. The process is agnostic to the type of encoding scheme used to compress the base mesh.

The displacements are generated using hierarchical subdivision process and represent the difference between the original mesh topology and previously reconstructed subdivided LoD. The first iteration of the displacements is using base mesh as input.

The displacements are processed by a hierarchical wavelet (or another) transform that recursively applies refinement layers (LoDs) to the reconstructed base mesh.

The wavelet coefficients are then quantized, packed into a 2D image/video, and can be compressed by using a traditional image/video encoder.

The reconstructed version of the wavelet coefficients is obtained by applying image unpacking and inverse quantization to the reconstructed wavelet coefficient image/video generated during the image/video decoding process.

Reconstructed displacements are then computed by applying the inverse wavelet transform to the reconstructed wavelet coefficients.

Wavelet coefficients are calculated in floating-point format and can be positive and negative. In some systems, to compose a 2D image, the coefficients are first converted to positive and mapped to a given bit-depth.

c' (i) =2^ [bit_depth-1] + [c (i) *2^bit_depth] / [c_max-c_min]

where c' (i) is integerized displacement coefficient value, c (i) is a current displacement coefficient, c_max is a maximum displacement coefficient value, c_min is a minimum displacement coefficient value, and bit_depth is a value that defines a number of fixed levels for image coding.

An example of geometry information in one mesh frame is provided in FIG. 8. The example is specifically an example for a mesh with attributes per vertex.

A surface example of a mesh consisting of four vertices and three faces is illustrated in FIG. 9, and a data structure example of a mesh consisting of four vertices and three faces is illustrated in FIG. 10. An example of a surface, represented by a mesh with color-per-vertex characteristics (FIG. 7) that consists of four vertices and three faces, is demonstrated in FIG. 8. Each vertex in space is described by its X, Y, Z position coordinates, and three color attributes R, G, B. As shown below, each face is defined by three vertex indices that form a triangle.

#---------------------------------

#Part 1:

#geometry information

#X Y Z a_1 a_2 a_3

#vertex 0 (v_idx_0)

v 0.0 0.0 0.0 127 127 127

#vertex 1 (v_idx_1)

v 1.0 0.0 0.0 127 127 127

#vertex 2 (v_idx_2)

v 0.0 1.0 0.0 127 127 127

#vertex 3 (v_idx_3)

v 0.0 0.0 1.0 127 127 127

#---------------------------------

#Part 2:

#connectivity information

#v_idx, v_idx, v_idx

#face 0 (f_idx_0)

f 01 2

#face 1 (f_idx_1)

f 03 1

#face 2 (f_idx_2)

f 02 3

#---------------------------------

A data structure example of a parameterized mesh with attribute texture maps is illustrated in FIG. 11.

An example of a surface, represented by a mesh with attribute mapping characteristics (FIG. 11) that consists of four vertices and three faces, is demonstrated in FIG. 12. Each vertex in space is described by its X, Y, Z position coordinates. (U, V) denotes attribute coordinates in the 2D texture vertex map. Each face is defined by three pairs of vertex indices and texture vertex coordinates that form a triangle in 3D space and a triangle in the 2D texture map.

#---------------------------------

#Part 1:

#geometry information

#X Y Z

#vertex 0 (v_idx_0)

v 0.0 0.0 0.0

#vertex 1 (v_idx_1)

v 1.0 0.0 0.0

#vertex 2 (v_idx_2)

v 0.0 1.0 0.0

#vertex 3 (v_idx_3)

v 0.0 0.0 1.0

#---------------------------------

#Part 2:

#mapping information

#U V

#texture vertex 0 (vt_0)

vt 0.500000 0.500000

#texture vertex 1 (vt_1)

vt 0.000000 0.500000

#texture vertex 2 (vt_2)

vt 0.500000 1.000000

#texture vertex 3 (vt_3)

VT 1.000000 0.500000

#texture vertex 4 (vt_4)

vt 0.500000 0.000000

#---------------------------------

#Part 3:

#connectivity information

#v_idx/vt_idx

#face 0 (f_idx_0)

f 0/0 1/2 2/1

#face 1 (f_idx_1)

f 0/0 3/2 1/3

#face 2 (f_idx_2)

f 0/0 2/3 3/4

In the embodiments of the disclosure, the orientation of a face is determined using the right-hand coordinate system. A face consists of three vertices that belong to three edges, and the three vertex indices describe each face.

A manifold mesh is a mesh where one edge belongs to two different faces at most, as illustrate illustrated in FIG. 13A.

A non-manifold mesh is a mesh with an edge that belongs to more than two faces, as illustrated in FIG. 13B.

Further, to encode the displacements component using existing video encoding standards, the transformed displacement components are mapped from a one-dimensional array to a 2-dimensional image, as illustrated in FIG. 14.

Each unit vector component is associated with a different color plane. Exemplarily, when the Normal unit vector is mapped to Y-plane; the Tangent unit vector is mapped to U-plane; the BiTangent unit vector is mapped to V-plane. In this case, YUV444 color mapping is used for encoding (fig 12) . An example of displacement coefficients 8x8 packing block of a video component is FIG. illustrated in 15A and FIG. 15B. FIG. 15A illustrates forward packing and FIG. 15B illustrates backward packing

The nature of the lifting transform used in displacement coefficient coding leads to independent subdivision levels with a separate set of coefficients used for reconstruction.

In the related art, as illustrated in FIG. 5, displacement coefficients are applied to a previously reconstructed LoD. In some instances, the displacements coefficients are often redundant at the high level, resulting in a high bit overhead and reducing the coding and decoding efficiency.

Based on this, the embodiments of the disclosure provide coding and decoding methods. By adding a first syntax element to indicate that recursive subdivision is applied to the current mesh, when it is determined that recursive subdivision is applied to the current mesh, a reconstructed mesh of a current level in the current mesh is subdivided, and a subdivided mesh of the current level is determined; displacement coefficients of the current level are determined by decoding the bitstream; and according to the subdivided mesh of the current level and the displacement coefficients of the current level, a reconstructed mesh of a next level in the current mesh is determined. In this way, when determining that recursive subdivision is applied to the current mesh, redundant displacement coefficients at a higher level may be removed, thereby reducing coded bits for the displacement coefficients and improving the coding and decoding efficiency.

In order to facilitate understanding of the technical solutions of the embodiments of the present disclosure, the technical solutions of the present disclosure will be described in detail by specific embodiments below. The above relevant technology as optional solutions may be combined with the technical solutions of the embodiments in any way, and shall fall within the scope of protection of the disclosure. The embodiments of the disclosure include at least some of the following.

In an embodiment of the disclosure, FIG. 16 is a first schematic flowchart of a decoding method according to an embodiment of the disclosure. As illustrated in FIG. 16, the method may include the following operations.

In S1601, a bitstream is decoded to determine a value of a first syntax element.

It should be noted that in the embodiment of the disclosure, the decoding method may refer to a mesh subdivision method, and in particular to a recursive subdivision method for dynamic mesh decoding, which can improve the coding and decoding efficiency.

It should also be noted that, in the embodiment of the disclosure, Video-based Dynamic Mesh Coding (VDMC) is a standard for compressing three-dimensional meshes, which mainly uses the existing Visual Volume Video-based Coding (V3C) standard to compress the three-dimensional meshes. In embodiments of the disclosure, the subdivision method shares functionality across all LoDs and is defined by two syntax elements asps_vdmc_ext_subdivision_method and asps_vdmc_ext_subdivision_iteration_count in the Atlas sequence parameter set VDMC extension RBSP syntax structure.

Table 1 provides a schematic syntax structure for Atlas sequence parameter set VDMC extension.

Table 1

In Table 1, asps_vdmc_ext_subdivision_method indicates an identifier of the method to subdivide the meshes associated with the current atlas sequence parameter set, asps_vdmc_ext_subdivision_iteration_count indicates the number of iterations used for the mesh subdivision, asps_vdmc_ext_displacement_coordinate_system indicates a coordinate system identifier of the mesh, and asps_vdmc_ext_transform_method indicates a wavelet transform identifier of the mesh.

For asps_vdmc_ext_subdivision_method, Table 2 describes a list of supported subdivision methods and their relationship with asps_vdmc_ext_subdivision_method.

Table 2

When asps_vdmc_ext_subdivision_iteration_count is not present, the value of asps_vdmc_ext_subdivision_iteration_count is inferred to be equal to 0.

Table 3 describes a list of supported coordinate systems and their relationship with asps_vdmc_ext_displacement_coordinate_system.

Table 3

Table 4 describes a list of supported wavelet transform methods and their relationship with asps_vdmc_ext_transform_method.

Table 4

In the embodiments of the disclosure, the first syntax element may be represented by asps_vdmc_ext_subdivision_iteration_count for indicating a mesh subdivision method, such as midpoint subdivision, Loop subdivision and the like. Due to specifics of the subdivision, the higher LoDs are often degenerated to “0” displacement, thus introducing significant redundancy in signaling and coding information.

Based on this, efficient coding of the geometry component of the mesh representation of the volumetric content can be achieved by removing redundant displacements in cases such as base mesh edge and inheriting value from the previous LoD. The embodiments of the disclosure provide a novel

adaptive subdivision method, in which edges adjacent to base points at higher LoD are always equal to 0.

It is to be understood that in embodiments of the disclosure, a new value is added for the first syntax element, so that the first syntax element may be used to indicate whether recursive subdivision is applied to the current mesh. The recursive subdivision may create same subdivision as the related technology, but the displacements neighboring to the base point (e.g., PB_1, PB_2, and PB_3) are inferred to be equal to zero. For each LoD, displacements (displacement coefficients) are calculated with respect to a subdivision vertex in a previously reconstructed LoD, so that the displacements are applied recursively to the previously reconstructed LoD. Therefore, the adaptive subdivision mode herein is referred to as "recursive subdivision mode" . Taking midpoint subdivision as an example, this recursive subdivision method may be referred to as midpoint recursive subdivision method.

In one possible implementation, as illustrated in Table 5, a new asps_vdmc_ext_subdivision_method equal to 2 is added. The new asps_vdmc_ext_subdivision_method may create same subdivision as the related technology, but the displacements neighboring to the base point (e.g., PB_1, PB_2, and PB_3) are inferred to be equal to zero, thereby reducing the coded bits for the displacements.

Table 5

In S1602, when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level.

It is be noted that in the embodiments of the disclosure, the first syntax element is used to indicate whether recursive subdivision is applied to the current mesh. When the value of the first syntax element is a first value, it is determined that the first syntax element indicates that no subdivision is applied to the current mesh. When the value of the first syntax element is a second value, it is determined that the first syntax element indicates that the midpoint subdivision mode is applied to the current mesh. When the value of the first syntax element is a third value, it is determined that the first syntax element indicates that the recursive subdivision mode is applied to the current mesh.

The first value, the second value and the third value are different. Exemplarily, the first value may be set to 0, the second value may be set to 1, and the third value may be set to 2. That is to say, according to different values of the first syntax element, it may be determined whether the recursive subdivision mode is applied to the current mesh.

In a specific embodiment, if the value of the first syntax element is 0, it may be determined that no subdivision mode (such as, midpoint subdivision, recursive subdivision, etc. ) is applied to the current mesh; if the value of the first syntax element is 1, it may be determined that the midpoint subdivision mode is applied to the current mesh; if the value of the first syntax element is 2, it may be determined that the recursive subdivision mode is applied to the current mesh.

It is to be noted that, in the embodiments of the disclosure, when the recursive subdivision mode is applied to the current mesh, reconstruction of the current mesh may be regarded as recursive reconstruction of at least one LoD. Specifically, after a reconstructed mesh of the current level in the current mesh is obtained, the reconstructed mesh of the current level may be subdivided to determine a subdivided mesh of the current level. Then combined with the displacement coefficients in the bitstream, a reconstructed mesh of the next level for the current level can be recursively obtained.

In some embodiments, a reconstructed mesh of the first level in the current mesh is determined firstly. The method may include the following operations. The bitstream is decoded to determine a reconstructed base mesh. The reconstructed base mesh is subdivided to determine an initial subdivided mesh. Displacement coefficients of the initial subdivided mesh are determined by decoding the bitstream. According to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh, a reconstructed mesh of a first level in the current mesh is determined.

In embodiments of the disclosure, the base mesh may also be referred to as a "simplified mesh" . In some embodiments, the reconstructed base mesh is determined by decoding the base mesh bitstream. Exemplarily, the base mesh bitstream may be decoded by a mesh decoder (e.g., edgebreaker, etc. ) to obtain the reconstructed base mesh.

In embodiments of the disclosure, the displacement coefficients for the first level refer to the difference between vertex coordinate information of the original mesh and vertex coordinate information of the initial subdivided mesh. In this way, after the displacement coefficients of the initial subdivided mesh are obtained, displacement calculation is performed on the vertex coordinate information of the initial subdivided mesh according to the displacement coefficients, to obtain the reconstructed mesh of the first level, i.e., the reconstructed mesh of LoD1. Then, the reconstructed mesh of LoD1 is subdivided to obtain a subdivided mesh of LoD1, and the reconstructed mesh of LoD2 may be determined according to the subdivided mesh of LoD1 and the corresponding displacement coefficients. The above process is repeated until the reconstruction is completed.

In S1603, displacement coefficients of the current level are determined by decoding the bitstream.

It is to be noted that in the embodiments of the disclosure, the displacement coefficients may include displacement components in one or more directions. As illustrated in FIG. 3, a displacement coefficient includes displacement components in three directions, i.e., in normal, tangent and bitangent directions. The displacement coefficients of the first level are calculated with respect to the subdivision vertex in the base mesh, and displacement coefficients of each of the subsequent levels are calculated with respect to the subdivision vertex in a previously reconstructed LoD, so that recursive subdivision of displacement coefficients may be realized.

In the embodiments of the disclosure, the displacement coefficients of the current level are obtained by decoding the bitstream. The bitstream may be a displacement bitstream. Decoding of the displacement bitstream may use multiple manners, such as video decoding, entropy decoding and so on. That is, in the embodiments of the disclosure, displacement information may be obtained through decoding by a video decoder, or displacement information may be obtained through decoding by an entropy decoder, which is not limited in the disclosure.

In some embodiments, as illustrated in FIG. 17, the displacement coefficients of the current level are determined as follows.

In S1701, decoded displacement coefficients of the current level are determined by decoding the bitstream.

In S1702, inverse quantization is performed on the decoded displacement coefficients of the current level according to a quantization parameter of the current level to determine dequantized displacement coefficients of the current level.

In S1703, inverse wavelet transform is performed on the dequantized displacement coefficients of the current level to determine the displacement coefficients of the current level.

It is to be noted that in the embodiments of the disclosure, after the decoded displacement coefficients of the current level are obtained, the decoded displacement coefficients of the current level may be preprocessed to determine the displacement coefficients of the current level. Preprocessing may include inverse quantization, inverse wavelet transform or the like.

In a specific embodiment, the operation that the decoded displacement coefficients of the current level are preprocessed to determine the displacement coefficients of the current level includes the following operations. A quantization parameter of the current level is determined. Inverse quantization is performed on the decoded displacement coefficients of the current level according to the quantization parameter of the current level to determine dequantized displacement coefficients of the current level. Inverse wavelet transform is performed on the dequantized displacement coefficients of the current level to determine the displacement coefficients of the current level.

In the embodiments of the disclosure, inverse quantization is the inverse process of quantization and is used for converting a quantized fixed point into a floating-point number. Inverse wavelet transform is the inverse process of wavelet transform and used to restore a signal in wavelet domain to the original time domain, so as to obtain the displacement coefficients of the current level.

In the embodiments of the disclosure, a quantization parameter (QP) reflects spatial detail compression. The smaller the value of QP, the finer the quantization, the higher the image quality and the longer the generated bitstream. If QP is small, most of the details may be retained. QP increases, some details are lost, and the bit rate decreases, but image distortion is enhanced and image quality is degraded. Specifically, QP is a serial number of the quantization step Qstep, and when the value of QP is 0, it indicates that the quantization is the finest, and when the value of QP is 51, it indicates that the quantization is coarser.

In one possible implementation, for the quantization parameter of the current level, the method may include the following operation. The bitstream is decoded to determine the quantization parameter of the current level.

In another possible implementation, for quantization parameter of the current level, the method may include the following operations. A quantization parameter of a previous level of the current level is determined. A quantization parameter increment of the current level is determined by decoding the bitstream. The quantization parameter of the current level is determined according to the quantization parameter of the previous level and the quantization parameter increment.

That is to say, the encoding end may write the quantization parameter directly into the bitstream, so that the decoding end may obtain the corresponding quantization parameter through decoding; or, the encoding end may write the quantization parameter increment into the bitstream, and then the decoding end may obtain the corresponding quantization parameter according to the quantization parameter increment obtained by decoding and the quantization parameter of the previous level.

Exemplarily, assuming that the quantization parameter of the current level may be represented by QP (i) , the quantization parameter of the previous level may be represented by QP (i-1) , and the quantization parameter increment may be represented by ΔQP, then QP (i) = QP (i-1) + ΔQP. For the quantization parameter of the first level, the quantization parameter QP (1) may be directly written into the bitstream.

In the embodiments of the disclosure, a reference quantization parameter may also be set at the encoding end and the decoding end, and then a quantization parameter increment between the quantization parameter of each level and the reference quantization parameter is written into the bitstream. In this way, after determining the quantization parameter increment of the current level by decoding the bitstream, the decoding end may determine the quantization parameter of the current level according to the quantization parameter increment of the current level and the reference quantization parameter, thereby reducing the signaling overhead for decoding the quantization parameter.

In some embodiments, the operation that the decoded displacement coefficients of the current level are determined by decoding the bitstream may include the following operations. The bitstream is decoded to determine a 2D image. Displacement coefficient extraction is performed on the 2D image according to a preset packing mode to obtain the decoded displacement coefficients of the current level.

In the embodiments of the disclosure, the decoded displacement coefficients of the current level may be determined by decoding the displacement bitstream. In addition, the preset packing mode includes a forward packing mode or a inverse packing mode. For determination of the preset packing mode, the decoding end and the encoding end may set the same packing mode, or the preset packing mode may be indicated according to the third syntax element in the bitstream.

In a specific embodiment, the method may include the following operations. The bitstream is decoded to determine a value of a third syntax element. The preset packing mode is determined according to the value of the third syntax element.

In the embodiments of the disclosure, the third syntax element may be represented by dmsps_packing_order. The preset packing mode may be a packing mode indicated by dmsps_packing_order in the bitstream.

Therefore, in the embodiments of the disclosure, the displacement bitstream may be decoded by the displacement decoder. If the encoding end compresses the displacements by video encoding, the decoding end decodes the displacements by a corresponding video decoder, and restores the displacements from the 2D image in a corresponding order according to the preset packing mode. Then, inverse quantization and inverse wavelet transform are performed to obtain the displacement coefficients consistent with the coding end. If the encoding end uses entropy coding for the displacement coefficients, the decoding end may directly perform entropy decoding, and then perform subsequent operations such as inverse quantization and inverse wavelet transform to obtain the displacement coefficients of the current level.

In S1604, a reconstructed mesh of a next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level.

In the embodiments of the disclosure, the operation that the reconstructed mesh of the next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level may include the following operation. Displacement calculation is performed on vertex coordinate information of the subdivided mesh of the current level according to the displacement coefficients, to obtain the reconstructed mesh of the next level in the current mesh.

That is to say, in the embodiments of the disclosure, the displacement coefficients are applied recursively to the previously reconstructed LoD, specifically, each vertex of the subdivided mesh of the current level is sequentially added with the respective decoded displacement coefficient, so the reconstructed mesh of the next level may be obtained.

In some embodiments, the method may further include the following operations. After the reconstructed mesh of the next level in the current mesh is determined, the reconstructed mesh of the next level is taken as a reconstructed mesh of the current level, and the operation of subdividing the reconstructed mesh of the current level to determine the subdivided mesh of the current level is returned and performed, until a reconstructed mesh of the L-th level in the current mesh is determined. The value of L is associated with a number of subdivision iterations for the current mesh.

It is to be noted that, in the embodiments of the disclosure, the number of subdivision iterations for the current mesh may be indicated by a second syntax element in the bitstream. In a specific embodiment, the method may include the following operations. The bitstream is decoded to determine a value of the second syntax element. The number of subdivision iterations for the current mesh is determined according to the value of the second syntax element.

In the embodiments of the disclosure, the second syntax element may be represented by asps_vmc_ext_subdivision_iteration_count. That is, the number of subdivision iterations for the current mesh may be a value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.

L represents the number of LoDs corresponding to the current mesh, and the value of L may be determined by the number of subdivision iterations for the current mesh, that is, the value of L is equal to the value indicated by asps_vmc_ext_subdivision_iteration_count.

In the embodiments of the disclosure, as illustrated in FIG. 18, first, the base mesh is subdivided to obtain an initial subdivided mesh, and after the displacement coefficients of the initial subdivided mesh are obtained by decoding, displacement calculation is performed on vertex coordinate information of the initial subdivided mesh according to the displacement coefficients, to obtain the reconstructed mesh of the first level, i.e., the reconstructed mesh of LoD1. Then, the reconstructed mesh of LoD1 is subdivided to obtain a subdivided mesh of LoD1, and after displacement coefficients of LoD1 are obtained by decoding, the reconstructed mesh of LoD2 may be determined according to the subdivided mesh of LoD1 and the displacement coefficients of LoD1. The process is repeated until the reconstructed mesh of LoD L is obtained, indicating that the reconstruction is completed. In FIG. 18, the black solid line (composed of vertices PB_1, PB_2 and PB_3) represents the base mesh, the black dotted line represents the subdivided mesh, the first bold solid line is LoD1 with applied displacements, and the second bold solid line is LoD2 with applied displacements.

In some embodiments, a detailed flowchart of recursive subdivision is illustrated in FIG. 19, which may include the following.

S1901: Mesh decimation.

S1902: Base mesh quantization.

S1903: Coding base mesh.

S1904: Decoding base mesh.

S1905: Mesh subdivision.

S1906: Calculate displacement coefficients.

S1907: Update interim mesh.

S1908: Save to memory.

S1909: Whether n < L-1?

S1910: n=n+1.

In FIG. 19, an original mesh is decimated to obtain a base mesh. Then the base mesh is quantized and encoded, and a reconstructed base mesh is obtained by decoding the bitstream. Initializing n=0, the reconstructed base mesh as an interim mesh is subdivided, and displacement coefficients are calculated according to the subdivided mesh and the original mesh. Then, the interim mesh is updated according to the subdivided mesh and the displacement coefficients, and the displacement coefficients are saved to memory. If the determination result is yes, i.e., n < L-1, then n=n+1, and S1905 to S1909 are continued to be executed. If the determination result is no, i.e., n < L-1 is not true, then the subdivision is completed. Finally, the displacement coefficients are encoded into the bitstream

It is to be understood that when recursive subdivision is not applied to the current mesh, the current mesh may use midpoint subdivision. In some embodiments, as illustrated in FIG. 20, the method may include the following operations.

In S2001, when the first syntax element indicates that a midpoint subdivision mode is applied to the current mesh, a reconstruction base mesh is determined by decoding the bitstream.

In S2002, the reconstructed base mesh is subdivided to determine a subdivided mesh of at least one level in the current mesh.

In S2003, displacement coefficients of the at least one level are determined by decoding the bitstream.

In S2004, a reconstructed mesh of the at least one level in the current mesh is determined according to the subdivided mesh and displacement coefficients of the at least one level.

In the embodiments of the disclosure, the number of levels in the at least one level is associated with the number of subdivision iterations for the current mesh. Here, L denotes the number of levels of the at least one level, i.e. the number of LoDs (LoD number) corresponding to the current mesh, and L may be equal to the value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.

It is also to be noted that, in the embodiments of the disclosure, as illustrated in FIG. 5 and FIG. 6, when the midpoint subdivision mode is applied to the current mesh, displacement coefficients of each level are obtained by displacement calculation on vertex coordinate information of the subdivided mesh and vertex coordinate information of the original mesh, instead of being recursively obtained based on the previously reconstructed LoD.

The embodiments of the disclosure provide a decoding method, in particular to a recursive subdivision method for a dynamic mesh. A bitstream is decoded to determine a value of a first syntax element; when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, and the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level; displacement coefficients of the current level are determined by decoding the bitstream; and according to the subdivided mesh of the current level and the displacement coefficients of the current level, a reconstructed mesh of a next level in the current mesh is determined. In this way, by adding the first syntax element to indicate that recursive subdivision is applied to the current mesh, the displacement coefficients are applied recursively to the previously reconstructed LoD when determining that recursive subdivision is applied to the current mesh, thereby removing redundant displacements at a higher level, reducing coded bits for the displacement coefficients, and further improving the coding and decoding efficiency.

In another embodiment of the disclosure, FIG. 21 is a schematic flowchart of a coding method according to an embodiment of the disclosure. As illustrated in FIG. 21, the method may include the following operations.

In S2101, when a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, and the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level.

It should be noted that in the embodiment of the disclosure, the coding method may refer to a mesh subdivision method, and in particular to a recursive subdivision method for dynamic mesh coding, which can improve the coding and decoding efficiency.

It should also be noted that, in the embodiment of the disclosure, Video-based Dynamic Mesh Coding (VDMC) is a standard for compressing three-dimensional meshes, which mainly uses the existing V3C standard to compress the three-dimensional meshes. In the embodiments of the disclosure, the subdivision method shares functionality across all LoDs and is defined by two syntax elements asps_vdmc_ext_subdivision_method and asps_vdmc_ext_subdivision_iteration_count in the Atlas sequence parameter set VDMC extension RBSP syntax structure, as illustrated in Table 1.

For mesh subdivision, due to specifics of the subdivision, the higher LoDs are often degenerated to “0” displacement, thus introducing significant redundancy in signaling and coding information. Based on this, efficient coding of the geometry component of the mesh representation of the volumetric content can be achieved by removing redundant displacements in cases such as base mesh edge and inheriting value from the previous LoD. The embodiments of the disclosure provide a novel adaptive subdivision method, in which edges adjacent to base points at a higher LoD are always equal to 0.

In the embodiments of the disclosure, a first syntax element may be set to indicate whether recursive subdivision is applied to the current mesh. The first syntax element may be represented by asps_vdmc_ext_subdivision_iteration_count for indicating the mesh subdivision method. Exemplarily, as illustrated in Table 2, if the value of the first syntax element is 0, it indicates that no subdivision is applied to the current mesh; if the value of the first syntax element is 1, it indicates that midpoint subdivision is applied to the current mesh.

It is to be understood that in embodiments of the disclosure, based on Table 2, a new value is added for the first syntax element, so that the first syntax element may be used to indicate whether recursive subdivision is applied to the current mesh. The recursive subdivision may create same subdivision as the related technology, but the displacements neighboring to the base point (e.g., PB_1, PB_2, and PB_3) are inferred to be equal to zero. For each LoD, displacements (displacement coefficients) are calculated with respect to a subdivision vertex in a previously reconstructed LoD, so that the displacements are applied recursively to the previously reconstructed LoD. Therefore, the adaptive subdivision mode herein is referred to as "recursive subdivision mode" . Taking midpoint subdivision as an example, this recursive subdivision method may be referred to as midpoint recursive subdivision method.

It is be noted that in the embodiments of the disclosure, the first syntax element is used to indicate whether recursive subdivision is applied to the current mesh. In some embodiments, the method may include the following operations. A value of the first syntax element is determined. The value of the first syntax element is encoded into a bitstream.

In a specific embodiment, when no subdivision is applied to the current mesh, the value of the first syntax element is determined to be the first value; when a midpoint subdivision mode is applied to the current mesh, the value of the first syntax element is determined to be the second value; and when the recursive subdivision mode is applied to the current mesh, the value of the first syntax element is determined to be the third value.

The first value, the second value and the third value are different. Exemplarily, the first value may be set to 0, the second value may be set to 1, and the third value may be set to 2. That is to say, it may be determined according to different values of the first syntax element whether the recursive subdivision mode is applied to the current mesh.

In some embodiments, a reconstructed mesh of the first level in the current mesh is determined firstly. The method may include the following operations. A base mesh of the current mesh is determined. Coding and decoding processing is performed on the base mesh to determine a reconstructed base mesh. The reconstructed base mesh is subdivided to determine an initial subdivided mesh. Displacement coefficients of the initial subdivided mesh are determined. A reconstructed mesh of a first level in the current mesh is determined according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh.

In embodiments of the disclosure, the base mesh can also be referred to as a "simplified mesh" . In some embodiments, the operation that the base mesh of the current mesh is determined may include the following operations. An original mesh of the current mesh is determined. The original mesh is downsampled to obtain the base mesh.

It is also to be noted that in embodiments of the disclosure, the current input mesh may be referred to as the original mesh. The input mesh is downsampled for mesh decimation, so as to obtain the base mesh.

In some embodiments, the method may further include the following operation. The base mesh is encoded into a bitstream.

Exemplarily, the bitstream may refer to the base mesh bitstream. The base mesh may be written into the base mesh bitstream by a mesh encoder (e.g., EdgeBreaker, etc. ) .

In some embodiments, for determining the displacement coefficients of the initial subdivided mesh, the method may include the following operation. Displacement calculation is performed on vertex coordinate information of the initial subdivided mesh and vertex coordinate information of the original mesh to determine the displacement coefficients of the initial subdivided mesh.

In S2102, displacement coefficients of the current level are determined.

In some embodiments, the operation that the displacement coefficients of the current level are determined may include the following operation. Displacement calculation is performed on vertex coordinate information of the initial subdivided mesh and vertex coordinate information of the original mesh to determine the displacement coefficients of the current level.

Further, in some embodiments, the method may further include the following operations. The displacement coefficient of the current level are preprocessed to determine quantized displacement coefficients of the current level. The quantized displacement coefficients of the current level are encoded into the bitstream.

In embodiments of the disclosure, after the subdivided mesh is generated, geometry displacement vectors may be calculated for each vertex of the subdivided mesh, so that the shape of the subdivided mesh is as similar as possible to the shape of the original mesh. These geometry displacement vectors are displacement coefficients. Specifically, there is a difference between geometry information of the vertices of the subdivided mesh and geometry information of vertices of the original mesh, and the difference is displacement coefficients.

It is also to be noted that, in the embodiments of the disclosure, after quantized displacement coefficients of the current level are determined, the quantized displacement coefficients of the current level are encoded into a bitstream. The bitstream may be a displacement bitstream. Coding of the displacement bitstream may use multiple manners, such as video coding, entropy coding and so on. That is, in the embodiments of the disclosure, displacement information may be written into the displacement bitstream by a video encoder, or displacement information may be written into the displacement bitstream by an entropy encoder, which is not limited in the disclosure.

It is to be noted that in the embodiments of the disclosure, after the displacement coefficients of the current level are obtained, the displacement coefficients of the current level may be preprocessed. Preprocessing may include wavelet transform, quantization, or the like.

In a specific embodiment, the operation that the displacement coefficients of the current level are preprocessed to determine the quantized displacement coefficients of the current level may include the following operations. Wavelet transform is performed on the displacement coefficients of the current level to determine the wavelet transform coefficients of the current level. The wavelet transform coefficients of the current level are quantized to determine the quantized displacement coefficients of the current level.

In the embodiments of the disclosure, wavelet transform is to convert the displacement coefficients of the current level into signals in wavelet domain, in which the wavelet transform coefficients are calculated in floating-point format. Quantization is to convert floating-point numbers into fixed-point numbers with preset precision. The preset precision may be a precision indicated in the coded bitstream at patch, picture, or sequence level.

In some embodiments, the operation that the wavelet transform coefficients of the current level are quantized to determine the quantized displacement coefficients of the current level may include the following operations. A quantization parameter of the current level is determined. The wavelet transform coefficients of the current level are quantized according to the quantization parameter of the current level, to determine the quantized displacement coefficients of the current level.

In the embodiments of the disclosure, a quantization parameter (QP) reflects spatial detail compression. The smaller the value of QP, the finer the quantization, the higher the image quality and the longer the generated bitstream. If QP is small, most of the details may be retained. As QP increases, some details are lost, and the bit rate decreases, but image distortion is enhanced and image quality is degraded. Specifically, QP is a serial number of the quantization step Qstep, and when the value of QP is 0, it indicates that the quantization is the finest, and when the value of QP is 51, it indicates that the quantization is coarser.

In some embodiments, the quantization parameter for the current level may be determined as follows. A plurality of candidate quantization parameters for the current level are determined. Quantization and coding cost calculation are performed on the wavelet transform coefficients of the current level based on the plurality of candidate quantization parameters to determine respective cost results of the plurality of candidate quantization parameters. A minimum cost result is determined from cost results of the plurality of candidate quantization parameters, and a candidate quantization parameter corresponding to the minimum cost result is determined as the quantization parameter of the current level.

In the embodiments of the disclosure, for each LoD, there may be a plurality of candidate quantization parameters, and according to the respective cost results of the plurality of candidate quantization parameters, the candidate quantization parameter corresponding to the minimum cost result is selected as the quantization parameter of the current level. The cost calculation may include at least one of the following: Rate-Distortion Optimization (RDO) , Mean Square Error (MSE) , Sum of Squared Difference (SSD) , Sum of Absolute Difference (SAD) , Sum of Absolute Transformed Difference (SATD) , Peak Signal to Noise Ratio (PSNR) , or the like.

It is also to be noted that in embodiments of the disclosure, the quantization parameter may be represented by vmc_transform_lifting_quantization_parameters [ltpIndex] [i] [j] . Wavelet transform coefficients are converted to a fix-point representation with a precision indicated in the coded bitstream at either patch, picture, or sequence level depending on the quantization parameter vmc_transform_lifting_quantization_parameters [ltpIndex] [i] [j] for a corresponding coordinate i and LoD j as signaled in the bitstream. i represents component information of displacement coefficients (x, y, z for canonical coordinate system, and n, t, bt for local coordinate system) , j represents the number of LoD levels, and ltpIndex represents an application level, where 0 represents sequence level, 1 represents picture level, and 2 represents patch level.

In one possible implementation, for the quantization parameter of the current level, the method may include the following operation. The quantization parameter of the current level is encoded into the bitstream.

In another possible implementation, for quantization parameters of the current level, the method may include the following operations. A quantization parameter of a previous level of the current level is determined. A quantization parameter increment of the current level is determined according to the quantization parameter of the previous level and the quantization parameter of the current level. the quantization parameter increment of the current level is encoded into the bitstream.

That is to say, in the embodiments of the disclosure, the encoding end may directly write quantization parameters into the bitstream, so that the decoding end may obtain the corresponding quantization parameters through decoding; or, the encoding end may write the quantization parameter increment into the bitstream, and then the decoding end may obtain the corresponding quantization parameter according to the quantization parameter increment obtained by decoding and the quantization parameter of the previous level.

In a specific embodiment, the quantization parameter increment of the current level may be determined by subtracting the quantization parameter of the previous level from the quantization parameter of the current level. Exemplarily, assuming that the quantization parameter of the current level may be represented by QP (i) , the quantization parameter of the previous level may be represented by QP (i-1) , and the quantization parameter increment may be represented by ΔQP, then ΔQP = QP (i) -QP (i-1) . For the quantization parameter of the first level, the quantization parameter QP (1) may be directly written into the bitstream.

In the embodiments of the disclosure, a reference quantization parameter may also be set at the encoding end and the decoding end, and then a quantization parameter increment between the quantization parameter of each level and the reference quantization parameter is written into the bitstream. In this way, after determining the quantization parameter increment of the current level by decoding the bitstream, the decoding end may determine the quantization parameter of the current level according to the quantization parameter increment of the current level and the reference quantization parameter, thereby saving coded bits for quantization parameters in the bitstream.

In some embodiments, the method may further include the following operations. Quantized displacement coefficients of at least one level in the current mesh are determined. The at least one level includes the current level. The quantized displacement coefficients of the at least one level are packed into a 2D image according to a preset packing mode. The 2D image is encoded into the bitstream.

In the embodiment of the disclosure, the preset packing mode includes a forward packing mode or an inverse packing mode. For determination of the preset packing mode, the decoding end and the encoding end may set the same packing mode, or the preset packing mode may be indicated according to the third syntax element in the bitstream.

In a specific embodiment, the method may further include the following operations. A value of the third syntax element is determined according to the preset packing mode. The value of the third syntax element is encoded into the bitstream.

Therefore, if the encoding end compresses the displacements by video encoding, the decoding end decodes the displacements by a corresponding video decoder, and restores the displacements from the 2D image in a corresponding order according to the preset packing mode. Then, inverse quantization and inverse wavelet transform are performed to obtain the displacement coefficients consistent with the coding end. If the encoding end uses entropy coding for the displacement coefficients, the decoding end may directly perform entropy decoding, and then perform subsequent operations such as inverse quantization and inverse wavelet transform to obtain the displacement coefficients of the current level.

As an example, FIG. 22A is a schematic diagram of a packing mode for displacement coefficients in a two-dimensional image according to an embodiment of the disclosure, and FIG. 22B is a schematic diagram of another packing mode for displacement coefficients in a two-dimensional image according to an embodiment of the disclosure. FIG. 22A is an exemplary forward packing mode (or referred to as "continuous-packing" ) , and FIG. 22B is an exemplary inverse packing.

In the embodiments of the disclosure, the quantized displacement coefficients are scanned along a 3D space scanning pattern (e.g. Morton, Hilbert, or along other space filling curve) within each LoD, forming three 1-dimensional arrays per each component (FIG. 22A and FIG. 22B) . The quantized displacement coefficients are converted to a 2D image according to LoD and the prest packing order indicated by the syntax element dmsps_packing_order. The unoccupied symbols in a coding tree unit (CTU) are padded using one of the padding methods (e.g. zero-padding) .

In S2103, a reconstructed mesh of a next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level.

It is to be noted that, in the embodiments of the disclosure, the number of subdivision iterations for the current mesh may be indicated by a second syntax element in the bitstream. In a specific embodiment, the method may include the following operations. The number of subdivision iterations for the current mesh is determined. A value of a second syntax element is determined according to the number of subdivision iterations for the current mesh. The value of the second syntax element is encoded into the bitstream.

In the embodiments of the disclosure, as illustrated in FIG. 18 and FIG. 19, first, the base mesh is subdivided to obtain an initial subdivided mesh, and after the displacement coefficients of the initial subdivided mesh are determined, displacement calculation is performed on vertex coordinate information of the initial subdivided mesh according to the displacement coefficients, to obtain the reconstructed mesh of the first level, i.e., the reconstructed mesh of LoD1. Then, the reconstructed mesh of LoD1 is subdivided to obtain a subdivided mesh of LoD1, and after displacement coefficients of LoD1 are determined, the reconstructed mesh of LoD2 may be determined according to the subdivided mesh of LoD1 and the displacement coefficients of LoD1. The process is repeated until the reconstructed mesh of LoD L is obtained, indicating that the reconstruction is completed. In FIG. 18, the black solid line (composed of vertices PB_1, PB_2 and PB_3) represents the base mesh, the black dotted line represents the subdivided mesh, the first bold solid line is LoD1 with applied displacements, and the second bold solid line is LoD2 with applied displacements.

It is to be understood that when recursive subdivision is not applied to the current mesh, the current mesh may use midpoint subdivision. In some embodiments, the method may include the following operations. When the midpoint subdivision mode is applied to the current mesh, a reconstruction base mesh is determined. The reconstructed base mesh is subdivided to determine a subdivided mesh of at least one level in the current mesh. Displacement coefficients of the at least one level are determined. A reconstructed mesh of the at least one level in the current mesh is determined according to the subdivided mesh and displacement coefficients of the at least one level.

In the embodiments of the disclosure, the number of levels in the at least one level is associated with the number of subdivision iterations for the current mesh. Here, L denotes the number of levels in the at least one level, i.e. the number of LoDs corresponding to the current mesh, and L may be equal to the value indicated by asps_vmc_ext_subdivision_iteration_count in the bitstream.

The embodiments of the disclosure also provide a bitstream. The bitstream is generated by encoding (bit coding) according to information to be encoded. The information to be encoded includes at least one of the following: a base mesh of a current mesh, displacement coefficients of at least one level in the current mesh, a quantization parameter of at least one level in the current mesh, a quantization parameter increment of at least one level in the current mesh, a value of a first syntax element, a value of a second syntax element or a value of a third syntax element.

The embodiments of the disclosure provide a coding method, in particular to a recursive subdivision method for a dynamic mesh. When a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, and the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level; displacement coefficients of the current level are determined; and a reconstructed mesh of a next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level. In this way, by adding the first syntax element to indicate that recursive subdivision is applied to the current mesh, the displacement coefficients are applied recursively to the previously reconstructed LoD when determining that recursive subdivision is applied to the current mesh, thereby removing redundant displacements at a higher level, reducing coded bits for the displacement coefficients, and further improving the coding and decoding efficiency.

In another embodiment of the disclosure, based on the coding and decoding methods described in the above embodiments, the displacement coefficients are applied to the previously reconstructed LoD as illustrated in FIG. 6. The displacements coefficients are often redundant at the high level, in some instances.

In some systems, the subdivision method shares functionality across all LoDs and is defined by two syntax elements asps_vdmc_ext_subdivision_method and asps_vdmc_ext_subdivision_iteration_count in the Atlas sequence parameter set VDMC extension RBSP syntax structure, as illustrated in Table 1.

asps_vdmc_ext_subdivision_method indicates the identifier of the method to subdivide the meshes associated with the current atlas sequence parameter set. Table 2 describes the list of supported subdivision methods and their relationship with asps_vdmc_ext_subdivision_method.

asps_vdmc_ext_subdivision_iteration_count indicates the number of iterations used for the subdivision. When sps_vdmc_ext_subdivision_iteration_count is not present, the value of asps_vdmc_ext_subdivision_iteration_count is inferred to be equal to 0.

Due to specifics of the subdivision, the higher levels of details are often degenerated to “0” displacement, thus introducing significant redundancy in signaling and coding information. The solutions provided by the embodiments of the disclosure introduces efficient and flexible method and apparatus for efficient coding of the geometry component of the mesh representation of the volumetric content by removing redundant displacements in cases such as base mesh edge and inheriting value from previous level of details.

To overcome the issue of signaling the degenerated edges, the embodiments of the disclosure provide a new adaptive subdivision method, in which edges adjacent to the base points at higher level of details shall be always equal to 0. One way of implementing this is by adding a new asps_vdmc_ext_subdivision_method equal to 2. The new asps_vdmc_ext_subdivision_method may create same subdivision as the original method, but the displacements neighboring to the base point (e.g., PB_1, PB_2, and PB_3) are inferred to be equal to zero. The specific process is illustrated in FIG. 18 and FIG. 19.

In a specific embodiment, the method to efficiently code geometry displacement coefficients in the mesh at the encoding end includes a subdivision process with a recursive subdivision update, and the displacement coefficients are calculated with respect to a subdivision vertex in a previously reconstructed LoD.

Stage 1: Mesh segmentation is a step that creates segments or blocks of mesh content representing individual objects/regions of interest/volumetric tiles, semantic blocks, etc. The number of level of details subdivisions is defined by asps_vmc_ext_subdivision_iteration_count.

Stage 2: Mesh decimation creates a base mesh, and the base mesh is coded with an undefined static mesh encoder. The base mesh is decoded and recursively subdivided to the number of level of details.

Stage 3: Mesh displacements are calculated between the previous updated subdivided level of details for mesh and the original surface for mesh for each level of details when the value of the syntax element asps_vdmc_ext_subdivision_method is equal to 2. The displacements are then processed with a wavelet transform.

Stage 4: Wavelet transform coefficients are converted to a fix-point representation with a precision indicated in the coded bitstream at either patch, picture, or sequence level (ltpIndex) depending on the quantization parameter vmc_transform_lifting_quantization_parameters [ltpIndex] [i] [j] for a corresponding coordinate i and level of details j as signaled in the bitstream.

Stage 5: The quantized wavelet coefficients are scanned along a 3D space scanning pattern (e.g. Morton, Hilbert, or along other space filling curve) within each LoD, forming three 1-dimensional arrays per each component (as illustrated in FIG. 22A and FIG. 22B) .

The coefficients are converted to a 2D image according to LoD and selected packing order indicated by a flag dmsps_packing_order. The unoccupied symbols in CTU are padded using one of the padding methods (e.g. zero-padding) ..

In another specific embodiment, the decoding process is the inverse of the encoding process and includes five stages as follows.

Stage 1: The base mesh is decoded from geometry bitstream and recursively subdivided to the level of details defined by the encoder.

Stage 2: A coded bitstream for geometry displacements is obtained and decoded with a codec corresponding to the dmsps_mesh_codec_id decoder.

Stage 3: The displacement wavelet coefficients are dequantized using the quantization parameter signaled in the bitstream.

Stage 4: The dequantized displacement wavelet coefficients are processed with an inverse wavelet transform.

Stage 5: Mesh displacements are applied to the subdivided base mesh at each transform level recursively to generate the reconstructed mesh consisting of blocks representing individual objects/regions of interest/volumetric tiles, semantic blocks, etc.

When the value of asps_vdmc_ext_subdivision_method is equal to 2, the displacements are applied recursively to the previously reconstructed level of details.

The specific implementation of the above embodiments has been described in detail. According to the technical solution of the above embodiments, by adding a first syntax element to indicate that recursive subdivision is applied to the current mesh, the displacement coefficients are applied recursively to the previously reconstructed level of details (LoD) when determining that recursive subdivision is applied to the current mesh, thereby removing redundant displacements at a higher level, reducing coded bits for the displacement coefficients, and further improving the coding and decoding efficiency.

In another embodiment of the disclosure, based on the same inventive concept as the above embodiments, FIG. 23 is a schematic structure diagram of an encoder according to an embodiment of the disclosure. As illustrated in FIG. 23, the encoder 230 may include a first subdivision unit 2301, a first determination unit 2302, and a first reconstruction unit 2303.

The first subdivision unit 2301 is configured to determine, when a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, and subdivide the reconstructed mesh to determine a subdivided mesh of the current level.

The first determination unit 2302 is configured to determine displacement coefficients of the current level.

The first reconstruction unit 2303 is configured to determine a reconstructed mesh of a next level in the current mesh according to the subdivided mesh of the current level and the displacement coefficients of the current level.

In some embodiments, the first determination unit 2302 is further configured to determine a base mesh of the current mesh. The first reconstruction unit 2303 is further configured to perform coding and decoding processing on the base mesh to determine a reconstructed base mesh. The first subdivision unit 2301 is further configured to subdivide the reconstructed base mesh to determine an initial subdivided mesh. The first determination unit 2302 is further configured to determine displacement coefficients of the initial subdivided mesh, and determine a reconstructed mesh of a first level in the current mesh according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh.

In some embodiments the first determination unit 2302 is further configured to determine an original mesh of the current mesh, and perform downsampling on the original mesh to obtain the base mesh.

In some embodiments, the first determination unit 2302 is further configured to perform displacement calculation on vertex coordinate information of the initial subdivided mesh and vertex coordinate information of the original mesh to determine the displacement coefficients of the initial subdivided mesh.

In some embodiments, referring to FIG. 23, the encoder 230 may also include an encoding unit 2304. The encoding unit 2304 is configured to encode the base mesh into a the bitstream.

In some embodiments, the first reconstruction unit 2303 is further configured to, after determining the reconstructed mesh of the next level in the current mesh, take the reconstructed mesh of the next level as a reconstructed mesh of a current level, and continue to subdivide the reconstructed mesh of the current level to determine the subdivided mesh of the current level, until a reconstructed mesh of an L-th level in the current mesh is determined. The value of L is associated with the number of subdivision iterations for the current mesh.

In some embodiments, the first determination unit 2302 is further configured to determine the number of subdivision iterations for the current mesh, and determine a value of a second syntax element according to the number of subdivision iterations for the current mesh. The encoding unit 2304 is further configured to encode the value of the second syntax element into a bitstream.

In some embodiments, the first determination unit 2302 is further configured to perform displacement calculation on vertex coordinate information of the subdivided mesh of the current level and vertex coordinate information of the original mesh to determine the displacement coefficients of the current level.

In some embodiments, the first determination unit 2302 is further configured to perform preprocessing on the displacement coefficients of the current level to determine quantized displacement coefficients of the current level. The encoding unit 2304 is further configured to encode the quantized displacement coefficients of the current level into a bitstream.

In some embodiments, the first determination unit 2302 is further configured to perform wavelet transform on the displacement coefficients of the current level to determine wavelet transform coefficients of the current level, and quantize the wavelet transform coefficients of the current level to determine the quantized displacement coefficients of the current level.

In some embodiments, the first determination unit 2302 is further configured to determine a quantization parameter of the current level, and quantize the wavelet transform coefficients of the current level according to the quantization parameter of the current level to determine the quantized displacement coefficients of the current level.

In some embodiments, the encoding unit 2304 is further configured to encode the quantization parameter of the current level into the bitstream.

In some embodiments, the first determination unit 2302 is further configured to determine a quantization parameter of a previous level of the current level, and determine a quantization parameter increment of the current level according to the quantization parameter of the previous level and the quantization parameter of the current level. The encoding unit 2304 is further configured to encode the quantization parameter increment of the current level into the bitstream.

In some embodiments, the first determination unit 2302 is further configured to determine quantized displacement coefficients of at least one level in the current mesh, where the at least one level includes the current level and pack the quantized displacement coefficients of the at least one level into a two-dimensional image according to a preset packing mode. The encoding unit 2304 is further configured to encode the two-dimensional image into the bitstream.

In some embodiments, the first determination unit 2302 is further configured to determine a value of a third syntax element according to the preset packing mode. The encoding unit 2304 is further configured to encode the value of the third syntax element into the bitstream.

In some embodiments, the first determination unit 2302 is further configured to perform displacement calculation on vertex coordinate information of the subdivided mesh of the current level according to the displacement coefficients to obtain the reconstructed mesh of the next level in the current mesh.

In some embodiments, the first determination unit 2302 is further configured to determine a value of a first syntax element. The first syntax element indicates whether the recursive subdivision mode is applied to the current mesh. The encoding unit 2304 is further configured to encode the value of the first syntax element into the bitstream.

In some embodiments, the first determination unit 2302 is further configured to determine the value of the first syntax element to be a first value when no subdivision is applied to the current mesh, determine the value of the first syntax element to be a second value when a midpoint subdivision mode is applied to the current mesh, and determine the value of the first syntax element to be a third value when the recursive subdivision mode is applied to the current mesh.

In some embodiments, the first determination unit 2302 is further configured to determine, when the midpoint subdivision mode is applied to the current mesh, a reconstruction base mesh. The first subdivision unit 2301 is further configured to subdivide the reconstructed base mesh to determine a subdivided mesh of at least one level in the current mesh. The first determination unit 2302 is further configured to determine displacement coefficients of the at least one level and determine a reconstructed mesh of the at least one level in the current mesh according to the subdivided mesh and displacement coefficients of the at least one level in the current mesh.

In some embodiments, the number of levels in the at least one level is associated with the number of subdivision iterations for the current mesh.

It is to be understood that, in the embodiment of the disclosure, "unit" may be part of a circuit, part of a processor, part of a program or software and the like, of course, may also be modular and may also be non-modular. In addition, each component in the embodiment may be integrated into a processing unit, each unit may also exist independently, and two or more than two units may also be integrated into a unit. The integrated unit may be implemented in a hardware form and may also be implemented in form of software function module.

When implemented in form of software function module and sold or used not as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment substantially or parts making contributions to the conventional art or all or part of the technical solution may be embodied in form of software product, and the computer software product is stored in a storage medium, including a plurality of instructions configured to enable a computer device (which may be a personal computer, a server, a network device or the like) or a processor to execute all or part of the operations of the method in the embodiments. The storage medium includes: various media capable of storing program codes such as a U disk, a mobile hard disk, a Read Only Memory (ROM) , a Random Access Memory (RAM) , a magnetic disk or an optical disk.

Therefore, the embodiments of the disclosure provide a computer-readable storage medium, applied to the encoder 230. The computer-readable storage medium stores a computer program that implements the method of any of the above embodiments when executed by a first processor.

Based on the composition of the encoder 230 and the computer-readable storage medium, FIG. 24 is a schematic diagram of hardware structure of an encoder according to an embodiment of the disclosure. As illustrated in FIG. 24, the encoder 230 may include a first communication interface 2401, a first memory 2402 and a first processor 2403. The components are coupled together by a first bus system 2404. It can be understood that the bus system 2404 is configured to implement connection communication among these components. The bus system 2404 includes a data bus and further includes a power bus, a control bus and a state signal bus. However, for clear description, various buses in FIG. 24 are marked as the bus system 2404.

The first communication interface 2401 is configured to receive and send a signal in a process of receiving and sending information with another external network element.

The first memory 2402 is configured to store a computer program capable of running in the processor 2403.

The first processor 2403 is configured to run the computer program to execute the following operations: determining, when a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, and subdividing the reconstructed mesh to determine a subdivided mesh of the current level; determining displacement coefficients of the current level; and determining, according to the subdivided mesh of the current level and the displacement coefficients of the current level, a reconstructed mesh of a next level in the current mesh.

It can be understood that the first memory 2402 in the embodiment of the disclosure may be a volatile memory or a nonvolatile memory, or may include both the volatile and nonvolatile memories. The nonvolatile memory may be a Read-Only Memory (ROM) , a Programmable ROM (PROM) , an Erasable PROM (EPROM) , an Electrically EPROM (EEPROM) or a flash memory. The volatile memory may be a Random Access Memory (RAM) , and is used as an external high-speed cache. It is exemplarily but unlimitedly described that RAMs in various forms may be adopted, such as a Static RAM (SRAM) , a Dynamic RAM (DRAM) , a Synchronous DRAM (SDRAM) , a Double Data Rate SDRAM (DDRSDRAM) , an Enhanced SDRAM (ESDRAM) , a Synchlink DRAM (SLDRAM) and a Direct Rambus RAM (DRRAM) . The first memory 2402 of the system and method described in the disclosure is intended to include, but not limited to, memories of these and any other proper types.

The first processor 2403 may be an integrated circuit chip with a signal processing capability. In an implementation process, each operation of the method may be completed by an integrated logic circuit of hardware in the first processor 2403 or an instruction in a software form. The first processor 2403 may be a universal processor, a Digital Signal Processor (DSP) , an Application Specific Integrated Circuit (ASIC) , a Field Programmable Gate Array (FPGA) or another Programmable Logic Device (PLD) , discrete gate or transistor logical device and discrete hardware component. Each method, step and logical block diagram disclosed in the embodiments of the disclosure may be implemented or executed. The universal processor may be a microprocessor or the processor may also be any conventional processor and the like. The operations of the method disclosed in combination with the embodiments of the disclosure may be directly embodied to be executed and completed by a hardware decoding processor or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in this field such as a RAM, a flash memory, a ROM, a PROM or EEPROM and a register. The storage medium is located in the first memory 2402. The first processor 2403 reads information in the first memory 2402 and completes the operations of the method in combination with hardware.

It can be understood that these embodiments described in the disclosure may be implemented by hardware, software, firmware, middleware, a microcode or a combination thereof. In case of implementation with the hardware, the processing unit may be implemented in one or more ASICs, DSPs, DSP Devices (DSPDs) , PLDs, FPGAs, universal processors, controllers, microcontrollers, microprocessors, other electronic units configured to execute the functions in the disclosure or combinations thereof. In case of implementation with the software, the technology of the disclosure may be implemented through the modules (for example, processes and functions) executing the functions in the disclosure. The software code may be stored in the memory and executed by the processor. The memory may be implemented inside the processor or outside the processor.

As another embodiment, the first processor 2403 is further configured to run the computer program to execute the method described in any one of the above embodiments.

The embodiments provide an encoder. In the encoder, by adding a first syntax element indicating that recursive subdivision is applied to the current mesh, the displacement coefficients are applied recursively to the previously reconstructed LoD when determining that recursive subdivision is applied to the current mesh, thereby removing redundant displacements at a higher level, reducing coded bits for the displacement coefficients, and further improving the coding and decoding efficiency.

In another embodiment of the disclosure, based on the same inventive concept as the preceding embodiments, FIG. 25 is a schematic structure diagram of a decoder according to an embodiment of the disclosure. As illustrated in FIG. 25, the decoder 250 may include a decoding unit 2501, a second subdivision unit 2502 and a second reconstruction unit 2503.

The decoding unit 2501 is configured to decode a bitstream to determine a value of a first syntax element.

The second subdivision unit 2502 is configured to determine, when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, and subdivide the reconstructed mesh to determine a subdivided mesh of the current level.

The decoding unit 2501 is further configured to decode the bitstream to determine displacement coefficients of the current level.

The second reconstruction unit 2503 is configured to determine a reconstructed mesh of a next level in the current mesh according to the subdivided mesh of the current level and the displacement coefficients of the current level.

In some embodiments, the decoding unit 2501 is further configured to decode the bitstream to determine a reconstructed base mesh. The second subdivision unit 2502 is further configured to subdivide the reconstructed base mesh to determine an initial subdivided mesh. The decoding unit 2501 is further configured to decode the bitstream to determine displacement coefficients of the initial subdivided mesh. The second reconstruction unit 2503 is further configured to determine a reconstructed mesh of a first level in the current mesh according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh.

In some embodiments, the second reconstruction unit 2503 is further configured to, after determining the reconstructed mesh of the next level in the current mesh, take the reconstructed mesh of the next level as a reconstructed mesh of a current level, and continue to subdivide the reconstructed mesh of the current level to determine the subdivided mesh of the current level, until a reconstructed mesh of an L-th level in the current mesh is determined. The value of L is associated with the number of subdivision iterations for the current mesh.

In some embodiments, referring to FIG. 25, the decoder 250 may also include a second determination unit 2504.

The decoding unit 2501 is further configured to decode the bitstream to determine a value of a second syntax element. The second determination unit 2504 is configured to determine the number of subdivision iterations for the current mesh according to the value of the second syntax element.

In some embodiments, the decoding unit 2501 further configured to decode the bitstream to determine decoded displacement coefficients of the current level. The second determination unit 2504 is further configured to preprocess the decoded displacement coefficients of the current level to determine the displacement coefficients of the current level.

In some embodiments, the second determination unit 2504 is further configured to determine a quantization parameter of the current level, perform inverse quantization on the decoded displacement coefficients of the current level according to the quantization parameter of the current level to determine dequantized displacement coefficients of the current level, and perform inverse wavelet transform on the dequantized displacement coefficients of the current level to determine the displacement coefficients of the current level

In some embodiments, the decoding unit 2501 is further configured to decode the bitstream to determine quantization parameter of the current level.

In some embodiments, the second determination unit 2504 is further configured to determine a quantization parameter of a previous level of the current level. The decoding unit 2501 is further configured to decode the bitstream to determine a quantization parameter increment of the current level. The second determination unit 2504 is further configured to determine the quantization parameter of the current level according to the quantization parameter of the previous level and the quantization parameter increment.

In some embodiments, the decoding unit 2501 is further configured to decode the bitstream to determine a two-dimensional image. The second determination unit 2504 is further configured to perform displacement coefficient extraction on the two-dimensional image according to a preset packing mode to obtain the decoded displacement coefficients of the current level.

In some embodiments, the decoding unit 2501 further configured to decode the bitstream to determine a value of a third syntax element. The second determination unit 2504 is further configured to determine the preset packing mode according to the value of the third syntax element.

In some embodiments, the second determination unit 2504 is further configured to perform displacement calculation on vertex coordinate information of the subdivided mesh of the current level according to the displacement coefficients to obtain the reconstructed mesh of the next level in the current mesh.

In some embodiments, the decoding unit 2501 is further configured to decode the bitstream to determine a reconstruction base mesh when the first syntax element indicates that a midpoint subdivision mode is applied to the current mesh. The second subdivision unit 2502 is further configured to subdivide the reconstructed base mesh to determine a subdivided mesh of at least one level in the current mesh. The decoding unit 2501 is further configured to decode the bitstream to determine displacement coefficients of the at least one level in the current mesh. The second reconstruction unit 2503 is further configured to determine a reconstructed mesh of the at least one level in the current mesh according to the subdivided mesh and displacement coefficients of the at least one level .

In some embodiments, the second determination unit 2504 is further configured to determine, when the value of the first syntax element is a first value, that the first syntax element indicates that no subdivision is applied to the current mesh; determine, when the value of the first syntax element is a second value, that the first syntax element indicates that a midpoint subdivision mode is applied to the current mesh; and determine, when the value of the first syntax element is a third value, that the first syntax element indicates that the recursive subdivision mode is applied to the current mesh.

When implemented in form of software function module and sold or used not as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on such an understanding, the embodiments of the disclosure provide a computer-readable storage medium, applied to the decoder 250. The computer-readable storage medium stores a computer program that implements the method of any of the above embodiments when executed by a second processor.

Based on the composition of the decoder 250 and the computer-readable storage medium, FIG. 26 is a schematic diagram of hardware structure of a decoder according to an embodiment of the disclosure. As illustrated in FIG. 26, the decoder 250 may include a second communication interface 2601, a second memory 2602 and a second processor 2603. The components are coupled together by a first bus system 2604. It can be understood that the bus system 2604 is configured to implement connection communication among these components. The bus system 2604 includes a data bus and further includes a power bus, a control bus and a state signal bus. However, for clear description, various buses in FIG. 26 are marked as the bus system 2604.

The second communication interface 2601 is configured to receive and send a signal in a process of receiving and sending information with another external network element.

The second memory 2602 is configured to store a computer program capable of running in the second processor 2603.

The second processor 2603 is configured to run the computer program to execute the following operations: decoding a bitstream to determine a value of a first syntax element; determining, when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, and subdividing the reconstructed mesh to determine a subdivided mesh of the current level; determining displacement coefficients of the current level by decoding the bitstream; and determining, according to the subdivided mesh of the current level and the displacement coefficients of the current level, a reconstructed mesh of a next level in the current mesh.

As another embodiment, the second processor 2603 is further configured to run the computer program to execute the method described in any one of the above embodiments.

It can be understood that the second memory 2602 has similar hardware functions to the first memory 2402 and the second processor 2603 has similar hardware functions to the first processor 2403, which are not elaborated herein.

The embodiments provide a decoder. In the decoder, by adding a first syntax element indicating that recursive subdivision is applied to the current mesh, the displacement coefficients are applied recursively to the previously reconstructed LoD when determining that recursive subdivision is applied to the current mesh, thereby removing redundant displacements at a higher level, reducing coded bits for the displacement coefficients, and further improving the coding and decoding efficiency.

In another embodiment of the disclosure, FIG. 27 is a schematic structure diagram of a codec system according to an embodiment of the disclosure. As illustrated in FIG. 27, the codec system 270 may include an encoder 2701 and a decoder 2702.

In the embodiment of the disclosure, the encoder 2701 may be an encoder as described in any of the above embodiments, and the decoder 2702 may be a decoder as described in any of the above embodiments.

It is to be noted that, in the embodiments of the disclosure, terms "include" and "contain" or any other variation thereof are intended to cover nonexclusive inclusions, so that a process, method, object or device including a series of elements not only includes those elements but also includes other elements which are not clearly listed or further includes elements intrinsic to the process, the method, the object or the device. Under the condition of no more limitations, an element defined by the statement "including a/an. . . " does not exclude existence of the same other elements in a process, method, object or device including the element.

It should also be noted that the embodiments of the disclosure also provide a computer program product, including a computer program or instructions.

In some embodiments, the computer program product may be applied to the encoder/decoder in the embodiments of the disclosure. The computer program or instructions enable a computer to execute corresponding flows implemented by the encoder/decoder in each method of the embodiments of the disclosure, which will not be elaborated here for brief description.

It should also be noted that the embodiments of the disclosure also provide a computer program.

In some embodiments, the computer program may be applied to the encoder/decoder in the embodiments of the disclosure. The computer program, when run on a computer, enables a computer to execute corresponding flows implemented by the encoder/decoder in each method of the embodiments of the disclosure, which will not be elaborated here for brief description.

Those of ordinary skill in the art may realize that the units and algorithm operations of each example described in combination with the embodiments disclosed in the disclosure may be implemented by electronic hardware or a combination of computer software and the electronic hardware. Whether these functions are executed by hardware or software depends on specific applications and design constraints of the technical solutions. Professionals may realize the described functions for each specific application by use of different methods, but such realization shall fall within the scope of the disclosure.

Those skilled in the art may clearly learn about that specific working processes of the device and unit described above may refer to the corresponding processes in the method embodiments and will not be elaborated herein for convenient and brief description.

The sequence numbers of the embodiments of the disclosure are adopted not to represent superiority-inferiority of the embodiments but only for description.

The methods disclosed in some method embodiments provided in the disclosure may be freely combined without conflicts to obtain new method embodiments.

The features disclosed in some product embodiments provided in the disclosure may be freely combined without conflicts to obtain new product embodiments.

The features disclosed in some method or device embodiments provided in the disclosure may be freely combined without conflicts to obtain new method embodiments or device embodiments.

The above is only the specific implementation of the disclosure and not intended to limit the scope of protection of the disclosure. Any variations or replacements apparent to those skilled in the art within the technical scope disclosed by the disclosure shall fall within the scope of protection of the disclosure. Therefore, the scope of protection of the disclosure shall be subject to the scope of protection of the claims.

INDUSTRIAL APPLICABILITY

In the embodiments of the disclosure, at the encoding end, when a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, and the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level; displacement coefficients of the current level are determined; and a reconstructed mesh of a next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level. At the decoding end, a bitstream is decoded to determine a value of a first syntax element; when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh is determined, and the reconstructed mesh of the current level is subdivided to determine a subdivided mesh of the current level; displacement coefficients of the current level are determined by decoding the bitstream; and a reconstructed mesh of a next level in the current mesh is determined according to the subdivided mesh of the current level and the displacement coefficients of the current level. In this way, by adding a first syntax element indicating that recursive subdivision is applied to the current mesh, the displacement coefficients are applied recursively to the previously reconstructed LoD when it is determined that recursive subdivision is applied to the current mesh, thereby removing redundant displacements at a higher level, reducing coded bits for the displacement coefficients, and further improving the coding and decoding efficiency.

Claims

A decoding method, applicable to a decoder, the method comprising:

decoding a bitstream to determine a value of a first syntax element;

determining, when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, and subdividing the reconstructed mesh to determine a subdivided mesh of the current level;

determining displacement coefficients of the current level by decoding the bitstream; and

determining, according to the subdivided mesh of the current level and the displacement coefficients of the current level, a reconstructed mesh of a next level in the current mesh.
The method of claim 1, wherein determining the reconstructed mesh of the current level in the current mesh comprises:

determining a reconstructed base mesh by decoding the bitstream;

subdividing the reconstructed base mesh to determine an initial subdivided mesh;

determining displacement coefficients of the initial subdivided mesh by decoding the bitstream; and

determining, according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh, a reconstructed mesh of a first level in the current mesh.
The method of claim 1, further comprising: after determining the reconstructed mesh of the next level in the current mesh,

taking the reconstructed mesh of the next level as a reconstructed mesh of a current level, and continuing to subdivide the reconstructed mesh of the current level to determine the subdivided mesh of the current level, until a reconstructed mesh of an L-th level in the current mesh is determined;

wherein a value of L is associated with a number of subdivision iterations for the current mesh.
The method of claim 3, further comprising:

decoding the bitstream to determine a value of a second syntax element; and

determining the number of subdivision iterations for the current mesh according to the value of the second syntax element.
The method of any one of claims 1 to 4, wherein determining the displacement coefficients of the current level by decoding the bitstream comprises:

determining decoded displacement coefficients of the current level by decoding the bitstream; and

performing preprocessing on the decoded displacement coefficients of the current level to determine the displacement coefficients of the current level.
The method of claim 5, wherein performing preprocessing on the decoded displacement coefficients of the current level to determine the displacement coefficients of the current level comprises:

determining a quantization parameter of the current level;

performing inverse quantization on the decoded displacement coefficients of the current level according to the quantization parameter of the current level to determine dequantized displacement coefficients of the current level; and

performing inverse wavelet transform on the dequantized displacement coefficients of the current level to determine the displacement coefficients of the current level.
The method of claim 6, wherein determining the quantization parameter of the current level comprises:

determining the quantization parameter of the current level by decoding the bitstream.
The method of claim 6, wherein determining the quantization parameter of the current level comprises:

determining a quantization parameter of a previous level of the current level;

determining a quantization parameter increment of the current level by decoding the bitstream; and

determining the quantization parameter of the current level according to the quantization parameter of the previous level and the quantization parameter increment.
The method of claim 5, wherein determining the decoded displacement coefficients of the current level by decoding the bitstream comprises:

decoding the bitstream to determine a two-dimensional image; and

performing displacement coefficient extraction on the two-dimensional image according to a preset packing mode to obtain the decoded displacement coefficients of the current level.
The method of claim 9, further comprising:

decoding the bitstream to determine a value of a third syntax element; and

determining the preset packing mode according to the value of the third syntax element.
The method of any one of claims 1 to 10, wherein determining, according to the subdivided mesh of the current level and the displacement coefficients of the current level, the reconstructed mesh of the next level in the current mesh comprises:

performing displacement calculation on vertex coordinate information of the subdivided mesh of the current level according to the displacement coefficients to obtain the reconstructed mesh of the next level in the current mesh.
The method of claim 1, further comprising:

when the first syntax element indicates that a midpoint subdivision mode is applied to the current mesh, determining a reconstruction base mesh by decoding the bitstream;

subdividing the reconstructed base mesh to determine a subdivided mesh of at least one level in the current mesh;

determining displacement coefficients of the at least one level by decoding the bitstream; and

determining, according to the subdivided mesh and displacement coefficients of the at least one level, a reconstructed mesh of the at least one level in the current mesh.
The method of claim 12, wherein a number of levels in the at least one level is associated with a number of subdivision iterations for the current mesh.
The method of claim 1, further comprising:

when the value of the first syntax element is a first value, determining that the first syntax element indicates that no subdivision is applied to the current mesh;

when the value of the first syntax element is a second value, determining that the first syntax element indicates that a midpoint subdivision mode is applied to the current mesh; and

when the value of the first syntax element is a third value, determining that the first syntax element indicates that the recursive subdivision mode is applied to the current mesh.
A coding method, applicable to an encoder, the method comprising:

determining, when a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, and subdividing the reconstructed mesh to determine a subdivided mesh of the current level;

determining displacement coefficients of the current level; and

determining, according to the subdivided mesh of the current level and the displacement coefficients of the current level, a reconstructed mesh of a next level in the current mesh.
The method of claim 15, wherein determining the reconstructed mesh of the current level in the current mesh comprises:

determining a base mesh of the current mesh;

performing coding and decoding processing on the base mesh to determine a reconstructed base mesh;

subdividing the reconstructed base mesh to determine an initial subdivided mesh;

determining displacement coefficients of the initial subdivided mesh; and

determining, according to the initial subdivided mesh and the displacement coefficients of the initial subdivided mesh, a reconstructed mesh of a first level in the current mesh.
The method of claim 16, wherein determining the base mesh of the current mesh comprises:

determining an original mesh of the current mesh; and

performing downsampling on the original mesh to obtain the base mesh.
The method of claim 17, wherein determining the displacement coefficients of the initial subdivided mesh comprises:

performing displacement calculation on vertex coordinate information of the initial subdivided mesh and vertex coordinate information of the original mesh to determine the displacement coefficients of the initial subdivided mesh.
The method of claim 16, further comprising:

encoding the base mesh into a bitstream.
The method of claim 15, further comprising: after determining the reconstructed mesh of the next level in the current mesh,

taking the reconstructed mesh of the next level as a reconstructed mesh of a current level, and continuing to subdivide the reconstructed mesh of the current level to determine the subdivided mesh of the current level, until a reconstructed mesh of an L-th level in the current mesh is determined;

wherein a value of L is associated with a number of subdivision iterations for the current mesh.
The method of claim 20, further comprising:

determining the number of subdivision iterations for the current mesh;

determining a value of a second syntax element according to the number of subdivision iterations for the current mesh; and

encoding the value of the second syntax element into a bitstream.
The method of claim 17, wherein determining the displacement coefficients of the current level comprises:

performing displacement calculation on vertex coordinate information of the subdivided mesh of the current level and vertex coordinate information of the original mesh to determine the displacement coefficients of the current level.
The method of any one of claims 15 to 22, further comprising:

performing preprocessing on the displacement coefficients of the current level to determine quantized displacement coefficients of the current level; and

encoding the quantized displacement coefficients of the current level into a bitstream.
The method of claim 23, wherein performing preprocessing on the displacement coefficients of the current level to determine the quantized displacement coefficients of the current level comprises:

performing wavelet transform on the displacement coefficients of the current level to determine wavelet transform coefficients of the current level; and

quantizing the wavelet transform coefficients of the current level to determine the quantized displacement coefficients of the current level.
The method of claim 24, wherein quantizing the wavelet transform coefficients of the current level to determine the quantized displacement coefficients of the current level comprises:

determining a quantization parameter of the current level;

quantizing the wavelet transform coefficients of the current level according to the quantization parameter of the current level to determine the quantized displacement coefficients of the current level.
The method of claim 25, further comprising:

encoding the quantization parameter of the current level into the bitstream.
The method of claim 25, further comprising:

determining a quantization parameter of a previous level of the current level;

determining a quantization parameter increment of the current level according to the quantization parameter of the previous level and the quantization parameter of the current level; and

encoding the quantization parameter increment of the current level into the bitstream.
The method of claim 23, further comprising:

determining quantized displacement coefficients of at least one level in the current mesh, wherein the at least one level comprises the current level;

packing the quantized displacement coefficients of the at least one level into a two-dimensional image according to a preset packing mode; and

encoding the two-dimensional image into the bitstream.
The method of claim 28, further comprising:

determining a value of a third syntax element according to the preset packing mode; and

encoding the value of the third syntax element into the bitstream.
The method of any one of claims 15 to 29, wherein determining, according to the subdivided mesh of the current level and the displacement coefficients of the current level, the reconstructed mesh of the next level in the current mesh comprises:

performing displacement calculation on vertex coordinate information of the subdivided mesh of the current level according to the displacement coefficients to obtain the reconstructed mesh of the next level in the current mesh.
The method of any one of claims 15 to 29, further comprising:

determining a value of a first syntax element, wherein the first syntax element indicates whether the recursive subdivision mode is applied to the current mesh; and

encoding the value of the first syntax element into a bitstream.
The method of claim 31, wherein determining the value of the first syntax element comprises:

when no subdivision is applied to the current mesh, determining the value of the first syntax element to be a first value;

when a midpoint subdivision mode is applied to the current mesh, determining the value of the first syntax element to be a second value; and

when the recursive subdivision mode is applied to the current mesh, determining the value of the first syntax element to be a third value.
The method of claim 32, further comprising:

when the midpoint subdivision mode is applied to the current mesh, determining a reconstruction base mesh;

subdividing the reconstructed base mesh to determine a subdivided mesh of at least one level in the current mesh;

determining displacement coefficients of the at least one level; and

determining, according to the subdivided mesh and displacement coefficients of the at least one level, a reconstructed mesh of the at least one level in the current mesh.
The method of claim 33, wherein a number of levels in the at least one level is associated with a number of subdivision iterations for the current mesh.
A bitstream, generated by encoding according to information to be encoded, wherein the information to be encoded includes at least one of:

a base mesh of a current mesh, displacement coefficients of at least one level in the current mesh, a quantization parameter of at least one level in the current mesh, a quantization parameter increment of at least one level in the current mesh, a value of a first syntax element, a value of a second syntax element or a value of a third syntax element;

wherein the value of the first syntax element indicates whether a recursive subdivision mode is applied to the current mesh, the value of the second syntax element indicates a number of subdivision iterations for the current mesh, and the value of the third syntax element indicates a preset packing mode.
An encoder, comprising:

a first subdivision unit, configured to determine, when a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, and subdivide the reconstructed mesh to determine a subdivided mesh of the current level;

a first determination unit, configured to determine displacement coefficients of the current level; and

a first reconstruction unit, configured to determine a reconstructed mesh of a next level in the current mesh according to the subdivided mesh of the current level and the displacement coefficients of the current level.
An encoder, comprising a first processor and a first memory for storing a computer program executable by the first processor;

wherein the first processor, when executing the computer program, performs the method of any one of claims 15 to 34.
A decoder, comprising a decoding unit, a second subdivision unit and a second reconstruction unit, wherein:

the decoding unit is configured to decode a bitstream to determine a value of a first syntax element;

the second subdivision unit is configured to determine, when the first syntax element indicates that a recursive subdivision mode is applied to a current mesh, a reconstructed mesh of a current level in the current mesh, and subdivide the reconstructed mesh to determine a subdivided mesh of the current level;

the decoding unit is further configured to decode the bitstream to determine displacement coefficients of the current level; and

the second reconstruction unit is configured to determine a reconstructed mesh of a next level in the current mesh according to the subdivided mesh of the current level and the displacement coefficients of the current level.
A decoder, comprising a second processor and a second memory for storing a computer program executable by the second processor;

wherein the second processor, when executing the computer program, performs the method of any one of claims 1 to 14.
A computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by at least one processor, implements the method of any one of claims 1 to 14, or the method of any one of claims 15 to 34.
A computer program product, comprising a computer program or instructions, wherein the computer program or instructions, when executed by at least one processor, implement the method of any one of claims 1 to 14, or the method of any one of claims 15 to 34.