WO2022267256A1 - Vr图像的压缩传输方法、系统 - Google Patents

Vr图像的压缩传输方法、系统 Download PDF

Info

Publication number
WO2022267256A1
WO2022267256A1 PCT/CN2021/121005 CN2021121005W WO2022267256A1 WO 2022267256 A1 WO2022267256 A1 WO 2022267256A1 CN 2021121005 W CN2021121005 W CN 2021121005W WO 2022267256 A1 WO2022267256 A1 WO 2022267256A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
video image
compression
compressed
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/121005
Other languages
English (en)
French (fr)
Inventor
于东壮
崔新宇
吴健
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Pico Technology Co Ltd
Original Assignee
Qingdao Pico Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Pico Technology Co Ltd filed Critical Qingdao Pico Technology Co Ltd
Priority to EP21946739.6A priority Critical patent/EP4325866A4/en
Priority to US17/886,063 priority patent/US11748915B2/en
Publication of WO2022267256A1 publication Critical patent/WO2022267256A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194—Transmission of image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present disclosure relates to the technical field of virtual reality, and more specifically, to a method and system for compressing and transmitting VR images.
  • virtual reality systems Due to the advancement of technology and the diversified development of market demand, virtual reality systems are becoming more and more common and applied in many fields, such as computer games, health and safety, industry and education and training. To give a few examples, mixed virtual reality systems are being integrated into mobile communication devices, game consoles, personal computers, movie theaters, theme parks, university laboratories, student classrooms, hospital exercise gyms and other corners of life.
  • VR streaming and VR live broadcasting The core of VR streaming and VR live broadcasting is to encode and transmit VR images acquired in different places to the local area, unpack, recombine, decode and redisplay them. Therefore, it is of great significance to ensure the clarity of images while reducing the transmission traffic.
  • most videos are encoded and compressed by h264 and h265. Although they also have good effects, due to the large image size and high resolution, there is still the problem of low transmission efficiency, which affects the operating experience of VR live broadcast or VR streaming. .
  • the purpose of this disclosure is to provide a compression transmission method and system for VR images, so as to solve the problem that the existing videos mostly use h264 and h265 encoding and compression. If the transmission rate is too high, there is still the problem of low transmission efficiency, which affects the operating experience of VR live broadcast or VR streaming.
  • the present disclosure provides a VR image compression transmission method, which includes:
  • the server Acquiring a video image to be sent by the server, and performing localized compression on the video image according to a compression range to form a first compressed video image; the compression range is determined by transmitting a sample image;
  • the process of determining the compression range through the transmission samples includes:
  • the user's eyes correspond to the left and right lens barrels worn by the VR headset;
  • the size of the sample image is adjusted so that the blind area exits the user's field of vision.
  • the process of performing localized compression on the video image according to the compression range to form the first compressed video image includes:
  • the compressed edge area and the central restored area are used as a first compressed video image.
  • the preset multiple is 2 times.
  • the first compressed video image when performing video coding processing on the first compressed video image to form a transmission video image, it also includes:
  • the video image is saved, and the compression range and compression ratio of the first compressed video image are recorded in the self-defined information frame of the video image.
  • the video coding adopts h264 or h265 video.
  • the resolution of the second compressed video is the same as that of the first compressed video.
  • the process of decompressing the second compressed video image to form a display video image includes:
  • the real edge area is fitted with the real central area to form a display video image.
  • the present disclosure also provides a VR image compression transmission system to realize the aforementioned VR image compression transmission method, which includes:
  • the edge compression module is configured to obtain the video image to be sent by the server, and perform localized compression on the video image according to the compression range to form the first compressed video image; the compression range is determined by transmitting the sample image;
  • a video encoding module configured to perform video encoding processing on the first compressed video image to form a transmission video image
  • a video decoding module configured to transmit the transmission video to the client, and perform video decoding processing on the transmission video image to restore the transmission video image to a second compressed video image;
  • the video restoration module is configured to decompress the second compressed video image to form a presentation video image.
  • the edge compression module includes an image shader, wherein,
  • a first image coloring program is set in the image shader, and the first image coloring program is configured to perform a preset multiple compression sampling on the edge area divided according to the compression range to form a compressed edge area.
  • the VR image compression transmission method and system provided by the present disclosure first acquires the video image to be sent by the server, and performs localized compression on the video image according to the compression range to form the first compressed video image, Then perform video encoding processing on the first compressed video image to form a transmission video image, then transmit the transmission video to the client, and perform video decoding processing on the transmission video image to restore the transmission video image to the second compressed video image , and then decompress the second compressed video image to form a display video image, that is, use d3d or opengl calculation as the medium, and use an image coloring program to compress the image data within the compression range of the edge, thereby compressing the edge image Resolution to reduce the resolution of the overall image, thereby reducing the image size, increasing the transmission rate, and thus improving the stability of VR live streaming and VR streaming.
  • FIG. 1 is a flowchart of a method for compressing and transmitting VR images according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of a compression range involved in a VR image compression transmission method according to an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of a system for compressing and transmitting VR images according to an embodiment of the disclosure.
  • the present disclosure provides a VR image compression transmission method and system, and specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
  • Fig. 1 is an example of the compression transmission method of VR images in the embodiment of the disclosure
  • Fig. 2 is an example of the compression transmission system of VR images in the embodiment of the disclosure sex mark.
  • the method for compressing and transmitting VR images according to the embodiments of the present disclosure provided by the present disclosure includes:
  • S1 Obtain a video image to be sent by the server, and perform localized compression on the video image according to the compression range to form a first compressed video image; the compression range is determined by transmitting sample images;
  • S3 Transmit the transmission video to the client, and perform video decoding processing on the transmission video image to restore the transmission video image to a second compressed video image;
  • step S1 is to obtain the video image to be sent by the server, and perform localized compression on the video image according to the compression range to form the first compressed video image; the compression range is determined by transmitting the sample image; wherein, S0: by The transmission sample determines the compression range process, including:
  • S02 capture the sample image sent by the server through the left lens barrel and the right lens barrel and marked with the area of the blind spot;
  • S04 Adjust the size of the sample image so that the blind spot area is out of the user's field of vision.
  • the process of performing localized compression on the video image according to the compression range to form the first compressed video image includes:
  • S11 Perform area locking on the video image according to the compression range to divide the video image into an edge area and a central area;
  • the compression range is first determined through step S0.
  • the video generation sender is the server, and the VR head Wear it as the client, after the server connects with the client, the user can wear the VR headset normally, and the boot program of the client prompts the user: confirm that the wearing is normal, and then click the confirmation button of the VR headset to confirm the left and right sides of the edge; then the server First, preset a position on the left and right sides.
  • the resolution of the video image is 1920
  • the client prompts the user to click the button "+” or the button “-” to increase or decrease the red area until the red area is only visible Click Confirm when a little or nothing is visible and does not affect the overall image look and feel.
  • the client sends a "+" or "-” command to the server, and the server adjusts the initial border of the red part according to the command until the user clicks OK to record the left and right compression borders, that is, repeat the process for the upper and lower borders to confirm, thereby
  • the compression range is obtained, that is to say, the finally confirmed red area in this embodiment is the compression range, so that the video image to be sent by the server is locally compressed according to the compression range to form the first compressed video image.
  • the preset multiplier described in step S1 is 2 times.
  • the first image coloring program of the image shader is written.
  • the first image coloring program is a shader program, and the shader program uses sampling to reconstruct an image.
  • the horizontal direction is 0-
  • the 480 part is the edge area, which is sampled every 2 pixels in this area, among which the cardinal points are sampled in the odd lines, and the even points are sampled in the even lines;
  • the formed first compressed video image is reconstructed to a size of 1440*1440.
  • the compressed part has a size of 240 for the left, right, upper, lower, and upper edge areas respectively, and the central area of the uncompressed part in the middle is 960.
  • the first compressed video image becomes 0.5625 times of the original video image, and the volume is significantly reduced.
  • step S2 is to perform video encoding processing on the first compressed video image to form a transmission video image, and while performing video encoding processing on the first compressed video image to form a transmission video image, it also includes :
  • the video image is saved, and the compression range and the compression range of the first compressed video image are recorded in the custom information frame of the video image.
  • the video encoding adopts h264 or h265 video encoding.
  • the first compressed video image is video encoded, such as h264 or h265 video, and the boundary information (compression range) and compression ratio are recorded in the SEI (custom information) frame at the same time, so as to restore the video image later.
  • SEI custom information
  • you save the original image you can record the boundary information and compression ratio in the same folder or name the bitmap according to the beginning of the original resolution_left and right boundary_left and right compression ratio_upper and lower boundaries_upper and lower compression ratio. For example 1920#1920_480_2_480_2_custom name.bmp.
  • step S3 is to transmit the transmission video to the client, and perform video decoding processing on the transmission video image to restore the transmission video image to the second compressed video image, that is, after the first compression of the compressed edge area
  • video decoding needs to be performed after being transmitted to the client to obtain the decoded second compressed video image
  • the decoded second compressed video image is the same as the first compressed video image before video encoding
  • the video images are completely the same, that is, the resolution of the second compressed video is the same as that of the first compressed video, so as to prevent video distortion.
  • step S4 is to decompress the second compressed video image to form a display video image
  • the process of decompressing the second compressed video image to form a display video image includes:
  • S42 Perform differential sampling on the compressed part by the second image coloring program to restore the compressed part to a real edge area, and perform point-to-point sampling on the original part to form a real central area;
  • Step S4 is to decompress the edge-compressed second compressed video image (equal to the first compressed video image).
  • the boundary information (compression range) and the original Image resolution if it is a file that is decompressed and stored, it can be extracted from the video stream SEI frame or extracted from the picture name.
  • Decompression needs to write the second image coloring program in the image shader.
  • the second image The coloring program is a shader program, which uses sampling to reconstruct the picture, that is, for the compressed part, the pigment of two points is expanded from the original image by using difference sampling, that is to say, the sampler is set to difference sampling, and the compressed part is restored to the original size.
  • the sampler is set to point-to-point sampling; wherein, no matter the first image coloring program playing the role of compression or the second image coloring program playing the role of decompression, opengl or d3d technology is used as Call the medium of the shader program to ensure the running speed.
  • the user can adjust the size of the compression range (boundary information) at any time, and then perform the localized compression in steps S1-S4 according to the compressed range and the localized compression for the localized compression
  • the decompression operation so that the transmission video is transmitted at a small resolution, the transmission rate is increased without affecting the image quality and clarity of the video image, and the quality and stability of real-time streaming or real-time live broadcasting are guaranteed.
  • the VR image compression transmission method first obtains the video image to be sent by the server, and performs localized compression on the video image according to the compression range to form the first compressed video image, and then compresses the first compressed video image Perform video encoding processing to form a transmission video image, then transmit the transmission video to the client, and perform video decoding processing on the transmission video image to restore the transmission video image to a second compressed video image, and then compress the second
  • the video image is decompressed to form a display video image, that is, using d3d or opengl computing as the medium, and using the image coloring program to compress the image data within the compression range of the edge, thereby reducing the overall image by compressing the resolution of the edge image resolution, thereby reducing the image size and increasing the transmission rate, thereby improving the stability of VR live streaming and VR streaming.
  • the present disclosure also provides a VR image compression transmission system 100 to implement the aforementioned VR image compression transmission method, which includes:
  • the edge compression module 101 is configured to acquire the video image to be sent by the server, and perform localized compression on the video image according to the compression range to form the first compressed video image; the compression range is determined by transmitting the sample image;
  • the video encoding module 102 is configured to perform video encoding processing on the first compressed video image to form a transmission video image;
  • the video decoding module 103 is configured to transmit the transmission video to the client, and perform video decoding processing on the transmission video image to restore the transmission video image to the second compressed video image;
  • the video restoration module 104 is configured to decompress the second compressed video image to form a presentation video image.
  • the edge compression module 101 includes an image shader, wherein,
  • a first image coloring program is set in the image shader, and the first image coloring program is configured to perform a preset multiplier compression sampling on the edge area divided according to the compression range to form a compressed edge area;
  • the image shader is also included in the video restoration module 104, that is, the image shader is also included in the video restoration module, and the image shader also includes a second image rendering program, and the second image rendering program is set to the second
  • the compressed portion in the compressed video image is differentially sampled to restore the compressed portion to a real edge area, and the original portion in the second compressed video image is sampled point-to-point to form a real center area, thereby forming a display video image, so that the The video image sent by the server is completely restored and displayed, which improves the transmission rate without affecting the stability of the video image.
  • the VR image compression transmission system first obtains the video image to be sent by the server through the edge compression module 101, and performs localized compression on the video image according to the compression range to form the first compressed video image, and then carry out video encoding processing to the first compressed video image by the video encoding module 102 to form a transmission video image, then transmit the transmission video to the client by the video decoding module 103, and perform video decoding processing on the transmission video image to The transmission video image is restored to a second compressed video image, and then the second compressed video image is decompressed by the video restoration module 104 to form a display video image, that is, using d3d or opengl calculation as a medium, and using an image coloring program to The image data within the compression range of the edge is compressed, thereby reducing the resolution of the overall image by compressing the resolution of the edge image, thereby reducing the image volume, increasing the transmission rate, and improving the stability of VR live broadcast and VR streaming.
  • each module or each step of the above-mentioned disclosure can be realized by a general-purpose computing device, and they can be concentrated on a single computing device, or distributed in a network composed of multiple computing devices In fact, they can be implemented in program code executable by a computing device, and thus, they can be stored in a storage device to be executed by a computing device, and in some cases, can be executed in an order different from that shown here. Or described steps, or they are fabricated into individual integrated circuit modules, or multiple modules or steps among them are fabricated into a single integrated circuit module for implementation. As such, the present disclosure is not limited to any specific combination of hardware and software.
  • the VR image compression transmission method has the following beneficial effects: the resolution of the overall image is reduced by compressing the resolution of the edge image, thereby reducing the image volume, increasing the transmission rate, and further improving VR Stability of live streaming and VR streaming.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本公开提供一种VR图像的压缩传输方法、系统，首先获取服务器待发送的视频图像，并根据压缩范围对所述视频图像进行定域压缩以形成第一压缩视频图像，而后对第一压缩视频图像进行视频编码处理以形成传输视频图像，再将该传输视频传输至客户端，并对该传输视频图像进行视频解码处理以将该传输视频图像还原为第二压缩视频图像，再对该第二压缩视频图像进行解压缩处理以形成展示视频图像，即使用d3d或opengl计算为介质，利用图像着色程序对边缘的压缩范围内的图像数据进行压缩，从而通过压缩边缘图像的分辨率来减小整体图像的分辨率，从而减小图像体积，提高传输速率，进而提高VR直播、VR串流的稳定性。

Description

VR图像的压缩传输方法、系统 技术领域
本公开涉及虚拟现实技术领域，更为具体地，涉及一种VR图像的压缩传输方法、系统。
背景技术
由于科技的进步，市场需求的多元化发展，虚拟现实系统正变得越来越普遍，应用在许多领域，如电脑游戏，健康和安全，工业和教育培训。举几个例子，混合虚拟现实系统正在被整合到移动通讯设备、游戏机、个人电脑、电影院，主题公园，大学实验室，学生教室，医院锻炼健身室等生活各个角落。
VR串流、VR直播的核心都是将异地获取的VR图像经过编码传输发送到本地，进行拆包重组和解码后重新显示，因此在降低发送流量的同时保证图像的清晰度具有重要意义。目前视频多使用h264，h265编码压缩，虽然也具有不错的效果，但是由于图像体积过大，分辨率过高，仍然存在着传输效率低的问题，以至于影响VR直播或VR串流的操作体验。
因此，亟需一种减少图像、视频传输体积、提高视频传输速率的VR图像的压缩传输方法、系统。
发明内容
鉴于上述问题，本公开的目的是提供一种VR图像的压缩传输方法、系统，以解决现有目前视频多使用h264，h265编码压缩，虽然也具有不错的效果，但是由于图像体积过大，分辨率过高，仍然存在着传输效率低的问题，以至于影响VR直播或VR串流的操作体验的问题。
本公开提供的一种VR图像的压缩传输方法，其中，包括：
获取服务器待发送的视频图像，并根据压缩范围对所述视频图像进行定域压缩以形成第一压缩视频图像；所述压缩范围通过传输样本图像确定；
对所述第一压缩视频图像进行视频编码处理以形成传输视频图像；
将所述传输视频传输至客户端，并对所述传输视频图像进行视频解码处理以将所述传输视频图像还原为第二压缩视频图像；
对所述第二压缩视频图像进行解压缩处理以形成展示视频图像。
优选地，通过所述传输样本确定所述压缩范围的过程，包括：
通过引导程序使用户的双眼与VR头戴的左镜筒、右镜筒相对应；
通过所述左镜筒、右镜筒捕捉由所述服务器发送的标记有盲区范围区域的样本图像；
调节所述样本图像的大小使所述盲区范围区域退出所述用户的视野。
优选地，根据压缩范围对所述视频图像进行定域压缩以形成第一压缩视频图像的过程，包括：
按照所述压缩范围对所述视频图像进行区域锁定以将所述视频图像划分为边缘区域和中心区域；
通过第一图像着色程序对所述边缘区域进行预设倍数压缩采样形成压缩边缘区域；对所述中心区域进行真实采样以形成中心还原区域；
将所述压缩边缘区域与所述中心还原区域作为第一压缩视频图像。
优选地，所述预设倍数为2倍。
优选地，对所述第一压缩视频图像进行视频编码处理以形成传输视频图像的同时，还包括：
保存所述视频图像，并将所述第一压缩视频图像的压缩范围和压缩比例记录在所述视频图像的自定义信息帧内。
优选地，所述视频编码采用h264或h265视频。
优选地，所述第二压缩视频与所述第一压缩视频的分辨率相同。
优选地，对所述第二压缩视频图像进行解压缩处理以形成展示视频图像的过程，包括：
获取所述第二压缩视频图像的压缩部分与原始部分；其中，所述压缩部分与所述压缩边缘区域相一致；所述原始部分与所述中心还原区域相一致；
通过第二图像着色程序对所述压缩部分进行差值采样以将所述压缩部分还原为真实边缘区域，对所述原始部分进行点对点采样已形成真实中心区域；
将所述真实边缘区域与所述真实中心区域进行拟合处理以形成展示视频图像。
本公开还提供一种VR图像的压缩传输系统，实现前述的VR图像的压缩传输方法，其中，包括：
边缘压缩模块，设置为获取服务器待发送的视频图像，并根据压缩范围对所述视频图像进行定域压缩以形成第一压缩视频图像；所述压缩范围通过传输样本图像确定；
视频编码模块，设置为对所述第一压缩视频图像进行视频编码处理以形成传输视频图像；
视频解码模块，设置为将所述传输视频传输至客户端，并对所述传输视频图像进行视频解码处理以将所述传输视频图像还原为第二压缩视频图像；
视频还原模块，设置为对所述第二压缩视频图像进行解压缩处理以形成展示视频图像。
优选地，所述边缘压缩模块包括图像着色器，其中，
在所述图像着色器中设置有第一图像着色程序，所述第一图像着色程序设置为对根据所述压缩范围划分的边缘区域进行预设倍数压缩采样以形成压缩边缘区域。
从上面的技术方案可知，本公开提供的VR图像的压缩传输方法、系统，首先获取服务器待发送的视频图像，并根据压缩范围对所述视频图像 进行定域压缩以形成第一压缩视频图像，而后对第一压缩视频图像进行视频编码处理以形成传输视频图像，再将该传输视频传输至客户端，并对该传输视频图像进行视频解码处理以将该传输视频图像还原为第二压缩视频图像，再对该第二压缩视频图像进行解压缩处理以形成展示视频图像，即使用d3d或opengl计算为介质，利用图像着色程序对边缘的压缩范围内的图像数据进行压缩，从而通过压缩边缘图像的分辨率来减小整体图像的分辨率，从而减小图像体积，提高传输速率，进而提高VR直播、VR串流的稳定性。
附图说明
图1为根据本公开实施例的VR图像的压缩传输方法的流程图；
图2为根据本公开实施例的VR图像的压缩传输方法中涉及的压缩范围示意图；
图3为根据本公开实施例的VR图像的压缩传输系统的示意图。
具体实施方式
目前视频多使用h264，h265编码压缩，虽然也具有不错的效果，但是由于图像体积过大，分辨率过高，仍然存在着传输效率低的问题，以至于影响VR直播或VR串流的操作体验。
针对上述问题，本公开提供一种VR图像的压缩传输方法、系统，以下将结合附图对本公开的具体实施例进行详细描述。
为了说明本公开提供的VR图像的压缩传输方法、系统，图1对本公开实施例的VR图像的压缩传输方法进行了示例性标示；图2对本公开实施例的VR图像的压缩传输系统进行了示例性标示。
以下示例性实施例的描述实际上仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。对于相关领域普通技术人员已知的技术和设备可能不作详细讨论，但在适当情况下，所述技术和设备应当被视为说明书的一部分。
如图1所示，本公开提供的本公开实施例的VR图像的压缩传输方法，包括：
S1：获取服务器待发送的视频图像，并根据压缩范围对该视频图像进行定域压缩以形成第一压缩视频图像；该压缩范围通过传输样本图像确定；
S2：对该第一压缩视频图像进行视频编码处理以形成传输视频图像；
S3：将该传输视频传输至客户端，并对该传输视频图像进行视频解码处理以将该传输视频图像还原为第二压缩视频图像；
S4：对该第二压缩视频图像进行解压缩处理以形成展示视频图像。
如图1所示，步骤S1为获取服务器待发送的视频图像，并根据压缩范围对视频图像进行定域压缩以形成第一压缩视频图像；该压缩范围通过传输样本图像确定；其中，S0：通过该传输样本确定压缩范围的过程，包括：
S01：通过引导程序使用户的双眼与VR头戴的左镜筒、右镜筒相对应；
S02：通过该左镜筒、右镜筒捕捉由该服务器发送的标记有盲区范围区域的样本图像；
S04：调节该样本图像的大小使该盲区范围区域退出该用户的视野。
根据压缩范围对该视频图像进行定域压缩以形成第一压缩视频图像的过程，包括：
S11：按照该压缩范围对该视频图像进行区域锁定以将该视频图像划分为边缘区域和中心区域；
S12：通过第一图像着色程序对该边缘区域进行预设倍数压缩采样形成压缩边缘区域；对该中心区域进行真实采样以形成中心还原区域；
S13：将该压缩边缘区域与该中心还原区域作为第一压缩视频图像。
具体的，如图2所示，由于VR头戴是通过两个近似圆形的左镜筒、右镜筒看一个长方形的屏幕，正常佩戴时长方形屏幕的边缘是看不到的 (图中的黑色部分)，仅仅在视线尽力偏移或头戴急速朝一侧运动时才能看到边缘部分的图像(黑色部分)，此时由于视线异常或由于急速运动，对该部分图像的显示效果很难正常感受，因此可以通过压缩边缘图像的分辨率来减小整体图像的分辨率，从而减小图像体积，故首先通过步骤S0确定压缩范围，在本实施例中，视频产生发送方为服务器，VR头戴为客户端，服务器端与客户端连通后，使用户正常佩戴VR头戴，客户端的引导程序提示用户：确定佩戴正常，观察正常后点击VR头戴的确认按钮，确定边缘左右侧；而后服务器首先预设一个左右侧的位置，比如视频图像为1920的分辨率，预设在480和1440(1920-480＝1440)的位置，0-480和1440-1920分别渲染成纯红色，视频图像的其他部分不变，并将该图像传输到客户端显示，记录原始图像分辨率；此时客户端提示用户点击按钮“+”或按钮“-”以增加或减小红色区域，直至红色区域仅仅看见少许或看不到且不影响图像整体观感时点击确认。并且客户端发送“+”或“-”的指令到服务器端，服务器根据该指令调节红色部分的起始边界，直至用户点击确认，记录左右压缩边界，即上下边界重复该过程，进行确认，从而获取压缩范围，也就是说在本实施例中最终确认的红色区域即为压缩范围，从而根据该压缩范围对服务器待发送的视频图像进行定域压缩以形成第一压缩视频图像。
并且，在本实施例中，步骤S1中所述的预设倍数为2倍，具体的，在本实施例中，确定压缩范围后，编写图像着色器的第一图像着色程序，在本实施例中，该第一图像着色程序为shader程序，该shader程序利用采样重建图片，以1920*1920分辨率的视频图像为例，假设左右上下压缩边界都是480，采用2倍压缩，则横向0-480部分为边缘区域，对该区域每2个像素采样一次，其中奇数行采样基数点，偶数行采样偶数点；480-1440部分为中心区域，对该区域进行一比一真实采样；经过压缩后形成的第一压缩视频图像被重构为1440*1440大小，此时压缩后部分分别是左右上下的边缘区域各为240，中间未压缩部分的中心区域为960。经过处理，该第一压缩视频图像变为原来的视频图像的0.5625倍，体积缩小明显。
在图1所示的实施例中，步骤S2为对第一压缩视频图像进行视频编码处理以形成传输视频图像，在对第一压缩视频图像进行视频编码处理以形成传输视频图像的同时，还包括：
保存视频图像，并将第一压缩视频图像的压缩范围和压缩范围记录在视频图像的自定义信息帧内。
具体的，在将步骤S1形成的第一压缩视频图像传输至客户端的过程中，需对该第一压缩视频图像进行视频编码，在本实施例中该视频编码采用h264或h265视频编码，如果对第一压缩视频图像进行视频编码，比如h264或h265视频，则同时将边界信息(压缩范围)和压缩比例记录在SEI(自定义信息)帧内，以便于后期还原视频图像。同时，若保存原始图像，可以将边界信息和压缩比例记录在同一文件夹下或在bitmap命名时按原分辨率_左右边界_左右压缩比例_上下边界_上下压缩比例的开头进行命名。例如1920#1920_480_2_480_2_自定义名称.bmp。
如图1所示，步骤S3为将传输视频传输至客户端，并对传输视频图像进行视频解码处理以将传输视频图像还原为第二压缩视频图像，即在对压缩了边缘区域的第一压缩视频图像进行视频编码后，传输到客户端后需进行视频解码以获取解码后的第二压缩视频图像，并且在本实施例中，解码后的第二压缩视频图像与视频编码前的第一压缩视频图像是完全相同的，即该第二压缩视频与该第一压缩视频的分辨率相同，以防止视频失真。
如图1所示，步骤S4为对第二压缩视频图像进行解压缩处理以形成展示视频图像，其中，对该第二压缩视频图像进行解压缩处理以形成展示视频图像的过程，包括：
S41：获取该第二压缩视频图像的压缩部分与原始部分；其中，该压缩部分与该压缩边缘区域相一致；该原始部分与该中心还原区域相一致；
S42：通过第二图像着色程序对该压缩部分进行差值采样以将该压缩部分还原为真实边缘区域，对该原始部分进行点对点采样已形成真实中心区域；
S43：将该真实边缘区域与该真实中心区域进行拟合处理以形成展示视频图像；
步骤S4即对进行边缘压缩了的第二压缩视频图像(等同于第一压缩视频图像)进行解压缩，具体的，在实时串流或实时直播时，已经记录了边界信息(压缩范围)和原始图像分辨率，如果是解压缩存储的文件可以从视频流SEI帧内提取或从图片命名中提取，解压缩需要编写图像着色器中的第二图像着色程序，在本实施例中该第二图像着色程序为shader程序，该程序利用采样重建图片，即对于压缩部分利用差值采样将两个点的色素从原图进行扩展，也就是说设置采样器为差值采样，将压缩过的部分恢复到原始大小。对原始部分(图像未压缩部分)，设置采样器为点对点采样；其中，无论是起到压缩作用的第一图像着色程序还是起到解压缩作用的第二图像着色程序均采用opengl或d3d技术为调用shader程序的媒介，从而确保运行速度。
并且，需要说明的是，在实时串流或直播过程中，用户可以随时调节压缩范围(边界信息)的大小，而后根据该压缩范围进行步骤S1-步骤S4的定域压缩和针对该定域压缩的解压缩操作，如此使传输视频以小分辨率进行传输，提高传输速率且不影响视频图像的画质和清晰度，保证实时串流或实时直播的质量和稳定性。
如上所述，本公开提供的VR图像的压缩传输方法，首先获取服务器待发送的视频图像，并根据压缩范围对视频图像进行定域压缩以形成第一压缩视频图像，而后对第一压缩视频图像进行视频编码处理以形成传输视频图像，再将该传输视频传输至客户端，并对该传输视频图像进行视频解码处理以将该传输视频图像还原为第二压缩视频图像，再对该第二压缩视频图像进行解压缩处理以形成展示视频图像，即使用d3d或opengl计算为介质，利用图像着色程序对边缘的压缩范围内的图像数据进行压缩，从而通过压缩边缘图像的分辨率来减小整体图像的分辨率，从而减小图像体积，提高传输速率，进而提高VR直播、VR串流的稳定性。
如图3所示，本公开还提供一种VR图像的压缩传输系统100，实现前述的VR图像的压缩传输方法，其中，包括：
边缘压缩模块101，设置为获取服务器待发送的视频图像，并根据压缩范围对视频图像进行定域压缩以形成第一压缩视频图像；该压缩范围通过传输样本图像确定；
视频编码模块102，设置为对第一压缩视频图像进行视频编码处理以形成传输视频图像；
视频解码模块103，设置为将传输视频传输至客户端，并对传输视频图像进行视频解码处理以将传输视频图像还原为第二压缩视频图像；
视频还原模块104，设置为对第二压缩视频图像进行解压缩处理以形成展示视频图像。
其中，该边缘压缩模块101包括图像着色器，其中，
在该图像着色器中设置有第一图像着色程序，该第一图像着色程序设置为对根据该压缩范围划分的边缘区域进行预设倍数压缩采样以形成压缩边缘区域；
该图像着色器也包括在视频还原模块104中，即在该视频还原模块中也包括图像着色器，在该图像着色器还包括第二图像着色程序，该第二图像着色程序设置为对第二压缩视频图像中的压缩部分进行差值采样以将该压缩部分还原为真实边缘区域，对第二压缩视频图像中的原始部分进行点对点采样已形成真实中心区域，从而形成展示视频图像，如此对该服务器发送的视频图像进行完全还原展示，提高传输速率且不影响视频图像的稳定性。
通过上述实施方式可以看出，本公开提供的VR图像的压缩传输系统，首先通过边缘压缩模块101获取服务器待发送的视频图像，并根据压缩范围对视频图像进行定域压缩以形成第一压缩视频图像，而后通过视频编码模块102对第一压缩视频图像进行视频编码处理以形成传输视频图像，再通过视频解码模块103将该传输视频传输至客户端，并对该传输视频图像 进行视频解码处理以将该传输视频图像还原为第二压缩视频图像，再通过视频还原模块104对该第二压缩视频图像进行解压缩处理以形成展示视频图像，即使用d3d或opengl计算为介质，利用图像着色程序对边缘的压缩范围内的图像数据进行压缩，从而通过压缩边缘图像的分辨率来减小整体图像的分辨率，从而减小图像体积，提高传输速率，进而提高VR直播、VR串流的稳定性。
如上参照附图以示例的方式描述了根据本公开提出的VR图像的压缩传输方法、系统。但是，本领域技术人员应当理解，对于上述本公开所提出的VR图像的压缩传输方法、系统，还可以在不脱离本公开内容的基础上做出各种改进。因此，本公开的保护范围应当由所附的权利要求书的内容确定。
本实施例中的具体示例可以参考上述实施例及示例性实施方式中所描述的示例，本实施例在此不再赘述。
显然，本领域的技术人员应该明白，上述的本公开的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本公开不限制于任何特定的硬件和软件结合。
以上所述仅为本公开的优选实施例而已，并不设置为限制本公开，对于本领域的技术人员来说，本公开可以有各种更改和变化。凡在本公开的原则之内，所作的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。
工业实用性
如上所述，本公开实施例提供的VR图像的压缩传输方法具有以下有 益效果：通过压缩边缘图像的分辨率来减小整体图像的分辨率，从而减小图像体积，提高传输速率，进而提高VR直播、VR串流的稳定性。

Claims (12)

  1. 一种VR图像的压缩传输方法，包括：
    获取服务器待发送的视频图像，并根据压缩范围对所述视频图像进行定域压缩以形成第一压缩视频图像；所述压缩范围通过传输样本图像确定；
    对所述第一压缩视频图像进行视频编码处理以形成传输视频图像；
    将所述传输视频传输至客户端，并对所述传输视频图像进行视频解码处理以将所述传输视频图像还原为第二压缩视频图像；
    对所述第二压缩视频图像进行解压缩处理以形成展示视频图像。
  2. 如权利要求1所述的VR图像的压缩传输方法，其中，通过所述传输样本确定所述压缩范围的过程，包括：
    通过引导程序使用户的双眼与VR头戴的左镜筒、右镜筒相对应；
    通过所述左镜筒、右镜筒捕捉由所述服务器发送的标记有盲区范围区域的样本图像；
    调节所述样本图像的大小使所述盲区范围区域退出所述用户的视野。
  3. 如权利要求2所述的VR图像的压缩传输方法，其中，根据压缩范围对所述视频图像进行定域压缩以形成第一压缩视频图像的过程，包括：
    按照所述压缩范围对所述视频图像进行区域锁定以将所述视频图像划分为边缘区域和中心区域；
    通过第一图像着色程序对所述边缘区域进行预设倍数压缩采样形成压缩边缘区域；对所述中心区域进行真实采样以形成中心还原区域；
    将所述压缩边缘区域与所述中心还原区域作为第一压缩视频图像。
  4. 如权利要求3所述的VR图像的压缩传输方法，其中，
    所述预设倍数为2倍。
  5. 如权利要求3所述的VR图像的压缩传输方法，其中，对所述第一压缩视频图像进行视频编码处理以形成传输视频图像的同时，还包括：
    保存所述视频图像，并将所述第一压缩视频图像的压缩范围和压缩比 例记录在所述视频图像的自定义信息帧内。
  6. 如权利要求5所述的VR图像的压缩传输方法，其中，
    所述视频编码采用h264或h265视频。
  7. 如权利要求3所述的VR图像的压缩传输方法，其中，
    所述第二压缩视频与所述第一压缩视频的分辨率相同。
  8. 如权利要求7所述的VR图像的压缩传输方法，其中，对所述第二压缩视频图像进行解压缩处理以形成展示视频图像的过程，包括：
    获取所述第二压缩视频图像的压缩部分与原始部分；其中，所述压缩部分与所述压缩边缘区域相一致；所述原始部分与所述中心还原区域相一致；
    通过第二图像着色程序对所述压缩部分进行差值采样以将所述压缩部分还原为真实边缘区域，对所述原始部分进行点对点采样已形成真实中心区域；
    将所述真实边缘区域与所述真实中心区域进行拟合处理以形成展示视频图像。
  9. 一种VR图像的压缩传输系统，实现如权利要求1-8任一所述的VR图像的压缩传输方法，包括：
    边缘压缩模块，设置为获取服务器待发送的视频图像，并根据压缩范围对所述视频图像进行定域压缩以形成第一压缩视频图像；所述压缩范围通过传输样本图像确定；
    视频编码模块，设置为对所述第一压缩视频图像进行视频编码处理以形成传输视频图像；
    视频解码模块，设置为将所述传输视频传输至客户端，并对所述传输视频图像进行视频解码处理以将所述传输视频图像还原为第二压缩视频图像；
    视频还原模块，设置为对所述第二压缩视频图像进行解压缩处理以形 成展示视频图像。
  10. 如权利要求9所述的VR图像的压缩传输系统，其中，所述边缘压缩模块包括图像着色器，其中，
    在所述图像着色器中设置有第一图像着色程序，所述第一图像着色程序设置为对根据所述压缩范围划分的边缘区域进行预设倍数压缩采样以形成压缩边缘区域。
  11. 一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序被处理器执行时实现所述权利要求1-8任一项中所述的方法。
  12. 一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行所述权利要求1-8任一项中所述的方法。
PCT/CN2021/121005 2021-06-22 2021-09-27 Vr图像的压缩传输方法、系统 Ceased WO2022267256A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21946739.6A EP4325866A4 (en) 2021-06-22 2021-09-27 VR IMAGE COMPRESSION AND TRANSMISSION METHOD AND SYSTEM
US17/886,063 US11748915B2 (en) 2021-06-22 2022-08-11 VR image compression transmission method and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110692510.XA CN113542799A (zh) 2021-06-22 2021-06-22 Vr图像的压缩传输方法、系统
CN202110692510.X 2021-06-22

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/886,063 Continuation US11748915B2 (en) 2021-06-22 2022-08-11 VR image compression transmission method and system

Publications (1)

Publication Number Publication Date
WO2022267256A1 true WO2022267256A1 (zh) 2022-12-29

Family

ID=78125627

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/121005 Ceased WO2022267256A1 (zh) 2021-06-22 2021-09-27 Vr图像的压缩传输方法、系统

Country Status (2)

Country Link
CN (1) CN113542799A (zh)
WO (1) WO2022267256A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118590623A (zh) * 2024-08-06 2024-09-03 国网黑龙江省电力有限公司伊春供电公司 面向数字化电力系统的电力设备视频监控方法及监控设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114786037B (zh) * 2022-03-17 2024-04-12 青岛虚拟现实研究院有限公司 一种面向vr投影的自适应编码压缩方法
CN114998143B (zh) * 2022-06-10 2025-07-08 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003153263A (ja) * 2001-11-19 2003-05-23 Hitachi Ltd 映像表示システムおよびその方法
CN107040794A (zh) * 2017-04-26 2017-08-11 盯盯拍（深圳）技术股份有限公司 视频播放方法、服务器、虚拟现实设备以及全景虚拟现实播放系统
WO2017138994A1 (en) * 2016-02-12 2017-08-17 Qualcomm Incorporated Foveated video rendering
CN108322727A (zh) * 2018-02-28 2018-07-24 北京搜狐新媒体信息技术有限公司 一种全景视频传输方法及装置
CN108337497A (zh) * 2018-02-07 2018-07-27 刘智勇 一种虚拟现实视频/图像格式以及拍摄、处理、播放方法和装置
CN110572656A (zh) * 2019-09-19 2019-12-13 北京视博云科技有限公司 一种编码方法、图像处理方法、装置及系统

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107317987B (zh) * 2017-08-14 2020-07-03 歌尔股份有限公司 虚拟现实的显示数据压缩方法和设备、系统
US11194389B2 (en) * 2018-06-21 2021-12-07 Qualcomm Incorporated Foveated rendering of graphics content using a rendering command and subsequently received eye position data
CN112263214A (zh) * 2020-11-12 2021-01-26 中国人民解放军空军特色医学中心 一种载人离心机视觉功能障碍检测装置及检测方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003153263A (ja) * 2001-11-19 2003-05-23 Hitachi Ltd 映像表示システムおよびその方法
WO2017138994A1 (en) * 2016-02-12 2017-08-17 Qualcomm Incorporated Foveated video rendering
CN107040794A (zh) * 2017-04-26 2017-08-11 盯盯拍（深圳）技术股份有限公司 视频播放方法、服务器、虚拟现实设备以及全景虚拟现实播放系统
CN108337497A (zh) * 2018-02-07 2018-07-27 刘智勇 一种虚拟现实视频/图像格式以及拍摄、处理、播放方法和装置
CN108322727A (zh) * 2018-02-28 2018-07-24 北京搜狐新媒体信息技术有限公司 一种全景视频传输方法及装置
CN110572656A (zh) * 2019-09-19 2019-12-13 北京视博云科技有限公司 一种编码方法、图像处理方法、装置及系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118590623A (zh) * 2024-08-06 2024-09-03 国网黑龙江省电力有限公司伊春供电公司 面向数字化电力系统的电力设备视频监控方法及监控设备

Also Published As

Publication number Publication date
CN113542799A (zh) 2021-10-22

Similar Documents

Publication Publication Date Title
US11758187B2 (en) Methods, devices and stream for encoding and decoding volumetric video
CN110557649B (zh) 直播交互方法、直播系统、电子设备及存储介质
TWI656787B (zh) 具有應用於以投影格式表示的３６０度圖像／視訊內容的內容導向旋轉的旋轉資訊的語法元素信令的視訊編碼功能的方法和設備以及具有相關的視訊解碼功能的方法和設備
CN113243112B (zh) 流式传输体积视频和非体积视频
US10497180B1 (en) System and method for display of augmented reality
US10735826B2 (en) Free dimension format and codec
CN116962743A (zh) 视频图像编码、抠图方法和装置及直播系统
EP3557534A1 (en) Information processing method and apparatus
CN112585978A (zh) 生成用于在vr中显示的复合视频流
CA3018600C (en) Method, apparatus and stream of formatting an immersive video for legacy and immersive rendering devices
JP2020115299A (ja) 仮想空間情報処理装置、方法、プログラム
CN113542799A (zh) Vr图像的压缩传输方法、系统
CN112291587A (zh) 远程桌面的动态视频帧处理方法、系统、介质及服务器
CN110519247A (zh) 一种一对多虚拟现实展示方法及装置
WO2023241459A1 (zh) 数据通信方法、系统、电子设备和存储介质
US11748915B2 (en) VR image compression transmission method and system
CN113194326A (zh) 全景直播方法、装置、计算机设备及计算机可读存储介质
CN113099237A (zh) 视频处理方法和装置
CN110798715A (zh) 基于图像串的视频播放方法及系统
CN118764664B (zh) 画面处理方法与装置、电子设备及存储介质
CN113160342B (zh) 基于反馈的编码方法及装置、存储介质、电子设备
WO2023051705A1 (zh) 视频通讯方法及装置、电子设备、计算机可读介质
CN114666567A (zh) 图像处理方法及装置
CN118612446A (zh) 编解码方法、装置、设备、存储介质及计算机程序
CN119697394A (zh) 基于suvc分层编码的vr在线观看方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21946739

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2021946739

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2021946739

Country of ref document: EP

Effective date: 20231116

NENP Non-entry into the national phase

Ref country code: DE