WO2020259192A1 - 多媒体数据处理方法及装置、存储介质、电子设备 - Google Patents

多媒体数据处理方法及装置、存储介质、电子设备 Download PDF

Info

Publication number
WO2020259192A1
WO2020259192A1 PCT/CN2020/092750 CN2020092750W WO2020259192A1 WO 2020259192 A1 WO2020259192 A1 WO 2020259192A1 CN 2020092750 W CN2020092750 W CN 2020092750W WO 2020259192 A1 WO2020259192 A1 WO 2020259192A1
Authority
WO
WIPO (PCT)
Prior art keywords
multimedia data
processing module
session
media processing
terminal device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/092750
Other languages
English (en)
French (fr)
Inventor
杨勇
张云飞
樊华恒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to EP20832485.5A priority Critical patent/EP3993341B1/en
Priority to JP2021555311A priority patent/JP7410966B2/ja
Publication of WO2020259192A1 publication Critical patent/WO2020259192A1/zh
Priority to US17/469,872 priority patent/US11522922B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/222—Secondary servers, e.g. proxy server, cable television Head-end
    • H04N21/2225—Local VOD servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066—Session management
    • H04L65/1069—Session establishment or de-establishment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066—Session management
    • H04L65/1101—Session protocols
    • H04L65/1104—Session initiation protocol [SIP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/764—Media network packet handling at the destination 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/765—Media network packet handling intermediate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • This application relates to the field of communication technology, in particular to multimedia data processing technology.
  • Edge computing is a distributed computing architecture. Under this architecture, operations such as application processing and data analysis can be decomposed from large-scale services that are completely processed by remote nodes into smaller and easier-to-manage parts, and distributed to edge computing nodes for processing. Because the edge computing node is closer to the user terminal equipment, it can speed up the data processing and transmission speed and reduce the delay.
  • the embodiments of the application provide a multimedia data processing method, a multimedia data processing device, an electronic device, and a computer-readable storage medium, which can reduce the deployment complexity of an edge computing system and improve the efficiency of system operation and maintenance.
  • a multimedia data processing method which is executed by a server, and includes:
  • the media processing module deployed on the edge computing node After the media processing module establishes a session with the terminal device, it receives multimedia data sent by the terminal device and processes the multimedia data.
  • the establishing a session between the terminal device and the media processing module includes:
  • the method further includes:
  • a forwarding strategy is pre-configured on the user plane functional entity deployed on the edge computing node; the forwarding strategy includes: forwarding multimedia data with a receiving port in a preset range to the media processing module;
  • the data receiving address information includes a target receiving port within the preset range.
  • the data receiving address information includes a target receiving port; wherein, before sending the data sending address information to the media processing module, it further includes:
  • a forwarding strategy is configured on the user plane functional entity deployed on the edge computing node; the forwarding strategy includes: forwarding multimedia data whose receiving port is the target receiving port to the media processing module.
  • the method further includes:
  • the method further includes:
  • the multimedia data sent by the terminal device is forwarded to the media processing module through the user plane function entity according to the forwarding strategy.
  • controlling the session includes: maintaining the session and ending the session.
  • the processing the multimedia data includes:
  • the decoded data is calculated and analyzed to be converted into structured multimedia data.
  • the method further includes:
  • a multimedia data processing device including:
  • the signaling processing module is deployed at the remote node and is used to establish a session between the terminal device and the media processing module, and control the session;
  • the media processing module is deployed on the edge computing node, and is used to receive multimedia data sent by the terminal device after establishing a session with the terminal device, and process the multimedia data.
  • the device includes multiple signaling processing modules and multiple media processing modules, and the number of signaling processing modules is less than the number of media processing modules.
  • the device includes a signaling processing module and a plurality of media processing modules.
  • the remote node includes a regional network computing platform and/or an Internet computing platform.
  • the signaling processing module establishes a session between the terminal device and the media processing module through the following steps:
  • the signaling processing module is further configured to:
  • a forwarding strategy is pre-configured on the user plane functional entity deployed on the edge computing node; the forwarding strategy includes: forwarding multimedia data with a receiving port in a preset range to the media processing module; wherein the data receiving address information The target receiving port included in the preset range.
  • the data receiving address information includes a target receiving port; wherein, before sending the data sending address information to the media processing module, the signaling processing module is also used for :
  • a forwarding strategy is configured on the user plane functional entity deployed on the edge computing node; the forwarding strategy includes: forwarding multimedia data whose receiving port is the target receiving port to the media processing module.
  • the signaling processing module is further configured to:
  • the device includes:
  • the user plane function entity is configured to forward the multimedia data sent by the terminal device to the media processing module according to the forwarding strategy.
  • controlling the session includes: maintaining the session and ending the session.
  • the media processing module processes the multimedia data through the following steps:
  • the decoded data is calculated and analyzed to be converted into structured multimedia data.
  • the media processing module is further configured to:
  • an electronic device including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions Perform any of the methods described above.
  • a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the above is implemented.
  • a computer program product including instructions, which when run on a computer, cause the computer to execute the method described in any one of the above.
  • the media processing module is deployed at the edge computing node, the signaling processing module is deployed at the remote node, and the session between the terminal device and the media processing module is established through the signaling processing module , And control the session; through the media processing module, receive multimedia data sent by the terminal device and process the multimedia data.
  • the signaling processing module since the signaling processing module only needs to process control signaling, it occupies a small amount of network resources and computing resources; therefore, a signaling processing module can be responsible for multiple media processing modules scattered on different edge computing nodes at the same time. A session between two terminal devices.
  • the deployment mode of multimedia data processing applications can be simplified, and the deployment complexity of edge computing nodes can be significantly reduced; at the same time, the infrastructure resources required to deploy edge computing nodes can also be reduced. Reduce deployment costs.
  • the signaling processing module is deployed at the remote node, installation, upgrade, and maintenance become simpler and more convenient.
  • there are only stateless media processing modules in edge computing nodes and the edge computing services deployed in edge computing nodes will become a consistent single service type, and the management, operation and maintenance of edge computing nodes will become more efficient. Therefore, based on the method in the exemplary embodiment, the operation and maintenance efficiency of the entire system can be greatly improved, and the operation and maintenance cost can be reduced.
  • FIG. 1 shows a schematic diagram of an exemplary system architecture of a multimedia data processing method and device to which the embodiments of the present application can be applied;
  • FIG. 2 shows a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application
  • FIG. 3 schematically shows a schematic diagram of a deployment architecture of a multimedia data processing apparatus according to an embodiment of the present application
  • Fig. 4 schematically shows a schematic diagram of a deployment architecture of a multimedia data processing device according to an embodiment of the present application
  • FIG. 5 schematically shows a flowchart of a multimedia data processing method according to an embodiment of the present application
  • FIG. 6 schematically shows a flowchart of a process of establishing a multimedia session according to an embodiment of the present application
  • Fig. 7 schematically shows an interactive flowchart of a process of establishing a multimedia session according to an embodiment of the present application
  • FIG. 8 schematically shows an interaction flowchart of a process of establishing a multimedia session according to an embodiment of the present application
  • Fig. 9 schematically shows a flowchart of steps for processing multimedia data in an embodiment of the present application.
  • Fig. 10 schematically shows a block diagram of a multimedia data processing device according to an embodiment of the present application.
  • FIG. 1 shows a schematic diagram of an application environment system architecture of a multimedia data processing method provided by an embodiment of the present application.
  • the system architecture 100 may include one or more of terminal devices 101, 102, and 103, a base station 104, an edge computing node 105, a regional network computing platform 106, and an Internet computing platform 107.
  • the edge computing node 105 can be a computing platform close to the network access point, and it can be located in an edge data center (Edge Data Center, Edge DC for short); the regional network computing platform 106 and the Internet computing platform 107 can be used as remote nodes in this application
  • the regional network computing platform 106 may be located in a regional data center (Local Data Center, referred to as Local DC), and the Internet computing platform 107 may be located in an Internet data center (Internet Data Center, referred to as IDC).
  • IDC Internet Data Center
  • the edge computing node 105 may directly communicate with the Internet computing platform 107; it may also be a three-layer network structure.
  • the above network structure is not specifically limited in this exemplary embodiment.
  • the terminal devices 101, 102, 103 may be various electronic devices, including but not limited to cameras, desktop computers, portable computers, smart phones, tablet computers, and so on.
  • the edge computing node 105, the regional network computing platform 106, and the Internet computing platform 107 all include servers. It should be understood that the numbers of terminal devices, networks, and servers in FIG. 1 are merely illustrative, and any number of terminal devices, networks, and servers may be included according to implementation requirements.
  • the edge computing node 105, the regional network computing platform 106, and the Internet computing platform 107 can be a server cluster composed of multiple servers; for another example, the regional network computing platform 106 and the Internet computing platform 107 can also be cloud computing platforms to achieve Edge-cloud collaborative architecture; that is, edge computing nodes and remote cloud computing platforms work together.
  • the multimedia data processing method provided in the embodiments of the present application is generally executed by a server, and accordingly, the multimedia data processing device is generally set in the server.
  • an MEC (Mobile Edge Computing) application can be deployed on the edge computing node 105, so as to achieve rapid download of various content, services, and applications in the network, allowing consumers Enjoy uninterrupted high-quality network experience.
  • the network traffic processed by non-MEC applications can be forwarded by the user plane function entity (User Plane Function, UPF) in the edge computing node 105 to the user plane function entity in the regional network computing platform 106, and finally enters the Internet computing platform 107.
  • UPF User Plane Function
  • Fig. 2 shows a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
  • the computer system 200 includes a central processing unit (Central Processing Unit, CPU) 201, which can be loaded into a random storage unit according to a program stored in a read-only memory (Read-Only Memory, ROM) 202 or from a storage part 208. Access to the program in the memory (Random Access Memory, RAM) 203 to execute various appropriate actions and processing. In RAM 203, various programs and data required for system operation are also stored.
  • the CPU 201, ROM 202, and RAM 203 are connected to each other through a bus 204.
  • An input/output (Input/Output, I/O) interface 205 is also connected to the bus 204.
  • the following components are connected to the I/O interface 205: an input part 206 including a keyboard, a mouse, etc.; an output part 207 such as a cathode ray tube (Cathode Ray Tube, CRT), a liquid crystal display (LCD), and speakers 207 A storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN card, a modem, etc. The communication section 209 performs communication processing via a network such as the Internet.
  • the drive 210 is also connected to the I/O interface 205 as needed.
  • a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed, so that the computer program read from it is installed into the storage section 208 as needed.
  • the process described below with reference to the flowchart can be implemented as a computer software program.
  • the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from the network through the communication section 209, and/or installed from the removable medium 211.
  • the computer program is executed by the central processing unit (CPU) 201, various functions defined in the method and device of the present application are executed.
  • the computer system 200 may further include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
  • AI Artificial Intelligence
  • the computer-readable medium shown in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above.
  • Computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM or Flash), optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any of the above The right combination.
  • the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
  • the computer-readable medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
  • each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the above-mentioned module, program segment, or part of code contains one or more for realizing the specified logical function Executable instructions.
  • the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be It is realized by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.
  • this application also provides a computer-readable medium.
  • the computer-readable medium may be included in the electronic device described in the above-mentioned embodiments; or it may exist alone without being assembled into the electronic device. in.
  • the above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device realizes the method described in the following embodiments.
  • FIG. 3 An implementation solution provided by the inventor is shown in FIG. 3.
  • the signaling processing module and the media processing module are deployed on the edge computing node, and the video access is controlled through the signaling processing module, and the received video is processed based on GPU (Graphics Processing Unit, graphics processing) through the media processing module. ) And deep neural network video analysis, so as to output structured video data, and then send these structured video data to the data processing module deployed at the remote node, through the data processing module based on the structured video data Processing such as retrieval and analysis.
  • GPU Graphics Processing Unit, graphics processing
  • the video session signaling processing and media processing are carried out by the edge computing node deployed on the user side, which improves the efficiency of real-time video processing; on the other hand, the data processing module deployed on the remote node receives and Processing the video structured data sent by the edge computing node can avoid the transmission of original video media data between the edge computing node and the remote node, and effectively reduce the network bandwidth requirements between the edge computing node and the remote node.
  • an implementation solution provided by the inventor is to deploy the signaling processing module and the media processing module on the edge computing node including the streaming media server, and the CDN (Content Delivery Network)
  • the distribution network) management server is deployed on a remote node.
  • a user requests streaming media on-demand service
  • the request is first submitted to the CDN management server deployed on the remote node.
  • the CDN management server selects the streaming media server deployed in the corresponding edge computing node according to the corresponding strategy, and sends the user
  • the request is forwarded to the streaming media server to provide users with nearby high-quality, low-latency video-on-demand services; at the same time, it also reduces the video data transmission between the user terminal equipment and the remote node.
  • each edge computing node which includes a signaling processing module and a media processing module.
  • Deploying the signaling processing module and media processing module on edge computing nodes at the same time increases the complexity of system deployment and also increases the cost of system deployment; in addition, when the system is upgraded or expanded, it needs to be deployed in each edge computing node separately All of the signaling processing modules or media processing modules are upgraded or expanded, and the efficiency and cost of system operation and maintenance are relatively high.
  • the signaling processing module is responsible for the establishment of media sessions. From initiation to termination of a session, the entire signaling interaction occupies very little computing resources, and the network resources occupied by it are similar to those occupied by media transmission. The ratio can also be ignored.
  • this application provides a multimedia data processing method, which can be applied to the deployment architecture for multimedia session applications as shown in FIG. 4.
  • the multimedia data processing device is divided into two modules: a signaling processing module and a media processing module.
  • the signaling processing module is deployed on a remote node, for example, it can be deployed on the aforementioned regional network computing platform. 106 or Internet computing platform 107; the media processing module is deployed on the edge computing node 105, so as to realize a simple and flexible edge computing application.
  • the signaling processing of the multimedia session between the user terminal equipment and the multimedia data processing device is completed by the signaling processing module deployed at the remote node, and the multimedia processing between the user terminal equipment and the multimedia data processing device This is done by media processing modules deployed in edge computing nodes.
  • the method may include the following steps S510 to S520:
  • Step S510 Establish a session between the terminal device and the media processing module through the signaling processing module deployed on the remote node, and control the session.
  • a session refers to a process in which a terminal device communicates with a media processing module.
  • the signaling processing module can establish a session between the terminal device and the media processing module.
  • the signaling processing module can also control the session to be maintained and to end the session.
  • the signaling processing module is usually deployed in the remote computer room.
  • the protocols supported by the signaling processing module may include multimedia session protocols such as SIP (Session Initiation Protocol) and RTSP (Real-Time Stream Protocol, Real-Time Streaming Control Protocol).
  • multimedia session protocols such as SIP (Session Initiation Protocol) and RTSP (Real-Time Stream Protocol, Real-Time Streaming Control Protocol).
  • the protocol supported by the signaling processing module can also include other multimedia session protocols such as HLS (HTTP Live Streaming), RTMP (Real Time Messaging Protocol, real-time messaging protocol);
  • HLS HTTP Live Streaming
  • RTMP Real Time Messaging Protocol
  • real-time messaging protocol Real-time messaging protocol
  • the signaling processing module may also support a non-multimedia session protocol; this exemplary embodiment is not limited to this.
  • a session between the terminal device and the media processing module can be established through the following steps S610 to S640. in particular:
  • step S610 a call request including data receiving address information is initiated to the terminal device.
  • the signaling processing module can send a SIP ⁇ RTSP call request to the camera;
  • the call request can include data in the format such as SDP (Session Description Protocol, Session Description Protocol) Receiving address information;
  • the data receiving address information may include, for example, the IP address of the media processing module and the connection port number information.
  • step S620 receiving response information of the terminal device to the call request, where the response information includes data sending address information.
  • the camera can reply to the signaling processing module with response information;
  • the response information can include data sending address information such as SDP format;
  • the data sending address information can include camera identification (such as ID Etc.), the source IP address and sending port number of the multimedia data sent by the camera.
  • step S630 the data sending address information is sent to the media processing module, and the media processing module is notified to receive the multimedia data according to the data receiving address.
  • step S640 a session establishment completion message is sent to the terminal device.
  • the user plane function entity can directly exchange protocols such as SIP ⁇ RTSP of the control plane between the terminal equipment and the signaling processing module.
  • the user plane function entity may be a user plane device in a 5G network, but is not limited to this; for example, the user plane function entity may also be a wireless access gateway (such as a WiFi gateway), wired access Non-mobile communication gateway equipment such as gateways.
  • the forwarding strategy of the multimedia data by the user plane function entity may include a static strategy and a dynamic strategy. in particular:
  • a forwarding strategy needs to be pre-configured in the user plane functional entity deployed on the edge computing node; the forwarding strategy includes: forwarding multimedia data with a receiving port in a preset range to the media processing module; wherein, data receiving address information Include the target receiving port within the preset range.
  • the terminal device needs to be required to set the receiving port in a preset range.
  • the camera can be required to limit the target receiving port of the multimedia data sent to 10000 to 20000, and then configure the forwarding strategy in the user plane functional entity.
  • the target receiving port sent from the camera is between 10000 and 20000, and the type is RTP (Real-time Transport Protocol) IP packets, such as UDP (User Datagram Protocol, User Datagram Protocol) packets are uniformly forwarded to the media for processing Port corresponding to the module.
  • RTP Real-time Transport Protocol
  • FIG. 7 is a schematic diagram of a conversation process based on a static policy. Including step S710 to step S760. in particular:
  • step S710 after receiving the camera video access request initiated by the service manager, the signaling processing module sends a SIP ⁇ RTSP call request to the camera; the call request may include data in the format such as SDP (Session Description Protocol, Session Description Protocol) Receive address information.
  • SDP Session Description Protocol, Session Description Protocol
  • the data receiving address information needs to limit the data receiving port to a free port Port A in the range of 10000 to 20000; the connection address is designated as the IP address of the media processing module.
  • the camera may reply response information to the signaling processing module;
  • the response information may include data sending address information such as SDP format;
  • the data sending address information may include the camera identification (such as ID, etc.), the source IP address and sending port number of the multimedia data sent by the camera.
  • step S730 after receiving the response information from the camera, the signaling processing module sends instructions to the media processing module, including the camera identification, the source IP address of the multimedia data sent by the camera, and the sending port number, etc., to notify the media processing module to Port A starts to receive the multimedia data sent by the camera.
  • step S740 the signaling processing module sends session establishment completion information to the terminal device to confirm the session establishment.
  • step S750 the camera starts to send multimedia data packets to the user plane functional entity of the edge computing node; the target receiving port of the multimedia data packets is Port A.
  • step S760 after receiving the multimedia data packet sent by the camera, the user plane function entity of the edge computing node forwards the multimedia data packet to the port Port A of the media processing module according to a preset forwarding strategy.
  • the data receiving address information includes the target receiving port; wherein, before the step of sending the data sending address information to the media processing module, it further includes: configuring a forwarding strategy on the user plane function entity deployed on the edge computing node; the forwarding strategy includes : Forward multimedia data whose receiving port is the target receiving port to the media processing module.
  • a certain free port Port B can be randomly selected as the target receiving port during the SDP negotiation process; then the forwarding strategy is configured in the user plane functional entity, whichever receives the camera transmission.
  • the incoming IP packets whose destination receiving port is Port B and type RTP, such as UDP packets, are uniformly forwarded to Port B corresponding to the media processing module.
  • FIG 8 is a schematic diagram of a conversation process based on a dynamic strategy. Including step S810 to step S870. in particular:
  • step S810 after receiving the camera video access request initiated by the service manager, the signaling processing module sends a SIP ⁇ RTSP call request to the camera; the call request may include data receiving address information such as SDP format.
  • the data receiving address information needs to limit the data receiving port to a randomly selected free port Port B; the connection address is designated as the IP address of the media processing module.
  • the camera may reply to the signaling processing module with response information;
  • the response information may include data sending address information in SDP format;
  • the data sending address information may include camera identification (Such as ID, etc.), the source IP address and port number of the multimedia data sent by the camera.
  • step S830 after receiving the response information from the camera, the signaling processing module sends an instruction to the user plane functional entity to set the forwarding strategy of the user plane functional entity.
  • This forwarding strategy requires that all IP packets, such as UDP packets, whose destination receiving port is Port B and type RTP sent from the camera, are forwarded to the corresponding port Port B of the media processing module.
  • step S840 the signaling processing module sends an instruction to the media processing module, including the camera identification, the source IP address of the multimedia data sent by the camera, and the sending port number, etc., to notify the media processing module to start receiving information from the camera at Port B. Multimedia data.
  • step S850 the signaling processing module sends session establishment completion information to the terminal device to confirm the session establishment.
  • step S860 the camera starts to send multimedia data packets to the user plane functional entity of the edge computing node; the target receiving port of the multimedia data packets is Port B.
  • step S870 after receiving the multimedia data packet sent by the camera, the user plane function entity of the edge computing node forwards the multimedia data packet to the port B of the media processing module according to a preset forwarding strategy.
  • the dynamic forwarding strategy set above can be deleted to avoid affecting the establishment of subsequent sessions.
  • Step S520 through the media processing module deployed on the edge computing node, after the media processing module establishes a session with the terminal device, receiving multimedia data sent by the terminal device, and processing the multimedia data.
  • the media processing module is deployed on the edge computing node.
  • the media processing module is mainly responsible for the processing of multimedia data, including the reception of audio and video media packets, the decoding of audio and video data, and the analysis of audio and video content based on deep learning methods. If the deep learning method is used for audio and video content analysis, the media processing module usually needs GPU to accelerate the computing performance of media analysis.
  • the processing of the multimedia data may include step S910 and step S920. among them:
  • step S910 the multimedia data is decoded to obtain decoded data.
  • the multimedia data can be decoded losslessly, that is, the decoded image and the original image are strictly the same, and the compression is completely recoverable or unbiased, and there is no distortion.
  • lossy decoding can also be used.
  • the decoding module may be a software module or a hardware module; there is no special limitation on this in this exemplary embodiment.
  • step S920 the decoded data is calculated and analyzed to be converted into structured multimedia data.
  • the important information in the multimedia data content can be structured and extracted, and processing methods such as space-time segmentation, feature extraction, object recognition, and deep learning can be used to organize it into text information or visual graphics that can be understood by computers and humans. information.
  • processing methods such as space-time segmentation, feature extraction, object recognition, and deep learning can be used to organize it into text information or visual graphics that can be understood by computers and humans. information.
  • the structured multimedia data may also be sent to the remote node. For example, it is sent to the data processing module of the local area network computing platform 106 or the Internet computing platform 107.
  • the data processing module can use structured multimedia data to perform multimedia retrieval or recognition according to product requirements, such as face retrieval, face recognition, and human behavior analysis.
  • a structured data warehouse can be established based on the processing structure, such as a face photo database, a face feature database, a behavior picture and feature database, a vehicle image and a feature database, etc., and these data warehouses and associated video clip warehouses can be integrated to establish corresponding
  • the search engine realizes in-depth information mining of various data warehouses, gives full play to the role of big data, enhances the application value of video data, and improves the analysis and prediction functions of video data.
  • the media processing module is deployed at the edge computing node, and the signaling processing module is deployed at the remote node.
  • the number of media processing modules deployed depends on the number of connected terminal devices, such as the number of cameras.
  • the signaling processing module only needs to process control signaling, and it does not occupy a large amount of network resources and computing resources. Therefore, a signaling processing module can be responsible for multiple media processing modules and multiple terminal devices scattered at different edge computing nodes at the same time. Conversation between. For example, taking the SIP signaling processing module deployed on a general X86 architecture server as an example, it can generally support thousands of SIP calls at the same time, that is, it can handle thousands of simultaneous SIP-enabled cameras and media processing modules.
  • the signaling processing module can start one or several service instances to meet the service requirements of the media processing modules and terminal equipment of all edge computing nodes in the entire network. Therefore, based on the method in this example embodiment, the deployment mode of multimedia data processing applications can be simplified, and the deployment complexity of edge computing nodes can be significantly reduced; at the same time, the infrastructure resources required by edge computing nodes can be reduced, thereby reducing Deployment cost.
  • the signaling processing module is deployed at the remote node, installation, upgrade, and maintenance become simpler and more convenient.
  • the signaling processing module is a stateful computing service
  • the media processing module is a stateless computing service
  • the edge There are only stateless media processing modules in computing nodes, and the edge computing services deployed in edge computing nodes will become a consistent single service type, and the management, operation and maintenance of edge computing nodes will become more efficient. Therefore, based on the method in the exemplary embodiment, the operation and maintenance efficiency of the entire system can be greatly improved, and the operation and maintenance cost can be reduced.
  • the forwarding policy configuration of the user plane functional entity also changes. It is more simple and convenient, so it can also reduce the complexity of the entire system flow unloading function.
  • a video surveillance application scenario is taken as an example for description.
  • the multimedia data processing method of this application can also be applied to other application scenarios such as voice monitoring, Internet of Vehicles and autonomous driving applications, video on demand, game entertainment, etc., which also belong to the protection of this application. range.
  • the multimedia data processing apparatus 1000 may include a signaling processing module 1010 and a media processing module 1020. among them:
  • the signaling processing module 1010 is deployed at a remote node and is used to establish a session between the terminal device and the media processing module 1020, and to control the session;
  • the media processing module 1020 is deployed on an edge computing node, and is used to receive multimedia data sent by the terminal device after establishing a session with the terminal device, and process the multimedia data.
  • the device includes multiple signaling processing modules 1010 and multiple media processing modules 1020, and the number of signaling processing modules 1010 is less than the number of media processing modules 1020.
  • the device includes one signaling processing module 1010 and multiple media processing modules 1020.
  • the remote node includes a regional network computing platform and/or an Internet computing platform.
  • the signaling processing module 1010 establishes a session between the terminal device and the media processing module 1020 through the following steps:
  • the signaling processing module 1010 is also used to:
  • a forwarding strategy is pre-configured on the user plane function entity deployed on the edge computing node; the forwarding strategy includes: forwarding multimedia data with a receiving port in a preset range to the media processing module 1020; wherein, the data receiving address The information includes the target receiving port within the preset range.
  • the data receiving address information includes a target receiving port; wherein, before the step of sending the data sending address information to the media processing module 1020, the signaling processing module 1010 is also used to:
  • a forwarding strategy is configured on the user plane functional entity deployed on the edge computing node; the forwarding strategy includes: forwarding multimedia data whose receiving port is the target receiving port to the media processing module 1020.
  • the signaling processing module 1010 is also used to:
  • the device includes:
  • the user plane function entity is configured to forward the multimedia data sent by the terminal device to the media processing module 1020 according to the forwarding strategy.
  • controlling the session includes: maintaining the session and ending the session.
  • the media processing module 1020 processes the multimedia data through the following steps:
  • the decoded data is calculated and analyzed to be converted into structured multimedia data.
  • the media processing module 1020 is further configured to:
  • modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory.
  • the features and functions of two or more modules or units described above may be embodied in one module or unit.
  • the features and functions of a module or unit described above can be further divided into multiple modules or units to be embodied.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer And Data Communications (AREA)

Abstract

本公开提供一种多媒体数据处理方法及装置、电子设备、存储介质；涉及通信技术领域。所述多媒体数据处理方法包括：通过部署于远端节点的信令处理模块，建立终端设备和媒体处理模块之间的会话，并对所述会话进行控制；通过部署于边缘计算节点的所述媒体处理模块，在与所述终端设备建立会话后，接收所述终端设备发送的多媒体数据并对所述多媒体数据进行处理。本公开可以降低系统部署复杂度和部署成本，并且提高运维效率和降低运维成本。

Description

多媒体数据处理方法及装置、存储介质、电子设备
本申请要求于2019年06月28日提交中国专利局、申请号为201910578317.6、申请名称为“多媒体数据处理方法及装置、存储介质、电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及通信技术领域，具体涉及多媒体数据处理技术。
背景技术
边缘计算是一种分散式运算架构。在这种架构下，可以将应用程序处理、数据分析等运算，从完全由远端节点处理的大型服务分解成更小更容易管理的部分，分散到边缘计算节点处理。由于边缘计算节点更接近于用户终端设备，因此可以加快数据的处理与传送速度，减少延迟。
相关技术通常会将业务服务器部署在边缘计算节点，以实现高质量、低时延的服务。然而，目前基于边缘计算的多媒体数据处理方法，普遍存在部署复杂度高、系统运维效率低等问题。因此，有必要提供一种新的多媒体数据处理方法。
需要说明的是，在上述背景技术部分公开的信息仅用于加强对本申请背景的理解，因此可以包括不构成对本领域普通技术人员已知的现有技术的信息。
发明内容
本申请实施例提供了一种多媒体数据处理方法、多媒体数据处理装置、电子设备以及计算机可读存储介质，能够降低边缘计算系统的部署复杂度，提高系统运维效率。
根据本申请的第一方面，提供一种多媒体数据处理方法，由服务器执行，包括：
通过部署于远端节点的信令处理模块，建立终端设备和媒体处理模块之间的会话，并对所述会话进行控制；
通过部署于边缘计算节点的所述媒体处理模块，在所述媒体处理模块与所述终端设备建立会话后，接收所述终端设备发送的多媒体数据，并对所述多媒体数据进行处理。
在本申请的一种示例性实施例中，所述建立终端设备和媒体处理模块之间的会话，包括：
向所述终端设备发起包括数据接收地址信息的呼叫请求；
接收所述终端设备对于所述呼叫请求的响应信息，所述响应信息包括数据发送地址信息；
向所述媒体处理模块发送所述数据发送地址信息，并通知所述媒体处理 模块根据所述数据接收地址接收所述多媒体数据；
向所述终端设备发送会话建立完成信息。
在本申请的一种示例性实施例中，所述方法还包括：
在部署于所述边缘计算节点的用户面功能实体预先配置转发策略；所述转发策略包括：将接收端口在预设范围的多媒体数据转发至所述媒体处理模块；
其中，所述数据接收地址信息包括在所述预设范围内的目标接收端口。
在本申请的一种示例性实施例中，所述数据接收地址信息包括目标接收端口；其中，在向所述媒体处理模块发送所述数据发送地址信息之前，还包括：
在部署于所述边缘计算节点的用户面功能实体配置转发策略；所述转发策略包括：将接收端口为所述目标接收端口的多媒体数据转发至所述媒体处理模块。
在本申请的一种示例性实施例中，所述方法还包括：
在所述会话结束后，删除所述转发策略。
在本申请的一种示例性实施例中，所述方法还包括：
通过所述用户面功能实体，按照所述转发策略将所述终端设备发送的多媒体数据转发至所述媒体处理模块。
在本申请的一种示例性实施例中，对所述会话进行控制，包括：保持所述会话以及结束所述会话。
在本申请的一种示例性实施例中，所述对所述多媒体数据进行处理，包括：
对所述多媒体数据进行解码，得到解码数据；
对所述解码数据进行计算分析以转换为结构化多媒体数据。
在本申请的一种示例性实施例中，在所述对所述多媒体数据进行处理之后，还包括：
将所述结构化多媒体数据发送至所述远端节点。
根据本申请的第二方面，提供一种多媒体数据处理装置，包括：
信令处理模块，部署于远端节点，用于建立终端设备和媒体处理模块之间的会话，并对所述会话进行控制；
媒体处理模块，部署于边缘计算节点，用于在自身与所述终端设备建立会话后，接收所述终端设备发送的多媒体数据，并对所述多媒体数据进行处理。
在本申请的一种示例性实施例中，所述装置包括多个信令处理模块以及多个所述媒体处理模块，所述信令处理模块的数量少于所述媒体处理模块的数量。
在本申请的一种示例性实施例中，所述装置包括一个信令处理模块以及 多个所述媒体处理模块。
在本申请的一种示例性实施例中，所述远端节点包括区域网络计算平台和/或互联网计算平台。
在本申请的一种示例性实施例中，所述信令处理模块通过下述步骤建立终端设备和媒体处理模块之间的会话：
向所述终端设备发起包括数据接收地址信息的呼叫请求；
接收所述终端设备对于所述呼叫请求的响应信息，所述响应信息包括数据发送地址信息；
向所述媒体处理模块发送所述数据发送地址信息，并通知所述媒体处理模块根据所述数据接收地址接收所述多媒体数据；
向所述终端设备发送会话建立完成信息。
在本申请的一种示例性实施例中，所述信令处理模块还用于：
在部署于所述边缘计算节点的用户面功能实体预先配置转发策略；所述转发策略包括：将接收端口在预设范围的多媒体数据转发至所述媒体处理模块；其中，所述数据接收地址信息包括在所述预设范围内的目标接收端口。
在本申请的一种示例性实施例中，所述数据接收地址信息包括目标接收端口；其中，所述信令处理模块在向所述媒体处理模块发送所述数据发送地址信息之前，还用于：
在部署于所述边缘计算节点的用户面功能实体配置转发策略；所述转发策略包括：将接收端口为所述目标接收端口的多媒体数据转发至所述媒体处理模块。
在本申请的一种示例性实施例中，所述信令处理模块还用于：
在所述会话结束后，删除所述转发策略。
在本申请的一种示例性实施例中，所述装置包括：
用户面功能实体，用于按照所述转发策略将所述终端设备发送的多媒体数据转发至所述媒体处理模块。
在本申请的一种示例性实施例中，所述对所述会话进行控制，包括：保持所述会话以及结束所述会话。
在本申请的一种示例性实施例中，所述媒体处理模块通过如下步骤对所述多媒体数据进行处理：
对所述多媒体数据进行解码，得到解码数据；
对所述解码数据进行计算分析以转换为结构化多媒体数据。
在本申请的一种示例性实施例中，所述媒体处理模块还用于：
将所述结构化多媒体数据发送至所述远端节点。
根据本申请的第三方面，提供一种电子设备，包括：处理器；以及存储器，用于存储所述处理器的可执行指令；其中，所述处理器配置为经由执行所述可执行指令来执行上述任意一项所述的方法。
根据本申请的第四方面，提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任意一项所述的方法。
根据本申请的第五方面，提供一种计算机程序产品，包括指令，当其在计算机上运行时，使得计算机执行上述任意一项所述的方法。
本申请示例性实施例可以具有以下部分或全部有益效果：
在本申请实施例提供的多媒体数据处理方法中，将媒体处理模块部署在边缘计算节点，将信令处理模块部署在远端节点，通过信令处理模块建立终端设备和媒体处理模块之间的会话，并对所述会话进行控制；通过媒体处理模块，接收所述终端设备发送的多媒体数据并对所述多媒体数据进行处理。一方面，由于信令处理模块只需处理控制信令，占用的网络资源和计算资源都不大；因此可以由一个信令处理模块同时负责多个分散在不同边缘计算节点的媒体处理模块以及多个终端设备之间的会话。因此，基于本申请实施例中的方法，可以简化多媒体数据处理应用的部署模式，显著地降低边缘计算节点的部署复杂性；同时，还可以减少边缘计算节点所需部署的基础设施资源，从而可以降低部署成本。另一方面，由于信令处理模块部署在远端节点，安装、升级和维护也变得更加简单和便利。同时，边缘计算节点中只有无状态的媒体处理模块，进而边缘计算节点中部署的边缘计算服务将成为一致性的单一服务类型，边缘计算节点的管理、运维工作将变得更加高效。因此，基于本示例实施方式中的方法可以极大地提升整个系统的运维效率，降低运维成本。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了可以应用本申请实施例的一种多媒体数据处理方法及装置的示例性系统架构的示意图；
图2示出了适于用来实现本申请实施例的电子设备的计算机系统的结构示意图；
图3示意性示出了根据本申请的一个实施例的多媒体数据处理装置的部署架构的示意图；
图4示意性示出了根据本申请的一个实施例的多媒体数据处理装置的部署架构的示意图；
图5示意性示出了根据本申请的一个实施例的多媒体数据处理方法的流程图；
图6示意性示出了根据本申请的一个实施例中建立多媒体会话过程的流程图；
图7示意性示出了根据本申请的一个实施例中建立多媒体会话过程的交互流程图；
图8示意性示出了根据本申请的一个实施例中建立多媒体会话过程的交互流程图；
图9示意性示出了根据本申请的一个实施例中对多媒体数据进行处理的步骤的流程图；
图10示意性示出了根据本申请的一个实施例的多媒体数据处理装置的框图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本申请将更加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施方式中。在下面的描述中，提供许多具体细节从而给出对本申请的实施方式的充分理解。然而，本领域技术人员将意识到，可以实践本申请的技术方案而省略所述特定细节中的一个或更多，或者可以采用其它的方法、组元、装置、步骤等。在其它情况下，不详细示出或描述公知技术方案以避免喧宾夺主而使得本公开的各方面变得模糊。
此外，附图仅为本申请的示意性图解，并非一定是按比例绘制。图中相同的附图标记表示相同或类似的部分，因而将省略对它们的重复描述。附图中所示的一些方框图是功能实体，不一定必须与物理或逻辑上独立的实体相对应。可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
图1示出了本申请实施例提供的多媒体数据处理方法的应用环境系统架构示意图。
如图1所示，系统架构100可以包括终端设备101、102、103中的一个或多个，基站104、边缘计算节点105、区域网络计算平台106以及互联网计算平台107。边缘计算节点105可以为靠近网络接入地点的计算平台，其可以位于边缘数据中心(Edge Data Center，简称Edge DC)；区域网络计算平台106以及互联网计算平台107可以作为本申请中的远端节点，区域网络计算平台106可以位于区域数据中心(Local Data Center，简称Local DC)，互联网计算平台107可以位于互联网数据中心(Internet Data Center，简称IDC)。此外，在图1中为三层网络结构，但在其他示例性实施例中，也可以为两层网络结构，即可以是边缘计算节点105直接与互联网计算平台107 通信连接；也可以为三层以上的网络结构，本示例性实施例中对此不做特殊限定。
终端设备101、102、103可以是各种电子设备，包括但不限于摄像头、台式计算机、便携式计算机、智能手机和平板电脑等等。边缘计算节点105、区域网络计算平台106以及互联网计算平台107均包括服务器。应该理解，图1中的终端设备、网络和服务器的数目仅仅是示意性的，根据实现需要，可以包括任意数目的终端设备、网络和服务器。比如，边缘计算节点105、区域网络计算平台106以及互联网计算平台107可以是多个服务器组成的服务器集群等；再比如，区域网络计算平台106以及互联网计算平台107也可以为云计算平台，从而实现边云协同的架构；即边缘计算节点与远端云计算平台协同发挥作用。
本申请实施例提供的多媒体数据处理方法一般由服务器执行，相应地，多媒体数据处理装置一般设置于服务器中。举例而言，在一种示例性实施例中，可以在边缘计算节点105部署MEC(Mobile Edge Computing，移动边缘计算)应用，从而实现网络中各项内容、服务及应用的快速下载，让消费者享有不间断的高质量网络体验。非MEC应用处理的网络流量可以由边缘计算节点105中的用户面功能实体(User Plane Function，UPF)转发给区域网络计算平台106中的用户面功能实体，最终再进入互联网计算平台107。
图2示出了适用于实现本申请实施例的电子设备的计算机系统的结构示意图。
需要说明的是，图2示出的电子设备的计算机系统200仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图2所示，计算机系统200包括中央处理单元(Central Processing Unit，CPU)201，其可以根据存储在只读存储器(Read-Only Memory，ROM)202中的程序或者从存储部分208加载到随机访问存储器(Random Access Memory，RAM)203中的程序而执行各种适当的动作和处理。在RAM 203中，还存储有系统操作所需的各种程序和数据。CPU 201、ROM 202以及RAM 203通过总线204彼此相连。输入/输出(Input/Output，I/O)接口205也连接至总线204。
以下部件连接至I/O接口205：包括键盘、鼠标等的输入部分206；包括诸如阴极射线管(Cathode Ray Tube，CRT)、液晶显示器(Liquid Crystal Display，LCD)等以及扬声器等的输出部分207；包括硬盘等的存储部分208；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分209。通信部分209经由诸如因特网的网络执行通信处理。驱动器210也根据需要连接至I/O接口205。可拆卸介质211，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器210上，以便于从其上读出的计算机程序根据需要被安装入存储部分208。
特别地，根据本申请的实施例，下文参考流程图描述的过程可以被实现为计算机软件程序。例如，本申请的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分209从网络上被下载和安装，和/或从可拆卸介质211被安装。在该计算机程序被中央处理单元(CPU)201执行时，执行本申请的方法和装置中限定的各种功能。在一些实施例中，计算机系统200还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
需要说明的是，本申请所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
在本申请中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本申请中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、RF等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实 现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现，所描述的单元也可以设置在处理器中。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定。
作为另一方面，本申请还提供了一种计算机可读介质，该计算机可读介质可以是上述实施例中描述的电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被一个该电子设备执行时，使得该电子设备实现如下述实施例中所述的方法。
以下对本申请实施例的技术方案进行详细阐述：
基于图1中所示的边缘计算网络部署架构，针对不同的应用场景，可以有不同的解决方案。
以基于边缘计算的视频监控应用场景为例，发明人提供的一种实现方案如图3所示。在该实现方案中，将信令处理模块和媒体处理模块部署在边缘计算节点，通过信令处理模块控制视频的接入，通过媒体处理模块对接收的视频进行基于GPU(Graphics Processing Unit，图形处理器)和深度神经网络的视频分析，从而输出结构化的视频数据，然后将这些结构化的视频数据发送给部署在远端节点的数据处理模块，进行通过该数据处理模块基于结构化视频数据进行检索和分析等处理。
基于这种实现方案，一方面，由部署在用户侧边缘计算节点进行视频会话信令处理和媒体处理，提升了视频实时处理效率；另一方面，由部署在远端节点的数据处理模块接收并处理边缘计算节点发送过来的视频结构化数据，可以避免原始视频媒体数据在边缘计算节点和远端节点之间的传输，有效降低对于边缘计算节点与远端节点之间的网络带宽要求。
以基于边缘计算的视频点播应用场景为例，发明人提供的一种实现方案是，将信令处理模块和媒体处理模块部署在包括流媒体服务器的边缘计算节点，将CDN(Content Delivery Network，内容分发网络)管理服务器部署在远端节点。当某个用户请求流媒体点播服务时，请求首先被提交给部署在远端节点的CDN管理服务器，由CDN管理服务器根据相应策略，选择相应的边缘计算节点中部署的流媒体服务器，并将用户请求转发至该流媒体服务器，以便为用户提供就近的高质量、低时延的视频点播服务；同时，也减少了用户终端设备和远端节点之间的视频数据传输。
在以上两种场景的实现方案中，都需要在各个边缘计算节点部署完整的业务服务器，其中均包括信令处理模块和媒体处理模块。将信令处理模块和媒体处理模块同时部署在边缘计算节点，增加了系统部署的复杂度，而且也增加了系统部署成本；此外，在系统升级或扩容时，需要分别对各个边缘计算节点中部署的信令处理模块或媒体处理模块都进行升级或者扩容，系统运 维效率和成本也比较高。
在多媒体会话业务系统中，信令处理模块负责媒体会话的建立，一路会话从发起到结束，整个信令交互占用的计算资源很少，其所占用的网络资源同媒体传输所占用的网络资源相比也可以忽略不计。基于上述内容，本申请提供一种多媒体数据处理方法，该多媒体数据处理方法可以应用于如图4所示的面向多媒体会话类应用的部署架构。
参考图4所示，在该部署架构中，将多媒体数据处理装置分为信令处理模块和媒体处理模块两个模块，信令处理模块部署在远端节点，例如可以部署在上述区域网络计算平台106或者互联网计算平台107；媒体处理模块部署在边缘计算节点105，从而实现一种简单、灵活的边缘计算应用。基于该部署架构，用户终端设备和多媒体数据处理装置之间的多媒体会话的信令处理由部署在远端节点的信令处理模块来完成，而用户终端设备与多媒体数据处理装置之间的多媒体处理由分散部署在边缘计算节点的媒体处理模块来完成。
下面结合图5所示，对基于上述部署架构的多媒体数据处理方法进行介绍。参考图5所示，该方法可以包括以下步骤S510至步骤S520：
步骤S510，通过部署于远端节点的信令处理模块，建立终端设备和媒体处理模块之间的会话，并对所述会话进行控制。
本示例实施方式中，会话是指一个终端设备与媒体处理模块进行通讯的过程。基于信令处理模块可以建立终端设备和媒体处理模块之间的会话，除此之外，信令处理模块还可以控制该会话保持以及结束该会话。
信令处理模块通常部署在远端机房。以视频监控应用场景为例，信令处理模块支持的协议可以包括SIP(Session Initiation Protocol，会话启动协议)和RTSP(Real-Time Stream Protocol，实时流传输控制协议)等多媒体会话协议。当然，根据应用场景以及需求的不同，信令处理模块支持的协议也可以包括HLS(HTTP Live Streaming，流媒体传输协议)、RTMP(Real Time Messaging Protocol，实时消息传送协议)等其它多媒体会话协议；在非多媒体会话应用场景中，信令处理模块还可以支持非多媒体会话协议；本示例性实施例中并不以此为限。
参考图6所示，本示例实施方式中，可以通过下述步骤S610至步骤S640建立终端设备和媒体处理模块之间的会话。具体而言：
在步骤S610中，向所述终端设备发起包括数据接收地址信息的呼叫请求。
例如，在接收到业务管理员发起的摄像头视频接入请求之后，信令处理模块可以向摄像头发起如SIP\RTSP呼叫请求；呼叫请求可以包括如SDP(Session Description Protocol，会话描述协议)格式的数据接收地址信息；数据接收地址信息可以包括如媒体处理模块的IP地址以及连接端口号信息 等。
在步骤S620中，接收所述终端设备对于所述呼叫请求的响应信息，所述响应信息包括数据发送地址信息。
例如，摄像头在接收到信令处理模块的呼叫请求之后，可以向信令处理模块回复响应信息；响应信息可以包括如SDP格式的数据发送地址信息；数据发送地址信息可以包括如摄像头标识(如ID等)、摄像头发送多媒体数据的源IP地址和发送端口号等信息。
在步骤S630中，向所述媒体处理模块发送所述数据发送地址信息，并通知所述媒体处理模块根据所述数据接收地址接收所述多媒体数据。
在步骤S640中，向所述终端设备发送会话建立完成信息。
在上述会话过程中，根据配置策略，用户面功能实体可以将控制面的SIP\RTSP等协议直接在终端设备和信令处理模块之间进行交换。本示例实施方式中，用户面功能实体可以是5G网络中的用户面设备，但并不以此为限；例如，用户面功能实体也可以是无线接入网关(如WiFi网关)、有线接入网关等非移动通讯网关设备。用户面功能实体对多媒体数据的转发策略可以包括静态策略和动态策略。具体而言：
基于静态策略，需要在部署于所述边缘计算节点的用户面功能实体预先配置转发策略；该转发策略包括：将接收端口在预设范围的多媒体数据转发至媒体处理模块；其中，数据接收地址信息包括在该预设范围内的目标接收端口。
同时，在多媒体会话协商过程中，需要要求终端设备设定接收端口在预设范围。以上述视频监控应用场景为例，基于SIP协议，可以在SDP协商过程中，要求摄像头将发送的多媒体数据的目标接收端口限定为10000～20000，然后在用户面功能实体中配置转发策略，凡是接收到摄像头发送过来的目标接收端口在10000～20000、类型为RTP(Real-time Transport Protocol，实时传输协议)的IP包，如UDP(User Datagram Protocol，用户数据报协议)报文统一转发给媒体处理模块对应的端口。
参考图7所示，为一种基于静态策略的会话过程示意图。其中包括步骤S710至步骤S760。具体而言：
在步骤S710中，在接收到业务管理员发起摄像头视频接入请求之后，信令处理模块向摄像头发起SIP\RTSP呼叫请求；呼叫请求可以包括如SDP(Session Description Protocol，会话描述协议)格式的数据接收地址信息。数据接收地址信息需要限定数据接收端口为10000～20000范围内的某一空闲端口Port A；连接地址指定为媒体处理模块的IP地址。
在步骤S720中，摄像头在接收到信令处理模块的呼叫请求之后，可以向信令处理模块回复响应信息；响应信息可以包括如SDP格式的数据发送地址信息；数据发送地址信息可以包括摄像头标识(如ID等)、摄像头发 送多媒体数据的源IP地址和发送端口号等信息。
在步骤S730中，信令处理模块收到摄像头的响应信息之后，向媒体处理模块发送指令，包括摄像头标识、摄像头发送多媒体数据的源IP地址和发送端口号等信息，以通知媒体处理模块准备在Port A开始接收摄像头发送过来的多媒体数据。
在步骤S740中，信令处理模块向所述终端设备发送会话建立完成信息，以确认会话建立。
在步骤S750中，摄像头开始向边缘计算节点的用户面功能实体发送多媒体数据报文；多媒体数据报文的目标接收端口为Port A。
在步骤S760中，边缘计算节点的用户面功能实体在接收到摄像头发送的多媒体数据报文之后，根据预先设定的转发策略，向媒体处理模块的端口Port A转发多媒体数据报文。
基于动态策略，数据接收地址信息包括目标接收端口；其中，在向媒体处理模块发送数据发送地址信息步骤之前，还包括：在部署于边缘计算节点的用户面功能实体配置转发策略；该转发策略包括：将接收端口为目标接收端口的多媒体数据转发至媒体处理模块。
以上述视频监控应用场景为例，例如基于SIP协议，可以在SDP协商过程中，随机选择某一空闲端口Port B作为目标接收端口；然后在用户面功能实体中配置转发策略，凡是接收到摄像头发送过来的目标接收端口为Port B、类型为RTP的IP包，如UDP报文，统一转发给媒体处理模块对应的端口Port B。
参考图8所示，为一种基于动态策略的会话过程示意图。其中包括步骤S810至步骤S870。具体而言：
在步骤S810中，在接收到业务管理员发起摄像头视频接入请求之后，信令处理模块向摄像头发起SIP\RTSP呼叫请求；呼叫请求可以包括如SDP格式的数据接收地址信息。数据接收地址信息需要限定数据接收端口为随机选择的某一空闲端口Port B；连接地址指定为媒体处理模块的IP地址。
在步骤S820中，摄像头在接收到信令处理模块的呼叫请求之后，可以向信令处理模块回复响应信息；响应信息可以包括如SDP格式的数据发送地址信息；数据发送地址信息可以包括如摄像头标识(如ID等)、摄像头发送多媒体数据的源IP地址和发送端口号等信息。
在步骤S830中，信令处理模块收到摄像头的响应信息之后，向用户面功能实体发送指令，设置用户面功能实体的转发策略。该转发策略要求凡是接收到摄像头发送过来的目标接收端口为Port B、类型为RTP的IP包，如UDP报文，统一转发给媒体处理模块对应的端口Port B。
在步骤S840中，信令处理模块向媒体处理模块发送指令，包括摄像头标识、摄像头发送多媒体数据的源IP地址和发送端口号等信息，以通知媒 体处理模块准备在Port B开始接收摄像头发送过来的多媒体数据。
在步骤S850中，信令处理模块向所述终端设备发送会话建立完成信息，以确认会话建立。
在步骤S860中，摄像头开始向边缘计算节点的用户面功能实体发送多媒体数据报文；多媒体数据报文的目标接收端口为Port B。
在步骤S870中，边缘计算节点的用户面功能实体在接收到摄像头发送的多媒体数据报文之后，根据预先设定的转发策略，向媒体处理模块的端口Port B转发多媒体数据报文。
此外，在所述会话结束后，则可以删除上述设定的动态转发策略，以避免对后续会话的建立造成影响。
步骤S520，通过部署于边缘计算节点的所述媒体处理模块，在所述媒体处理模块与所述终端设备建立会话后，接收所述终端设备发送的多媒体数据，并对所述多媒体数据进行处理。
本示例实施方式中，媒体处理模块部署在边缘计算节点。媒体处理模块主要负责多媒体数据的处理，包括音视频媒体包的接收、音视频数据的解码以及基于深度学习方法的音视频内容分析等。如果采用深度学习方法进行音视频内容分析，媒体处理模块通常需要GPU来加速媒体分析的计算性能。基于此，本示例实施方式中，所述对所述多媒体数据进行处理可以包括步骤S910以及步骤S920。其中：
在步骤S910中，对所述多媒体数据进行解码，得到解码数据。
本示例实施方式中，可以对多媒体数据进行无损解码，即解码图像和原始图像严格相同，压缩是完全可恢复的或无偏差的，没有失真。当然，为了减少数据量，也可以采用有损解码。解码模块可以是软件模块，也可以是硬件模块；本示例性实施例中对此不做特殊限定。
在步骤S920中，对所述解码数据进行计算分析以转换为结构化多媒体数据。
本示例实施方式中，可以对多媒体数据内容中的重要信息进行结构化提取，采用时空分割、特征提取、对象识别、深度学习等处理手段，组织成可供计算机和人理解的文本信息或可视化图形信息。通过将多媒体数据结构化，可以将原本数量庞大、响应缓慢、且大多没有应用价值的监控视频进行精缩，从而极大提升搜索和排查效率，同时大幅度降低存储空间占用。
参考图7中步骤S770或者图8中步骤S880中，在所述媒体处理模块将多媒体数据转换为结构化多媒体数据之后，还可以将所述结构化多媒体数据发送至所述远端节点。例如，发送至区域网络计算平台106或者互联网计算平台107的数据处理模块。
本示例实施方式中，数据处理模块可以根据产品需要，利用结构化多媒体数据进行多媒体检索或识别等工作，例如人脸检索、人脸识别、人体行为 分析等。例如，可以基于处理结构建立结构化数据仓库，如人脸照片数据库、人脸特征数据库、行为图片及特征库、车辆图像及特征库等等，并综合这些数据仓库以及关联视频片段仓库建立相应的检索引擎，实现对各类的数据仓库的深度信息挖掘，充分发挥大数据作用，提升视频数据的应用价值，提高视频数据的分析和预测功能。
本示例实施方式中的多媒体数据处理方法中，将媒体处理模块部署在边缘计算节点，将信令处理模块部署在远端节点。媒体处理模块部署的数量取决于接入的终端设备的数目，如摄像头的数目。而信令处理模块只需处理控制信令，占用的网络资源和计算资源都不大，因此可以由一个信令处理模块同时负责多个分散在不同边缘计算节点的媒体处理模块与多个终端设备之间的会话。例如，以部署在一个通用的X86架构的服务器上SIP信令处理模块为例，一般可以同时支持数千路的SIP呼叫，即可以处理同时数千个支持SIP协议的摄像头与媒体处理模块之间的会话；例如信令处理模块启动一个或数个服务实例即可满足全网全部边缘计算节点的媒体处理模块与终端设备的服务要求。因此，基于本示例实施方式中的方法，可以简化多媒体数据处理应用的部署模式，显著地降低边缘计算节点的部署复杂性；同时，还可以减少边缘计算节点所需的基础设施资源，从而可以降低部署成本。
同时，由于信令处理模块部署在远端节点，安装、升级和维护也变得更加简单和便利。对于边缘计算节点而言，信令处理模块是一种有状态的计算服务，而媒体处理模块是一种无状态的计算服务；在将有状态的信令处理模块部署在远端节点之后，边缘计算节点中只有无状态的媒体处理模块，进而边缘计算节点中部署的边缘计算服务将成为一致性的单一服务类型，边缘计算节点的管理、运维工作将变得更加高效。因此，基于本示例实施方式中的方法可以极大地提升整个系统的运维效率，降低运维成本。
此外，在本示例实施方式中的多媒体数据处理方法中，对用户面功能实体而言，由于无需将信令控制流量卸载(Traffic Offload)到边缘计算节点，用户面功能实体的转发策略配置也变得更加简单便捷，因此还可以降低整个系统流量卸载功能的复杂性。
在上述示例性实施例中是以视频监控应用场景为例进行的说明。在本申请的其他示例性实施例中，本申请的多媒体数据处理方法也可以应用于语音监测、车联网和自动驾驶应用、视频点播、游戏娱乐等其他的应用场景，这同样属于本申请的保护范围。
应当注意，尽管在附图中以特定顺序描述了本申请中方法的各个步骤，但是，这并非要求或者暗示必须按照该特定顺序来执行这些步骤，或是必须执行全部所示的步骤才能实现期望的结果。附加的或备选的，可以省略某些步骤，将多个步骤合并为一个步骤执行，以及/或者将一个步骤分解为多个步骤执行等。
进一步的，本示例实施方式中，还提供了一种多媒体数据处理装置。参考图10所示，该多媒体数据处理装置1000可以包括信令处理模块1010以及媒体处理模块1020。其中：
所述信令处理模块1010，部署于远端节点，用于建立终端设备和媒体处理模块1020之间的会话，并对所述会话进行控制；
所述媒体处理模块1020，部署于边缘计算节点，用于自身在与所述终端设备建立会话后，接收所述终端设备发送的多媒体数据，并对所述多媒体数据进行处理。
本示例实施方式中，所述装置包括多个信令处理模块1010以及多个所述媒体处理模块1020，所述信令处理模块1010的数量少于所述媒体处理模块1020的数量。
本示例实施方式中，所述装置包括一个信令处理模块1010以及多个所述媒体处理模块1020。
本示例实施方式中，所述远端节点包括区域网络计算平台和/或互联网计算平台。
本示例实施方式中，所述信令处理模块1010通过下述步骤建立终端设备和媒体处理模块1020之间的会话：
向所述终端设备发起包括数据接收地址信息的呼叫请求；
接收所述终端设备对于所述呼叫请求的响应信息，所述响应信息包括数据发送地址信息；
向所述媒体处理模块1020发送所述数据发送地址信息，并通知所述媒体处理模块1020根据所述数据接收地址接收所述多媒体数据；
向所述终端设备发送会话建立完成信息。
本示例实施方式中，所述信令处理模块1010还用于：
在部署于所述边缘计算节点的用户面功能实体预先配置转发策略；所述转发策略包括：将接收端口在预设范围的多媒体数据转发至所述媒体处理模块1020；其中，所述数据接收地址信息包括在所述预设范围内的目标接收端口。
本示例实施方式中，所述数据接收地址信息包括目标接收端口；其中，所述信令处理模块1010在向所述媒体处理模块1020发送所述数据发送地址信息步骤之前还用于：
在部署于所述边缘计算节点的用户面功能实体配置转发策略；所述转发策略包括：将接收端口为所述目标接收端口的多媒体数据转发至所述媒体处理模块1020。
本示例实施方式中，所述信令处理模块1010还用于：
在所述会话结束后，删除所述转发策略。
本示例实施方式中，所述装置包括：
用户面功能实体，用于按照所述转发策略将所述终端设备发送的多媒体数据转发至所述媒体处理模块1020。
本示例实施方式中，对所述会话进行控制包括：保持所述会话以及结束所述会话。
本示例实施方式中，所述媒体处理模块1020通过如下步骤对所述多媒体数据进行处理：
对所述多媒体数据进行解码，得到解码数据；
对所述解码数据进行计算分析以转换为结构化多媒体数据。
本示例实施方式中，所述媒体处理模块1020还用于：
将所述结构化多媒体数据发送至所述远端节点。
上述多媒体数据处理装置中各模块或单元的具体细节已经在对应的多媒体数据处理方法中进行了详细的描述，因此此处不再赘述。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本申请的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
本领域技术人员在考虑说明书及实践这里申请的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。

Claims (16)

  1. 一种多媒体数据处理方法，由服务器执行，包括：
    通过部署于远端节点的信令处理模块，建立终端设备和媒体处理模块之间的会话，并对所述会话进行控制；
    通过部署于边缘计算节点的所述媒体处理模块，在所述媒体处理模块与所述终端设备建立会话后，接收所述终端设备发送的多媒体数据，并对所述多媒体数据进行处理。
  2. 根据权利要求1所述的多媒体数据处理方法，所述建立终端设备和媒体处理模块之间的会话，包括：
    向所述终端设备发起包括数据接收地址信息的呼叫请求；
    接收所述终端设备对于所述呼叫请求的响应信息，所述响应信息包括数据发送地址信息；
    向所述媒体处理模块发送所述数据发送地址信息，并通知所述媒体处理模块根据所述数据接收地址接收所述多媒体数据；
    向所述终端设备发送会话建立完成信息。
  3. 根据权利要求2所述的多媒体数据处理方法，所述方法还包括：
    在部署于所述边缘计算节点的用户面功能实体预先配置转发策略；所述转发策略包括：将接收端口在预设范围的多媒体数据转发至所述媒体处理模块；
    其中，所述数据接收地址信息包括在所述预设范围内的目标接收端口。
  4. 根据权利要求2所述的多媒体数据处理方法，所述数据接收地址信息包括目标接收端口；其中，在所述向所述媒体处理模块发送所述数据发送地址信息之前，还包括：
    在部署于所述边缘计算节点的用户面功能实体配置转发策略；所述转发策略包括：将接收端口为所述目标接收端口的多媒体数据转发至所述媒体处理模块。
  5. 根据权利要求4所述的多媒体数据处理方法，所述方法还包括：
    在所述会话结束后，删除所述转发策略。
  6. 根据权利要求3～5任一项所述的多媒体数据处理方法，所述方法还包括：
    通过所述用户面功能实体，按照所述转发策略将所述终端设备发送的多媒体数据转发至所述媒体处理模块。
  7. 根据权利要求1所述的多媒体数据处理方法，所述对所述会话进行控制，包括：保持所述会话以及结束所述会话。
  8. 根据权利要求1所述的多媒体数据处理方法，所述对所述多媒体数据进行处理，包括：
    对所述多媒体数据进行解码，得到解码数据；
    对所述解码数据进行计算分析以转换为结构化多媒体数据。
  9. 根据权利要求8所述的多媒体数据处理方法，在所述对所述多媒体数据进行处理之后，还包括：
    将所述结构化多媒体数据发送至所述远端节点。
  10. 一种多媒体数据处理装置，包括：
    信令处理模块，部署于远端节点，用于建立终端设备和媒体处理模块之间的会话，并对所述会话进行控制；
    媒体处理模块，部署于边缘计算节点，用于在自身与所述终端设备建立会话后，接收所述终端设备发送的多媒体数据，并对所述多媒体数据进行处理。
  11. 根据权利要求10所述的多媒体数据处理装置，所述装置包括多个信令处理模块以及多个所述媒体处理模块，所述信令处理模块的数量少于所述媒体处理模块的数量。
  12. 根据权利要求10所述的多媒体数据处理装置，所述装置包括一个信令处理模块以及多个所述媒体处理模块。
  13. 根据权利要求10所述的多媒体数据处理装置，所述远端节点包括区域网络计算平台和/或互联网计算平台。
  14. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现权利要求1-9任一项所述的方法。
  15. 一种电子设备，包括：
    处理器；以及
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1-9任一项所述的方法。
  16. 一种计算机程序产品，包括指令，当其在计算机上运行时，使得计算机执行如权利要求1-9中任一项所述的方法。
PCT/CN2020/092750 2019-06-28 2020-05-28 多媒体数据处理方法及装置、存储介质、电子设备 Ceased WO2020259192A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP20832485.5A EP3993341B1 (en) 2019-06-28 2020-05-28 Multimedia data processing method and apparatus, storage medium and electronic device
JP2021555311A JP7410966B2 (ja) 2019-06-28 2020-05-28 マルチメディアデータ処理方法、装置、電子機器及びコンピュータプログラム
US17/469,872 US11522922B2 (en) 2019-06-28 2021-09-08 Multimedia data processing method and apparatus, storage medium, and electronic device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910578317.6 2019-06-28
CN201910578317.6A CN110290140B (zh) 2019-06-28 2019-06-28 多媒体数据处理方法及装置、存储介质、电子设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/469,872 Continuation US11522922B2 (en) 2019-06-28 2021-09-08 Multimedia data processing method and apparatus, storage medium, and electronic device

Publications (1)

Publication Number Publication Date
WO2020259192A1 true WO2020259192A1 (zh) 2020-12-30

Family

ID=68020053

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/092750 Ceased WO2020259192A1 (zh) 2019-06-28 2020-05-28 多媒体数据处理方法及装置、存储介质、电子设备

Country Status (5)

Country Link
US (1) US11522922B2 (zh)
EP (1) EP3993341B1 (zh)
JP (1) JP7410966B2 (zh)
CN (1) CN110290140B (zh)
WO (1) WO2020259192A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110290140B (zh) 2019-06-28 2021-09-24 腾讯科技（深圳）有限公司 多媒体数据处理方法及装置、存储介质、电子设备
CN111225280B (zh) * 2020-01-22 2021-10-01 复旦大学 基于嵌入式平台的轻量级视频分析系统
CN113538024B (zh) * 2020-04-14 2023-12-26 中国移动通信集团浙江有限公司 广告管理方法、系统及内容传输网络设备
CN114268591B (zh) * 2020-09-16 2024-03-26 华为技术有限公司 数据分流方法和装置
CN113158717A (zh) * 2020-11-25 2021-07-23 杭州视洞科技有限公司 一种端及云人脸识别架构
CN115412587A (zh) 2021-05-26 2022-11-29 中兴通讯股份有限公司 多媒体云服务方法、通信云系统及计算机可读存储介质
CN116546102A (zh) * 2023-06-05 2023-08-04 平安银行股份有限公司 一种http请求灰度分发方法、装置和设备
CN116708441B (zh) * 2023-07-11 2026-03-03 山东亚华电子股份有限公司 一种用于智慧医院的媒体终端互通方法、设备及介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180049179A1 (en) * 2016-08-09 2018-02-15 Wipro Limited Method and a system for identifying operating modes of communications in mobile-edge computing environment
CN108063773A (zh) * 2018-02-11 2018-05-22 清华大学 一种基于移动边缘计算的应用服务接入方法及装置
CN108355350A (zh) * 2018-02-11 2018-08-03 清华大学 一种基于移动边缘计算的应用服务接入方法及装置
CN108496341A (zh) * 2017-06-30 2018-09-04 北京小米移动软件有限公司 蜂窝网络中实现边缘计算的方法、装置、设备及基站
US20180376444A1 (en) * 2017-06-21 2018-12-27 Lg Electronics Inc. Method for performing service request procedure and apparatus therefor in wireless communication system
CN110290140A (zh) * 2019-06-28 2019-09-27 腾讯科技（深圳）有限公司 多媒体数据处理方法及装置、存储介质、电子设备

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050021626A1 (en) * 2003-05-22 2005-01-27 Cisco Technology, Inc. Peer-to-peer dynamic web page sharing
US20050060411A1 (en) * 2003-09-16 2005-03-17 Stephane Coulombe System and method for adaptation of peer-to-peer multimedia sessions
CN100370872C (zh) * 2004-04-30 2008-02-20 华为技术有限公司 会话建立方法
CN100356737C (zh) * 2004-12-02 2007-12-19 华为技术有限公司 一种对媒体服务器进行大容量测试的系统及方法
CN101123718B (zh) * 2007-09-13 2011-12-21 华为技术有限公司 多媒体点播的方法及系统
US9479967B1 (en) * 2007-10-26 2016-10-25 Genband Us Llc Enhanced media gateway
US7979558B2 (en) * 2008-08-06 2011-07-12 Futurewei Technologies, Inc. Remote session control
CN102281251B (zh) * 2010-06-09 2014-12-17 中兴通讯股份有限公司 一种实现智能家居应用的装置、系统和方法
CN102984402B (zh) * 2011-09-06 2015-08-12 中兴通讯股份有限公司 一种VoIP数据包的处理方法及系统
CN103179213B (zh) * 2013-04-03 2016-05-18 深圳市闪联信息技术有限公司 基于点对点代理机制的家庭媒体资源传输系统
US10356059B2 (en) 2015-06-04 2019-07-16 Nagravision S.A. Methods and systems for communication-session arrangement on behalf of cryptographic endpoints
EP3326406B1 (en) * 2015-07-23 2020-05-27 Nokia Solutions and Networks Oy Acceleration facility control in a network
CN106713819A (zh) * 2015-07-24 2017-05-24 三亚中兴软件有限责任公司 用于视频会议的数据传输方法、装置及系统
US20170054770A1 (en) * 2015-08-23 2017-02-23 Tornaditech Llc Multimedia teleconference streaming architecture between heterogeneous computer systems
WO2017037776A1 (ja) 2015-08-28 2017-03-09 日本電気株式会社 端末とネットワークノードと通信制御方法並びにプログラム
CN107172664B (zh) 2016-03-07 2020-04-03 大唐移动通信设备有限公司 数据传输方法、装置及系统
CN107566437B (zh) * 2016-07-01 2020-04-14 大唐移动通信设备有限公司 信息传输方法及装置
CN108738104B (zh) * 2017-04-19 2021-11-19 华为技术有限公司 一种建立本地网络连接的方法、装置、系统和存储介质
US11082455B2 (en) * 2017-05-03 2021-08-03 T-Mobile Usa, Inc. Network gateway transcoder-utilization-aware session control
CN107466482B (zh) * 2017-06-07 2021-07-06 香港应用科技研究院有限公司 在蜂窝通信系统中联合确定计算卸载和内容预取的方法和系统
CN113194157B (zh) 2017-06-30 2022-10-28 华为技术有限公司 一种应用实例地址的转换方法和装置
CN109274634B (zh) * 2017-07-18 2021-06-11 腾讯科技（深圳）有限公司 多媒体通信方法及装置、存储介质
WO2019080105A1 (zh) * 2017-10-27 2019-05-02 北京小米移动软件有限公司 网络接入方法及装置
CN111684824B (zh) 2017-12-15 2023-04-11 索尼公司 增强的nef功能、mec和5g集成
CN108234511B (zh) * 2018-01-18 2021-07-13 携程旅游信息技术（上海）有限公司 多媒体数据传输的方法、系统、设备、存储介质及网关
US20190215729A1 (en) * 2018-03-15 2019-07-11 Intel Corporation Session description protocol mechanisms for signaling radio access network capabilities in multimedia telephony sessions
KR102882847B1 (ko) * 2019-01-03 2025-11-10 삼성전자 주식회사 IMS(IP multimedia subsystem) 서비스를 제공하는 전자 장치
CN110198363B (zh) * 2019-05-10 2021-05-18 深圳市腾讯计算机系统有限公司 一种移动边缘计算节点的选择方法、装置及系统
CN110912835B (zh) * 2019-11-08 2023-04-07 腾讯科技（深圳）有限公司 业务分流方法、装置及系统
US20220086218A1 (en) * 2020-12-23 2022-03-17 Dario Sabella Interoperable framework for secure dual mode edge application programming interface consumption in hybrid edge computing platforms

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180049179A1 (en) * 2016-08-09 2018-02-15 Wipro Limited Method and a system for identifying operating modes of communications in mobile-edge computing environment
US20180376444A1 (en) * 2017-06-21 2018-12-27 Lg Electronics Inc. Method for performing service request procedure and apparatus therefor in wireless communication system
CN108496341A (zh) * 2017-06-30 2018-09-04 北京小米移动软件有限公司 蜂窝网络中实现边缘计算的方法、装置、设备及基站
CN108063773A (zh) * 2018-02-11 2018-05-22 清华大学 一种基于移动边缘计算的应用服务接入方法及装置
CN108355350A (zh) * 2018-02-11 2018-08-03 清华大学 一种基于移动边缘计算的应用服务接入方法及装置
CN110290140A (zh) * 2019-06-28 2019-09-27 腾讯科技（深圳）有限公司 多媒体数据处理方法及装置、存储介质、电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3993341A4

Also Published As

Publication number Publication date
US20210409460A1 (en) 2021-12-30
EP3993341B1 (en) 2024-07-31
JP7410966B2 (ja) 2024-01-10
US11522922B2 (en) 2022-12-06
CN110290140B (zh) 2021-09-24
EP3993341A1 (en) 2022-05-04
EP3993341A4 (en) 2022-08-03
JP2022538953A (ja) 2022-09-07
CN110290140A (zh) 2019-09-27

Similar Documents

Publication Publication Date Title
WO2020259192A1 (zh) 多媒体数据处理方法及装置、存储介质、电子设备
CN113596191B (zh) 一种数据处理方法、网元设备以及可读存储介质
CN109640348B (zh) 多业务mec网络架构、多业务数据流的处理方法及装置
US9197701B1 (en) Consolidated peer-to-peer media sessions for audio and/or video communications
US9602553B2 (en) Method, apparatus, and system for implementing VOIP call in cloud computing environment
CN116546454A (zh) 网络辅助信息提供方法及相关设备
CN104394127B (zh) 一种多媒体分享方法、设备和系统
CN103348657B (zh) 流媒体播放方法、设备及系统
WO2023000940A1 (zh) 数据处理方法、装置、网元设备、存储介质及程序产品
CN105282624A (zh) 桌面云视频的播放处理方法及装置
WO2023000936A1 (zh) 一种数据处理方法、网元设备以及可读存储介质
CN115022725B (zh) 一种视频播放方法和装置
WO2018006678A1 (zh) 语音通话方法及装置
US9686334B2 (en) Transmission method for media data stream and thin client
CN103151041B (zh) 一种自动语音识别业务的实现方法、系统和媒体服务器
CN102571409A (zh) 一种用于实现业务跨终端使用的业务请求管理系统及方法
CN114629912B (zh) 基于mec的通信传输方法及装置
EP4401388B1 (en) Method and system for managing multimedia exchange in tele-robotics using dew computing
CN110708293A (zh) 多媒体业务的分流方法和装置
WO2023227059A1 (zh) 协商方法、装置、网络设备及终端
Collins II et al. 5G deployment considerations to enable consumer innovation
US12244659B2 (en) Multimedia cloud service method, communication cloud system and computer readable storage medium
CN112738026B (zh) 一种指挥调度方法
CN121397176A (zh) 一种视频交互方法、装置、存储介质及电子设备
Serrano et al. Kaleidoscope: A Cloud-Based Platform for Real-Time Video-Based Interaction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20832485

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021555311

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2020832485

Country of ref document: EP