WO2020233708A1 - 创建网络仿真平台的方法和网络仿真方法及相应装置 - Google Patents

创建网络仿真平台的方法和网络仿真方法及相应装置 Download PDF

Info

Publication number
WO2020233708A1
WO2020233708A1 PCT/CN2020/091813 CN2020091813W WO2020233708A1 WO 2020233708 A1 WO2020233708 A1 WO 2020233708A1 CN 2020091813 W CN2020091813 W CN 2020091813W WO 2020233708 A1 WO2020233708 A1 WO 2020233708A1
Authority
WO
WIPO (PCT)
Prior art keywords
network
black box
box model
network performance
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/091813
Other languages
English (en)
French (fr)
Inventor
叶国骏
成博
李汐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to KR1020217019934A priority Critical patent/KR102543557B1/ko
Priority to EP20808886.4A priority patent/EP3879758B1/en
Publication of WO2020233708A1 publication Critical patent/WO2020233708A1/zh
Priority to US17/345,273 priority patent/US11856424B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/06—Testing, supervising or monitoring using simulated traffic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852—Delays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852—Delays
    • H04L43/087—Jitter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H04L43/0888—Throughput
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W16/00—Network planning, e.g. coverage or traffic planning tools; Network deployment, e.g. resource partitioning or cells structures
    • H04W16/22—Traffic simulation tools or models

Definitions

  • This application relates to the field of simulation technology, and in particular to a method for creating a network simulation platform and a network simulation method and device.
  • Wireless communication simulation technology can be divided into link simulation, system simulation and network simulation.
  • Link simulation is channel-level simulation (such as physical channel coverage and capacity simulation)
  • system simulation is base station-level simulation (such as protocol flow and scheduling simulation)
  • network simulation is network-level simulation (such as network coverage and capacity simulation). Therefore, the simulation scale of network simulation is relatively large, usually at the level of thousands of stations or even 10,000 stations.
  • the embodiments of the present application provide a method for creating a network simulation platform, a network simulation method and corresponding devices, which can be applied to the network simulation of a 5G communication system.
  • an embodiment of the present application provides a method for creating a network simulation platform.
  • the network simulation platform includes at least one black box model, and the at least one black box model includes a first black box model.
  • the method includes: firstly, obtaining candidate network performance indicators and feature information related to the candidate network performance indicators; then, training a machine learning model to obtain a first black box model.
  • the output parameter of the first black box model is the performance index of the candidate network, and the input parameter of the first black box model is characteristic information related to the performance index of the candidate network.
  • the first black box model is used to simulate the performance index of the candidate network.
  • the candidate network performance indicators include: network performance indicators in the core network, network performance indicators in the bearer network, network performance indicators in the access network, or network performance indicators across networks.
  • the cross-network network performance index refers to the network performance index affected by at least two of the core network, the bearer network, and the access network.
  • a black box model for simulating one or more candidate network performance indicators in the core network, the bearer network, and/or the access network can be created.
  • each black box model is defined with input parameters and output parameters (that is, input interface and output interface). So far, it can be considered that a network simulation platform has been created.
  • the network simulation platform mainly focuses on candidate network performance indicators and related input feature information, and does not need to simulate the protocol process in the actual network, so the complexity is small and it can be applied to large-scale network simulation.
  • the characteristic information related to the candidate network performance index refers to part or all of the characteristic information required to obtain the candidate network performance index.
  • the at least one black box model further includes a second black box model.
  • the output parameter of the second black box model is one of the input parameters of the first black box model. That is to say, the embodiment of the present application supports a technical solution that uses the output parameter of one black box model as the input parameter of another black box model (that is, the black box models have an association relationship).
  • This optional technical solution provides a theoretical basis for "cascading multiple black box models during the simulation process to simulate a certain network performance index" and also provides a theoretical basis for implementing end-to-end simulation .
  • the output parameter of the first black box model is the network performance index of the first network
  • the output parameter of the second black box model is the network performance index of the second network.
  • the first network is a core network, a bearer network or an access network.
  • the second network is any network other than the first network among the core network, the bearer network, and the access network.
  • the embodiments of this application support the simulation of cross-network performance indicators, that is, support end-to-end simulation; in other words, the models in the access network part, the bearer network part, and the core network part can be coordinated to complete the matching Simulation of a certain network performance index.
  • the method further includes: obtaining training data.
  • the training data comes from the actual network or other simulation platforms except the simulation platform provided in the embodiment of the present application.
  • training the machine learning model to obtain the first black box model may include: training the machine learning model based on the training data to obtain the first black box model. It is understandable that if the training data comes from the actual network, it will help improve the accuracy of the simulation.
  • the machine learning model includes a neural network model.
  • the candidate network performance indicator includes a rank
  • the characteristic information related to the rank includes a channel matrix
  • the candidate network performance index includes a channel matrix
  • the characteristic information related to the channel matrix includes multipath parameters and antenna configuration parameters.
  • the candidate network performance index includes a multipath parameter
  • the feature information related to the multipath parameter includes at least one of an electronic map, an engineering parameter, and a feature type.
  • the candidate network performance index includes the channel state probability distribution parameter of the cell
  • the characteristic information related to the channel state probability distribution parameter of the cell includes the grid channel matrix of the cell and the channel state occupancy of the cell sequentially.
  • the candidate network performance index includes the precoding matrix of the cell, and the characteristic information related to the precoding matrix of the cell includes the grid channel matrix of the cell.
  • the black box model in the simulation stage, can be used to replace the message queue processing process in the traditional network simulation platform.
  • a black box model can be used to replace a certain network element in a traditional network simulation platform, or can be used to replace a certain functional module (such as a channel model, a scheduling model, a transmission model, etc.) in a network element.
  • the black box model can communicate with data in the traditional network simulation platform.
  • the black box model provided by the embodiment of the present application can be compatible with the traditional network simulation platform.
  • an embodiment of the present application provides a network simulation method, which is applied to a network simulation platform including at least two black box models.
  • the simulation platform may be a simulation platform created based on the foregoing first aspect or any possible design of the foregoing first aspect.
  • the method includes: firstly, determining a first network performance index to be simulated; then, from the at least two black box models, searching for a first black box model for simulating the first network performance index to be simulated.
  • the output parameter of the first black box model is the performance index of the first network to be simulated.
  • this technical solution can directly simulate the performance parameters of the simulated network, which is helpful to improve the simulation efficiency compared with the traditional technology that simulates the message processing process and then indirectly simulates the performance parameters of the simulated network.
  • the first network performance indicators to be simulated include: network performance indicators in the core network, network performance indicators in the bearer network, network performance indicators in the access network, or network performance indicators across networks.
  • the method further includes: searching for a second black box model for simulating the performance index of the second network to be simulated from the at least two black box models.
  • the output parameter of the second black box model is one of the input parameters of the first black box model.
  • the value of the input parameter of the second black box model is input to the second black box model to obtain the value of the output parameter of the second black box model, and the obtained value is used as the simulation result of the performance index of the second network to be simulated.
  • inputting the value of the input parameter of the first black box model into the first black box model includes: inputting the simulation result of the second network performance index to be simulated into the first black box model.
  • the embodiments of the present application support cascading between black box models, so as to realize the simulation of a certain network performance index.
  • the output parameter of the first black box model is a network performance index in the first network
  • the output parameter of the second black box model is a network performance index in the second network.
  • the first network is a core network, a bearer network, or an access network
  • the second network is any network other than the first network among the core network, the bearer network, and the access network.
  • the embodiments of this application support the simulation of cross-network performance indicators, that is, support end-to-end simulation; in other words, the models in the access network part, the bearer network part, and the core network part can be coordinated to complete the matching Simulation of a certain network performance index.
  • the first candidate network performance indicator to be simulated includes a rank
  • the characteristic information related to the rank includes a channel matrix
  • the first network performance indicator to be simulated includes a channel matrix
  • the output parameters of the first black box model include multipath parameters and antenna configuration parameters.
  • the first network performance indicator to be simulated includes multipath parameters
  • the output parameters of the first black box model include electronic maps, engineering parameters, and feature types.
  • the first network performance indicator to be simulated includes the channel state probability distribution parameter of the cell
  • the output parameter of the first black box model includes the grid channel matrix of the cell and the channel state occupation time sequence of the cell.
  • the first network performance indicator to be simulated includes the precoding matrix of the cell, and the output parameter of the first black box model includes the grid channel matrix of the cell.
  • an embodiment of the present application provides an apparatus for creating a network simulation platform, which may be used to execute any method provided in the foregoing first aspect or any possible design of the first aspect.
  • the device can be a server or a chip.
  • the device may be divided into functional modules according to the method provided in the first aspect or any of the possible designs of the first aspect.
  • each functional module may be divided corresponding to each function, or Integrate two or more functions into one processing module.
  • the device may include a memory and a processor.
  • the memory is used to store computer programs.
  • the processor is used to invoke the computer program to execute the first aspect or the method provided by any possible design of the first aspect.
  • an embodiment of the present application provides a network simulation device, which can be used to implement the foregoing second aspect or any method provided by any possible design of the second aspect.
  • the device can be a server or a chip.
  • the device can be divided into functional modules according to the method provided by the second aspect or any of the possible designs of the second aspect.
  • each functional module can be divided corresponding to each function, or Integrate two or more functions into one processing module.
  • the device may include a memory and a processor, and the memory is used to store a computer program.
  • the processor is used to invoke the computer program to execute the second aspect or any one of the possible designs provided by the second aspect.
  • the embodiments of the present application provide a computer-readable storage medium, such as a non-transitory computer-readable storage medium.
  • a computer program (or instruction) is stored thereon, and when the computer program (or instruction) runs on a computer, the computer executes any method provided by the first aspect or any possible design of the first aspect .
  • embodiments of the present application provide a computer-readable storage medium, such as a computer-readable storage medium that is non-transitory.
  • a computer program (or instruction) is stored thereon, and when the computer program (or instruction) runs on a computer, the computer executes any method provided by the second aspect or any possible design of the second aspect .
  • the embodiments of the present application provide a computer program product, which when running on a computer, enables any method provided in the first aspect or any one of the possible designs of the first aspect to be executed.
  • the embodiments of the present application provide a computer program product that, when it runs on a computer, enables any method provided in the second aspect or any possible design of the second aspect to be executed.
  • any of the above-provided devices for creating network simulation platforms, network simulation devices, computer storage media, or computer program products can be applied to the corresponding methods provided above, and therefore, what it can achieve For the beneficial effects, please refer to the beneficial effects in the corresponding method, which will not be repeated here.
  • Figure 1 is a schematic diagram of a traditional network simulation platform
  • FIG. 2 is a schematic structural diagram of a server applicable to an embodiment of the present application
  • FIG. 3 is a schematic flowchart of a method for creating a network simulation platform provided by an embodiment of the application
  • FIG. 4 is a schematic diagram of a network simulation platform provided by an embodiment of the application.
  • 5A is a schematic diagram of cascading between black box models provided by an embodiment of the application.
  • 5B is a schematic diagram of cascading between another black box model provided by an embodiment of the application.
  • FIG. 6 is a schematic diagram of a multipath parameter model and a channel matrix model provided by an embodiment of the application.
  • FIG. 7A is a schematic diagram of a network simulation platform applicable to embodiments of the present application.
  • FIG. 7B is a schematic diagram of another network simulation platform applicable to embodiments of the present application.
  • FIG. 8 is a schematic flowchart of a network simulation method provided by an embodiment of this application.
  • FIG. 9 is a schematic flowchart of another network simulation method provided by an embodiment of the application.
  • FIG. 10 is a schematic structural diagram of a device for creating a network simulation platform provided by an embodiment of the application.
  • FIG. 11 is a schematic structural diagram of a network simulation device provided by an embodiment of this application.
  • a network simulation platform which can also be called a network simulation system, refers to software running on a server (or called a simulation device) to perform network simulation tasks. Among them, performing a network simulation task can be understood as simulating one or more network performance indicators (such as channel matrix, multipath parameters, cell interference, etc.).
  • FIG. 1 it is a schematic diagram of a traditional network simulation platform.
  • the traditional network simulation platform shown in Figure 1 is implemented using system simulation technology.
  • the simulation process consists of antenna modeling (the creation process of the antenna array model in Figure 1) and channel modeling (the multi-model spectrum in Figure 1). Channel creation process), scheduling algorithm processing (not shown in FIG. 1, specifically may be included in the MAC layer), and protocol flow processing.
  • the network simulation platform needs to simulate the information interaction between the protocol layers, and process each data message based on the simulated protocol layer, and simulate the interaction of data messages between network elements.
  • the network simulation platform can also simulate the resource scheduling process and baseband processing process.
  • Figure 1 illustrates the protocol layer and the characteristic information of the wireless air interface that need to be simulated when communicating between network elements in a wireless communication network.
  • the network element includes: a remote host, a PDN gateway (PDN gateway, PGW)/serving gateway (SGW), a base station, and a terminal.
  • PDN is the English abbreviation for public data network.
  • the protocol layers that need to be simulated include: application layer, transmission control protocol (TCP)/user datagram protocol (UDP), and internet protocol (IP) layer .
  • TCP transmission control protocol
  • UDP user datagram protocol
  • IP internet protocol
  • the protocol layers that need to be simulated include: the first IP layer, the application layer, the GPRS tunneling protocol (GPRS tuning protocol, GTP) layer, the UDP layer, and the second IP layer.
  • GPRS is the English abbreviation for general packet radio service (general packet radio service).
  • the first IP layer corresponds to the IP layer of the remote host.
  • the protocol layers that need to be simulated include: IP, UDP, and GTP layers corresponding to the second IP layer, UDP layer, and GTP layer of the PGW/SGW, as well as the application layer and radio resource control (radio resource control).
  • the protocol layers that need to be simulated include: the PHY layer, MAC layer, RLC layer, PDCP layer, and RRC layer corresponding to the base station's PHY layer, MAC layer, RLC layer, PDCP layer, and RRC layer, and The IP layer, TCP/UDP layer and application layer of the end host correspond to the IP layer, TCP/UDP layer and application layer respectively.
  • the "correspondence between the two protocol layers” means that the processing performed by one of the two protocol layers is the inverse of the processing performed by the other protocol layer.
  • the characteristic information of the wireless air interface part needs to be simulated between the base station and the terminal, such as the need to simulate the spectral PHY layer, the multi-model spectral channel, the beamforming model (used to simulate beamforming technology), and the propagation loss model (used to simulate the information transmission process). Energy loss), mutual information error (error, miError) model and antenna array model (used to simulate the characteristics of the antenna array of base stations and terminals), etc.
  • system simulation technology can realize dynamic simulation, system simulation realizes simulation by simulating the message interaction mechanism between the network protocol layer and network elements, which will lead to a huge amount of calculation and low simulation efficiency, which makes traditional system simulation unable to be applied to large-scale applications.
  • system simulation technology is mainly used to simulate a single station, and focuses on the simulation of the wireless air interface part, and cannot achieve end-to-end simulation (ie, simulation of network performance indicators across networks).
  • the embodiments of the present application provide a method and device for creating a network simulation platform, and a network simulation method and device.
  • the technical solutions provided by the embodiments of the present application can be applied to various communication systems, for example, the 5th generation (5G) mobile communication system such as the 5G new radio (NR) system, the future evolution system or multiple communications In converged systems, etc., it can also be applied to existing communication systems.
  • the server (marked as the first server) used to execute the method of creating a network simulation platform and the server (marked as the second server) used to execute the network simulation method may be the same server or different servers.
  • the second server may directly or indirectly exchange information with the first server to obtain the network simulation platform created by the first server, and then perform network simulation based on the network simulation platform.
  • the server including the first server and the second server
  • FIG. 2 it is a schematic structural diagram of a server 20 (including the above-mentioned first server and second server) applicable to the embodiment of the present application.
  • the server 20 is used to execute the method for creating a network simulation platform and/or the network simulation method provided in the embodiments of the present application.
  • the server 20 may include at least one processor 201, a communication line 202, a memory 203, and at least one communication interface 204.
  • the processor 201 can be a general-purpose central processing unit (central processing unit, CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more programs for controlling the execution of the program of this application. integrated circuit.
  • CPU central processing unit
  • ASIC application-specific integrated circuit
  • the communication line 202 may include a path to transmit information between the aforementioned components (such as at least one processor 201, communication line 202, memory 203, and at least one communication interface 204).
  • the communication interface 204 uses any device such as a transceiver to communicate with other devices or communication networks, such as wide area networks (WAN), local area networks (LAN), and so on.
  • WAN wide area networks
  • LAN local area networks
  • the memory 203 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM), or other types that can store information and instructions
  • the dynamic storage device can also be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, optical disc storage (Including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired program codes in the form of instructions or data structures and can be used by a computer Any other media accessed, but not limited to this.
  • the memory 203 may exist independently and is connected to the processor 201 through a communication line 202.
  • the memory 203 may also be integrated with the processor 201.
  • the memory 203 provided by the embodiment of the present application may generally be non-volatile. Among them, the memory 203 is used to store computer instructions for executing the solution of the present application, and the processor 201 controls the execution.
  • the processor 201 is configured to execute computer instructions stored in the memory 203, so as to implement the method provided in the following embodiments of the present application.
  • the computer instructions in the embodiments of the present application may also be referred to as application program codes, which are not specifically limited in the embodiments of the present application.
  • the server 20 may include multiple processors, such as the processor 201 and the processor 207 in FIG. 2.
  • processors can be a single-CPU (single-CPU) processor or a multi-core (multi-CPU) processor.
  • the processor here may refer to one or more devices, circuits, and/or processing cores for processing data (for example, computer program instructions).
  • the server 20 may further include an output device 205 and/or an input device 206.
  • the output device 205 communicates with the processor 201 and can display information in a variety of ways.
  • the output device 205 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector (projector) Wait.
  • the input device 206 communicates with the processor 201 and can receive user input in a variety of ways.
  • the input device 206 may be a mouse, a keyboard, a touch screen device, a sensor device, or the like.
  • the black box model which can also be called the black box model (black box), is a type of model that is often used in environmental prediction work. It is created based on the input-output relationship (that is, input parameters and output parameters), reflecting the output A general direct causal relationship between parameters and input parameters.
  • a black box model generally has one output parameter and one or more input parameters.
  • End-to-end simulation can also be called cross-network network performance indicator simulation, or cross-layer network performance indicator simulation.
  • the cross-network (or cross-layer) network performance index refers to the network performance index that will be affected by multiple networks.
  • Various networks include core networks, bearer networks, and access networks.
  • the delay will be affected by the core network and the transmission network. Therefore, the delay can be regarded as a cross-network network performance index.
  • the delay can also be considered as a single network performance index, such as the delay in the core network or the delay in the transmission network.
  • At least one includes one or more.
  • Multiple means two or more.
  • at least one of A, B, and C includes: A alone, B alone, A and B at the same time, A and C at the same time, B and C at the same time, and A, B, and C at the same time.
  • "/" means or.
  • A/B can mean A or B.
  • "and/or" is only an association relationship describing the associated objects, which means that there can be three kinds of relationships, for example, A and/or B, which can mean: A alone exists, and both A and B exist. , There are three cases of B alone.
  • words such as “exemplary” or “for example” are used to represent examples, illustrations, or illustrations. Any embodiment or design solution described as “exemplary” or “for example” in this application should not be construed as being more preferable or advantageous than other embodiments or design solutions. To be precise, words such as “exemplary” or “for example” are used to present related concepts in a specific manner.
  • FIG. 3 it is a schematic flowchart of a method for creating a network simulation platform provided by an embodiment of this application.
  • the network simulation platform includes at least one black box model.
  • creating a network simulation platform specifically includes creating each of the at least one black box model.
  • the first black box model included in the creation of at least one black box model is taken as an example for description.
  • the first black box model may be any one of the at least one black box model.
  • the method shown in Figure 3 includes the following steps:
  • S101 The server obtains a candidate network performance index and characteristic information related to the candidate network performance index.
  • the server may receive the candidate network performance index and the characteristic information related to the candidate network performance index indicated by the user or sent by other devices.
  • the server may determine the candidate network performance index and characteristic information related to the candidate network performance index based on information indicated by the user or information sent by other devices.
  • the candidate network performance index may be any one of the core network, the bearer network, and the access network, or may be a cross-network network performance index.
  • the performance indicators in the core network may include at least one of the following: user distribution characteristics, user movement characteristics, service characteristics, time delay, quality of service (QoS), average subjective opinion score MOS (mean opinion score, MOS) score , Jitter, throughput, etc.
  • the performance indicators in the bearer network may include at least one of the following: delay, jitter, and throughput.
  • the performance indicators in the access network may include at least one of the following: interference, coverage, channel capacity, throughput, rank, jitter, and so on. Network performance indicators across networks can include delay, jitter, and throughput.
  • the candidate network performance index may be any network performance index that can be simulated in the existing network simulation technology. Of course, the embodiments of the present application are not limited to this.
  • the characteristic information related to a candidate network performance index refers to part or all of the characteristic information required to obtain the candidate network performance index.
  • the number of feature information related to a candidate network performance index may be one or more.
  • the specific parameters of the one or more parameters may be determined based on the degree of influence on the performance index of the candidate network, etc., for example, the one or more parameters may be one or more that have a greater impact on the performance index of the candidate network parameter.
  • candidate network performance indicators include rank
  • characteristic information related to rank includes channel matrices.
  • the channel matrix may be the channel matrix between the terminal and the main serving base station of the terminal.
  • candidate network performance indicators include a channel matrix
  • characteristic information related to the channel matrix includes multipath parameters and antenna configuration parameters.
  • the multipath parameter may be the multipath parameter of the channel between the terminal and the primary serving cell
  • the antenna configuration parameter may be the The antenna configuration parameters of the terminal and the antenna configuration parameters of the main serving base station.
  • Multipath parameters may include angle and/or delay spread.
  • the antenna configuration parameters may include information such as the number and orientation of the antennas included in the antenna array.
  • the candidate network performance indicators include multipath parameters
  • the feature information related to the multipath parameters includes at least one of an electronic map, an engineering parameter, and a feature type.
  • the multipath parameter can be the multipath parameter of the channel between the terminal and the main serving base station
  • the electronic map, engineering parameter, and feature type are used to characterize the location of the terminal and the main serving base station.
  • Electronic maps, engineering parameters and feature types of the physical environment In this way, the simulation accuracy of multipath parameters can be improved.
  • the candidate network performance index includes the channel state probability distribution parameter of the cell
  • the characteristic information related to the channel state probability distribution parameter of the cell includes the grid channel matrix of the cell and the channel state occupation time sequence of the cell.
  • the candidate network performance index includes the precoding matrix of the cell
  • the characteristic information related to the precoding matrix of the cell includes the grid channel matrix of the cell.
  • the server determines to acquire training data.
  • the training data includes multiple sets of values, and each set of values includes a value of the candidate network performance index and specific feature information used to obtain the value.
  • the specific feature information can be understood as an example of the feature information determined in S101.
  • the characteristic information determined in S101 includes a channel matrix
  • an example of the characteristic information is a specific matrix.
  • an example of the characteristic information is a multipath parameter of a channel between a specific terminal and the primary serving base station of the specific terminal.
  • the training data can include: ⁇ multipath parameter 1, antenna configuration parameter 1, channel matrix 1 ⁇ , ⁇ Multipath parameter 2, antenna configuration parameter 2, channel matrix 2 ⁇ .
  • ⁇ multipath parameter n, antenna configuration parameter n, channel matrix n ⁇ is a set of values in the training data, this set of values is used to indicate that when the value of the input parameter is the multipath parameter n and the antenna configuration parameter n, the output The value of the parameter is the channel matrix n.
  • n is a value greater than or equal to 1.
  • the training data in S102 comes from an actual network or other simulation platforms other than the simulation platform provided in this embodiment of the application.
  • the actual network refers to the actually deployed network, which may include network elements such as PGW, SGW, base station, and terminal.
  • the actual network may be a network actually deployed in a certain office (that is, an area such as a certain city).
  • the other simulation platform may be any network simulation platform in the prior art, for example, it may be the network simulation platform shown in FIG. 1. Alternatively, the other simulation platform may be a future network simulation platform.
  • S103 Based on the training data, train a machine learning model to obtain a first black box model.
  • the output parameter of the first black box model is the candidate network performance index
  • the input parameter of the first black box model is the feature information related to the candidate network performance index determined in S101.
  • the first black box model is used to simulate the performance index of the candidate network.
  • the type of the machine learning model may include a neural network model, such as a picture-based neural network model or a convolutional neural network (convolutional neural network, CNN).
  • the types of machine learning models may include statistical learning models, such as regression models or classification models.
  • S103 may include: selecting a machine learning model based on the candidate network performance index and feature information related to the candidate network performance index.
  • the selected machine learning model may be a deep learning model such as a picture-based neural network model.
  • the candidate network performance index is a channel matrix
  • the selected machine learning model may be a convolutional neural network (ie, CNN). Then, the machine learning model is trained based on the training data to obtain the first black box model.
  • the embodiment of the present application does not limit the specific implementation manner of training the machine learning model, for example, reference may be made to the prior art. Among them, the machine learning models used when simulating different network performance indicators may be the same or different.
  • a black box model for simulating a candidate network performance index can be created, and similar methods can be used to create a model for the core network, bearer network, and/or access network.
  • each black box model is defined with input parameters and output parameters (that is, input interface and output interface). So far, it can be considered that a network simulation platform has been created.
  • the network simulation platform shown in Figure 4 includes: an access network part, a bearer network part and a core network part.
  • the access network part includes: channel model, interference model, coverage model, channel capacity model and rank model, etc., which are respectively used to simulate network performance indicators such as channel matrix, interference, coverage, channel capacity and rank.
  • the bearer network part includes a delay model, a jitter model, and a transmission resource mapping model, which are used to simulate network performance indicators such as delay, jitter, and transmission resource mapping.
  • the core network part includes: user distribution model, user movement model, service model, QoS model and MOS model, etc., which are respectively used to simulate user distribution characteristics, user movement characteristics, service characteristics, QoS and MOS.
  • Each model in Figure 4 can be a black box model.
  • wireless operators will pay attention to performance indicators such as coverage, interference, and capacity for the access network. Based on this, coverage, interference, and capacity can be used as candidate network performance indicators, and S101 ⁇ S103 can be executed to create a black box model for simulating coverage, a black box model for simulating interference, and Black box model for capacity simulation.
  • This process can also be referred to as a process of black-boxing coverage, interference, and capacity.
  • the network simulation platform further includes a second black box model.
  • the output parameter of the second black box model is one of the input parameters of the first black box model. If the first black box model has multiple input parameters, any one or more of the multiple input parameters may be output parameters of a black box model provided by the embodiment of the present application.
  • the embodiment of the application supports a technical solution in which the output parameter of one black box model is used as the input parameter of another black box model (that is, the black box models have an association relationship).
  • the output parameters of the multipath parameter model ie, multipath parameters
  • the output parameters of the channel model ie, channel matrix
  • the interference model coverage model, channel capacity model, and rank The input parameters of the model.
  • This optional technical solution provides a theoretical basis for "cascading multiple black box models during the simulation process to simulate a certain network performance index" and also provides a theoretical basis for implementing end-to-end simulation .
  • the output parameter of the first black box model is a network performance index in the first network
  • the output parameter of the second black box model is a network performance index in the second network.
  • the first network is a core network, a bearer network, or an access network
  • the second network is any network other than the first network among the core network, the bearer network, and the access network.
  • the embodiments of this application support the simulation of cross-network performance indicators, that is, support end-to-end simulation; in other words, the models in the access network part, the bearer network part, and the core network part can be coordinated to complete the matching Simulation of a certain network performance index.
  • the channel capacity model (ie, channel capacity) and service model of the access network part, and the output parameters of the QoS model in the core network part (ie QoS requirements) can be used as the input of the delay model of the bearer network parameter.
  • the method for creating a network simulation platform mainly focuses on candidate network performance indicators and related input feature information, and does not need to simulate the protocol process in the actual network, so the complexity is small and it can be applied to large-scale networks simulation.
  • the goal of this embodiment is to create a black box model capable of simulating the realization channel matrix.
  • the candidate network performance index is the channel matrix
  • the characteristic information related to the channel matrix is the multipath parameter.
  • Step 1 Obtain training data.
  • the training data includes multiple sets of values.
  • Each set of values includes a channel matrix and the value of the multipath parameter used to obtain the channel matrix.
  • the value of the multipath parameter may be determined based on the multipath parameter model.
  • the multipath parameter model may be a multipath parameter model provided in traditional technology, such as a ray tracing model or a statistical model.
  • the multipath parameter model may be a black box model for simulating multipath parameters provided in the embodiments of the present application.
  • FIG. 6 a schematic diagram of a multipath parameter model and a channel matrix model provided by an embodiment of this application.
  • the value of the multipath parameter may be estimated based on actual measured data.
  • Step 2 Input the acquired training data into a neural network model (such as a 3GPP protocol model), and train the neural network model to obtain a black box model for simulating the channel matrix.
  • a neural network model such as a 3GPP protocol model
  • the multipath parameters can be obtained from ray tracing or actual measurement data, so the process of obtaining the multipath parameter model and obtaining the channel matrix can be trained separately, and training efficiency can be improved through parallel training.
  • the multipath parameters required for the simulation of the channel matrix are usually obtained based on a statistical model or a ray tracing model.
  • the multipath parameters required for the simulation are randomly generated through the statistical model, and therefore cannot accurately reflect the channel response in the real environment.
  • Obtaining the multipath parameters required for the simulation of the channel matrix based on the ray tracing model, etc. can include: based on the plane wave assumption, generating a large number of rays in the electronic map and tracking the reflection and scattering behavior of these rays through algorithms, and finally generating the simulation The required multipath parameters.
  • the multipath parameters are simulated based on the black box model.
  • the characteristics of the wireless propagation environment can be directly obtained from the electronic map and engineering parameters, and corresponding multipath parameters can be generated based on these characteristics, which can make the simulation more efficient. , So it can be applied to large-scale simulation scenarios.
  • the creation network simulation platform provided by the embodiment of the present application may provide an external calling interface in the form of a library.
  • the black box models in the network simulation platform (for example, the black box models shown in FIG. 4) can be called individually, or multiple black box models can be called jointly to complete the simulation of a certain network performance index.
  • the black box model in the simulation stage, can be used to replace the message queue processing process in the traditional network simulation platform.
  • a black box model can be used to replace a certain network element in a traditional network simulation platform, or can be used to replace a certain functional module (such as a channel model, a scheduling model, a transmission model, etc.) in a network element.
  • the output parameter of the black box model is the output parameter of a certain functional module of the network element, as shown in FIG. 7A.
  • the black box model in Figure 7A replaces the base station.
  • the input parameters of the black box model are the same as the input parameters of the function module, and the output parameters of the black box model are the same as the output parameters of the function module.
  • Figure 7B replaces the channel model.
  • Fig. 7A and Fig. 7B are drawn based on Fig. 1.
  • the black box model in FIG. 7B replaces the multi-model spectral channel (which can be understood as a channel model) included in the base station.
  • the black box model can communicate with the data in the traditional network simulation platform.
  • the black box model provided by the embodiment of the present application can be compatible with the traditional network simulation platform.
  • FIG. 8 it is a schematic flowchart of a network simulation method provided by an embodiment of this application.
  • the method is applied to a network simulation platform including at least two black box models.
  • the method of creating any black box model in the network simulation platform can refer to the above.
  • the method shown in Figure 8 includes the following steps:
  • the server determines the first network performance index to be simulated.
  • the first network performance indicator to be simulated may be any network performance indicator that needs to be simulated.
  • the server may receive a network performance index indicated by the user (or indicated by other devices), and use the network performance index as the first network performance index to be simulated, or analyze the network performance index, and compare it with the network performance index.
  • a network performance indicator related to the performance indicator serves as the first network performance indicator to be simulated.
  • the server searches for a first black box model for simulating the performance index of the first network to be simulated.
  • the output parameter of the first black box model is the performance index of the first network to be simulated.
  • the server may use the black box model whose output parameter is the first network performance indicator to be simulated among the at least two black box models as the first black box model.
  • S203 The server inputs the value of the input parameter of the first black box model into the first black box model, obtains the value of the output parameter of the first black box model, and uses the obtained value as the simulation result of the performance index of the first network to be simulated .
  • the value of the input parameter of the first black box model may include: the antenna of the terminal The value of the configuration parameter, the value of the antenna configuration parameter of the main serving base station, and the value of the multipath parameter between the terminal and the main serving base station, etc.
  • the simulation result of the first network performance index to be simulated is the channel matrix between the terminal and the main serving base station.
  • the network simulation method provided by the embodiments of the present application can directly simulate the simulated network performance parameters, which is helpful to improve the simulation efficiency compared with the technical solution in the traditional technology that simulates the message processing process and then indirectly simulates the simulated network performance parameters. .
  • FIG. 9 it is a schematic flowchart of a network simulation method provided by an embodiment of this application.
  • the method is applied to a network simulation platform including at least two black box models.
  • the method shown in Figure 9 includes the following steps:
  • S301 ⁇ S302 Refer to S201 ⁇ S202 above.
  • the embodiments of the present application are not limited to this.
  • S303 From the at least two black box models, the server searches for a second black box model for simulating the performance index of the second network to be simulated; the output parameter of the second black box model is one of the first black box models Input parameters.
  • the server determines that one of the input parameters of the first black box model is the output parameter of one of the at least two black box models, it will use the input parameter as the second network performance indicator to be simulated, and set This black box model serves as the second black box model.
  • the multipath parameter can be used as the performance of the second network to be simulated Index and the multipath parameter model as the second black box model.
  • S304 The server inputs the value of the input parameter of the second black box model into the second black box model, obtains the value of the output parameter of the second black box model, and uses the obtained value as the simulation result of the performance index of the second network to be simulated .
  • the server can be the physical environment where the terminal and the main serving base station are located
  • the electronic map, engineering parameters and feature types of the input are input into the second black box model.
  • the simulation result from the second network performance index to be simulated is the multipath parameter of the channel between the terminal and the main serving base station.
  • S302 can be executed first and then S303 ⁇ S304, or S303 ⁇ S304 can be executed before S302, or it can be executed in the process of executing S303 ⁇ S304. S302 and so on.
  • S305 The server inputs the simulation result of the second network performance indicator to be simulated into the first black box model, obtains the value of the output parameter of the first black box model, and uses the obtained value as the simulation result of the first network performance indicator to be simulated .
  • the server may also correspond to the simulation result of the second network performance indicator to be simulated
  • the value of the other input parameter is input to the first black box model, so as to obtain the value of the output parameter of the first black box model.
  • the embodiment of the present application may divide the server into functional modules according to the foregoing method examples.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.
  • FIG. 10 it is a schematic structural diagram of an apparatus 100 for creating a network simulation platform provided by an embodiment of this application.
  • the device 100 may be the server used to execute the method for creating a network simulation platform as described above.
  • the device 100 may be used to execute the steps executed by the server in the method shown in FIG. 3.
  • the network simulation platform includes at least one black box model, and the at least one black box model includes a first black box model.
  • the device 100 includes: an acquisition unit 1001 and a training unit 1002. Wherein, the obtaining unit 1001 is configured to obtain candidate network performance indicators and characteristic information related to the candidate network performance indicators.
  • the training unit 1002 is used to train the machine learning model to obtain the first black box model.
  • the output parameter of the first black box model is the performance index of the candidate network.
  • the input parameter of the first black box model is characteristic information related to the candidate network performance index.
  • the first black box model is used to simulate the performance index of the candidate network.
  • the characteristic information related to the candidate network performance index refers to part or all of the characteristic information required to obtain the candidate network performance index.
  • the acquiring unit 1001 may be specifically used to perform S101, and the training unit 1002 may be specifically used to perform S103.
  • the candidate network performance indicators include: network performance indicators in the core network, network performance indicators in the bearer network, network performance indicators in the access network, or network performance indicators across networks.
  • the at least one black box model further includes a second black box model.
  • the output parameter of the second black box model is one of the input parameters of the first black box model.
  • the output parameter of the first black box model is a network performance index in the first network
  • the output parameter of the second black box model is a network performance index in the second network.
  • the first network is a core network, a bearer network or an access network.
  • the second network is any network other than the first network among the core network, the bearer network, and the access network.
  • the obtaining unit 1001 is further configured to obtain training data.
  • the training data comes from the actual network or other simulation platforms except the simulation platform provided in the embodiment of the present application.
  • the training unit 1002 is specifically configured to train a machine learning model based on the training data, the first black box model.
  • the acquiring unit 1001 may be specifically configured to perform S102.
  • the machine learning model includes a neural network model.
  • the candidate network performance index includes a rank, and the characteristic information related to the rank includes a channel matrix.
  • the candidate network performance index includes a channel matrix, and the characteristic information related to the channel matrix includes multipath parameters and antenna configuration parameters.
  • the candidate network performance indicators include multipath parameters, and the feature information related to the multipath parameters includes electronic maps, engineering parameters, and feature types.
  • the candidate network performance index includes the channel state probability distribution parameter of the cell, and the characteristic information related to the channel state probability distribution parameter of the cell includes the grid channel matrix of the cell and the channel state occupation time sequence of the cell.
  • the candidate network performance index includes the precoding matrix of the cell, and the characteristic information related to the precoding matrix of the cell includes the grid channel matrix of the cell.
  • the above-mentioned acquisition unit 1001 and training unit 1002 can both be implemented by the processor 201 in FIG. 2 calling a computer program stored in the memory 203.
  • FIG. 11 it is a schematic structural diagram of a network simulation device 110 provided by an embodiment of this application.
  • the device can be applied to a network simulation platform including at least two black box models.
  • the device 100 may be the server for executing the network simulation method described above.
  • the device 100 may be used to execute the steps executed by the server in the method shown in FIG. 8 or FIG. 9.
  • the device 110 includes a determining unit 1101, a searching unit 1102, and a simulation unit 1103.
  • the determining unit 1101 is configured to determine the first network performance index to be simulated.
  • the searching unit 1102 is configured to search for the first black box model used to simulate the performance index of the first network to be simulated from the at least two black box models.
  • the output parameter of the first black box model is the performance index of the first network to be simulated.
  • the simulation unit 1103 is configured to input the value of the input parameter of the first black box model into the first black box model to obtain the value of the output parameter of the first black box model, and use the obtained value as the performance index of the first network to be simulated Simulation results. For example, with reference to FIG.
  • the determining unit 1101 may be used to perform S201, the searching unit 1102 may be used to perform S202, and the simulation unit 1103 may be used to perform S203.
  • the determining unit 1101 may be used to perform S301, the searching unit 1102 may be used to perform S302, and the simulation unit 1103 may be used to perform S304 and S305.
  • the first network performance indicators to be simulated include: network performance indicators in the core network, network performance indicators in the bearer network, or network performance indicators in the access network, or network performance indicators across networks.
  • the searching unit 1102 is further configured to, from the at least two black box models, search for a second black box model for simulating the performance index of the second network to be simulated.
  • the output parameter of the second black box model is one of the input parameters of the first black box model.
  • the simulation unit 1103 is also used to input the value of the input parameter of the second black box model into the second black box model to obtain the value of the output parameter of the second black box model, and use the obtained value as the second network performance to be simulated The simulation result of the indicator.
  • the simulation unit 1103 executes inputting the value of the input parameter of the first black box model into the first black box model, it is specifically configured to: input the simulation result of the second network performance index to be simulated into the first black box model.
  • the search unit 1102 may be used to perform S303, and the simulation unit 1103 may be used to perform S305.
  • the output parameter of the first black box model is a network performance index in the first network
  • the output parameter of the second black box model is a network performance index in the second network.
  • the first network is a core network, a bearer network or an access network.
  • the second network is any network other than the first network among the core network, the bearer network, and the access network.
  • the first network performance indicator to be simulated includes a rank
  • the characteristic information related to the rank includes a channel matrix
  • the first network performance indicator to be simulated includes a channel matrix
  • the output parameters of the first black box model include multipath parameters and antenna configuration parameters.
  • the first network performance indicator to be simulated includes multipath parameters
  • the output parameters of the first black box model include electronic maps, engineering parameters, and feature types.
  • the first network performance indicator to be simulated includes the channel state probability distribution parameter of the cell
  • the output parameter of the first black box model includes the grid channel matrix of the cell and the channel state occupation time sequence of the cell.
  • the first network performance indicator to be simulated includes the precoding matrix of the cell, and the output parameter of the first black box model includes the grid channel matrix of the cell.
  • the determination unit 1101, the search unit 1102, and the simulation unit 1103 may all be implemented by the processor 201 in FIG. 2 calling a computer program stored in the memory 203.
  • the processor described above can be implemented by hardware or software.
  • the processor may be a logic circuit, an integrated circuit, or the like.
  • the processor may be a general-purpose processor, which is implemented by reading software codes stored in the memory.
  • the memory may be integrated in the processor, or located outside the processor and exist independently.
  • the embodiment of the present application also provides a chip.
  • the chip integrates a circuit and one or more interfaces for realizing the functions of the above-mentioned processor.
  • the functions supported by the chip may include processing actions in the embodiment described in FIG. 3, FIG. 8 or FIG. 9, which will not be repeated here.
  • the program can be stored in a computer-readable storage medium.
  • the aforementioned storage medium may be a read-only memory, a random access memory, and the like.
  • the above-mentioned processing unit or processor may be a central processing unit, a general-purpose processor, an application specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.
  • the embodiments of the present application also provide a computer program product containing instructions, which when the instructions are run on a computer, cause the computer to execute any one of the methods in the foregoing embodiments.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part.
  • the computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • computer instructions may be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or may include one or more data storage devices such as a server or a data center that can be integrated with the medium.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
  • the foregoing devices for storing computer instructions or computer programs provided in the embodiments of the present application are non-transitory. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

本申请公开了创建网络仿真平台的方法和网络仿真方法及相应装置，涉及仿真技术领域，可以应用于5G通信系统的网络仿真中。网络仿真平台包括第一黑盒模型。创建网络仿真平台的方法包括：获取候选网络性能指标和与该候选网络性能指标相关的特征信息；与该候选网络性能指标相关的特征信息是指用于获得该候选网络性能指标所需的部分或全部特征信息；训练机器学习模型，得到第一黑盒模型；第一黑盒模型的输出参数是该候选网络性能指标，第一黑盒模型的输入参数是与该候选网络性能指标相关的特征信息；第一黑盒模型用于对该候选网络性能指标进行仿真。本申请提供的技术方案有助于降低仿真复杂度，可以适用于大规模的网络仿真。

Description

创建网络仿真平台的方法和网络仿真方法及相应装置
本申请要求于2019年05月22日提交国家知识产权局、申请号为201910430994.3、申请名称为“创建网络仿真平台的方法和网络仿真方法及相应装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及仿真技术领域，尤其涉及创建网络仿真平台的方法和网络仿真方法及装置。
背景技术
无线通信技术的研究和开发很大程度上依赖于仿真技术。无线通信仿真技术可以划分成链路仿真、系统仿真和网络仿真。链路仿真是信道级仿真(如物理信道覆盖及容量仿真)，系统仿真是基站级仿真(如协议流程及调度仿真)，网络仿真是网络级仿真(如网络覆盖及容量仿真)。因此，网络仿真的仿真规模较大，通常为千站甚至万站级别的仿真。
传统网络仿真通过静态快照实现，无法有效反映无线网络中的时变特性，而5G通信系统的频点较高、波束较窄，对环境变化更敏感，因此传统网络仿真不能适用于5G通信系统。
传统系统仿真虽然可以实现动态仿真，并且可以将系统仿真技术应用到网络仿真中，但是，传统系统仿真需要模拟网元的消息交互机制，这会导致仿真过程运算量大，无法快速得到仿真结果。尤其地，将系统仿真技术应用到网络仿真时，面对千站甚至万站级别的仿真，仿真所需要的时间将无法接受。
传统链路仿真技术无法应用到网络仿真中。
综上可知，传统的无线通信仿真技术无法有效地应用到5G通信系统的网络仿真中，因此，亟待提出新的网络仿真技术。
发明内容
本申请实施例提供了创建网络仿真平台的方法和网络仿真方法及相应装置，可以应用于5G通信系统的网络仿真中。
第一方面，本申请实施例提供了一种创建网络仿真平台的方法，网络仿真平台包括至少一个黑盒模型，该至少一个黑盒模型包括第一黑盒模型。该方法包括：首先，获取候选网络性能指标和与该候选网络性能指标相关的特征信息；然后，训练机器学习模型，得到第一黑盒模型。第一黑盒模型的输出参数是该候选网络性能指标，第一黑盒模型的输入参数是与该候选网络性能指标相关的特征信息。第一黑盒模型用于对该候选网络性能指标进行仿真。可选的，该候选网络性能指标包括：核心网中的网络性能指标、承载网中的网络性能指标、接入网中的网络性能指标或者跨网络的网络性能指标。其中，跨网络的网络性能指标是指受核心网、承载网和接入网中的至少两种网络的影响的网络性能指标。
可见，基于第一方面提供的方法，可以创建用于对核心网、承载网和/或接入网中的一个或多个候选网络性能指标进行仿真的黑盒模型。其中，每个黑盒模型均被定义 了输入参数和输出参数(即输入接口和输出接口)。至此，可以认为已创建好网络仿真平台。该网络仿真平台中，主要关注候选网络性能指标及其相关的输入特征信息，不需要模拟实际网络中的协议流程，因此复杂度较小，可以适用于大规模的网络仿真。
可选的，与该候选网络性能指标相关的特征信息是指用于获得该候选网络性能指标所需的部分或全部特征信息。
在一种可能的设计中，该至少一个黑盒模型还包括第二黑盒模型。第二黑盒模型的输出参数是第一黑盒模型的其中一个输入参数。也就是说，本申请实施例支持将一个黑盒模型的输出参数作为另一个黑盒模型的输入参数(即黑盒模型之间具有关联关系)的技术方案。该可选的技术方案为“在仿真过程中，将多个黑盒模型进行级联，从而对某个网络性能指标进行仿真”提供了理论依据，并且，为实现端到端仿真提供了理论依据。
在一种可能的设计中，第一黑盒模型的输出参数是第一网络中的网络性能指标，第二黑盒模型的输出参数是第二网络中的网络性能指标。其中，第一网络是核心网、承载网或接入网。第二网路是核心网、承载网和接入网中的除第一网络之外的任一网络。也就是说，本申请实施例支持对跨网络的性能指标的仿真，即支持端到端的仿真；换句话说，接入网部分、承载网部分和核心网部分中的模型之间可以协同完成对某个网络性能指标的仿真。
在一种可能的设计中，该方法还包括：获取训练数据。该训练数据来自实际网络或除本申请实施例提供的仿真平台之外的其他仿真平台。该情况下，训练机器学习模型，得到第一黑盒模型，可以包括：基于该训练数据，训练机器学习模型，得到第一黑盒模型。可以理解的是，如果训练数据来自实际网络，则有助于提高仿真的准确度。
在一种可能的设计中，机器学习模型包括神经网络模型。
在一种可能的设计中，该候选网络性能指标包括秩(rank)，与rank相关的特征信息包括信道矩阵。
在一种可能的设计中，该候选网络性能指标包括信道矩阵，与该信道矩阵相关的特征信息包括多径参数和天线配置参数。
在一种可能的设计中，该候选网络性能指标包括多径参数，与多径参数相关的特征信息包括电子地图、工程参数和地物类型中的至少一种。
在一种可能的设计中，该候选网络性能指标包括小区的信道状态概率分布参数，与该小区的信道状态概率分布参数相关的特征信息包括该小区的栅格信道矩阵和该小区的信道状态占用时间序列。
在一种可能的设计中，该候选网络性能指标包括小区的预编码矩阵，与该小区的预编码矩阵相关的特征信息包括该小区的栅格信道矩阵。
在一种可能的实现方式中，在仿真阶段，黑盒模型可以用于替换传统网络仿真平台中的消息队列处理过程。具体的，一个黑盒模型可以用于替换传统网络仿真平台中的某个网元，或者可以用于替换一个网元中的某个功能模块(如信道模型、调度模型、传输模型等)。可选的，在使用黑盒模型替换传统网络仿真平台中的网元或者功能模块之后，该黑盒模型与传统网络仿真平台中的数据可以互通。也就是说，本申请实施例提供的黑盒模型可以与传统网络仿真平台进行兼容。
第二方面，本申请实施例提供了一种网络仿真方法，应用于包括至少两个黑盒模型的网络仿真平台。可选的，该仿真平台可以是基于上述第一方面或上述第一方面的任一种可能的设计所创建的仿真平台。该方法包括：首先，确定第一待仿真网络性能指标；然后，从该至少两个黑盒模型中，查找用于对第一待仿真网络性能指标进行仿真的第一黑盒模型。第一黑盒模型的输出参数是第一待仿真网络性能指标。接着，将第一黑盒模型的输入参数的值输入第一黑盒模型，得到第一黑盒模型的输出参数的值，并将所得到的值作为第一待仿真网络性能指标的仿真结果。
可见，该技术方案中可以直接对待仿真网络性能参数进行仿真，与传统技术中通过模拟消息处理流程再间接对待仿真网络性能参数进行仿真的技术方案相比，有助于提高仿真效率。
在一种可能的设计中，第一待仿真网络性能指标包括：核心网中的网络性能指标、承载网中的网络性能指标、接入网中的网络性能指标或跨网络的网络性能指标。
在一种可能的设计中，该方法还包括：从该至少两个黑盒模型中，查找用于对第二待仿真网络性能指标进行仿真的第二黑盒模型。第二黑盒模型的输出参数是第一黑盒模型的其中一个输入参数。然后，将第二黑盒模型的输入参数的值输入第二黑盒模型，得到第二黑盒模型的输出参数的值，并将所得到的值作为第二待仿真网络性能指标的仿真结果。基于此，将第一黑盒模型的输入参数的值输入第一黑盒模型，包括：将第二待仿真网络性能指标的仿真结果输入第一黑盒模型。也就是说，本申请实施例支持黑盒模型之间进行级联，从而实现对某个网络性能指标进行仿真。
在一种可能的设计中，第一黑盒模型的输出参数是第一网络中的网络性能指标，所述第二黑盒模型的输出参数是第二网络中的网络性能指标。其中，第一网络是核心网、承载网或接入网；第二网路是核心网、承载网和接入网中的除第一网络之外的任一网络。也就是说，本申请实施例支持对跨网络的性能指标的仿真，即支持端到端的仿真；换句话说，接入网部分、承载网部分和核心网部分中的模型之间可以协同完成对某个网络性能指标的仿真。
在一种可能的设计中，第一待仿真候选网络性能指标包括rank，与rank相关的特征信息包括信道矩阵。
在一种可能的设计中，第一待仿真网络性能指标包括信道矩阵，第一黑盒模型的输出参数包括多径参数和天线配置参数。
在一种可能的设计中，第一待仿真网络性能指标包括多径参数，第一黑盒模型的输出参数包括电子地图、工程参数和地物类型。
在一种可能的设计中，第一待仿真网络性能指标包括小区的信道状态概率分布参数，第一黑盒模型的输出参数包括该小区的栅格信道矩阵和该小区的信道状态占用时间序列。
在一种可能的设计中，第一待仿真网络性能指标包括小区的预编码矩阵，第一黑盒模型的输出参数包括该小区的栅格信道矩阵。
第三方面，本申请实施例提供了一种创建网络仿真平台的装置，该装置可用于执行上述第一方面或第一方面的任一种可能的设计提供的任一种方法。该装置可以是服务器或芯片。
在一种可能的设计中，可以根据上述第一方面或第一方面的任一种可能的设计提供的方法对该装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。
在一种可能的设计中，该装置可以包括存储器和处理器。存储器用于存储计算机程序。处理器用于调用该计算机程序，以执行第一方面或第一方面的任一种可能的设计提供的方法。
第四方面，本申请实施例提供了一种网络仿真装置，该装置可用于执行上述第二方面或第二方面的任一种可能的设计提供的任一种方法。该装置可以是服务器或芯片。
在一种可能的设计中，可以根据上述第二方面或第二方面的任一种可能的设计提供的方法对该装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。
在一种可能的设计中，该装置可以包括存储器和处理器，存储器用于存储计算机程序。处理器用于调用该计算机程序，以执行上述第二方面或第二方面的任一种可能的设计提供的方法。
第五方面，本申请实施例提供了一种计算机可读存储介质，如计算机非瞬态的可读存储介质。其上储存有计算机程序(或指令)，当该计算机程序(或指令)在计算机上运行时，使得该计算机执行上述第一方面或第一方面的任一种可能的设计提供的任一种方法。
第六方面，本申请实施例提供了一种计算机可读存储介质，如计算机非瞬态的可读存储介质。其上储存有计算机程序(或指令)，当该计算机程序(或指令)在计算机上运行时，使得该计算机执行上述第二方面或第二方面的任一种可能的设计提供的任一种方法。
第七方面，本申请实施例提供了一种计算机程序产品，当其在计算机上运行时，使得第一方面或第一方面的任一种可能的设计提供的任一种方法被执行。
第八方面，本申请实施例提供了一种计算机程序产品，当其在计算机上运行时，使得第二方面或第二方面的任一种可能的设计提供的任一种方法被执行。
可以理解的是，上述提供的任一种创建网络仿真平台的装置、网络仿真装置、计算机存储介质或计算机程序产品等均可以应用于上文所提供的对应的方法，因此，其所能达到的有益效果可参考对应的方法中的有益效果，此处不再赘述。
附图说明
图1为一种传统网络仿真平台的示意图；
图2为可适用于本申请实施例的一种服务器的结构示意图；
图3为本申请实施例提供的一种创建网络仿真平台的方法的流程示意图；
图4为本申请实施例提供的一种网络仿真平台的示意图；
图5A为本申请实施例提供的一种黑盒模型之间进行级联的示意图；
图5B为本申请实施例提供的另一种黑盒模型之间进行级联的示意图；
图6为本申请实施例提供的多径参数模型和信道矩阵模型的示意图；
图7A为可适用于本申请实施例的一种网络仿真平台的示意图；
图7B为可适用于本申请实施例的另一种网络仿真平台的示意图；
图8为本申请实施例提供的一种网络仿真方法的流程示意图；
图9为本申请实施例提供的另一种网络仿真方法的流程示意图；
图10为本申请实施例提供的一种创建网络仿真平台的装置的结构示意图；
图11为本申请实施例提供的一种网络仿真装置的结构示意图。
具体实施方式
网络仿真平台，也可被称作网络仿真系统，是指运行在服务器(或称为仿真设备)上的软件，用于执行网络仿真任务。其中，执行网路仿真任务可以理解为对某一个或多个网络性能指标(如信道矩阵、多径参数、小区干扰等)进行仿真。
如图1所示，为一种传统网络仿真平台的示意图。图1所示的传统网络仿真平台是采用系统仿真技术实现的，其仿真过程由天线建模(如图1中的天线阵列模型的创建过程)、信道建模(如图1中的多模型光谱信道的创建过程)、调度算法处理(图1中未示出，具体可以包含在MAC层中)和协议流程处理等构成。在具体实现过程中，网络仿真平台需要模拟协议层之间的信息交互，并基于所模拟的协议层处理每一条数据报文，以及模拟数据报文在网元之间的交互。另外，网络仿真平台还可以模拟资源调度过程和基带处理过程等。
图1示意出了无线通信网络中的网元之间进行通信时需要模拟的协议层和无线空口部分的特征信息。其中，该网元包括：远程主机、PDN网关(PDN gateway，PGW)/服务网关(serving gateway，SGW)、基站以及终端。其中，PDN是公用数据网(public data network)的英文缩写。对于远端主机来说，需要模拟的协议层包括：应用层、传输控制协议(transmission control protocol，TCP)/用户数据报协议(user datagram protocol，UDP)，以及互联网协议(internet protocol，IP)层。对于PGW/SGW来说，需要模拟的协议层包括：第一IP层、应用层、GPRS隧道协议(GPRS tunelling protocol，GTP)层、UDP层和第二IP层。其中，GPRS是通用分组无线服务技术(general packet radio service)的英文缩写。第一IP层与远端主机的IP层对应。对于基站来说，需要模拟的协议层包括：与PGW/SGW的第二IP层、UDP层和GTP层分别对应的IP层、UDP层和GTP层，以及应用层、无线资源控制(radio resource control，RRC)层、分组数据汇聚协议(packet data convergence protocol，PDCP)层、无线链路控制(radio link control，RLC)层、介质访问控制(media access control，MAC)层和物理(physical)层。对于终端来说，需要模拟的协议层包括：与基站的PHY层、MAC层、RLC层、PDCP层和RRC层分别对应的PHY层、MAC层、RLC层、PDCP层和RRC层，以及与远端主机的IP层、TCP/UDP层和应用层分别对应的IP层、TCP/UDP层和应用层。
其中“两个协议层对应”是指这两个协议层的其中一个协议层所执行的处理过程是另一个协议层所执行的处理过程的逆过程。另外基站和终端之间还需要模拟无线空口部分的特征信息，例如需要模拟光谱PHY层、多模型光谱信道、波束成型模型(用于模拟波束成型技术)、传播损失模型(用于模拟信息传输过程中的能量损失)、交互信息错误(mutual information，error，miError)模型和天线阵列模型(用于模拟基站和终端的天线阵列的特征)等。
系统仿真技术虽然能够实现动态仿真，但是，系统仿真通过模拟网络协议层和网元间的消息交互机制来实现仿真，这会导致运算量巨大、仿真效率低，从而导致传统 系统仿真无法应用于大规模的网络仿真。另外，系统仿真技术主要用于对单站进行仿真，而且侧重无线空口部分的仿真，不能实现端到端仿真(即对跨网络的网络性能指标进行仿真)。
基于此，本申请实施例提供了创建网络仿真平台的方法及装置，及网络仿真方法及装置。
本申请实施例提供的技术方案可以应用于各种通信系统，例如、第五代(5th generation，5G)移动通信系统如5G新空口(new radio，NR)系统中，未来演进系统或多种通信融合系统等中，也可以应用于在现有通信系统中。用于执行创建网络仿真平台的方法的服务器(标记为第一服务器)和用于执行网络仿真方法的服务器(标记为第二服务器)可以是同一服务器，也可以是不同服务器。例如，第二服务器可以直接或间接地与第一服务器进行信息交互，以获得第一服务器所创建的网络仿真平台，然后基于该网络仿真平台进行网络仿真。在具体实现时，服务器(包括第一服务器和第二服务器)可以是(或集成在)实际网路中所部署的任意一个设备，或者可以是独立于实际网络中的每个设备的一个设备。
如图2所示，为可适用于本申请实施例的一种服务器20(包括上述第一服务器和第二服务器)的结构示意图。该服务器20用于执行本申请实施例提供的创建网络仿真平台的方法，和/或网络仿真方法。服务器20可以包括至少一个处理器201，通信线路202，存储器203以及至少一个通信接口204。
处理器201可以是一个通用中央处理器(central processing unit，CPU)，微处理器，特定应用集成电路(application-specific integrated circuit，ASIC)，或一个或多个用于控制本申请方案程序执行的集成电路。
通信线路202可包括一通路，在上述组件(如至少一个处理器201，通信线路202，存储器203以及至少一个通信接口204)之间传送信息。
通信接口204，使用任何收发器一类的装置，用于与其他设备或通信网络通信，如广域网(wide area network，WAN)，局域网(local area networks，LAN)等。
存储器203可以是只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)或者可存储信息和指令的其他类型的动态存储设备，也可以是电可擦可编程只读存储器(electrically erasable programmable read-only memory，EEPROM)、只读光盘(compact disc read-only memory，CD-ROM)或其他光盘存储、光碟存储(包括压缩光碟、激光碟、光碟、数字通用光碟、蓝光光碟等)、磁盘存储介质或者其他磁存储设备、或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。存储器203可以是独立存在，通过通信线路202与处理器201相连接。存储器203也可以和处理器201集成在一起。本申请实施例提供的存储器203通常可以具有非易失性。其中，存储器203用于存储执行本申请方案的计算机指令，并由处理器201来控制执行。处理器201用于执行存储器203中存储的计算机指令，从而实现本申请下述实施例提供的方法。
可选的，本申请实施例中的计算机指令也可以称之为应用程序代码，本申请实施例对此不作具体限定。
在具体实现中，作为一种实施例，服务器20可以包括多个处理器，例如图2中的 处理器201和处理器207。这些处理器中的每一个可以是一个单核(single-CPU)处理器，也可以是一个多核(multi-CPU)处理器。这里的处理器可以指一个或多个设备、电路、和/或用于处理数据(例如计算机程序指令)的处理核。
在具体实现中，作为一种实施例，服务器20还可以包括输出设备205和/或输入设备206。输出设备205和处理器201通信，可以以多种方式来显示信息。例如，输出设备205可以是液晶显示器(liquid crystal display，LCD)，发光二级管(light emitting diode，LED)显示设备，阴极射线管(cathode ray tube，CRT)显示设备，或投影仪(projector)等。输入设备206和处理器201通信，可以以多种方式接收用户的输入。例如，输入设备206可以是鼠标、键盘、触摸屏设备或传感设备等。
以下，对本申请实施例中所涉及的术语进行解释说明，以方便读者理解。
黑盒模型，也可以被称作黑箱模型(black box)，是环境预测工作中应用较多的一类模型，它是根据输入-输出关系(即输入参数和输出参数)创建的，反映了输出参数与输入参数之间的一种笼统的直接因果关系。一个黑盒模型一般有一个输出参数，以及一个或多个输入参数。
端到端仿真，也可以被称为跨网络的网络性能指标仿真，或者跨层的网络性能指标仿真。其中，跨网络(或跨层)的网络性能指标，是指会受到多种网络的影响的网络性能指标。多种网路(或多层网络)包括核心网、承载网和接入网。例如，时延会受到核心网和传输网的影响，因此，时延可以认为是一种跨网络的网络性能指标。当然，在一些实现方式中，时延也可以认为是一种单网络的网络性能指标，如核心网中的时延，或者传输网中的时延。
在本申请的描述中，“至少一个”包括一个或多个。“多个”是指两个或两个以上。例如，A、B和C中的至少一个，包括：单独存在A、单独存在B、同时存在A和B、同时存在A和C、同时存在B和C，以及同时存在A、B和C。在本申请的描述中，“/”表示或的意思，例如，A/B可以表示A或B。在本申请的描述中，“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。
本申请的描述中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其他实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
以下，结合附图对本申请实施例提供的技术方案进行说明。需要说明的是，本申请实施例描述的业务场景是为了更清楚地说明本申请实施例的技术方案，并不构成对本申请实施例提供的技术方案的限制。本领域普通技术人员可知，随着网络架构的演变和新业务场景的出现，本申请实施例提供的技术方案对于类似技术问题同样适用。
如图3所示，为本申请实施例提供的一种创建网络仿真平台的方法的流程示意图。该网络仿真平台包括至少一个黑盒模型。在本申请实施例中，创建网络仿真平台具体包括创建该至少一个黑盒模型中的每个黑盒模型。本实施例中以创建至少一个黑盒模型包括的第一黑盒模型为例进行说明的。第一黑盒模型可以是该至少一个黑盒模型中的任意一个黑盒模型。图3所示的方法包括如下步骤：
S101：服务器获取候选网络性能指标和与该候选网络性能指标相关的特征信息。
本申请实施例对服务器获取候选网络性能指标和与该候选网络性能指标相关的特征信息的具体实现方式不进行限定。例如，服务器可以接收用户指示的或其他设备发送的该候选网络性能指标和与该候选网络性能指标相关的特征信息。又如，服务器可以基于用户指示的信息或其他设备发送的信息，确定该候选网络性能指标和与该候选网络性能指标相关的特征信息。
候选网络性能指标，可以是核心网、承载网和接入网中的任意一个网络性能指标，或者可以是跨网络的网络性能指标。核心网中的性能指标可以包括以下至少一种：用户分布特征、用户运动特征、业务特征、时延、服务质量(quality of service，QoS)、平均主观意见得分MOS(mean opinion score，MOS)分、抖动和吞吐量等。承载网中的性能指标可以包括以下至少一种：时延、抖动和吞吐量等。接入网中的性能指标可以包括以下至少一种：干扰、覆盖、信道容量、吞吐量、rank和抖动等。跨网络的网络性能指标可以包括时延、抖动和吞吐量等。在具体实现时，候选网络性能指标可以是现有网络仿真技术中能够仿真的任意一个网络性能指标。当然本申请实施例不限于此。
与一个候选网络性能指标相关的特征信息是指用于获得该候选网络性能指标所需的部分或全部特征信息。与一个候选网络性能指标相关的特征信息的数量可以是一个或多个。该一个或多个参数具体是哪些参数可以是基于对候选网络性能指标的影响程度等所确定的，例如，该一个或多个参数可以是对该候选网络性能指标影响较大的一个或多个参数。
例如，候选网络性能指标包括rank，与rank相关的特征信息包括信道矩阵等。其中，当该rank是终端的rank时，该信道矩阵可以是该终端与该终端的主服务基站之间的信道矩阵。
例如，候选网络性能指标包括信道矩阵，与信道矩阵相关的特征信息包括多径参数和天线配置参数等。其中，当信道矩阵是终端与该终端的主服务小区之间的信道矩阵时，该多径参数可以是该终端与该主服务小区之间的信道的多径参数，该天线配置参数可以是该终端的天线配置参数和该主服务基站的天线配置参数。多径参数可以包括角度和/或时延扩展等。天线配置参数可以包括天线阵列所包括的天线的个数、方位等信息。
例如，候选网络性能指标包括多径参数，与多径参数相关的特征信息包括电子地图、工程参数和地物类型中的至少一种。其中，当多径参数该多径参数可以是该终端与该主服务基站之间的信道的多径参数时，电子地图、工程参数和地物类型分别是用于表征该终端与其主服务基站所在的物理环境的电子地图、工程参数和地物类型。这样，可以提高对多径参数的仿真精确度。
例如，候选网络性能指标包括小区的信道状态概率分布参数，与该小区的信道状态概率分布参数相关的特征信息包括该小区的栅格信道矩阵和该小区的信道状态占用时间序列等。
例如，候选网络性能指标包括小区的预编码矩阵，与该小区的预编码矩阵相关的特征信息包括该小区的栅格信道矩阵等。
S102：服务器确定获取训练数据。该训练数据包括多组值，每组值包括该候选网络性能指标的一个值和用于获得该值时所采用的具体特征信息。其中，具体特征信息，可以理解为S101中所确定的特征信息的实例。例如，当S101中所确定的特征信息包括信道矩阵时，该特征信息的实例是一个具体的矩阵。又如，当S101中所确定的特征信息包括多径参数时，该特征信息的实例是特定终端与该特定终端的主服务基站之间的信道的多径参数。
例如，如果候选网络性能指标包括信道矩阵，与信道矩阵相关的特征信息包括多径参数和天线配置参数，那么，训练数据可以包括：{多径参数1，天线配置参数1，信道矩阵1}、{多径参数2，天线配置参数2，信道矩阵2}……。其中，{多径参数n，天线配置参数n，信道矩阵n}是训练数据中的一组值，该组值用于表示当输入参数的值是多径参数n和天线配置参数n时，输出参数的值是信道矩阵n。n是大于等于1的值。
可选的，S102中的训练数据来自实际网络或除本申请实施例提供的仿真平台之外的其他仿真平台。其中，实际网络是指实际部署的网络，该网络中可以包括：PGW、SGW、基站和终端等网元。例如该实际网络可以是某个局点(即一个区域如某个城市)中实际部署的网络。该其他仿真平台可以是现有技术中的任意一种网络仿真平台，例如可以是如图1所示的网络仿真平台。或者，该其他仿真平台可以是未来的网络仿真平台。
S103：基于该训练数据，训练机器学习模型，得到第一黑盒模型。第一黑盒模型的输出参数是该候选网络性能指标，第一黑盒模型的输入参数是与S101中所确定的该候选网络性能指标相关的特征信息。第一黑盒模型用于对该候选网络性能指标进行仿真。
其中，可选的，机器学习模型的类型可以包括神经网络模型，如基于图片的神经网络模型或卷积神经网络(convolutional neural network，CNN)等。另外，机器学习模型的类型可以包括统计学习模型，如回归模型或分类模型等。
S103可以包括：基于该候选网络性能指标和与该候选网络性能指标相关的特征信息选择机器学习模型。例如，当候选网络性能指标是多径参数时，所选择的机器学习模型可以是深度学习模型如基于图片的神经网络模型。又如，当候选网络性能指标是信道矩阵时，所选择的机器学习模型可以是卷积神经网络(即CNN)。然后，基于训练数据训练该机器学习模型，得到第一黑盒模型。本申请实施例对训练机器学习模型的具体实现方式不进行限定，例如可以参考现有技术。其中，对不同网络性能指标进行仿真时所使用的机器学习模型可以相同，也可以不同。
可以理解的是，基于上述S101～S103可以创建用于对一个候选网络性能指标进行仿真的黑盒模型，采用类似的方法可以分别创建用于对核心网、承载网和/或接入网中的一个或多个候选网络性能指标进行仿真的黑盒模型。其中，每个黑盒模型均被定义了输入参数和输出参数(即输入接口和输出接口)。至此，可以认为已创建好网络仿真平台。
如图4所示，为本申请实施例提供的一种网络仿真平台的示意图。图4中所示的网络仿真平台包括：接入网部分、承载网部分和核心网部分。接入网部分包括：信道 模型、干扰模型、覆盖模型、信道容量模型和rank模型等，分别用于对信道矩阵、干扰、覆盖、信道容量和rank等网络性能指标进行仿真。承载网部分包括时延模型、抖动模型和传输资源映射模型等，分别用于对时延、抖动和传输资源映射等网络性能指标进行仿真。核心网部分包括：用户分布模型、用户运动模型、业务模型、QoS模型和MOS模型等，分别用于对用户分布特征、用户运动特征、业务特征、QoS以及MOS进行仿真。图4中的每个模型均可以是一个黑盒模型。
在具体实现的过程中，针对接入网，无线运营商会关注覆盖、干扰和容量等性能指标。基于此，可以将覆盖、干扰和容量分别作为候选网络性能指标，从而执行S101～S103，以创建用于对覆盖进行仿真的黑盒模型、用于对干扰进行仿真的黑盒模型，以及用于对容量进行仿真的黑盒模型。该过程也可以被称为是对覆盖、干扰和容量进行黑盒化的过程。
可选的，该网络仿真平台还包括第二黑盒模型。第二黑盒模型的输出参数是第一黑盒模型的其中一个输入参数。如果第一黑盒模型有多个输入参数，则该多个输入参数中的任意一个或多个输入参数均可以是由本申请实施例提供的一个黑盒模型的输出参数。
该可选的技术方案说明，本申请实施例支持将一个黑盒模型的输出参数作为另一个黑盒模型的输入参数(即黑盒模型之间具有关联关系)的技术方案。例如，参见图5A，多径参数模型的输出参数(即多径参数)可以作为信道模型的输入参数；信道模型的输出参数(即信道矩阵)可以作为干扰模型、覆盖模型、信道容量模型和rank模型的输入参数。
该可选的技术方案为“在仿真过程中，将多个黑盒模型进行级联，从而对某个网络性能指标进行仿真”提供了理论依据，并且，为实现端到端仿真提供了理论依据。
可选的，第一黑盒模型的输出参数是第一网络中的网络性能指标，第二黑盒模型的输出参数是第二网络中的网络性能指标。其中，第一网络是核心网、承载网或接入网；第二网路是核心网、承载网和接入网中的除第一网络之外的任一网络。也就是说，本申请实施例支持对跨网络的性能指标的仿真，即支持端到端的仿真；换句话说，接入网部分、承载网部分和核心网部分中的模型之间可以协同完成对某个网络性能指标的仿真。
例如，参见图5B，接入网部分的信道容量模型(即信道容量)和业务模型，以及核心网部分中的QoS模型的输出参数(即QoS需求)，可以作为承载网的时延模型的输入参数。
本申请实施例提供的创建网络仿真平台的方法，主要关注候选网络性能指标及其相关的输入特征信息，不需要模拟实际网络中的协议流程，因此复杂度较小，可以适用于大规模的网络仿真。
以下，通过一个具体实施例对上文中所提供的创建网络仿真平台的方法进行说明。
本实施例的目标是：创建能够对实现信道矩阵进行仿真的黑盒模型。基于此，候选网络性能指标是信道矩阵，与信道矩阵相关的特征信息是多径参数。本申请实施例提供的用于对信道矩阵进行仿真的黑盒模型的创建方法如下：
第一步：获取训练数据。该训练数据包括多组值。每组值包括一个信道矩阵，以及用 于得到该信道矩阵的多径参数的值。
在一些实现方式中，多径参数的值可以是基于多径参数模型确定的。其中，该多径参数模型可以是传统技术中提供的多径参数模型，如射线追踪模型或统计模型。或者，该多径参数模型可以是本申请实施例中提供的用于对多径参数进行仿真的黑盒模型。如图6所示，为本申请实施例提供的多径参数模型和信道矩阵模型的示意图。
在另一些实现方式中，多径参数的值可以是基于实测数据估计得到的。
第二步：将所获取的训练数据输入神经网络模型(如3GPP协议模型)，并训练该神经网络模型，得到用于对信道矩阵进行仿真的黑盒模型。
可以理解的是，多径参数能够在射线追踪或实测数据中获取，因此获得多径参数模型和获得信道矩阵的过程可以单独训练，通过并行训练能够提升训练效率。
需要说明的是，传统技术中，通常基于统计模型或射线追踪模型等来获得对信道矩阵进行仿真时所需的多径参数。其中，在基于统计模型获得对信道矩阵进行仿真时所需的多径参数时，仿真所需的多径参数是通过统计模型随机产生的，因此无法准确反映真实环境下的信道响应。基于射线追踪模型等来获得对信道矩阵进行仿真时所需的多径参数，可以包括：基于平面波假设，在电子地图中产生大量射线并通过算法追踪这些射线的反射、散射行为，最终产生仿真所需的多径参数。该方法计算效率比较低，无法适用于大规模仿真场景中。而本实施例中，基于黑盒模型对多径参数进行仿真，可以直接从电子地图和工程参数中获取无线传播环境的特征，并根据这些特征产生对应的多径参数，这样可以使仿真更高效，因此可以适用于大规模仿真场景中。
在具体实现的过程中，本申请实施例提供的创建网络仿真平台可以以库的形式对外提供调用接口。该网络仿真平台中的黑盒模型(例如图4中所示的各黑盒模型)可以被单独调用，也可以多个黑盒模型被联合调用，从而完成对某个网络性能指标的仿真。另外还可以直接端到端的调用来完成跨网络的网络性能指标的仿真。用户可以根据仿真需求(如用户所关注的指标、仿真效率等)来灵活选择不同的调用方式。
在具体实现的过程中，在仿真阶段，黑盒模型可以用于替换传统网络仿真平台中的消息队列处理过程。具体的，一个黑盒模型可以用于替换传统网络仿真平台中的某个网元，或者可以用于替换一个网元中的某个功能模块(如信道模型、调度模型、传输模型等)。
当使用黑盒模型替换传统网络仿真平台中的某个网元时，该黑盒模型的输出参数是该网元的某个功能模块的输出参数，如图7A所示。图7A中黑盒模型替换了基站。当使用黑盒模型替换传统网络仿真平台中的某个功能模块时，该黑盒模型的输入参数与该功能模块的输入参数相同，该黑盒模型的输出参数与该功能模块的输出参数相同，如图7B所示。图7B中黑盒模型替换了信道模型。图7A和图7B是基于图1进行绘制的。图7B中黑盒模型替换了基站所包括的多模型光谱信道(可以理解为信道模型)。
在使用黑盒模型替换传统网络仿真平台中的网元或者功能模块之后，该黑盒模型与传统网络仿真平台中的数据可以互通。也就是说，本申请实施例提供的黑盒模型可以与传统网络仿真平台进行兼容。
如图8所示，为本申请实施例提供的一种网络仿真方法的流程示意图。该方法应用于包括至少两个黑盒模型的网络仿真平台。可选的，创建该网络仿真平台中的任意 一个黑盒模型的方法可以参考上文。图8所示的方法包括如下步骤：
S201：服务器确定第一待仿真网络性能指标。其中，第一待仿真网络性能指标可以是任意一个需要进行仿真的网络性能指标。
示例的，服务器可以接收用户指示的(或其他设备指示的)一个网络性能指标，并将该网络性能指标作为第一待仿真网络性能指标，或者对该网络性能指标进行分析，从而将与该网络性能指标相关的一个网络性能指标作为第一待仿真网络性能指标。
S202：服务器从该至少两个黑盒模型中，查找用于对第一待仿真网络性能指标进行仿真的第一黑盒模型。其中，第一黑盒模型的输出参数是第一待仿真网络性能指标。
具体的，服务器可以将该至少两个黑盒模型中，输出参数是第一待仿真网络性能指标的黑盒模型作为第一黑盒模型。
S203：服务器将第一黑盒模型的输入参数的值输入第一黑盒模型，得到第一黑盒模型的输出参数的值，并将所得到的值作为第一待仿真网络性能指标的仿真结果。
例如，假设第一黑盒模型是信道模型，用于对终端和该终端的主服务基站之间的信道矩阵进行仿真，那么，第一黑盒模型的输入参数的值可以包括：该终端的天线配置参数的值、该主服务基站的天线配置参数的值，以及该终端和该主服务基站之间的多径参数的值等。基于此，第一待仿真网络性能指标的仿真结果是该终端和该主服务基站之间的信道矩阵。
本申请实施例提供的网络仿真方法，可以直接对待仿真网络性能参数进行仿真，与传统技术中通过模拟消息处理流程再间接对待仿真网络性能参数进行仿真的技术方案相比，有助于提高仿真效率。
如图9所示，为本申请实施例提供的一种网络仿真方法的流程示意图。该方法应用于包括至少两个黑盒模型的网络仿真平台。图9所示的方法包括如下步骤：
S301～S302：可以参考上述S201～S202。当然本申请实施例不限于此。
S303：服务器从该至少两个黑盒模型中，查找用于对第二待仿真网络性能指标进行仿真的第二黑盒模型；第二黑盒模型的输出参数是第一黑盒模型的其中一个输入参数。
具体的，服务器如果确定第一黑盒模型的其中一个输入参数是该至少两个黑盒模型中的一个黑盒模型的输出参数，则将该输入参数作为第二待仿真网络性能指标，并将该黑盒模型作为第二黑盒模型。
例如，当第一待仿真网络性能指标是信道矩阵时，由于信道模型(即第一黑盒模型)的其中一个输入参数是多径参数，因此，可以将多径参数作为第二待仿真网络性能指标，并将多径参数模型作为第二黑盒模型。
S304：服务器将第二黑盒模型的输入参数的值输入第二黑盒模型，得到第二黑盒模型的输出参数的值，并将所得到的值作为第二待仿真网络性能指标的仿真结果。
例如，假设第二黑盒模型是多径参数模型，并且，第一待仿真网络性能指标是终端与其主服务基站之间的信道矩阵，则服务器可以将该终端与该主服务基站所在的物理环境的电子地图、工程参数和地物类型输入第二黑盒模型。基于此，从第二待仿真网络性能指标的仿真结果为该终端与该主服务基站之间的信道的多径参数。
本申请实施例对S302与S303～S304的执行顺序不进行限定，例如可以先执行S302 再执行S303～S304，或者可以先执行S303～S304再执行S302，或者可以在执行S303～S304的过程中执行S302等。
S305：服务器将第二待仿真网络性能指标的仿真结果输入第一黑盒模型，得到第一黑盒模型的输出参数的值，并将所得到的值作为第一待仿真网络性能指标的仿真结果。
具体的，如果第一黑盒模型还有其他输入参数(即除第二黑盒模型的输出参数之外的其他输入参数)，则服务器还可以将第二待仿真网络性能指标的仿真结果所对应的该其他输入参数的值输入第一黑盒模型，从而得到第一黑盒模型的输出参数的值。具体示例可以参考上述S203中的具体示例。
上述主要从方法的角度对本申请实施例提供的方案进行了介绍。为了实现上述功能，其包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对服务器进行功能模块的划分，例如可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
如图10所示，为本申请实施例提供的一种创建网络仿真平台的装置100的结构示意图。作为一个示例，该装置100可以是上文中的用于执行创建网络仿真平台的方法的服务器。作为一个示例，该装置100可以用于执行图3所示的方法中服务器所执行的步骤。
该网络仿真平台包括至少一个黑盒模型，该至少一个黑盒模型包括第一黑盒模型。该装置100包括：获取单元1001和训练单元1002。其中，获取单元1001，用于获取候选网络性能指标和与该候选网络性能指标相关的特征信息。训练单元1002，用于训练机器学习模型，得到第一黑盒模型。第一黑盒模型的输出参数是该候选网络性能指标。第一黑盒模型的输入参数是与该候选网络性能指标相关的特征信息。第一黑盒模型用于对该候选网络性能指标进行仿真。可选的，与该候选网络性能指标相关的特征信息是指用于获得该候选网络性能指标所需的部分或全部特征信息。
例如，结合图3，获取单元1001具体可以用于执行S101，训练单元1002具体可以用于执行S103。
可选的，该候选网络性能指标包括：核心网中的网络性能指标、承载网中的网络性能指标、接入网中的网络性能指标或者跨网络的网络性能指标。
可选的，该至少一个黑盒模型还包括第二黑盒模型。第二黑盒模型的输出参数是所述第一黑盒模型的其中一个输入参数。
可选的，第一黑盒模型的输出参数是第一网络中的网络性能指标，第二黑盒模型 的输出参数是第二网络中的网络性能指标。其中，第一网络是核心网、承载网或接入网。第二网路是核心网、承载网和接入网中的除第一网络之外的任一网络。
可选的，获取单元1001还用于，获取训练数据。该训练数据来自实际网络或除本申请实施例提供的仿真平台之外的其他仿真平台。训练单元1002具体用于，基于该训练数据训练机器学习模型，所述第一黑盒模型。例如，结合图3，获取单元1001具体可以用于执行S102。
可选的，机器学习模型包括神经网络模型。
可选的，该候选网络性能指标包括rank，与rank相关的特征信息包括信道矩阵。可选的，该候选网络性能指标包括信道矩阵，与信道矩阵相关的特征信息包括多径参数和天线配置参数。可选的，该候选网络性能指标包括多径参数，与多径参数相关的特征信息包括电子地图、工程参数和地物类型。可选的，该候选网络性能指标包括小区的信道状态概率分布参数，与小区的信道状态概率分布参数相关的特征信息包括小区的栅格信道矩阵和小区的信道状态占用时间序列。可选的，该候选网络性能指标包括小区的预编码矩阵，与小区的预编码矩阵相关的特征信息包括小区的栅格信道矩阵。
在一个示例中，参见图2，上述获取单元1001和训练单元1002均可以由图2中的处理器201调用存储器203中存储的计算机程序实现。
关于上述可选方式的具体描述参见前述的方法实施例，此处不再赘述。此外，上述提供的任一种装置100的解释以及有益效果的描述均可参考上述对应的方法实施例，不予赘述。
需要说明的是，上述各个单元对应执行的动作仅是具体举例，各个单元实际执行的动作参照上述基于图3所述的实施例的描述中提及的动作或步骤。
如图11所示，为本申请实施例提供的一种网络仿真装置110的结构示意图。该装置可以应用于包括至少两个黑盒模型的网络仿真平台。作为一个示例，该装置100可以是上文中的用于执行网络仿真方法的服务器。作为一个示例，该装置100可以用于执行图8或图9所示的方法中服务器所执行的步骤。
该装置110包括：确定单元1101、查找单元1102和仿真单元1103。确定单元1101，用于确定第一待仿真网络性能指标。查找单元1102，用于从该至少两个黑盒模型中，查找用于对第一待仿真网络性能指标进行仿真的第一黑盒模型。第一黑盒模型的输出参数是第一待仿真网络性能指标。仿真单元1103，用于将第一黑盒模型的输入参数的值输入第一黑盒模型，得到第一黑盒模型的输出参数的值，将所得到的值作为第一待仿真网络性能指标的仿真结果。例如，结合图8，确定单元1101可以用于执行S201，查找单元1102可以用于执行S202，仿真单元1103可以用于执行S203。又如，结合图9，确定单元1101可以用于执行S301，查找单元1102可以用于执行S302，仿真单元1103可以用于执行S304和S305。
可选的，第一待仿真网络性能指标包括：核心网中的网络性能指标、承载网中的网络性能指标或接入网中的网络性能指标或跨网络的网络性能指标。
可选的，查找单元1102还用于，从该至少两个黑盒模型中，查找用于对第二待仿真网络性能指标进行仿真的第二黑盒模型。第二黑盒模型的输出参数是第一黑盒模型的其中一个输入参数。仿真单元1103还用于，将第二黑盒模型的输入参数的值输入第 二黑盒模型，得到第二黑盒模型的输出参数的值，并将所得到的值作为第二待仿真网络性能指标的仿真结果。仿真单元1103在执行将第一黑盒模型的输入参数的值输入第一黑盒模型时，具体用于：将第二待仿真网络性能指标的仿真结果输入第一黑盒模型。例如，结合图9，查找单元1102可以用于执行S303，仿真单元1103可以用于执行S305。
可选的，第一黑盒模型的输出参数是第一网络中的网络性能指标，第二黑盒模型的输出参数是第二网络中的网络性能指标。其中，第一网络是核心网、承载网或接入网。第二网路是核心网、承载网和接入网中的除第一网络之外的任一网络。
可选的，第一待仿真网络性能指标包括rank，与rank相关的特征信息包括信道矩阵。
可选的，第一待仿真网络性能指标包括信道矩阵，第一黑盒模型的输出参数包括多径参数和天线配置参数。
可选的，第一待仿真网络性能指标包括多径参数，第一黑盒模型的输出参数包括电子地图、工程参数和地物类型。
可选的，第一待仿真网络性能指标包括小区的信道状态概率分布参数，第一黑盒模型的输出参数包括小区的栅格信道矩阵和小区的信道状态占用时间序列。
可选的，第一待仿真网络性能指标包括小区的预编码矩阵，第一黑盒模型的输出参数包括小区的栅格信道矩阵。
在一个示例中，参见图2，上述确定单元1101、查找单元1102和仿真单元1103均可以由图2中的处理器201调用存储器203中存储的计算机程序实现。
关于上述可选方式的具体描述参见前述的方法实施例，此处不再赘述。此外，上述提供的任一种装置110的解释以及有益效果的描述均可参考上述对应的方法实施例，不再赘述。
需要说明的是，上述各个单元对应执行的动作仅是具体举例，各个单元实际执行的动作参照上述基于图8或图9所述的实施例的描述中提及的动作或步骤。
需要说明的是，上文中所描述的处理器可以通过硬件来实现也可以通过软件来实现。当通过硬件实现时，该处理器可以是逻辑电路、集成电路等。当通过软件来实现时，该处理器可以是一个通用处理器，通过读取存储器中存储的软件代码来实现。该存储器可以集成在处理器中，也可以位于处理器之外，独立存在。
本申请实施例还提供了一种芯片。该芯片中集成了用于实现上述处理器的功能的电路和一个或者多个接口。可选的，该芯片支持的功能可以包括基于图3、图8或图9所述的实施例中的处理动作，此处不再赘述。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可通过程序来指令相关的硬件完成。所述的程序可以存储于一种计算机可读存储介质中。上述提到的存储介质可以是只读存储器，随机接入存储器等。上述处理单元或处理器可以是中央处理器，通用处理器、特定集成电路(application specific integrated circuit，ASIC)、微处理器(digital signal processor，DSP)，现场可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。
本申请实施例还提供了一种包含指令的计算机程序产品，当该指令在计算机上运行时，使得计算机执行上述实施例中的任意一种方法。该计算机程序产品包括一个或 多个计算机指令。在计算机上加载和执行计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令可以从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可以用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质(例如，软盘、硬盘、磁带)，光介质(例如，DVD)、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
应注意，本申请实施例提供的上述用于存储计算机指令或者计算机程序的器件，例如但不限于，上述存储器、计算机可读存储介质和通信芯片等，均具有非易失性(non-transitory)。
在实施所要求保护的本申请过程中，本领域技术人员通过查看附图、公开内容、以及所附权利要求书，可理解并实现公开实施例的其他变化。在权利要求中，“包括”(comprising)一词不排除其他组成部分或步骤，“一”或“一个”不排除多个的情况。单个处理器或其他单元可以实现权利要求中列举的若干项功能。相互不同的从属权利要求中记载了某些措施，但这并不表示这些措施不能组合起来产生良好的效果。
尽管结合具体特征及其实施例对本申请进行了描述，在不脱离本申请的精神和范围的情况下，可对其进行各种修改和组合。相应地，本说明书和附图仅仅是所附权利要求所界定的本申请的示例性说明，且视为已覆盖本申请范围内的任意和所有修改、变化、组合或等同物。

Claims (26)

  1. 一种创建网络仿真平台的方法，其特征在于，所述网络仿真平台包括至少一个黑盒模型，所述至少一个黑盒模型包括第一黑盒模型；所述方法包括：
    获取候选网络性能指标和与所述候选网络性能指标相关的特征信息；与所述候选网络性能指标相关的特征信息是指用于获得所述候选网络性能指标所需的部分或全部特征信息；
    训练机器学习模型，得到所述第一黑盒模型；其中，所述第一黑盒模型的输出参数是所述候选网络性能指标，所述第一黑盒模型的输入参数是所述与所述候选网络性能指标相关的特征信息；所述第一黑盒模型用于对所述候选网络性能指标进行仿真。
  2. 根据权利要求1所述的方法，其特征在于，所述候选网络性能指标包括：核心网中的网络性能指标、承载网中的网络性能指标、接入网中的网络性能指标或者跨网络的网络性能指标。
  3. 根据权利要求1或2所述的方法，其特征在于，所述至少一个黑盒模型还包括第二黑盒模型；所述第二黑盒模型的输出参数是所述第一黑盒模型的其中一个输入参数。
  4. 根据权利要求3所述的方法，其特征在于，所述第一黑盒模型的输出参数是第一网络中的网络性能指标，所述第二黑盒模型的输出参数是第二网络中的网络性能指标；
    其中，所述第一网络是核心网、承载网或接入网；所述第二网路是核心网、承载网和接入网中的除所述第一网络之外的任一网络。
  5. 根据权利要求1至4任一项所述的方法，其特征在于，所述方法还包括：
    获取训练数据；所述训练数据来自实际网络或除所述仿真平台之外的其他仿真平台；
    所述训练机器学习模型，得到所述第一黑盒模型，包括：
    基于所述训练数据，训练机器学习模型，得到所述第一黑盒模型。
  6. 根据权利要求1至5任一项所述的方法，其特征在于，所述机器学习模型包括神经网络模型。
  7. 根据权利要求1至6任一项所述的方法，其特征在于，
    所述候选网络性能指标包括秩rank，与所述rank相关的特征信息包括信道矩阵；
    或者，所述候选网络性能指标包括信道矩阵，与所述信道矩阵相关的特征信息包括多径参数和天线配置参数；
    或者，所述候选网络性能指标包括多径参数，与所述多径参数相关的特征信息包括电子地图、工程参数和地物类型；
    或者，所述候选网络性能指标包括小区的信道状态概率分布参数，与所述小区的信道状态概率分布参数相关的特征信息包括所述小区的栅格信道矩阵和所述小区的信道状态占用时间序列；
    或者，所述候选网络性能指标包括小区的预编码矩阵，与所述小区的预编码矩阵相关的特征信息包括所述小区的栅格信道矩阵。
  8. 一种网络仿真方法，其特征在于，应用于包括至少两个黑盒模型的网络仿真平 台，所述方法包括：
    确定第一待仿真网络性能指标；
    从所述至少两个黑盒模型中，查找用于对所述第一待仿真网络性能指标进行仿真的第一黑盒模型；其中，所述第一黑盒模型的输出参数是所述第一待仿真网络性能指标；
    将所述第一黑盒模型的输入参数的值输入所述第一黑盒模型，得到所述第一黑盒模型的输出参数的值，并将所得到的值作为所述第一待仿真网络性能指标的仿真结果。
  9. 根据权利要求8所述的方法，其特征在于，所述第一待仿真网络性能指标包括：核心网中的网络性能指标、承载网中的网络性能指标、接入网中的网络性能指标或跨网络的网络性能指标。
  10. 根据权利要求8或9所述的方法，其特征在于，所述方法还包括：
    从所述至少两个黑盒模型中，查找用于对第二待仿真网络性能指标进行仿真的第二黑盒模型；所述第二黑盒模型的输出参数是所述第一黑盒模型的其中一个输入参数；
    将所述第二黑盒模型的输入参数的值输入所述第二黑盒模型，得到所述第二黑盒模型的输出参数的值，并将所得到的值作为所述第二待仿真网络性能指标的仿真结果；
    所述将所述第一黑盒模型的输入参数的值输入所述第一黑盒模型，包括：
    将所述第二待仿真网络性能指标的仿真结果输入所述第一黑盒模型。
  11. 根据权利要求10所述的方法，其特征在于，所述第一黑盒模型的输出参数是第一网络中的网络性能指标，所述第二黑盒模型的输出参数是第二网络中的网络性能指标；
    其中，所述第一网络是核心网、承载网或接入网；所述第二网路是核心网、承载网和接入网中的除所述第一网络之外的任一网络。
  12. 根据权利要求8至11任一项所述的方法，其特征在于，
    所述第一待仿真网络性能指标包括秩rank，与所述rank相关的特征信息包括信道矩阵；
    或者，所述第一待仿真网络性能指标包括信道矩阵，所述第一黑盒模型的输出参数包括多径参数和天线配置参数；
    或者，所述第一待仿真网络性能指标包括多径参数，所述第一黑盒模型的输出参数包括电子地图、工程参数和地物类型；
    或者，所述第一待仿真网络性能指标包括小区的信道状态概率分布参数，所述第一黑盒模型的输出参数包括所述小区的栅格信道矩阵和所述小区的信道状态占用时间序列；
    或者，所述第一待仿真网络性能指标包括小区的预编码矩阵，所述第一黑盒模型的输出参数包括所述小区的栅格信道矩阵。
  13. 一种创建网络仿真平台的装置，其特征在于，所述网络仿真平台包括至少一个黑盒模型，所述至少一个黑盒模型包括第一黑盒模型；所述装置包括：
    获取单元，用于获取候选网络性能指标和与所述候选网络性能指标相关的特征信息；其中，与所述候选网络性能指标相关的特征信息是指用于获得所述候选网络性能指标所需的部分或全部特征信息；
    训练单元，用于训练机器学习模型，得到所述第一黑盒模型；其中，所述第一黑盒模型的输出参数是所述候选网络性能指标，所述第一黑盒模型的输入参数是所述与所述候选网络性能指标相关的特征信息；所述第一黑盒模型用于对所述候选网络性能指标进行仿真。
  14. 根据权利要求13所述的装置，其特征在于，所述候选网络性能指标包括：核心网中的网络性能指标、承载网中的网络性能指标、接入网中的网络性能指标或者跨网络的网络性能指标。
  15. 根据权利要求13或14所述的装置，其特征在于，所述至少一个黑盒模型还包括第二黑盒模型；所述第二黑盒模型的输出参数是所述第一黑盒模型的其中一个输入参数。
  16. 根据权利要求15所述的装置，其特征在于，所述第一黑盒模型的输出参数是第一网络中的网络性能指标，所述第二黑盒模型的输出参数是第二网络中的网络性能指标；
    其中，所述第一网络是核心网、承载网或接入网；所述第二网路是核心网、承载网和接入网中的除所述第一网络之外的任一网络。
  17. 根据权利要求13至16任一项所述的装置，其特征在于，
    所述获取单元还用于，获取训练数据；所述训练数据来自实际网络或除所述仿真平台之外的其他仿真平台；
    所述训练单元具体用于，基于所述训练数据训练机器学习模型，得到所述第一黑盒模型。
  18. 根据权利要求13至17任一项所述的装置，其特征在于，所述机器学习模型包括神经网络模型。
  19. 根据权利要求13至18任一项所述的装置，其特征在于，
    所述候选网络性能指标包括秩rank，与所述rank相关的特征信息包括信道矩阵；
    或者，所述候选网络性能指标包括信道矩阵，与所述信道矩阵相关的特征信息包括多径参数和天线配置参数；
    或者，所述候选网络性能指标包括多径参数，与所述多径参数相关的特征信息包括电子地图、工程参数和地物类型；
    或者，所述候选网络性能指标包括小区的信道状态概率分布参数，与所述小区的信道状态概率分布参数相关的特征信息包括所述小区的栅格信道矩阵和所述小区的信道状态占用时间序列；
    或者，所述候选网络性能指标包括小区的预编码矩阵，与所述小区的预编码矩阵相关的特征信息包括所述小区的栅格信道矩阵。
  20. 一种网络仿真装置，其特征在于，应用于包括至少两个黑盒模型的网络仿真平台，所述装置包括：
    确定单元，用于确定第一待仿真网络性能指标；
    查找单元，用于从所述至少两个黑盒模型中，查找用于对所述第一待仿真网络性能指标进行仿真的第一黑盒模型；所述第一黑盒模型的输出参数是所述第一待仿真网络性能指标；
    仿真单元，用于将所述第一黑盒模型的输入参数的值输入所述第一黑盒模型，得到所述第一黑盒模型的输出参数的值，将所得到的值作为所述第一待仿真网络性能指标的仿真结果。
  21. 根据权利要求20所述的装置，其特征在于，所述第一待仿真网络性能指标包括：核心网中的网络性能指标、承载网中的网络性能指标、接入网中的网络性能指标或跨网络的网络性能指标。
  22. 根据权利要求20或21所述的装置，其特征在于，
    所述查找单元还用于，从所述至少两个黑盒模型中，查找用于对第二待仿真网络性能指标进行仿真的第二黑盒模型；所述第二黑盒模型的输出参数是所述第一黑盒模型的其中一个输入参数；
    所述仿真单元还用于，将所述第二黑盒模型的输入参数的值输入所述第二黑盒模型，得到所述第二黑盒模型的输出参数的值，并将所得到的值作为所述第二待仿真网络性能指标的仿真结果；
    所述仿真单元在执行所述将所述第一黑盒模型的输入参数的值输入所述第一黑盒模型时，具体用于：将所述第二待仿真网络性能指标的仿真结果输入所述第一黑盒模型。
  23. 根据权利要求22所述的装置，其特征在于，所述第一黑盒模型的输出参数是第一网络中的网络性能指标，所述第二黑盒模型的输出参数是第二网络中的网络性能指标；
    其中，所述第一网络是核心网、承载网或接入网；所述第二网路是核心网、承载网和接入网中的除所述第一网络之外的任一网络。
  24. 根据权利要求20至23任一项所述的装置，其特征在于，
    所述第一待仿真网络性能指标包括秩rank，与所述rank相关的特征信息包括信道矩阵；
    或者，所述第一待仿真网络性能指标包括信道矩阵，所述第一黑盒模型的输出参数包括多径参数和天线配置参数；
    或者，所述第一待仿真网络性能指标包括多径参数，所述第一黑盒模型的输出参数包括电子地图、工程参数和地物类型；
    或者，所述第一待仿真网络性能指标包括小区的信道状态概率分布参数，所述第一黑盒模型的输出参数包括所述小区的栅格信道矩阵和所述小区的信道状态占用时间序列；
    或者，所述第一待仿真网络性能指标包括小区的预编码矩阵，所述第一黑盒模型的输出参数包括所述小区的栅格信道矩阵。
  25. 一种创建网络仿真平台的装置，其特征在于，包括：存储器和处理器，所述存储器用于存储计算机程序，所述处理器用于执行所述计算机程序，以执行权利要求1至7任一项所述的方法。
  26. 一种网络仿真装置，其特征在于，包括：存储器和处理器，所述存储器用于存储计算机程序，所述处理器用于执行所述计算机程序，以执行权利要求8至12任一项所述的方法。
PCT/CN2020/091813 2019-05-22 2020-05-22 创建网络仿真平台的方法和网络仿真方法及相应装置 Ceased WO2020233708A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
KR1020217019934A KR102543557B1 (ko) 2019-05-22 2020-05-22 네트워크 시뮬레이션 플랫폼 생성 방법, 네트워크 시뮬레이션 방법 및 대응하는 장치
EP20808886.4A EP3879758B1 (en) 2019-05-22 2020-05-22 Method for creating network simulation platform, and network simulation method and corresponding device
US17/345,273 US11856424B2 (en) 2019-05-22 2021-06-11 Method for creating network simulation platform, network simulation method, and corresponding apparatuses

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910430994.3 2019-05-22
CN201910430994.3A CN111988156B (zh) 2019-05-22 2019-05-22 创建网络仿真平台的方法和网络仿真方法及相应装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/345,273 Continuation US11856424B2 (en) 2019-05-22 2021-06-11 Method for creating network simulation platform, network simulation method, and corresponding apparatuses

Publications (1)

Publication Number Publication Date
WO2020233708A1 true WO2020233708A1 (zh) 2020-11-26

Family

ID=73435998

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/091813 Ceased WO2020233708A1 (zh) 2019-05-22 2020-05-22 创建网络仿真平台的方法和网络仿真方法及相应装置

Country Status (5)

Country Link
US (1) US11856424B2 (zh)
EP (1) EP3879758B1 (zh)
KR (1) KR102543557B1 (zh)
CN (1) CN111988156B (zh)
WO (1) WO2020233708A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11424991B2 (en) 2020-11-06 2022-08-23 Google Llc Change impact simulation analysis
CN115941004A (zh) * 2021-08-11 2023-04-07 华为技术有限公司 一种通信方法及装置
US20230156482A1 (en) * 2021-11-12 2023-05-18 Verizon Patent And Licensing Inc. Systems and methods for feature importance determination in a wireless network modeling and simulation system
US12206723B1 (en) * 2023-07-06 2025-01-21 Keysight Technologies, Inc. Systems and methods for testing video streaming on wireless networks
US12501291B2 (en) * 2023-09-06 2025-12-16 Rohde & Schwarz Gmbh & Co. Kg Method and test system for establishing an over-the-air communication connection between a DUT and a mobile communication tester
CN119728454B (zh) * 2025-02-27 2025-07-04 北京邮电大学 大模型训练的网络流量模拟方法、装置、设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105491599A (zh) * 2015-12-21 2016-04-13 南京华苏科技股份有限公司 预测lte网络性能指标的新型回归系统
CN105634787A (zh) * 2014-11-26 2016-06-01 华为技术有限公司 网络关键指标的评估方法、预测方法及装置和系统
CN108234198A (zh) * 2017-12-19 2018-06-29 清华大学 一种基站流量预测方法和设备
CN109217955A (zh) * 2018-07-13 2019-01-15 北京交通大学 基于机器学习的无线环境电磁参数拟合方法
CA2977300A1 (en) * 2017-08-25 2019-02-25 Vahid POURAHMADI System and methods for channel modeling/estimation in a wireless communication network

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101237395A (zh) * 2007-02-01 2008-08-06 北京邮电大学 宽带移动通信网络性能的分层动态仿真实现方法
US9439081B1 (en) * 2013-02-04 2016-09-06 Further LLC Systems and methods for network performance forecasting
CN103747455B (zh) 2013-12-27 2018-03-09 南京信息工程大学 基于非均匀散射体分布的信道建模方法及参数匹配方法
CN105553584A (zh) 2015-12-10 2016-05-04 国网山东省电力公司烟台供电公司 一种3d mimo信道建模的方法
TWI647934B (zh) * 2017-04-21 2019-01-11 思銳科技股份有限公司 網路拓樸實機模擬方法與系統
CN106961122A (zh) 2017-05-08 2017-07-18 河海大学常州校区 一种基于特征模型的微电网动态等效建模方法
CN108564241A (zh) 2018-01-09 2018-09-21 河海大学常州校区 一种基于特征模型的微电网群等效聚合方法
CN108365903B (zh) 2018-01-29 2021-02-02 哈尔滨工程大学 一种基于随机散射簇的三维Massive MIMO信道建模方法
CN108512621B (zh) 2018-03-02 2020-12-29 东南大学 一种基于神经网络的无线信道建模方法
US10477426B1 (en) * 2019-02-06 2019-11-12 Accenture Global Solutions Limited Identifying a cell site as a target for utilizing 5th generation (5G) network technologies and upgrading the cell site to implement the 5G network technologies
US20200327449A1 (en) * 2019-04-15 2020-10-15 Accenture Global Solutions Limited User retention platform

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105634787A (zh) * 2014-11-26 2016-06-01 华为技术有限公司 网络关键指标的评估方法、预测方法及装置和系统
CN105491599A (zh) * 2015-12-21 2016-04-13 南京华苏科技股份有限公司 预测lte网络性能指标的新型回归系统
CA2977300A1 (en) * 2017-08-25 2019-02-25 Vahid POURAHMADI System and methods for channel modeling/estimation in a wireless communication network
CN108234198A (zh) * 2017-12-19 2018-06-29 清华大学 一种基站流量预测方法和设备
CN109217955A (zh) * 2018-07-13 2019-01-15 北京交通大学 基于机器学习的无线环境电磁参数拟合方法

Also Published As

Publication number Publication date
EP3879758A4 (en) 2022-03-09
KR20210092305A (ko) 2021-07-23
CN111988156A (zh) 2020-11-24
EP3879758A1 (en) 2021-09-15
CN111988156B (zh) 2022-04-05
US20210306881A1 (en) 2021-09-30
KR102543557B1 (ko) 2023-06-13
US11856424B2 (en) 2023-12-26
EP3879758B1 (en) 2025-08-27

Similar Documents

Publication Publication Date Title
WO2020233708A1 (zh) 创建网络仿真平台的方法和网络仿真方法及相应装置
US10805171B1 (en) Understanding network entity relationships using emulation based continuous learning
US11539590B2 (en) Detect impact of network maintenance in software defined infrastructure
Zeng et al. Emuedge: A hybrid emulator for reproducible and realistic edge computing experiments
US9456365B2 (en) Virtualized wireless communication system and establishing method thereof
US20230019202A1 (en) Method and electronic device for generating molecule set, and storage medium thereof
CN114548416A (zh) 数据模型训练方法及装置
JP2025500937A (ja) ユーザ機器人工知能機械学習能力のカテゴリ化システム、方法、デバイス、およびプログラム
CN116418664A (zh) 自动化网络设备模型创建的方法及设备、系统及存储介质
CN115002215A (zh) 面向云上政企的资源分配模型训练方法以及资源分配方法
Aral et al. Simulators and emulators for edge computing
CN113890831B (zh) 一种模拟网络设备的方法、装置及存储介质
CN109308243A (zh) 数据处理方法、装置、计算机设备和介质
CN109361547A (zh) 一种网络切片链路部署方法与装置
Liu et al. A discrete-event-based simulator for distributed deep learning
CN113704765A (zh) 基于人工智能的操作系统识别方法、装置及电子设备
Murgod et al. A comparative study of different network simulation tools and experimentation platforms for underwater communication
Mittal Opnet: An integrated design paradigm for simulations
CN113992520B (zh) 一种虚拟网络资源的部署方法和系统
Maller et al. Cloud-in-the-Loop simulation of C-V2X application relocation distortions in Kubernetes based Edge Cloud environment
US20260089521A1 (en) Systems and methods for backhaul bandwidth and core capacity estimation for dedicated networks
Li et al. Fast AI Model Splitting over Edge Networks
CN112738815A (zh) 一种可接入用户数的评估方法和装置
US12132603B2 (en) Data processing method, apparatus, and system, and storage medium
US20250063396A1 (en) Computerized system, method and apparatus for integrated device testing, verification and configuration

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20808886

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020808886

Country of ref document: EP

Effective date: 20210610

ENP Entry into the national phase

Ref document number: 20217019934

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 2020808886

Country of ref document: EP