WO2014161281A1 - 联机处理数据的方法、设备及系统 - Google Patents
联机处理数据的方法、设备及系统 Download PDFInfo
- Publication number
- WO2014161281A1 WO2014161281A1 PCT/CN2013/084374 CN2013084374W WO2014161281A1 WO 2014161281 A1 WO2014161281 A1 WO 2014161281A1 CN 2013084374 W CN2013084374 W CN 2013084374W WO 2014161281 A1 WO2014161281 A1 WO 2014161281A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- data
- business volume
- business
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
Definitions
- the present invention relates to data processing techniques, and more particularly to a method, apparatus, and system for processing data online. Background technique
- An On-Line Transaction Processing (OLAP) server is usually set up in each subnet of the operator to obtain the traffic of each subnet.
- OLAP On-Line Transaction Processing
- an OLAP server in each subnet obtains the traffic of each user, and then manually exports the traffic data of all users in the OLAP server as A Comma Separated Value (CSV) file, manually importing the CSV file into a separate database, and summarizing the traffic data of all users in the independent database to generate a central report.
- CSV Comma Separated Value
- Embodiments of the present invention provide a method, device, and system for processing data online to improve the efficiency of processing data online.
- an embodiment of the present invention provides a method for processing data online, including: sending a data request message to at least one sub-line analysis processing OLAP server; receiving user service data information sent by the at least one sub-OLAP server, The user service data information includes the traffic volume information of the roaming user and the traffic volume information of the non-roaming user that satisfies the first condition parameter, where the traffic volume information includes the user identifier and the corresponding traffic volume;
- the service total amount information of each user that meets the second condition parameter is identified by the sum of the corresponding traffic volume, where the service total amount information includes the user identifier and the corresponding sum of the service volumes.
- the service quantity summation is performed based on the user identifier and the corresponding traffic volume, and the sum of the service volumes corresponding to the user identifier is satisfied.
- the total amount of business information of each user of the second condition parameter includes:
- the total traffic volume information of each user that satisfies the second condition parameter is determined.
- the data request message carries a data reporting parameter, where the data reporting parameter includes any one of the following or The combination is: a service type parameter to be counted, a statistical time period parameter, and the first condition parameter.
- the first condition parameter is that the traffic volume is greater than the first a condition parameter of a preset threshold, or a condition parameter in which the traffic is ranked before the first preset ranking among the traffic of all non-roaming users;
- the second condition parameter is a condition parameter that the sum of the traffic volume is greater than the second preset threshold, or the condition parameter that the sum of the traffic volume is ranked before the second preset ranking in the sum of the traffic of all users.
- the user identifier and the corresponding The service volume is summed with the traffic, and after the sum of the traffic volume corresponding to the user identifier and the service total information of each user that meets the second condition parameter is obtained, the method further includes:
- a multi-dimensional traffic graph is generated based on the total amount of business information of each user that satisfies the second condition parameter.
- each of the sub-line analysis processing OLAP servers are set in different geography region.
- an embodiment of the present invention provides a method for processing data online, including: receiving a data request message sent by a central online analysis processing OLAP server, and acquiring the data request with the data request The data reporting parameter associated with the message;
- the user service data information includes the traffic volume information of the roaming user and the non-roaming user that satisfies the first condition parameter.
- Traffic information where the traffic information includes a user identifier and a corresponding traffic volume;
- the calculating, by the data reporting parameter, the traffic volume of the roaming user and the non-roaming user, before obtaining the user service data information :
- each user traffic information includes a user identifier of the user, a single traffic volume of the user, a home location identifier of the user, and generating the single traffic volume Geographical area identifier;
- Determining that the user is a roaming user and determining, according to the data reporting parameter, all user services including the user identifier, if the home location identifier of the user is inconsistent with the geographic area identifier that generates the single service volume.
- the single traffic in the quantity information is counted, and the traffic information of the roaming user is obtained;
- Determining that the user is a non-roaming user and determining, according to the data reporting parameter, all users that include the user identifier, if the home location identifier of the user is consistent with the geographic area identifier that generates the single-time traffic volume.
- the traffic of the non-roaming user in the traffic information is obtained, and the traffic volume of the non-roaming user is obtained according to the traffic volume of all the non-roaming users.
- the data reporting parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, the first Conditional parameter
- the data reporting parameter includes a service type parameter to be counted
- the data is based on the data
- the reporting parameter is used to collect statistics on the service volume of the roaming user and the non-roaming user
- the information about the user service data includes: collecting statistics on the service quantity of the service type parameter to be counted by the roaming user and the non-roaming user, and obtaining the user service data information.
- the data reporting parameter includes the statistic time period parameter
- the statistic is performed on the service quantity of the roaming user and the non-roaming user according to the data reporting parameter, and the user service data information is obtained by: the roaming user and the non-roaming user are corresponding to:
- the traffic of the statistical time period parameter is counted to obtain user service data information.
- the first condition parameter is that the traffic volume is greater than the first a condition parameter of a preset threshold, or a condition parameter in which the traffic is ranked before the first preset ranking among the traffic of all non-roaming users;
- the second condition parameter is a condition parameter that the sum of the traffic volume is greater than the second preset threshold, or the condition parameter that the sum of the traffic volume is ranked before the second preset ranking in the sum of the traffic of all users.
- the acquiring the data associated with the data request message includes:
- the obtaining the data report parameter associated with the data request message includes:
- Extracting the carried data reporting parameter from the data request message Extracting the carried data reporting parameter from the data request message.
- the embodiment of the present invention provides a central online analysis processing server, including: a sending module, configured to send a data request message to at least one sub-line analysis processing OLAP server;
- a receiving module configured to receive user service data information sent by the at least one sub-OLAP server, where the user service data information includes traffic information of the roaming user and traffic information of the non-roaming user that satisfies the first condition parameter,
- the traffic information includes the user identifier and the corresponding traffic volume;
- a processing module configured to perform traffic summation based on the user identifier and the corresponding traffic volume, and obtain a total traffic volume of each user that satisfies the second condition parameter by the total traffic volume corresponding to the user identifier
- the information, the service total information includes the user identifier and the corresponding sum of the traffic.
- the processing module is specifically configured to:
- the total traffic volume information of each user that satisfies the second condition parameter is determined.
- the sending module is specifically configured to: send a data request that carries a data reporting parameter to the at least one sub OLAP server
- the data reporting parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, and the first condition parameter.
- the processing module is further configured to: The total amount of business information of each user of the second condition parameter generates a multi-dimensional traffic chart.
- the embodiment of the present invention provides a sub-line analysis processing server, which includes:
- a receiving module configured to receive a data request message sent by the central online analysis processing OLAP server, and obtain a data reporting parameter associated with the data request message;
- the processing module is configured to perform statistics on the service volume of the roaming user and the non-roaming user according to the data reporting parameter, and obtain user service data information, where the user service data information includes the traffic quantity information of the roaming user and the first condition parameter is met. Traffic information of the non-roaming user, the traffic information includes a user identifier and a corresponding traffic volume;
- a sending module configured to send the user service data information to the central OLAP server, so that the central OLAP server performs service summation based on the user identifier and the corresponding traffic volume, and obtains the user identifier corresponding to The traffic volume sum meets the service total information of each user that satisfies the second condition parameter, and the service total amount information includes the user identifier and the corresponding sum of the traffic amounts.
- the at least one user traffic information sent by the deep packet parsing device is received, where each user traffic information includes a user identifier of the user, and the user's ticket Secondary traffic, the attribution of the user, and the generation of the single industry Geographical area identification of traffic;
- the processing module is specifically configured to:
- Determining that the user is a roaming user and determining, according to the data reporting parameter, all user services including the user identifier, if the home location identifier of the user is inconsistent with the geographic area identifier that generates the single service volume.
- the single traffic in the quantity information is counted, and the traffic information of the roaming user is obtained;
- Determining that the user is a non-roaming user and determining, according to the data reporting parameter, all users that include the user identifier, if the home location identifier of the user is consistent with the geographic area identifier that generates the single-time traffic volume.
- the traffic of the non-roaming user in the traffic information is obtained, and the traffic volume of the non-roaming user is obtained according to the traffic volume of all the non-roaming users.
- the data reporting parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, the first The processing module is specifically configured to: if the data reporting parameter includes a service type parameter to be counted, perform statistics on the service quantity of the service type parameter to be counted by the roaming user and the non-roaming user, and obtain the user service Data information, the user service data information includes traffic information of the roaming user, and traffic information of the non-roaming user that satisfies the first condition parameter, where the traffic information includes the user identifier and the corresponding traffic volume;
- the data reporting parameter includes a statistical time period parameter
- statistics are performed on the service quantity of the statistical time period parameter corresponding to the roaming user and the non-roaming user, to obtain user service data information, where the user service data information includes roaming
- the receiving module is specifically configured to receive a central online analysis processing and send the OLAP server a data request message, according to a predetermined mapping relationship between the data request message and the data report parameter, acquiring a data report parameter associated with the data request message;
- an embodiment of the present invention provides an online processing data system, including a central online analytical processing OLAP server and at least one sub-line analytical processing OLAP server, where:
- the central OLAP server is configured to: send a data request message to the at least one child OLAP server; receive user service data information sent by the at least one child OLAP server, where the user service data information includes traffic information of the roaming user and meets the a traffic parameter information of the non-roaming user of the condition parameter, the traffic volume information includes a user identifier and a corresponding traffic volume; and the traffic volume is summed based on the user identifier and the corresponding traffic volume, and the user identifier is obtained.
- the traffic total sum meets the total service information of each user of the second condition parameter, where the total service information includes a sum of the user identifier and the corresponding traffic volume;
- the sub-OLAP server is configured to: receive a data request message sent by the central OLAP server, and obtain a data report parameter corresponding to the data request message; and perform services on the roaming user and the non-roaming user according to the data reporting parameter And performing statistics on the user to obtain user service data information; and transmitting the user service data information to the central OLAP server, where the user service data information includes traffic information of the roaming user and a service of the non-roaming user that satisfies the first condition parameter
- the quantity information includes the user identifier and the corresponding traffic volume.
- the central OLAP server is specifically configured to:
- the sub OLAP server is specifically configured to:
- each user traffic information includes a user identifier of the user, a single traffic volume of the user, a home location identifier of the user, and generating the single traffic volume Geographical area identifier;
- Determining that the user is a roaming user and determining, according to the data reporting parameter, all user services including the user identifier, if the home location identifier of the user is inconsistent with the geographic area identifier that generates the single service volume.
- the quantity of the traffic in the quantity information is collected to obtain the traffic volume information of the roaming user, where the traffic volume information includes the user identifier and the corresponding traffic volume;
- Determining that the user is a non-roaming user and determining, according to the data reporting parameter, all users that include the user identifier, if the home location identifier of the user is consistent with the geographic area identifier that generates the single-time traffic volume.
- the traffic of the non-roaming user is obtained according to the traffic of the non-roaming user, and the traffic information of the non-roaming user that satisfies the first condition parameter is determined according to the traffic volume of all the non-roaming users, where the service
- the quantity information includes a user identifier and a corresponding traffic volume; the user service data information is sent to the central OLAP server to enable the central
- the OLAP server performs the service summation based on the user identifier and the corresponding traffic volume, and obtains the service total information of each user that satisfies the second condition parameter, and the total service information.
- the user identification and the corresponding sum of the traffic are included.
- the sub OLAP server is specifically configured to:
- the data reporting parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, and the first condition parameter;
- the service quantity of the service type parameter that is required to be counted is calculated by the roaming user and the non-roaming user, and the user service data information is obtained, where the user service data information includes roaming.
- User's business volume information and meeting the first condition The traffic information of the non-roaming user of the parameter, where the traffic information includes the user identifier and the corresponding traffic volume; or
- the data reporting parameter includes a statistical time period parameter
- the traffic volume of the roaming user and the non-roaming user corresponding to the statistical time period parameter is collected, and the user service data information is obtained, where the user service data information includes the traffic volume of the roaming user.
- the traffic information of the non-roaming user that satisfies the first condition parameter where the traffic information includes the user identifier and the corresponding traffic volume;
- An embodiment of the present invention provides a method, device, and system for processing data online, where a central OLAP server sends a data request message to at least one sub OLAP server, and the central OLAP server receives user service data information sent by at least one sub OLAP server.
- the user service data information includes traffic information of the roaming user and traffic information of the non-roaming user that satisfies the first condition parameter, that is, the child OLAP server filters the non-roaming user, and the central OLAP server only receives the traffic information of the part of the non-roaming user.
- the central OLAP server sums the traffic volume according to the user, and obtains the traffic total information of each user that satisfies the second condition parameter, and the central OLAP server only needs the traffic volume of the roaming user reported by the sub-OLAP server. And the traffic of the non-roaming users satisfying the first condition parameter is summed, that is, the sub-OLAP server performs preliminary statistics first, and the central OLAP server further performs statistics based on the preliminary statistics of the sub-OLAP server, and the data of the central OLAP server Reducing the amount of data processing efficiency.
- Embodiment 1 is a flowchart of Embodiment 1 of a method for processing data online according to the present invention
- Embodiment 2 is a flowchart of Embodiment 2 of a method for processing data online according to the present invention
- Embodiment 3 is a flowchart of Embodiment 3 of a method for processing data online according to the present invention
- 4 is a flowchart of Embodiment 4 of a method for processing data online according to the present invention.
- FIG. 5 is a schematic structural diagram of Embodiment 1 of a central online analysis processing server according to the present invention
- FIG. 6 is a schematic structural diagram of Embodiment 1 of a sub-line analysis processing server according to the present invention.
- FIG. 7 is a schematic structural diagram of Embodiment 2 of a central online analysis processing server according to the present invention
- FIG. 8 is a schematic structural diagram of Embodiment 2 of a sub-line analysis processing server according to the present invention.
- Embodiment 9 is a schematic structural diagram of Embodiment 1 of an online processing data system according to the present invention.
- FIG. 10 is a schematic diagram of an application of an online processing data system according to the present invention. detailed description
- FIG. 1 is a flowchart of Embodiment 1 of a method for processing data online according to the present invention.
- the execution host of the present implementation is an On-Line Transaction Processing (OLAP) server, and the central online analytical processing server can be implemented by software and/or hardware.
- OLAP On-Line Transaction Processing
- the method in this embodiment may include:
- Step 101 Send a data request message to at least one child OLAP server.
- the query request can be triggered to the central OLAP server.
- the query request can carry the data parameters that the central OLAP server needs to count, that is, Data statistics parameters.
- the data statistics parameter may generally include any one of a service type parameter, a statistical time period parameter, and a first condition parameter to be counted.
- the data statistics parameter may be carried by the query request, and may also be a preset default parameter, and may also determine all parameters according to the mapping relationship between the partial parameters carried in the query request and the preset partial parameters and other parameters.
- the data statistics parameter carried in the query request is a service type parameter to be counted
- the preset part parameter is a statistical time period parameter
- the central OLAP server can map the relationship between the service type parameter and the statistical time period parameter according to the statistics. , determine the statistical time period parameters.
- the central OLAP server After receiving the query request, the central OLAP server sends a data request message to at least one child OLAP server, so that the child OLAP server performs data statistics.
- the statistical user service data information is reported to the central OLAP server.
- the data request message may carry a data report parameter or may not carry a data report parameter.
- the data report parameter may be the same as the data statistics parameter, or may be different from the data statistics parameter.
- the statistics in this embodiment include activities such as collection, collation, calculation, and analysis of related data.
- the child OLAP server can perform data statistics according to the data reporting parameters.
- the sub-OLAP server first obtains the data reporting parameter associated with the data request message, which may be implemented in the following two manners.
- One possible implementation manner is a mapping relationship between the predetermined data request message and the data reporting parameter. And obtaining the data reporting parameter associated with the data request message, that is, the sub-OLAP server, after receiving the data request message, acquiring the data reporting parameter according to the mapping relationship.
- Another possible implementation manner is that if the data request message carries the data report parameter, the carried data report parameter is extracted from the data request message.
- the data request message can also carry a part of the data reporting parameter, and the sub OLAP server determines all the data reporting parameters according to the mapping relationship between the predetermined data request message and the partial data reporting parameter.
- the data reporting parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, and a first condition parameter.
- the service type parameter to be counted may include a traffic service, a video service, a voice service, and the like;
- the statistical time period parameter includes a start point and an end point of the statistical time, for example, from 7:00 pm to 9:00 pm on 2012-12-18, for specific
- the statistic time period parameter is not limited in this embodiment;
- the first condition parameter may be a condition parameter that is specifically satisfied by the traffic volume of the non-roaming user returned by the child OLAP server to the central OLAP server.
- the data reporting message carries the data reporting parameter, and the central OLAP server can uniformly manage the data statistics.
- the sub-OLAP server does not need to store the data reporting parameter, which can reduce the data storage capacity of each sub-OLAP server.
- Step 102 Receive user service data information that is sent by the at least one sub-OLAP server, where the user service data information includes traffic information of the roaming user and traffic information of the non-roaming user that satisfies the first condition parameter, where the service volume
- the information includes the user identification and the corresponding amount of traffic.
- the sub-OLAP server may be distributed in a subnet deployed by the operator in different geographical areas, and the subnet may be in a city, and the user may be a user who uses the mobile terminal.
- User service data information is sent to the central OLAP server.
- the central OLAP receives user service data information sent by at least one sub-OLAP server.
- the user identifier may be various identifiers of the mobile terminal, such as an international mobile subscriber identity code or a mobile phone number, and the traffic volume is a traffic volume corresponding to the subscriber identity.
- the sub-OLAP server is located in Guangzhou.
- the service type parameter to be counted is the traffic service, and the statistical time period parameter is the third quarter.
- the first condition parameter is the condition parameter whose traffic is greater than the first preset threshold.
- the user service data information includes: traffic information of the roaming user, such as a user whose home location is Shenzhen, when the user roams to Guangzhou in the third quarter, as the traffic information of the roaming user using the traffic service in Guangzhou and generating traffic.
- traffic information of the roaming user such as a user whose home location is Shenzhen, when the user roams to Guangzhou in the third quarter, as the traffic information of the roaming user using the traffic service in Guangzhou and generating traffic.
- the user service data information also includes the traffic information of the non-roaming user that satisfies the first condition parameter, that is, the user whose home belongs to Guangzhou, uses the traffic service locally in Guangzhou, and generates traffic information of the traffic.
- the traffic information of the non-roaming user that satisfies the first condition parameter, that is, the user whose home belongs to Guangzhou, uses the traffic service locally in Guangzhou, and generates traffic information of the traffic.
- the traffic information of the non-roaming user that satisfies the first condition parameter, that is, the user whose home belongs to Guangzhou, uses the traffic service locally in Guangzhou, and generates traffic information of the traffic.
- statistics are collected for these users. The obtained statistical results are for the operators to understand the network. The actual operation situation does not contribute much.
- a large amount of data will increase the processing burden of the central OLAP server.
- the sub-OLAP server filters the non-roaming users to obtain non-roaming users that satisfy the first condition parameter, for example, when non-roaming When the traffic of the user is greater than the first preset threshold, the non-roaming user is a roaming user that satisfies the first condition parameter.
- the first condition parameter may also be a condition parameter in which the traffic volume ranks before the first preset rank among the traffic of all non-roaming users. For example, when the first preset rank is 1000, the non-roaming users whose total traffic is in the top 1000 in the third quarter are non-roaming users who satisfy the first condition parameter.
- the traffic statistics received by the central OLAP server are the traffic statistics sent by the multiple sub-OLAP servers, and the traffic statistics sent by each sub-OLAP server are implemented in Guangzhou with the above sub-OLAP servers. For example, this embodiment is not illustrated here.
- Step 103 Perform service summation based on the user identifier and the corresponding traffic volume, and obtain total service information of each user that meets the second condition parameter, and the total service volume.
- the information includes the user identification and the corresponding sum of the traffic.
- the central OLAP server After receiving the user service data information sent by the at least one sub-OLAP server, the central OLAP server sums the traffic volume of each user based on the user identifier, obtains the sum of the traffic of each user, and obtains each of the second condition parameters. The total amount of business information of the user.
- the central OLAP server when the user uses the service at the home location and roams to multiple other geographical areas to use the service, the central OLAP server will receive the corresponding multiple geographic areas to which the user roams.
- the plurality of sub-OLAP servers respectively send the user as the traffic information of the roaming user.
- the central OLAP server will also receive the traffic information of the user as the non-roaming user sent by the sub-OLAP server corresponding to the home location of the user.
- the total amount of traffic of the user as the roaming user and the non-roaming user may be counted based on the user identifier of the user, that is, The sum of all the traffic corresponding to the same user identifier is obtained, and the total amount of services of the user is obtained.
- the sum of the traffic of the user is the traffic of the user as the non-roaming user.
- the second condition parameter may be a preset second condition parameter, and may also be a second condition parameter carried in the query request when the operator triggers the query request.
- the embodiment is not particularly limited herein.
- the second condition parameter is a condition parameter that the sum of the traffic volume is greater than the second preset threshold, or the condition parameter that the sum of the traffic volume is ranked before the second preset ranking in the total traffic volume of all users.
- the central OLAP server further filters the user, and the service type parameter to be counted is the traffic service parameter. For example, when the total traffic of each user is greater than the second preset threshold, or the total traffic of the user is ranked second. Before the ranking, the user is a user who satisfies the second condition parameter, and stores the total amount of service information of the user that meets the second condition parameter, where the service total amount information includes the user identifier and the sum of the traffic volume corresponding to the user identifier. .
- the method for processing data online by the embodiment of the present invention sends a data request message to the at least one sub-OLAP server through the central OLAP server, and the central OLAP server receives the user service data information sent by the at least one sub-OLAP server, where the user service data information includes the roaming user.
- the OLAP server filters the non-roaming users.
- the central OLAP server only receives the traffic information of some non-roaming users.
- the central OLAP server sums the traffic volume according to the user, and obtains the total traffic volume of each user that satisfies the second condition parameter.
- Information the central OLAP server only needs to sum the traffic of the roaming users reported by the sub-OLAP server and the traffic of the non-roaming users satisfying the first condition parameter, that is, the sub-OLAP server first performs preliminary statistics, and the central OLAP server re- Based on the preliminary statistics of the preliminary statistics of the sub-OLAP server, the data processing capacity of the central OLAP server is reduced, and the data processing efficiency is improved.
- the central OLAP server performs summation of traffic based on the user identifier and the corresponding traffic volume, and obtains a sum of traffic corresponding to the user identifier to satisfy the second condition.
- the parameter information of each user's business total information it also includes:
- a multi-dimensional traffic graph is generated according to the total amount of business information of each user that satisfies the second condition parameter.
- the central OLAP server can model the multi-angle thinking mode of the decision-makers according to the multi-angle thinking mode of the decision-makers, and generate a multi-dimensional traffic graph according to the total business information of each user satisfying the second condition parameter, wherein the dimension refers to the analysis angle.
- the statistical time period parameter is a dimension
- the business type parameter and the user ID are also one dimension.
- the online processing data method provided by the present embodiment can quickly obtain data from various analysis angles according to the multi-dimensional traffic chart, and can also dynamically switch between various angles or perform multi-angle comprehensive analysis, which has great analysis flexibility. .
- FIG. 2 is a flowchart of Embodiment 2 of a method for processing data online according to the present invention.
- the execution body of this embodiment is a child OLAP server, and the child OLAP server can be implemented by software and/or hardware.
- the method in this embodiment may include:
- Step 201 Receive a data request message sent by the central OLAP server, and obtain a data report parameter associated with the data request message.
- the child OLAP server receives the data request message sent by the central OLAP server for data statistics.
- the sub-OLAP server needs to obtain the data reporting parameters associated with the data request message before performing data statistics.
- the manner in which the sub-OLAP server obtains the data reporting parameters associated with the data request message includes the following possible implementation manners.
- One possible implementation manner is to obtain a mapping relationship between the data request message and the data reporting parameter according to the predetermined data request message.
- the data reporting parameter that is, the child OLAP server, after receiving the data request message, acquires the number according to the mapping relationship According to the reported parameters.
- Another possible implementation manner is that if the data request message carries the data report parameter, the carried data report parameter is extracted from the data request message.
- Step 202 Perform statistics on the traffic volume of the roaming user and the non-roaming user according to the data reporting parameter, and obtain user service data information, where the user service data information includes the traffic volume information of the roaming user and the non-first parameter The traffic information of the roaming user, where the traffic information includes a user identifier and a corresponding traffic volume.
- the sub-OLAP server performs statistics on the traffic volume of the roaming user and the non-roaming user according to the data reporting parameter, and obtains the traffic information of the roaming user and the traffic information of the non-roaming user that satisfies the first preset condition parameter.
- the data reporting parameter includes any one of the following or a combination thereof: a service type parameter to be counted, a statistical time period ⁇ t, and a first condition parameter.
- the sub-OLAP server can perform statistics according to the specific content of the data reporting parameters. For example, if the data reporting parameter includes the service type parameter to be counted, the step 202 includes: performing statistics on the service quantity of the service type parameter to be counted by the roaming user and the non-roaming user according to the data reporting parameter, and obtaining the user service data information; Or
- the step 202 includes: collecting statistics on the service quantity corresponding to the statistical time period parameter of the roaming user and the non-roaming user according to the data reporting parameter, and obtaining the user service data information.
- the statistical time period parameter and the service type parameter can be combined to count the traffic volume of the specific service type in the statistical time period of the roaming user and the non-roaming user.
- the child OLAP server In order to improve the processing efficiency of the central OLAP server, the child OLAP server must also determine the non-roaming users that satisfy the first condition parameter.
- the first condition parameter is a condition parameter whose traffic is greater than the first preset threshold, or a condition parameter whose traffic is ranked before the first preset ranking among the traffic of all non-roaming users.
- Step 203 Send the user service data information to the central OLAP server, so that the central OLAP server performs service summation based on the user identifier and the corresponding traffic volume, and obtains a service corresponding to the user identifier.
- the total amount of services of each user that satisfies the second condition parameter, and the total amount of service information includes the sum of the user identifiers and the corresponding traffic volume.
- the second condition parameter is that the sum of the traffic is greater than the second preset threshold.
- the condition parameter, or the sum of the traffic is the condition parameter before the second preset rank in the sum of the traffic of all users.
- the method for processing data online by the embodiment of the present invention receives the data request message sent by the central OLAP server through the sub-OLAP server, and performs statistics on the service volume of the roaming user and the non-roaming user, and obtains user service data information, where the user service data information includes The traffic information of the roaming user and the traffic information of the non-roaming user satisfying the first condition parameter, the traffic information includes the user identifier and the traffic volume, and the user service data information is sent to the central OLAP server, that is, the sub-OLAP server performs the non-roaming user.
- the sub-OLAP server sends only the traffic information of the non-roaming users to the central OLAP server, so that the central OLAP server only needs the traffic of the roaming users reported by the sub-OLAP server and the non-roaming users that satisfy the first condition parameter.
- the traffic is summed, that is, the sub-OLAP server performs preliminary statistics first, and the central OLAP server further statistics based on the preliminary statistics of the sub-OLAP server.
- the data processing volume of the central OLAP server is reduced, and the data processing efficiency is improved. .
- FIG. 3 is a flowchart of Embodiment 3 of a method for processing data online according to the present invention. This embodiment will be described in detail in step 202 of the embodiment of Fig. 2 on the basis of the embodiment of Fig. 2.
- step 301 is further included.
- the sub-OLAP server receives at least one piece of user traffic information sent by the deep packet parsing device, where each user traffic information includes a user identifier of the user, a single traffic volume of the user, a home location identifier of the user, and generation. Geographical area ID for a single transaction.
- Deep Packet Inspection (DPI) devices can sense network applications and detect traffic based on the application layer.
- the DPI device may be connected to a gateway device set in each geographical area for a server having a DPI function, and on the other hand, because the access of the mobile terminal of the user in each geographical area is finally aggregated to
- the DPI module can be set in the gateway to make the gateway have DPI capability and act as a DPI device.
- each sub-OLAP server has a gateway connected thereto.
- the DPI device learns that the user uses the service in the area corresponding to the gateway, the DPI device sends at least one user traffic information to the sub-OLAP server, the service.
- the quantity information includes the user identification of the user, a single transaction amount, a attribution identification, and a geographical area identification that generates a single transaction amount.
- the geographic area identifier of the traffic may be the place where the user generates a single traffic when using the service.
- the user's single-time traffic can be used by the user to use the traffic generated by the service once, for example, the traffic generated by the user using the primary traffic service, or the duration of the call generated by the user using the primary call service, etc., for a single time
- the specific manner of the traffic is not limited in this embodiment.
- the attribution identifier and the geographic area identifier that generates a single amount of traffic may be the device number of the corresponding DPI device.
- the sub-OLAP servers located in different geographical areas have corresponding DPI devices.
- the device number of the DPI device corresponding to the sub-OLAP server in Shenzhen is 1001, and the DPI device corresponding to the sub-OLAP server located in Guangzhou.
- the device number is 1002, the device number of the DPI device corresponding to the sub-OLAP server in Zhuhai is 1003, and the device number of the DPI device corresponding to the Dongguan sub-OLAP server is 1004.
- the mapping information in Table 1 can be used as metadata and maintained by each DPI device.
- the attribution identifier is 1001
- the service is used in Guangzhou
- the corresponding traffic is generated (the geographical area identifier 1002 that generates a single transaction amount)
- the DPI device in Guangzhou senses the behavior of the user, and sends user traffic information to the sub-OLAP server connected thereto, the information including the mobile phone number of the user, the single transaction volume, the attribution identifier 1001, and the generation of a single service.
- the sub-OLAP server when receiving the user traffic information sent by the DPI device, can receive the real-time information one by one, or can receive multiple user traffic information at the same time in large quantities, and the specific information received by the sub-OLAP server. The implementation manner is not repeated here.
- the sub-OLAP server after receiving the user traffic information sent by the DPI device, saves the user traffic information.
- a user information table may be newly created, and the user information table is newly created. It is used to store user traffic information, such as the identity of the user, the attribution identifier, the geographic area identifier that generates a single traffic, and the traffic volume.
- the child OLAP server may insert the user traffic information in the user information table, in order to distinguish each user traffic information of the user and determine
- a time point field for generating a single transaction amount may be added to the user information table for recording the time for generating a single transaction amount.
- step 202 specifically includes:
- Step 2021 Determine whether the home location identifier of the user is consistent with the geographic area identifier that generates a single traffic volume. If not, go to step 2022, and if yes, go to step 2023.
- Step 2022 Determine that the user is a roaming user, and perform statistics on the single traffic volume of all the user traffic information including the user identifier according to the data reporting parameter, and obtain the traffic volume information of the roaming user.
- Step 2023 Determine that the user is a non-roaming user. And collecting, according to the data reporting parameter, a single traffic volume of all user traffic information including the user identifier, and obtaining the traffic volume of the non-roaming user; determining, according to the traffic volume of the non-roaming user, the non-roaming user that meets the first condition parameter Traffic information.
- the sub-OLAP server can directly obtain relevant information from the user information table.
- the child OLAP server may determine that the user is a roaming user or a non-roaming user according to the home location identifier in the user information table and the geographic area identifier that generates the single traffic.
- Table 2 lists only some of the fields in the user information table: user ID, attribution, and geographic area identifier that generates a single transaction.
- the sub-OLAP server determines whether the home location identifier is consistent with the geographic area identifier that generates the single traffic volume, that is, when the child OLAP server determines that the same user in the second table generates a single traffic volume, the user's home location identifier and the single service are generated. Whether the geographic area identifiers of the quantity are consistent. For example, for the user 13512345678, the home identity 1001 and the geographic area identifier 1002 that generates a single traffic are inconsistent, then the user is determined to be a roaming user, and step 2022 is performed. For the user 1322345678, the home identity 1002 and the single service are generated. The geographic area identifier 1002 is determined, and the user is determined to be a non-roaming user, and step 2023 is performed.
- the roaming information table and the non-roaming information table may be newly created according to the user information table.
- the roaming information table may only include The user ID of the roaming user, the single traffic volume of the roaming user, and the time when the single traffic volume is generated; correspondingly, the non-roaming information table may also include only the user identity of the non-roaming user, the single traffic volume of the roaming user, and the generation The time of this single transaction. It can be understood by those skilled in the art that there are various ways of constructing the table in this embodiment, and the specific manner of constructing the table in this embodiment is not particularly limited.
- step 2022 it is determined that when the current user is a roaming user, the single traffic volume of all user service information including the user identifier is counted according to the data reporting parameter, and the traffic volume of the roaming user is obtained.
- the statistics may be performed according to the parameters in the data reporting parameters.
- the statistical time period parameter indicates that the statistical time period is the second week of May, and when the service type parameter to be counted is the traffic service parameter, the user is in the All the single traffic belonging to the traffic service generated during this time period is accumulated.
- step 2023 when it is determined that the current user is a non-roaming user, the single-time traffic in all the user traffic information including the user identifier is counted according to the data reporting parameter, and the traffic of the non-roaming user is obtained, according to all non-roaming.
- the user's traffic volume determines the user identifier and the traffic volume of the non-roaming user that meets the first condition parameter. Specifically, the user may perform statistics according to each parameter in the data reporting parameter.
- the traffic belonging to the traffic service of each non-roaming user in the statistical time period from 7:00 to 8:00 is counted, and the non-roaming that satisfies the first condition parameter is determined according to the single traffic volume of the traffic service of each non-roaming user.
- the first condition parameter may be a condition parameter whose traffic volume is greater than the first preset threshold.
- the online processing data method of the present example has a strong pertinence by using the roaming user to determine the traffic volume information and the non-roaming user based on the first condition parameter, thereby avoiding the meaningless system resource waste.
- the performance of online processing of data methods has been greatly improved.
- FIG. 4 is a flowchart of Embodiment 4 of a method for processing data online according to the present invention.
- the two sub-OLAP servers are used as an example, that is, after the operator decision maker triggers the query request, the online processing data method of this embodiment includes the following steps:
- Step 401 The decision maker triggers a query request to the central OLAP server
- Step 402 The central OLAP server sends a data request message to the child OLAP server 1;
- Step 403 The central OLAP server sends a data request message to the child OLAP server 2; those skilled in the art may understand that step 402 and step 403 have no strict timing relationship. Both can be done simultaneously.
- Step 405 The sub-OLAP server 2 obtains the data reporting parameter associated with the data request message, and performs statistics on the traffic volume of the roaming user and the non-roaming user according to the data reporting parameter, to obtain user service data information.
- step 404 and step 405 there is no strict timing relationship between step 404 and step 405, and the two can be performed simultaneously.
- Step 406 The child OLAP server 1 sends the user service data information to the central OLAP server.
- Step 407 the child OLAP server 2 sends the user service data information to the central OLAP server.
- steps 406 and 407 have no strict timing. Relationship, the two can be carried out simultaneously.
- Step 408 The central OLAP server performs traffic summation based on the user identifier, and obtains service total information of each user that satisfies the second condition parameter.
- Step 409 The central OLAP server generates a multi-dimensional traffic graph according to the total service information of each user that satisfies the second condition parameter.
- each sub-OLAP server may refer to the embodiment shown in FIG. 2 and FIG. 3.
- the online processing data method performed by the central OLAP server can be referred to the embodiment shown in FIG.
- each sub-OLAP server includes 1 million users, that is, the total user is 4 million, assuming 15% of the total users. Roaming to the other three areas, 20% of the total users roam to the other two areas, 25% of the total users roam to the other area, and the other 40% of the total users do not roam.
- the service type is a traffic service
- the statistical time period is from January 1, 2012 to January 31, 2012
- the online data processing method of this embodiment is used, and the user trajectory information finally calculated by the central OLAP server is used.
- the time for generating multidimensional traffic charts is 121 seconds.
- the final statistics of the central OLAP server are about 400,000 under the same scenarios and conditions.
- the time for generating the multidimensional traffic chart is 517 seconds, compared with the prior art, the online processing data method provided by this embodiment has a performance improvement of 76.6%.
- FIG. 5 is a schematic structural diagram of Embodiment 1 of a central online analysis processing server according to the present invention.
- the central OLAP server 50 provided in this embodiment includes a sending module 501, a receiving module 502, and a processing module 503.
- the sending module 501 is configured to send a data request message to the at least one sub-line analysis processing OLAP server;
- the receiving module 502 is configured to receive user service data information that is sent by the at least one sub-OLAP server, where the user service data information includes traffic information of the roaming user and traffic information of the non-roaming user that satisfies the first condition parameter,
- the traffic information includes the user identifier and the corresponding traffic volume;
- the processing module 503 is configured to perform traffic summation based on the user identifier and the corresponding traffic volume, and obtain service total information of each user that satisfies the second condition parameter in the total traffic volume corresponding to the user identifier, where the service is The total amount information includes the user identification and the corresponding sum of the traffic amounts.
- the central online analysis processing server of this embodiment may be used to implement the technical solution of the embodiment shown in FIG. 1.
- the implementation principle and technical effects are similar, and details are not described herein again.
- processing module 503 is specifically configured to:
- the total traffic volume information of each user that satisfies the second condition parameter is determined.
- the sending module 501 is specifically configured to: send, to the at least one sub-OLAP server, a data request message that carries a data reporting parameter, where the data reporting parameter includes any one of the following or a combination thereof: a service type parameter to be counted And a statistical time period parameter and the first condition parameter.
- the first condition parameter is a condition parameter that the traffic volume is greater than the first preset threshold, or a condition parameter that the traffic volume is ranked before the first preset ranking in the traffic volume of all non-roaming users;
- the condition parameter is a condition parameter whose sum of traffic is greater than a second preset threshold, or a condition parameter in which the sum of traffic is ranked before the second preset rank in the sum of traffic of all users.
- the processing module 503 is further configured to:
- a multi-dimensional traffic graph is generated based on the total amount of business information of each user that satisfies the second condition parameter.
- each of the sub-OLAP servers is disposed in a different geographical area.
- the central online analysis processing server of this embodiment may be used to implement the technical solution of the method for processing the online processing data.
- the implementation principle and technical effects are similar, and details are not described herein again.
- FIG. 6 is a schematic structural diagram of Embodiment 1 of a sub-line analysis processing server according to the present invention.
- the sub-OLAP server 60 provided in this embodiment includes a receiving module 601, a processing module 602, and a sending module 603.
- the receiving module 601 is configured to receive a data request message sent by the central online analysis processing OLAP server, and obtain a data reporting parameter associated with the data request message.
- the processing module 602 is configured to perform statistics on the service volume of the roaming user and the non-roaming user according to the data reporting parameter, and obtain user service data information, where the user service data information includes the traffic quantity information of the roaming user and the first condition parameter is met. Traffic information of the non-roaming user, the traffic information includes a user identifier and a corresponding traffic volume;
- the sending module 603 is configured to send the user service data information to the central OLAP server, so that the central OLAP server performs service summation based on the user identifier and the corresponding traffic volume, and obtains the user identifier corresponding to The traffic volume sum meets the service total information of each user that satisfies the second condition parameter, and the service total amount information includes the user identifier and the corresponding sum of the traffic amounts.
- the sub-line analysis processing server of this embodiment may be used to implement the technical solution of the embodiment of FIG. 2, and the implementation principle and technical effects thereof are similar, and details are not described herein again.
- the receiving module 601 is further configured to:
- each user traffic information includes a user identifier of the user, a single traffic volume of the user, a home location identifier of the user, and generating the single traffic volume Geographical area identifier;
- the processing module 602 is specifically configured to:
- Determining that the user is a roaming user and determining, according to the data reporting parameter, all user services including the user identifier, if the home location identifier of the user is inconsistent with the geographic area identifier that generates the single service volume.
- the single traffic in the quantity information is counted, and the roaming user is obtained.
- Determining that the user is a non-roaming user and determining, according to the data reporting parameter, all users that include the user identifier, if the home location identifier of the user is consistent with the geographic area identifier that generates the single-time traffic volume.
- the traffic of the non-roaming user in the traffic information is obtained, and the traffic volume of the non-roaming user is obtained according to the traffic volume of all the non-roaming users.
- the data reporting parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, and the first condition parameter;
- the processing module 602 is specifically configured to: if the data reporting parameter includes a service type parameter to be counted, perform statistics on the service quantity of the service type parameter to be counted by the roaming user and the non-roaming user, and obtain user service data information.
- the user service data information includes the traffic volume information of the roaming user and the traffic volume information of the non-roaming user that meets the first condition parameter, where the traffic volume information includes the user identifier and the corresponding traffic volume;
- the data reporting parameter includes a statistical time period parameter
- the first condition parameter is a condition parameter that the traffic volume is greater than the first preset threshold, or a condition parameter that the traffic volume is ranked before the first preset ranking in the traffic volume of all non-roaming users;
- the condition parameter is a condition parameter whose sum of traffic is greater than a second preset threshold, or a condition parameter in which the sum of traffic is ranked before the second preset rank in the sum of traffic of all users.
- the receiving module 601 is specifically configured to receive a data request message sent by the central online analysis processing OLAP server, and acquire data associated with the data request message according to a predetermined mapping relationship between the data request message and the data reporting parameter. Reporting parameters;
- the method is: receiving the data request message sent by the OLAP server by the central online analysis processing, and if the data request message carries the data reporting parameter, extracting the carried data reporting parameter from the data request message.
- FIG. 7 is a schematic structural diagram of Embodiment 2 of a central online analysis processing server according to the present invention.
- the central OLAP server 70 includes: at least one processor 701, such as a CPU, at least one network interface 704 or other user interface 703, a memory 705, and at least one communication bus 702. Communication bus 702 is used to implement connection communication between these components.
- the central OLAP server 70 optionally includes a user interface 703, including a display, keyboard or pointing device (e.g., a mouse, trackball, touchpad or tactile display).
- the memory 705 may include a high speed RAM memory and may also include a non-volatile memory such as at least one disk memory.
- the memory 705 can optionally include at least one storage device located remotely from the aforementioned processor 701.
- memory 705 stores the following elements, executable modules or data structures, or a subset thereof, or their extension set:
- the operating system 7051 which contains various system programs for implementing various basic services and handling hardware-based tasks;
- the application module 7052 includes various applications for implementing various application services.
- the application module 7052 includes but is not limited to the sending module 501, the receiving module 502, and the processing module.
- each module in the application module 7052 refers to the corresponding modules in the embodiment shown in FIG. 5, and details are not described herein.
- the processor 701 is configured to: send a data request message to the at least one sub-line analysis processing OLAP server;
- the user service data information includes traffic information of the roaming user and traffic information of the non-roaming user that satisfies the first condition parameter, where the service volume information includes the user Identification and corresponding business volume;
- the summation of the traffic is performed based on the user identifier and the corresponding traffic volume, and the total traffic information of each user that satisfies the second condition parameter is obtained, including:
- the total traffic volume information of each user that satisfies the second condition parameter is determined.
- the data request message carries a data report parameter
- the data report parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, and the first condition parameter.
- the first condition parameter is a condition parameter that the traffic volume is greater than the first preset threshold, or a condition parameter that the traffic volume is ranked before the first preset ranking in the traffic volume of all non-roaming users;
- the condition parameter is a condition parameter whose sum of traffic is greater than a second preset threshold, or a condition parameter in which the sum of traffic is ranked before the second preset rank in the sum of traffic of all users.
- the method further includes :
- a multi-dimensional traffic graph is generated based on the total amount of business information of each user that satisfies the second condition parameter.
- the central online analysis processing server of this embodiment may be used to implement the technical solution of the method for processing the online processing data.
- the implementation principle and technical effects are similar, and details are not described herein again.
- FIG. 8 is a schematic structural diagram of Embodiment 2 of a sub-line analysis processing server according to the present invention.
- the sub-OLAP server 80 includes: at least one processor 801, such as a CPU, at least one network interface 804 or other user interface 803, a memory 805, and at least one communication bus 802. Communication bus 802 is used to implement connection communication between these components.
- the sub-OLAP server 80 optionally includes a user interface 803 including a display, a keyboard or a pointing device (e.g., a mouse, a trackball, a touchpad, or a touch sensitive display).
- the memory 805 may include a high speed RAM memory and may also include a non-volatile memory such as at least one disk memory.
- the memory 805 can optionally include at least one storage device located remotely from the aforementioned processor 801.
- memory 805 stores the following elements, executable modules or data structures, or a subset thereof, or their extension set:
- Operating system 8051 which contains various system programs for implementing various basic services and processing hardware-based tasks;
- the application module 8052 includes various applications for implementing various application services.
- the application module 8052 includes but is not limited to the receiving module 601, the processing module 602, and the sending module.
- each module in the application module 8052 refers to the corresponding module in the embodiment shown in FIG. 6, which is not mentioned herein.
- the processor 801 is configured to: receive a data request message sent by the central OLAP, and obtain a data report parameter associated with the data request message;
- the user service data information includes the traffic volume information of the roaming user and the non-roaming user that satisfies the first condition parameter.
- Traffic information where the traffic information includes a user identifier and a corresponding traffic volume;
- the method further includes:
- each user traffic information includes a user identifier of the user, a single traffic volume of the user, a home location identifier of the user, and generating the single traffic volume Geographical area identifier;
- Determining that the user is a roaming user and determining, according to the data reporting parameter, all user services including the user identifier, if the home location identifier of the user is inconsistent with the geographic area identifier that generates the single service volume.
- the single traffic in the quantity information is counted, and the traffic information of the roaming user is obtained;
- Determining that the user is a non-roaming user and determining, according to the data reporting parameter, all users that include the user identifier, if the home location identifier of the user is consistent with the geographic area identifier that generates the single-time traffic volume.
- the single-time traffic in the traffic information is collected, and the traffic of the non-roaming user is obtained; and the non-roaming user's traffic is determined according to the non-roaming user's traffic.
- the traffic information of the roaming user is determining, according to the data reporting parameter, all users that include the user identifier, if the home location identifier of the user is consistent with the geographic area identifier that generates the single-time traffic volume.
- the data reporting parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, and the first condition parameter;
- the data reporting parameter includes a service type parameter to be counted
- the statistic is performed on the service quantity of the roaming user and the non-roaming user according to the data reporting parameter, and the user service data information is obtained: the roaming user and the non-roaming
- the user performs statistics according to the service quantity of the service type parameter to be counted, and obtains user service data information;
- the data reporting parameter includes the statistic time period parameter
- the statistic is performed on the service quantity of the roaming user and the non-roaming user according to the data reporting parameter, and the user service data information is obtained by: the roaming user and the non-roaming user are corresponding to:
- the traffic of the statistical time period parameter is counted to obtain user service data information.
- the first condition parameter is a condition parameter that the traffic volume is greater than the first preset threshold, or a condition parameter that the traffic volume is ranked before the first preset ranking in the traffic volume of all non-roaming users;
- the condition parameter is a condition parameter whose sum of traffic is greater than a second preset threshold, or a condition parameter in which the sum of traffic is ranked before the second preset rank in the sum of traffic of all users.
- the obtaining the data reporting parameter associated with the data request message includes: obtaining, according to a predetermined mapping relationship between the data request message and the data reporting parameter, a data reporting parameter associated with the data request message;
- the obtaining the data report parameter associated with the data request message includes:
- Extracting the carried data reporting parameter from the data request message Extracting the carried data reporting parameter from the data request message.
- the sub-line analysis processing server of this embodiment may be used to implement the technical solution of the method for processing the online processing data.
- the implementation principle and technical effects are similar, and details are not described herein again.
- FIG. 9 is a schematic structural diagram of Embodiment 1 of an online processing data system according to the present invention.
- the online processing data system 90 provided by this embodiment includes a central OLAP server and at least one sub OLAP server.
- the online processing system 90 includes a central OLAP server 901, a first sub-OLAP server 902, a second sub-OLAP server 903, a third sub-OLAP server 904, and a fourth sub-OLAP server 905. .
- OLAPs in the system of the embodiment of the present invention is scalable; the OLAP series is also scalable; and the single OLAP load can be uniformly extended.
- a disk array can also be built in or externally, and the disk array is used to store various user information tables.
- the central OLAP server 901 can be a central OLAP server as shown in FIG.
- Each sub-OLAP server may be a structure of a sub-OLAP server as shown in FIG. 6.
- the central OLAP server 901 is configured to send a data request message to the at least one sub OLAP server, and receive user service data information sent by the at least one sub OLAP server, where the user service data information includes a roaming user.
- the sub-OLAP server is configured to: receive a data request message sent by the central OLAP server 901, and obtain a data report parameter corresponding to the data request message; and perform traffic to the roaming user and the non-roaming user according to the data reporting parameter Performing statistics to obtain user service data information; and transmitting the user service data information to the central OLAP server 901, where the user service data information includes traffic quantity information of the roaming user and traffic information of the non-roaming user that satisfies the first condition parameter
- the traffic information includes a user identifier and a corresponding traffic volume.
- central OLAP server 901 is specifically configured to:
- sub OLAP server is specifically configured to:
- each user traffic information includes a user identifier of the user, a single traffic volume of the user, a home location identifier of the user, and generating the single traffic volume Geographical area identifier;
- Determining that the user is a roaming user and determining, according to the data reporting parameter, all user services including the user identifier, if the home location identifier of the user is inconsistent with the geographic area identifier that generates the single service volume.
- the quantity of the traffic in the quantity information is collected to obtain the traffic volume information of the roaming user, where the traffic volume information includes the user identifier and the corresponding traffic volume;
- Determining that the user is a non-roaming user and determining, according to the data reporting parameter, all users that include the user identifier, if the home location identifier of the user is consistent with the geographic area identifier that generates the single-time traffic volume.
- the traffic of the non-roaming user is obtained according to the traffic of the non-roaming user, and the traffic information of the non-roaming user that satisfies the first condition parameter is determined according to the traffic volume of all the non-roaming users, where the service The quantity information includes the user identifier and the corresponding traffic volume; the user service data information is sent to the central OLAP server 901, so that the central OLAP server 901 performs the summation of the traffic based on the user identifier and the corresponding traffic volume, and obtains the The total amount of traffic corresponding to the user identifier satisfies the total service information of each user that satisfies the second condition parameter, and the total service information includes the sum of the user identifier and the corresponding traffic volume.
- sub OLAP server is specifically configured to:
- the data reporting parameter includes any one or a combination of the following: a service type parameter to be counted, a statistical time period parameter, and the first condition parameter;
- the service quantity of the service type parameter that is required to be counted is calculated by the roaming user and the non-roaming user, and the user service data information is obtained, where the user service data information includes roaming.
- the data reporting parameter includes a statistical time period parameter
- the traffic volume of the roaming user and the non-roaming user corresponding to the statistical time period parameter is collected, and the user service data information is obtained, where the user service data information includes the traffic volume of the roaming user.
- the traffic information of the non-roaming user that satisfies the first condition parameter where the traffic information includes the user identifier and the corresponding traffic volume;
- the central OLAP server 901 Sending the user service data information to the central OLAP server 901, so that the central OLAP server 901 performs the summation of the traffic based on the user identifier and the corresponding traffic volume, and obtains the sum of the traffic amounts corresponding to the user identifiers to satisfy the second.
- the central OLAP server 901 can be implemented by the structure shown in FIG. 7, and each sub-OLAP server can be implemented by the structure shown in FIG.
- the online processing data system of this embodiment may be used to implement the foregoing technical solution of the method for processing data on the line.
- the implementation principle and technical effects are similar, and details are not described herein again.
- the online processing data system of this embodiment may be used to implement the foregoing technical solution of the method for processing data on the line.
- the implementation principle and technical effects are similar, and details are not described herein again.
- FIG. 10 is a schematic diagram of an application of an online processing data system according to the present invention.
- the sub-OLAP server can also be connected to the DPI device during the specific application process. As shown in Figure 10, the first child OLAP server
- the second sub-OLAP server 1103, the third sub-OLAP server 1104, and the fourth sub-OLAP server 1105 are respectively connected to the DPI devices numbered 1001, 1002, 1003, and 1004.
- the DPI device learns that the user uses the service in the geographical area corresponding to the gateway, the DPI device sends at least one user traffic information to the child OLAP.
- the DPI device sends the user traffic information of the local user in the Guangzhou area to the second sub-OLAP server.
- the traffic information for the user who is roaming to the Guangzhou area and whose attribution is the Shenzhen area is sent to the second sub-OLAP server 1103.
- Each sub-OLAP server receives at least one piece of user traffic information sent by the DPI device.
- a storage device such as a disk array, may be built in or externally used to store the traffic including the user.
- a variety of user information tables for information may be used.
- each sub-OLAP server When each sub-OLAP server receives the data request message sent by the central OLAP server 1101, each sub-OLAP server obtains the traffic quantity information of the roaming user and the first condition parameter according to the stored service quantity information of the user according to the data request message. Traffic volume letter for non-roaming users Interest. The specific implementation process is shown in FIG. 3, and details are not described herein again.
- Each sub-OLAP server sends user service data information to the central OLAP server 1101, including traffic information of the roaming user and a traffic volume of the non-roaming user satisfying the first condition parameter.
- the central OLAP server 1101 is based on the user identifier and the corresponding traffic volume.
- the traffic summation is performed, and the total amount of traffic corresponding to the user identifier is obtained, and the total service information of each user that satisfies the second condition parameter is included, where the total service information includes a sum of the user identifier and the traffic volume.
- the online processing data system sends a data request message to the at least one sub-OLAP server through the central OLAP server, and the central OLAP server receives the user service data information sent by the at least one sub-OLAP server, where the user service data information includes the service of the roaming user.
- the quantity information and the traffic information of the non-roaming users satisfying the first condition parameter, that is, the sub-OLAP server filters the non-roaming users, and the central OLAP server only receives the traffic information of some non-roaming users; the central OLAP server performs the traffic according to the users.
- the central OLAP server only needs the traffic of the roaming user reported by the sub-OLAP server and the service of the non-roaming user satisfying the first condition parameter.
- the quantity is summed, that is, the sub-OLAP server performs preliminary statistics first, and the central OLAP server further statistics based on the preliminary statistics of the sub-OLAP server, the data processing volume of the central OLAP server is reduced, and the data processing efficiency is improved.
- the aforementioned program can be stored in a computer readable storage medium.
- the program when executed, performs the steps including the foregoing method embodiments; and the foregoing storage medium includes: a medium that can store program codes, such as a ROM, a RAM, a magnetic disk, or an optical disk.
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computing Systems (AREA)
- Environmental & Geological Engineering (AREA)
- Mobile Radio Communication Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Telephonic Communication Services (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims
Priority Applications (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2015524630A JP6100900B2 (ja) | 2013-04-01 | 2013-09-27 | データのオンライン処理のための方法、デバイスおよびシステム |
| KR1020157001746A KR20150023857A (ko) | 2013-04-01 | 2013-09-27 | 데이터의 온라인 처리를 위한 방법, 장치 및 시스템 |
| RU2015105984A RU2612570C2 (ru) | 2013-04-01 | 2013-09-27 | Способ, устройство и система обработки данных в режиме реального времени |
| EP13881196.3A EP2857992A4 (en) | 2013-04-01 | 2013-09-27 | METHOD, DEVICE AND SYSTEM FOR PROCESSING ONLINE DATA |
| US14/868,000 US20160020982A1 (en) | 2013-04-01 | 2015-09-28 | Method, device and system for online processing of data |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310111060.6 | 2013-04-01 | ||
| CN2013101110606A CN103235793A (zh) | 2013-04-01 | 2013-04-01 | 联机处理数据的方法、设备及系统 |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US14/868,000 Continuation US20160020982A1 (en) | 2013-04-01 | 2015-09-28 | Method, device and system for online processing of data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2014161281A1 true WO2014161281A1 (zh) | 2014-10-09 |
Family
ID=48883835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2013/084374 Ceased WO2014161281A1 (zh) | 2013-04-01 | 2013-09-27 | 联机处理数据的方法、设备及系统 |
Country Status (7)
| Country | Link |
|---|---|
| US (1) | US20160020982A1 (zh) |
| EP (1) | EP2857992A4 (zh) |
| JP (1) | JP6100900B2 (zh) |
| KR (1) | KR20150023857A (zh) |
| CN (1) | CN103235793A (zh) |
| RU (1) | RU2612570C2 (zh) |
| WO (1) | WO2014161281A1 (zh) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103235793A (zh) * | 2013-04-01 | 2013-08-07 | 华为技术有限公司 | 联机处理数据的方法、设备及系统 |
| CN104572740B (zh) * | 2013-10-23 | 2019-09-13 | 华为技术有限公司 | 一种存储数据的方法和装置 |
| CN103995879B (zh) * | 2014-05-27 | 2017-12-15 | 华为技术有限公司 | 基于olap系统的数据查询方法、装置及系统 |
| CN105138691B (zh) * | 2015-09-18 | 2018-10-02 | 北京百度网讯科技有限公司 | 分析用户业务量的方法和系统 |
| CN105553690A (zh) * | 2015-12-07 | 2016-05-04 | 北京奇虎科技有限公司 | 业务访问信息的统计方法、装置及系统 |
| CN105677698B (zh) * | 2015-12-11 | 2019-03-05 | 重庆川仪自动化股份有限公司 | 一种报表数据导出方法 |
| CN113285816B (zh) | 2020-02-19 | 2022-10-28 | 华为技术有限公司 | 一种基于键值配置的控制请求发送方法、装置和系统 |
| CN112579287B (zh) * | 2020-12-16 | 2024-07-30 | 跬云(上海)信息科技有限公司 | 一种基于读写分离及自动伸缩的云编排系统及方法 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1688998A (zh) * | 2002-10-18 | 2005-10-26 | 国际商业机器公司 | 用于联机分析处理的方法和系统 |
| US20070022120A1 (en) * | 2005-07-25 | 2007-01-25 | Microsoft Corporation | Caching and modifying portions of a multi-dimensional database on a user device |
| CN102938097A (zh) * | 2012-09-28 | 2013-02-20 | 用友软件股份有限公司 | 用于联机分析处理系统的数据处理装置和数据处理方法 |
| CN103235793A (zh) * | 2013-04-01 | 2013-08-07 | 华为技术有限公司 | 联机处理数据的方法、设备及系统 |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6385604B1 (en) * | 1999-08-04 | 2002-05-07 | Hyperroll, Israel Limited | Relational database management system having integrated non-relational multi-dimensional data store of aggregated data elements |
| US7007020B1 (en) * | 2000-03-10 | 2006-02-28 | Hewlett-Packard Development Company, L.P. | Distributed OLAP-based association rule generation method and system |
| JP2002108899A (ja) * | 2000-09-27 | 2002-04-12 | Hitachi Kokusai Electric Inc | データベースアクセス方法及び多次元データベースアクセスシステム |
| US7506046B2 (en) * | 2001-07-31 | 2009-03-17 | Hewlett-Packard Development Company, L.P. | Network usage analysis system and method for updating statistical models |
| US7627555B2 (en) * | 2004-06-22 | 2009-12-01 | Microsoft Corporation | Combining multidimensional expressions and data mining extensions to mine OLAP cubes |
| CN1588358A (zh) * | 2004-08-26 | 2005-03-02 | 陈红 | 对mdx多维数据查询语句的处理方法和系统 |
-
2013
- 2013-04-01 CN CN2013101110606A patent/CN103235793A/zh active Pending
- 2013-09-27 JP JP2015524630A patent/JP6100900B2/ja not_active Expired - Fee Related
- 2013-09-27 RU RU2015105984A patent/RU2612570C2/ru not_active IP Right Cessation
- 2013-09-27 KR KR1020157001746A patent/KR20150023857A/ko not_active Ceased
- 2013-09-27 WO PCT/CN2013/084374 patent/WO2014161281A1/zh not_active Ceased
- 2013-09-27 EP EP13881196.3A patent/EP2857992A4/en not_active Withdrawn
-
2015
- 2015-09-28 US US14/868,000 patent/US20160020982A1/en not_active Abandoned
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1688998A (zh) * | 2002-10-18 | 2005-10-26 | 国际商业机器公司 | 用于联机分析处理的方法和系统 |
| US20070022120A1 (en) * | 2005-07-25 | 2007-01-25 | Microsoft Corporation | Caching and modifying portions of a multi-dimensional database on a user device |
| CN102938097A (zh) * | 2012-09-28 | 2013-02-20 | 用友软件股份有限公司 | 用于联机分析处理系统的数据处理装置和数据处理方法 |
| CN103235793A (zh) * | 2013-04-01 | 2013-08-07 | 华为技术有限公司 | 联机处理数据的方法、设备及系统 |
Non-Patent Citations (1)
| Title |
|---|
| See also references of EP2857992A4 * |
Also Published As
| Publication number | Publication date |
|---|---|
| US20160020982A1 (en) | 2016-01-21 |
| EP2857992A1 (en) | 2015-04-08 |
| RU2612570C2 (ru) | 2017-03-09 |
| KR20150023857A (ko) | 2015-03-05 |
| JP2015527657A (ja) | 2015-09-17 |
| RU2015105984A (ru) | 2016-09-10 |
| JP6100900B2 (ja) | 2017-03-22 |
| EP2857992A4 (en) | 2015-07-15 |
| CN103235793A (zh) | 2013-08-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2014161281A1 (zh) | 联机处理数据的方法、设备及系统 | |
| CN107547633B (zh) | 一种用户常驻点的处理方法、装置和存储介质 | |
| CN111263409B (zh) | 提供网络功能服务的元数据信息的方法、系统以及相关设备 | |
| CN103605791B (zh) | 信息推送系统和信息推送方法 | |
| CN102577494B (zh) | 支持通信网络中的社交网络分析的方法和装置 | |
| CN104598557B (zh) | 数据栅格化、用户行为分析的方法和装置 | |
| CN104424229A (zh) | 一种多维度拆分的计算方法及系统 | |
| CN103902636B (zh) | 基于过滤聚类法推送信息的方法和服务器 | |
| WO2017067505A1 (zh) | 网络扩容方法及装置 | |
| WO2012106861A1 (zh) | 终端分布信息获取方法、数据获取装置以及通信系统 | |
| WO2019137195A1 (zh) | 一种获取业务数据的方法及装置 | |
| CN101951623B (zh) | 一种基于用户事件的用户行为统计方法及装置 | |
| CN105049298A (zh) | 一种云资源监控方法和系统 | |
| CN105550248A (zh) | 用户信息挖掘方法及装置 | |
| CN110727752B (zh) | 位置指纹库处理方法、设备及计算机可读存储介质 | |
| CN104486769B (zh) | 一种价值小区的选取方法及装置 | |
| WO2023045434A1 (zh) | 访问检测方法、系统及装置 | |
| CN102905236A (zh) | 一种垃圾短消息监控方法、装置及系统 | |
| CN106027692B (zh) | 信息获取方法、装置及服务器 | |
| CN116723339B (zh) | 内容数据的分发方法和装置、存储介质及电子设备 | |
| JP7823624B2 (ja) | ネットワークノード、情報処理システム、情報処理方法、およびコンピュータプログラム | |
| CN107481154A (zh) | 社交网络人际关系的分析方法及装置 | |
| CN116016651B (zh) | 一种经纬度校正方法、装置、电子设备及存储介质 | |
| CN111510940A (zh) | 信令分析方法及设备 | |
| CN117768183A (zh) | 一种识别移动网络资产的方法、装置、设备及存储介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13881196 Country of ref document: EP Kind code of ref document: A1 |
|
| REEP | Request for entry into the european phase |
Ref document number: 2013881196 Country of ref document: EP |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2013881196 Country of ref document: EP |
|
| ENP | Entry into the national phase |
Ref document number: 20157001746 Country of ref document: KR Kind code of ref document: A |
|
| ENP | Entry into the national phase |
Ref document number: 2015524630 Country of ref document: JP Kind code of ref document: A |
|
| ENP | Entry into the national phase |
Ref document number: 2015105984 Country of ref document: RU Kind code of ref document: A |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |

