WO2022061807A1 - 一种数据的存储方法及设备 - Google Patents

一种数据的存储方法及设备 Download PDF

Info

Publication number
WO2022061807A1
WO2022061807A1 PCT/CN2020/118086 CN2020118086W WO2022061807A1 WO 2022061807 A1 WO2022061807 A1 WO 2022061807A1 CN 2020118086 W CN2020118086 W CN 2020118086W WO 2022061807 A1 WO2022061807 A1 WO 2022061807A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
node device
node
storage module
request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/118086
Other languages
English (en)
French (fr)
Inventor
李小杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202080004307.2A priority Critical patent/CN112513832B/zh
Priority to PCT/CN2020/118086 priority patent/WO2022061807A1/zh
Priority to EP20954658.9A priority patent/EP4213036A4/en
Publication of WO2022061807A1 publication Critical patent/WO2022061807A1/zh
Priority to US18/190,323 priority patent/US12229025B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/16Error detection or correction of the data by redundancy in hardware
    • G06F11/1658Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/275Synchronous replication
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/0766Error or fault reporting or storing
    • G06F11/0772Means for error signaling, e.g. using interrupts, exception flags, dedicated error registers
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/273Asynchronous replication or reconciliation

Definitions

  • the present application relates to the technical field of data storage, and in particular, to a data storage method and device.
  • Figure 1 takes the deployment of three node devices (also referred to as nodes) as an example to illustrate, as the reader of each application (application, APP) of the client /
  • the request for writing data is processed by the process S2 running on the master node (leader), and then the process S2 sends a synchronization command request (synchronization, SYNC) to the processes of other nodes in the cluster (such as process S1 and process S3) , synchronize the data of the storage modules on each node (implemented by the database engine on the node, also referred to as the engine);
  • Figure 2 still takes the deployment of three nodes as an example to illustrate, as a customer Each APP on the terminal accesses the node nearby.
  • the above deployment methods all rely on distributed consensus algorithms (such as paxos algorithm and its evolution algorithm, raft algorithm, etc.), and these algorithms all have a key constraint: all election processes (such as the process of electing a leader), data writing Operations, etc. require that more than half of the processes in all nodes in the cluster agree to be considered as reaching a consensus. Due to the rigid requirements of more than half, the deployment method of more than 3 odd nodes is generally adopted.
  • the cost of automotive components is strictly controlled and the data reliability is high.
  • the above-mentioned three-node deployment method has a lot of redundant data backup and large resource consumption, resulting in high hardware costs, making it impossible to meet the data reliability requirements at the same time. High performance and low cost requirements, especially many mid-to-low-end models cannot afford the increase in these hardware costs.
  • the embodiments of the present application provide a method and device for storing data, which can be used for deploying two node devices on a wheeled mobile device (eg, an autonomous vehicle), through strong consistency (that is, at any time under the premise of deploying two node devices) , the data in the first node device and the second node device in the system are the same) the data synchronization process realizes the real-time backup of the data, which ensures the reliability of the data.
  • a wheeled mobile device eg, an autonomous vehicle
  • strong consistency that is, at any time under the premise of deploying two node devices
  • the data in the first node device and the second node device in the system are the same
  • the data synchronization process realizes the real-time backup of the data, which ensures the reliability of the data.
  • at least 3 nodes are deployed
  • the method of deploying two node devices in the present application reduces the hardware cost of the wheeled mobile device, that is, high-reliability data storage is realized in the hardware
  • the embodiments of the present application provide the following technical solutions, wherein, in the following embodiments of the present application, the first node device is referred to as the first node for short, and the second node device is referred to as the second node for short.
  • an embodiment of the present application first provides a method for storing data.
  • the method includes: first, a first node, as a master node, receives a first request sent by a first APP through a first process running on the first node , the first APP can be an APP corresponding to the first process, wherein the first process refers to a process used for processing data storage related, and the first APP is used to obtain the perception information of the surrounding environment of the wheeled mobile device (such as , the perception information can be obtained through sensors such as cameras and radars deployed on the wheeled mobile device), and based on the perception information, an operation instruction (ie a first request) for the data is generated, and the first request is sent to the first request via the first APP.
  • the first request is used to instruct to write target data (which may be referred to as first data) to a storage module (eg, hard disk, memory, etc.).
  • the first node obtains the first request sent by the first APP, and in response to the first request, the first process can write the first request to the first storage module on the first node based on the first database engine on the first process. data.
  • the first request is used to instruct the storage module to write the first data, and does not limit which storage module to write the first data to.
  • the first node calls the first database engine through the first process
  • the first data is written to the first storage module on the first node according to the first request.
  • the second node calls the second database engine through the second process
  • the first data is written to the second storage module on the second node according to the first request.
  • the first node will send the first request to the second node through the first process, and the purpose of sending the first request to the second node will be The method is to make the second node also write the first data to the second storage module on the second node according to the second request.
  • the second node After receiving the first request sent by the first node through the first process, the second node will write the request to the second storage module on the second node by calling the second database engine on the second process (running on the second node). the first data, so that the updated data of the second storage module on the second node and the first storage module can be kept completely consistent.
  • the second process also refers to a process for processing data storage.
  • the second node When the operation of writing the first data to the second storage module on the second node by calling the second database engine on the second process is completed, the second node will send the first response message to the first node through the second process , the first response message is used to indicate that the operation of writing the first data to the second storage module has been completed.
  • the first node When the first node receives the first response message sent by the second node through the second process within the preset time period through the first process, the first node can determine that the second node has also completed the writing operation of the first data, Then the first node will send a second response message to the first application through the first process, and the second response message is used to instruct the operation of writing the first data to the first storage module and writing the first data to the second storage module. The manipulation of the data has been completed.
  • the method can be applied to a wheeled mobile device, which can then include the first node and the second node described above.
  • the wheeled mobile device eg, self-driving vehicle, smart car, connected car, etc.
  • the first node (as the master node) sends the first node
  • the above-mentioned strong consistency that is, at any time, the data in the first node device and the second node device in the system
  • the data synchronization process is the same) to realize the real-time backup of data, that is, the reliability of the data is guaranteed.
  • the method of deploying 2 node devices in the present application reduces the The hardware cost of the wheeled mobile device is reduced, that is, high-reliability data storage is realized in the hardware architecture of the low-cost wheeled mobile device.
  • the second node will immediately replace the first node to continue to provide services, and cache data, and the cached data is used by the second node to replace the first node as the The updated data in the second storage module after the main node, and after the first node resumes normal use, the second process sends the cached data to the first node, so that the first node calls the first database engine to cache the data.
  • the data of the node is updated to its own first storage module, thereby realizing data synchronization between the two nodes.
  • the second node if the first node fails (the first node acts as the master node, and the second node acts as the standby node), the second node is immediately promoted to the master node and replaces the first node to perform services, thereby ensuring data storage. Availability of the service system, no data loss.
  • the second node to replace the first node to continue to provide services may specifically include the following replacement methods: a.
  • the second node replaces the first node to provide services until the first node returns to normal use , that is to say, when the first node resumes normal use, the second node will no longer be replaced, and the second node will still work as a standby node.
  • the first node is a high configuration
  • the second node is a low configuration, that is The hardware configurations of the first node and the second node are different.
  • the main database engine runs on the first node with high configuration.
  • the second node with low configuration can replace the first node to provide services in a short period of time. After the first node resumes normal use, the first node will continue to provide services as the master node, and this method of setting high and low configuration nodes is also to save costs.
  • the second node always replaces the first node to provide services, that is, after the second node replaces the first node to provide services as the master node, regardless of whether the first node resumes normal use, the second node will always serve as the master node. Service, if the first node resumes normal use, it will always be used as a standby node. Generally, the way of always replacing is when there is no difference in the hardware configuration of the first node and the second node.
  • the judgment of the failure of the first node may specifically include the following methods: a.
  • the heartbeat sent by the first node to the second node is abnormal.
  • the frequency that a node sends a heartbeat is normally once every 300 milliseconds, because the purpose of the first node sending a heartbeat to the second node is to inform the peer that it is running normally, but within a preset period of time (for example, within 1 second) the second node If the heartbeat sent by the first node is not received, the second node considers the first node to be faulty.
  • b. The first node actively sends a fault notification message to the second node.
  • the first node when the software on the first process running on the first node exits normally or abnormally, the first node will actively send a fault notification message to the second node; For another example, when the third process that monitors the first process on the first node monitors that the first process is abnormal, the first node will also actively send a failure notification message to the second node; for example, if the communication of the first node is abnormal, Then, the first node will also actively send a failure notification message to the second node, which will not be illustrated by an example this time.
  • the above two fault sensing methods are both to enable the second node to quickly perceive that the first node has a fault, and to shorten the fault detection delay.
  • the second fault sensing method compared with the first fault sensing method, enables the second node to sense that the first node has failed more quickly, without having to wait for a preset period of time when the second node does not receive the first node. The heartbeat of the first node determines that the first node is faulty, thereby further shortening the fault detection delay.
  • the first node may set the peer to an abnormal state, while the first node continues to provide services as a master node and cache data (ie, the second data ), the cached data is the data obtained after the first node writes the first data to the first storage module through the first process, and after the second node resumes normal use, the cached data is stored by the first process Send the data to the second node, so that the second node updates the cached data to its own second storage module by invoking the second database engine, thereby realizing data synchronization between the two nodes.
  • the cached data is the data obtained after the first node writes the first data to the first storage module through the first process
  • the cached data is stored by the first process Send the data to the second node, so that the second node updates the cached data to its own second storage module by invoking the second database engine, thereby realizing data synchronization between the two nodes.
  • the first node will continue to provide services as the master node, and after the second node resumes normal use, the cached second data will be stored in the first process through the first process.
  • Send the data to the second node so that the second node can update the cached second data to its own second storage module by invoking the second database engine, so as to realize data synchronization between the two nodes, which is achievable.
  • the judgment of the failure of the second node may include the following methods: a.
  • the heartbeat sent by the second node to the first node is abnormal.
  • the frequency that a node sends a heartbeat is normally once every 300 milliseconds, because the purpose of the second node sending a heartbeat to the first node is to inform the peer that it is running normally, but within a preset time period (for example, within 1 second) the first node If the heartbeat sent by the second node is not received, the first node considers the second node to be faulty.
  • the second node actively sends a fault notification message to the first node.
  • the second node when the software on the second process running on the second node exits normally or abnormally, the second node will actively send a fault notification message to the first node; For another example, when the fourth process monitoring the second process on the second node monitors that the second process is abnormal, the second node will also actively send a failure notification message to the first node; for example, the communication of the second node is abnormal, Then, the second node will also actively send a failure notification message to the first node, which will not be illustrated by an example this time.
  • the above two fault sensing methods are both to enable the first node to quickly perceive that the second node has a fault, thereby shortening the fault detection delay.
  • the second fault sensing method can make the first node sense that the second node has failed more quickly, without waiting for the preset time period when the first node does not receive the second node.
  • the heartbeat of the second node is determined to be faulty, thereby further shortening the fault detection delay.
  • the first storage module includes a first in-memory database; and/or the second storage module includes a second in-memory database.
  • the first storage module and the second storage module may be in-memory databases.
  • the reading speed of in-memory databases is faster, especially in the field of intelligent driving, due to the operation of the vehicle.
  • Data such as status data, perception data, intermediate calculation results, and SOA information need to be read frequently. If the first storage module and the second storage module are hard disks, the repeated and frequent reading/writing of data will shorten the life of the hard disks, which will lead to automotive use. The replacement cost of the device increases.
  • the first node may further pass the first process.
  • a process records (eg, updates) the current data version number, which may be referred to as the first data version number, which is used to represent the updated data version of the first storage module data (which may be referred to as the first data version) ), the updated data version is the data version under the condition that the operation of writing the first data to the first storage module has been completed. For example, assuming that the data version number before this data write operation is P, then after the current write operation to the first data is completed, the data in the current first storage module is the latest version, and the recordable data version number is P+1 .
  • the version number of the updated data is also recorded, so that it is easy to find out which version of the updated data is and which data has been updated. Wait.
  • the second node may further use the second process to write the first data to the second storage module on the second node.
  • the second process records (eg, updates) the current data version number, which may be referred to as the second data version number, which is used to indicate the updated data version of the second storage module data (which may be referred to as the second data version) ), the updated data version is the data version under the condition that the operation of writing the first data to the second storage module has been completed.
  • the data version number before the data write operation is Q
  • the data in the current second storage module is the latest version
  • the recordable data version number is Q+1 , so that it is easy to find which version of the data updated each time, which data has been updated, etc.
  • the first storage module on the first node and the second storage module on the second node are updated synchronously, so the values of P and Q are substantially the same.
  • the version number of the updated data is also recorded, so that it is convenient to find out which version of the updated data is and which data has been updated. Wait.
  • the wheeled mobile device may be an autonomous vehicle, which may be a car, truck, motorcycle, bus, boat, airplane, helicopter, lawn mower, recreational vehicle, playground Vehicles, construction equipment, trams, golf carts, trains, and trolleys, etc., are not particularly limited in the embodiments of the present application.
  • the wheeled mobile device may be an autonomous driving vehicle, which is feasible.
  • a second aspect of the embodiments of the present application further provides a data storage method, which is applied to a wheeled mobile device, where the wheeled mobile device includes a first node and a second node, and the method includes: first, the second node is used as a standby node, A second request sent by a second APP corresponding to the second process is received by a second process running on the second node, where the second process refers to a process used to process data storage, and the second APP is used for Obtain the perception information of the surrounding environment of the wheeled mobile device (for example, the perception information can be obtained through sensors such as cameras and radars deployed on the wheeled mobile device), and generate operation instructions for the data (ie, the second request) based on the perception information, The second request is sent to the second process running on the second node corresponding to the second APP via the second APP.
  • the perception information can be obtained through sensors such as cameras and radars deployed on the wheeled mobile device
  • the second request is sent to the second process
  • the second request is used to instruct to write target data (which may be referred to as third data) to a storage module (eg, hard disk, memory, etc.).
  • a storage module eg, hard disk, memory, etc.
  • the second node since the second node is used as a standby node, the second node needs to forward the second request to the first node serving as the master node after receiving the second request sent by the corresponding second APP. deal with.
  • the first node receives, through the first process running on the first node, the second request forwarded by the second process running on the second node.
  • the first node obtains the second request sent by the first APP, and according to the second request, the first process can call the first database engine on the first process to write the third data to the first storage module on the first node .
  • the second request is used to instruct the storage module to write the third data, and does not limit which storage module the third data is to be written to. For example, if the first node calls the first database engine through the first process, Then, the third data is written to the first storage module on the first node according to the second request. For another example, if the second node calls the second database engine through the second process, the third data is written to the second storage module on the second node according to the second request.
  • the first node When the operation of writing the third data to the first storage module is completed, the first node will send a third response message to the second node through the first process, and the third response message is used to instruct the second node according to the For the second request, the second process calls the second database engine on the second process to write the third data to the second storage module on the second node. After receiving the second request sent by the first node through the first process, the second node will perform the write operation of the third data on the second storage module on the second node by calling the second database engine on the second process, In this way, the updated data of the second storage module on the second node and the first storage module can be kept completely consistent.
  • the second process also refers to a process for processing data storage.
  • the second node When the operation of writing the third data to the second storage module on the second node by calling the second database engine on the second process is completed, the second node will send a fourth response message to the first node through the second process , the fourth response message is used to indicate that the operation of writing the third data to the second storage module has been completed.
  • the first node receives, through the first process, the fourth response message sent by the second node through the second process within a preset time period, the first node sends a fifth response message to the second node through the first process, and the fifth response message is sent to the second node through the first process.
  • the response message is used to indicate that both the operation of writing the third data to the first storage module and the operation of writing the third data to the second storage module have been completed.
  • the second node receives the fifth response message sent by the first node through the first process through the second process, and will forward the fifth response message to the second application, so that the second application can perceive the first Both the node and the second node have completed the write operation to the third data.
  • the above-mentioned strong consistency that is, at any time, the data in the first node device and the second node device in the system are both The same
  • data synchronization process realizes real-time backup of data, that is, the reliability of data is guaranteed, and at the same time, compared with the way of deploying at least 3 nodes in the prior art, the way of deploying 2 nodes in the present application reduces the number of wheels
  • the hardware cost of mobile devices that is, the realization of high-reliability data storage in the hardware architecture of low-cost wheeled mobile devices.
  • the second node will immediately replace the first node to continue to provide services, and cache data, and the cached data is used by the second node to replace the first node as the The updated data in the second storage module after the main node, and after the first node resumes normal use, the second process sends the cached data to the first node, so that the first node calls the first database engine to cache the data.
  • the data of the node is updated to its own first storage module, thereby realizing data synchronization between the two nodes.
  • the second node if the first node fails (the first node acts as the master node, and the second node acts as the standby node), the second node is immediately promoted to the master node and replaces the first node to perform services, thereby ensuring data storage. Availability of the service system, no data loss.
  • the second node to replace the first node to continue to provide services may specifically include the following replacement methods: a.
  • the second node replaces the first node to provide services until the first node returns to normal use , that is to say, when the first node resumes normal use, the second node will no longer be replaced, and the second node will still work as a standby node.
  • the first node is a high configuration
  • the second node is a low configuration, that is The hardware configurations of the first node and the second node are different.
  • the main database engine runs on the first node with high configuration.
  • the second node with low configuration can replace the first node to provide services in a short period of time. After the first node resumes normal use, the first node will continue to provide services as the master node, and this method of setting high and low configuration nodes is also to save costs.
  • the second node always replaces the first node to provide services, that is, after the second node replaces the first node to provide services as the master node, regardless of whether the first node resumes normal use, the second node will always serve as the master node. Service, if the first node resumes normal use, it will always be used as a standby node. Generally, the way of always replacing is when there is no difference in the hardware configuration of the first node and the second node.
  • the judgment of the failure of the first node may specifically include the following methods: a.
  • the heartbeat sent by the first node to the second node is abnormal.
  • the frequency that a node sends a heartbeat is normally once every 300 milliseconds, because the purpose of the first node sending a heartbeat to the second node is to inform the peer that it is running normally, but within a preset period of time (for example, within 1 second) the second node If the heartbeat sent by the first node is not received, the second node considers the first node to be faulty.
  • b. The first node actively sends a fault notification message to the second node.
  • the first node when the software on the first process running on the first node exits normally or abnormally, the first node will actively send a fault notification message to the second node; For another example, when the third process that monitors the first process on the first node monitors that the first process is abnormal, the first node will also actively send a failure notification message to the second node; for example, if the communication of the first node is abnormal, Then, the first node will also actively send a failure notification message to the second node, which will not be illustrated by an example this time.
  • the above two fault sensing methods are both to enable the second node to quickly perceive that the first node has a fault, and to shorten the fault detection delay.
  • the second fault sensing method compared with the first fault sensing method, enables the second node to sense that the first node has failed more quickly, without having to wait for a preset period of time when the second node does not receive the first node. The heartbeat of the first node determines that the first node is faulty, thereby further shortening the fault detection delay.
  • the first node may set the peer to an abnormal state, while the first node continues to provide services as a master node and cache data (ie, the second data ), the cached data is the data obtained after the first node writes the first data to the first storage module through the first process, and after the second node resumes normal use, the cached data is stored by the first process Send the data to the second node, so that the second node updates the cached data to its own second storage module by invoking the second database engine, thereby realizing data synchronization between the two nodes.
  • the cached data is the data obtained after the first node writes the first data to the first storage module through the first process
  • the cached data is stored by the first process Send the data to the second node, so that the second node updates the cached data to its own second storage module by invoking the second database engine, thereby realizing data synchronization between the two nodes.
  • the first node will continue to provide services as the master node, and after the second node resumes normal use, the cached second data will be stored in the first process through the first process.
  • Send the data to the second node so that the second node can update the cached second data to its own second storage module by invoking the second database engine, so as to realize data synchronization between the two nodes, which is achievable.
  • the judgment of the failure of the second node may include the following methods: a.
  • the heartbeat sent by the second node to the first node is abnormal.
  • the frequency that a node sends a heartbeat is normally once every 300 milliseconds, because the purpose of the second node sending a heartbeat to the first node is to inform the peer that it is running normally, but within a preset time period (for example, within 1 second) the first node If the heartbeat sent by the second node is not received, the first node considers the second node to be faulty.
  • the second node actively sends a fault notification message to the first node.
  • the second node when the software on the second process running on the second node exits normally or abnormally, the second node will actively send a fault notification message to the first node; For another example, when the fourth process monitoring the second process on the second node monitors that the second process is abnormal, the second node will also actively send a failure notification message to the first node; for example, the communication of the second node is abnormal, Then, the second node will also actively send a failure notification message to the first node, which will not be illustrated by an example this time.
  • the above two fault sensing methods are both to enable the first node to quickly perceive that the second node has a fault, thereby shortening the fault detection delay.
  • the second fault sensing method can make the first node sense that the second node has failed more quickly, without waiting for the preset time period when the first node does not receive the second node.
  • the heartbeat of the second node is determined to be faulty, thereby further shortening the fault detection delay.
  • the first storage module includes a first in-memory database; and/or the second storage module includes a second in-memory database.
  • the first storage module and the second storage module may be in-memory databases.
  • the reading speed of in-memory databases is faster, especially in the field of intelligent driving, due to the operation of the vehicle.
  • Data such as status data, perception data, intermediate calculation results, and SOA information need to be read frequently. If the first storage module and the second storage module are hard disks, the repeated and frequent reading/writing of data will shorten the life of the hard disks, which will lead to automotive use. The replacement cost of the device increases.
  • the first node may further use the first The process records (eg, updates) the current data version number, which may be referred to as the first data version number, which is used to represent the updated data version of the first storage module data (which may be referred to as the first data version)
  • the updated data version is the data version under the condition that the operation of writing the third data to the first storage module has been completed. For example, assuming that the data version number before this data write operation is X, then after the current third data write operation is completed, the data in the current first storage module is the latest version, and the recordable data version number is X+1 .
  • the version number of the updated data is also recorded, so that it is easy to find out which version of the updated data is and which data has been updated. Wait.
  • the second node may further use the second process to write the third data to the second storage module.
  • the process records (eg, updates) the current data version number, which may be referred to as the second data version number, which is used to represent the updated data version of the second storage module data (which may be referred to as the second data version)
  • the updated data version is the data version under the condition that the operation of writing the third data to the second storage module has been completed.
  • the data in the current second storage module is the latest version, and the recordable data version number is Y+1 , so that it is easy to find which version of the data updated each time, which data has been updated, etc. It should be noted here that because each time a node writes data, the first storage module on the first node and the second storage module on the second node are updated synchronously, so the values of X and Y are substantially the same.
  • the version number of the updated data is also recorded, so that it is easy to find out which version of the data is updated each time, and which data has been updated. Wait.
  • the wheeled mobile device may be an autonomous vehicle, which may be a car, truck, motorcycle, bus, boat, airplane, helicopter, lawn mower, recreational vehicle, playground Vehicles, construction equipment, trams, golf carts, trains, and trolleys, etc., are not particularly limited in the embodiments of the present application.
  • the wheeled mobile device may be an autonomous driving vehicle, which is feasible.
  • a third aspect of the embodiments of the present application provides a data storage service system.
  • the system is deployed on a wheeled mobile device.
  • the system includes a first node and a second node, and the first node and the second node have functions to implement the first aspect. Or the function of the method in any possible implementation manner of the first aspect.
  • a fourth aspect of the embodiments of the present application further provides a data storage service system.
  • the system is deployed on a wheeled mobile device.
  • the system includes a first node and a second node, and the first node and the second node have functions to implement the second A function of the method of any possible implementation of the aspect or the second aspect.
  • a fifth aspect of the embodiments of the present application further provides a node, the node is applied to a data storage service system, the system is deployed on a wheeled mobile device, the system includes a first node and a second node, and the node serves as the first node A node, having the function of the first node in the method for implementing the first aspect or any possible implementation manner of the first aspect.
  • This function can be implemented by hardware or by executing corresponding software by hardware.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • a sixth aspect of the embodiments of the present application further provides a node, the node is applied to a data storage service system, the system is deployed on a wheeled mobile device, the system includes a first node and a second node, and the node serves as the first node
  • the node has the function of the first node in the method for implementing the second aspect or any possible implementation manner of the second aspect. This function can be implemented by hardware or by executing corresponding software by hardware.
  • the hardware or software includes one or more modules corresponding to the above functions.
  • a seventh aspect of an embodiment of the present application provides a data storage service system, the system is deployed on a wheeled mobile device, the system includes a first node and a second node, the first node includes a first processor and a first memory, the first The processor is coupled to the first memory, the second node includes a second processor and a second memory, and the second processor is coupled to the second memory; wherein the first memory and the second memory are used to store the first program and the second memory, respectively A program, where the first processor is configured to call the first program stored in the first memory to execute the steps executed by the first node in the first aspect or any possible implementation manner of the first aspect of the embodiments of the present application, or, the first A processor is used to call the first program stored in the first memory to execute the steps performed by the first node in the second aspect or any possible implementation manner of the second aspect of the embodiments of the present application; the second processor is used to call The second program stored in the second memory is used to execute the steps executed by the second node in the first aspect or any
  • An eighth aspect of the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium runs on a computer, the computer can execute the first aspect or any one of the possible implementations of the first aspect.
  • a ninth aspect of the embodiments of the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method of the first aspect or any possible implementation manner of the first aspect, or enables the computer to execute The second aspect or the method of any possible implementation manner of the second aspect.
  • FIG. 1 is a schematic diagram of a deployment mode of a data storage service system provided by an embodiment of the present application
  • FIG. 2 is another schematic diagram of a deployment mode of a data storage service system provided by an embodiment of the present application
  • FIG. 3 is a schematic flowchart of a data storage method provided by an embodiment of the present application.
  • FIG. 4 is another schematic flowchart of a data storage method according to an embodiment of the present application.
  • FIG. 5 is another schematic flowchart of a data storage method provided by an embodiment of the present application.
  • FIG. 6 is another schematic flowchart of a data storage method provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a data storage service system provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a first node according to an embodiment of the present application.
  • FIG. 9 is another schematic structural diagram of the first node according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a data storage service system provided by an embodiment of the present application.
  • the embodiments of the present application provide a method and device for storing data, which can be used for deploying two node devices on a wheeled mobile device (eg, an autonomous vehicle), through strong consistency (that is, at any time under the premise of deploying two node devices) , the data in the first node device and the second node device in the system are the same) the data synchronization process realizes the real-time backup of the data, which ensures the reliability of the data.
  • a wheeled mobile device eg, an autonomous vehicle
  • strong consistency that is, at any time under the premise of deploying two node devices
  • the data in the first node device and the second node device in the system are the same
  • the data synchronization process realizes the real-time backup of the data, which ensures the reliability of the data.
  • at least 3 nodes are deployed
  • the method of deploying two node devices in the present application reduces the hardware cost of the wheeled mobile device, that is, high-reliability data storage is realized in the hardware
  • wheeled mobile robot It is a comprehensive system integrating environmental perception, dynamic decision-making and planning, behavior control and execution, etc. It can also be called a wheeled mobile robot or a wheeled intelligent body. For example, it can be wheeled construction equipment, self-driving vehicles, A driving assistance vehicle or the like is referred to as the wheeled mobile device described in this application as long as it is a wheeled mobile device.
  • the wheeled mobile device is taken as an example of an automatic driving vehicle for illustration, and the automatic driving vehicle can be a car, a truck, a motorcycle, a bus, a ship, an airplane, a helicopter, Lawn mowers, recreational vehicles, playground vehicles, construction equipment, trams, golf carts, trains, and carts, etc., are not particularly limited in the embodiments of the present application.
  • a node can also be called a computer network node or a physical node.
  • a physical node can be a data circuit terminating device, such as a modem, hub, bridge, or switch; it can also be a data terminal device, such as a digital mobile phone, A printer or host (such as a router, workstation, or server).
  • the node specifically refers to a physical hardware unit, which may be, for example, a host.
  • a node is also referred to as a node device, and in this embodiment of the present application, the first node device and the second node device are both simply referred to as the first node and the second node, respectively.
  • a process is the running activity of a program in a computer on a data set, the basic unit of resource allocation and scheduling in the system, and the basis of the operating system (OS) structure.
  • OS operating system
  • the process is the basic execution entity of the program; in the contemporary computer structure of thread-oriented design, the process is the container of the thread.
  • a program is a description of instructions, data, and their organization, and a process is the entity of a program. It can apply for and own system resources and is an active entity. It is not just the code of the program, but also the current activity, represented by the value of the program counter and the contents of the processing registers.
  • the concept of a process has two main points: First, a process is an entity.
  • Each process has its own address space, in general, including the text region (text region), data region (data region) and stack (stack region).
  • the text area stores code executed by the processor; the data area stores variables and dynamically allocated memory used during process execution; the stack area stores instructions and local variables for active procedure calls.
  • a process is a "program in execution". A program is an inanimate entity. Only when the processor gives the program life (the operating system executes it), it can become an active entity and is called a process.
  • a database engine is the core service used to store, process, and protect data. Utilize the database engine to control access permissions and process transactions quickly, meeting the requirements of most data-intensive applications. Use the database engine to create relational databases for online transaction processing or online analytical processing of data, including creating tables for storing data and database objects (such as indexes, views, and stored procedures) for viewing, managing, and securing data. In this embodiment of the present application, each process runs a lightweight database engine.
  • Consistency refers to the consistency of data.
  • data cannot only exist on a single node, otherwise a single point of failure may occur, resulting in data loss; 2) multiple nodes need to ensure that they have the same data, aiming at Ensure data reliability.
  • Distributed consistency algorithms can be roughly divided into two categories, one is called strong consistency algorithm, that is, the system guarantees that the state of data in each node in the cluster will be changed immediately after the change is submitted.
  • the representative algorithms include paxos algorithm, raft algorithm, ZAB algorithm, etc. ;
  • the other type is called weak consistency algorithm, which can also be called final consistency algorithm, that is, the system does not guarantee that the state of the data in each node in the cluster will be changed immediately after the change is submitted, but the final data in each node will be finalized as time goes by.
  • the state is consistent, which means that the algorithm has the Gossip protocol.
  • the so-called strong consistency can also be called atomic consistency or linear consistency, which has two requirements: 1) any data read can read the most recently written data of a certain data; 2) each data in the system
  • the operation order of all processes of the node is the same as that under the global clock.
  • the data in the first node and the second node in the system are the same.
  • the following takes the raft algorithm as an example and the deployment method shown in Figure 1 as an example to introduce how to realize data synchronization among multiple nodes in a distributed system (ie, a cluster). It should be noted here that the premise is that each node A corresponding process is run on each node, and all processes are implemented by the processes running on their respective nodes.
  • the raft algorithm is a simplification and optimization of the paxos algorithm.
  • the algorithm distinguishes each node in the distributed system into a master node (leader) and a standby node (also known as a follower node).
  • the master node is responsible for issuing proposals
  • the essence of a proposal is a request to read/write data in a distributed system.
  • the request is generally made by the client.
  • the proposal can be expressed as [proposal number n, proposal content value], and the standby node is responsible for the received proposal sent by the master node.
  • the voting principle is: if you do not agree with a proposal with a smaller number than the proposal number you have received before, all other proposals agree.
  • the basic logic of the raft algorithm is briefly introduced as follows: First, the master node election process. Each standby node holds a timer. When the timer expires and there is still no master node in the cluster, the standby node will declare itself as the master node candidate. and participate in the master node election, and at the same time send the election message to other standby nodes to win their votes. If other nodes do not respond to the master node candidate for a long time, the master node candidate will resend the election information.
  • M-term is the latest term, provided that the M-1 If any master node fails or loses master node qualification), the master node during the term will continue to send heartbeats to other nodes (that is, the standby node of any M) to prove that it is still alive, and other nodes will clear their timers after receiving the heartbeat. And reply the heartbeat of the master node, this mechanism is used to ensure that other nodes will not participate in the master node election during the term of the master node.
  • the master node When the master node fails and the master node is disconnected, other nodes that have not received the heartbeat will prepare to become master node candidates and enter the next round of master node election. If two master node candidates send election messages to other nodes at the same time and obtain the same number of support votes, the two master node candidates will randomly delay for a period of time before sending election messages to other nodes, which ensures that the election message will be sent to other nodes again. There is no conflict after the pick message is sent. Secondly, it is the replication process of the data state of each node. In this deployment method in Figure 1, the master node is responsible for receiving proposals from the application (that is, requests to read/write data), and the content of the proposals will be included in the data sent by the master node.
  • the standby node In the next heartbeat, after the standby node receives the heartbeat of the master node, if the standby node agrees with the proposal, it will reply to the heartbeat of the master node. After the master node receives the replies from more than half of the standby nodes, it confirms that the proposal is passed, and then the proposal is passed. Write to its own storage space and reply to the application. At the same time, the master node notifies the standby node to confirm the proposal and write to its own storage space. Finally, all nodes in the distributed system have the same data.
  • the above is only to illustrate the basic logic of the distributed consensus algorithm by taking the raft algorithm as an example.
  • the logic of other algorithms is similar, that is, to realize the data synchronization of each node in the distributed system, more than half of the nodes need to agree. , that is to say, at least 3 nodes need to be deployed in each distributed system (the principle of super-majority agreement requires an odd number of nodes).
  • An embodiment of the present application provides a data storage method, and the method can be applied to a wheeled mobile device (eg, an autonomous vehicle, a smart car, a connected car, etc.).
  • the wheeled mobile device includes two nodes, which are The first node and the second node, in this embodiment of the present application, the first APP and the first process run on the same OS, generally refer to running on the same node (ie, the first node), where the first node As the master node, the second node serves as the standby node.
  • FIG. 3 is a schematic flowchart of a data storage method provided by an embodiment of the present application, which may include the following steps:
  • the first node receives, through a first process running on the first node, a first request sent by a first APP.
  • the first node receives the first request sent by the first APP through the first process running on the first node.
  • the APP is used to obtain the perception information of the surrounding environment of the wheeled mobile device (for example, the perception information can be obtained through sensors such as cameras and radars deployed on the wheeled mobile device), and generate operation instructions for the data based on the perception information (that is, the first request), the first request is sent to the first process running on the first node corresponding to the first APP via the first APP.
  • the first request is used to instruct to write target data (which may be referred to as first data) to a storage module (eg, hard disk, memory, etc.).
  • a storage module eg, hard disk, memory, etc.
  • reading data refers to reading the data in the storage module (ie, "checking")
  • the APP reads the data in the storage module.
  • reading data refers to reading the data in the storage module
  • writing the data refers to adding, deleting or modifying the data in the storage module. and other operations (ie “add”, “delete”, “modify")
  • the APP running on each node can request to add, delete or modify the data in the storage module, these operations will make the data in the storage module The data is changed.
  • the operation instruction of the APP on the data only includes the data write operation to the storage module, that is, the first request is used to instruct the storage module to write the first data.
  • the write data is "add” data
  • the first data is the data that needs to be added to the storage module, which is included in the first request and sent by the first APP to the first node
  • the write data is "delete” data
  • the first data is the corresponding data to be deleted in the storage module, and the relevant identification information used to indicate the first data is included in the first request and sent to the first node by the first APP
  • the write data is "Modify” data
  • the first data is the corresponding data to be modified in the storage module, and the specific modification information and relevant identification information corresponding to the first data are also included in the first request from the first APP to the first node. send.
  • the first APP corresponds to the first process.
  • the first APP corresponding to the first process generally means that the first process and the first APP are running on On the same OS, or, the first process and the first APP both run on the first node, which is not specifically limited here.
  • the first process refers to a process for processing data storage related processes.
  • other processes may also be running at the same time.
  • the objects to be processed are different.
  • only one process related to data storage is running on a node.
  • the first node In response to the first request and based on the first database engine on the first process, the first node writes the first data to the first storage module on the first node through the first process.
  • the first node obtains the first request sent by the first APP, and according to the first request, the first database engine on the first process can be called through the first process, and the first database engine on the first node can be written to the first storage module on the first node. first data.
  • the first request is used to instruct the storage module to write the first data, and there is no limit to which storage module to write the first data to.
  • the first node calls the first database engine through the first process
  • the first data is written to the first storage module on the first node according to the first request.
  • the second node calls the second database engine through the second process
  • the first data is written to the second storage module on the second node according to the first request.
  • the first node may further pass The first process records (eg, updates) the current data version number, which may be referred to as the first data version number, which is used to represent the updated data version of the first storage module data (which may be referred to as the first data version). version), the updated data version is the data version when the operation of writing the first data to the first storage module has been completed.
  • the first process records (eg, updates) the current data version number, which may be referred to as the first data version number, which is used to represent the updated data version of the first storage module data (which may be referred to as the first data version). version), the updated data version is the data version when the operation of writing the first data to the first storage module has been completed.
  • the data version number before this data write operation is P
  • the data in the current first storage module is the latest version
  • the recordable data version number is P+1 , so that it is easy to find which version of the data updated each time, which data has been updated, etc.
  • the first node sends the first request to the second node through the first process.
  • the first node When the operation of writing the first data to the first storage module has been completed, the first node will send the first request to the second node through the first process, and the purpose of sending the first request to the second node will be The method is to make the second node also write the first data to the second storage module on the second node according to the second request.
  • the second node In response to the received first request, the second node writes the first data to the second storage module on the second node through the second process based on the second database engine on the second process.
  • the second node After receiving the first request sent by the first node through the first process, the second node will write to the second storage module on the second node by calling the second database engine on the second process (running on the second node). For the first data, the updated data of the second storage module on the second node and the first storage module can be kept completely consistent.
  • the second process also refers to a process for processing data storage.
  • the second node calls the second database engine through the second process to write the first data to the second storage module on the second node.
  • the operation of writing the first data is completed.
  • the current data version number can also be further recorded (eg, updated) through the second process, and the current data version number can be referred to as the second data version number, which is used to represent the updated data version of the second storage module data ( It may be referred to as the second data version), and the updated data version is the data version under the condition that the operation of writing the first data to the second storage module has been completed.
  • the data version number before the data write operation is Q
  • the data in the current second storage module is the latest version
  • the recordable data version number is Q+1 , so that it is easy to find which version of the data updated each time, which data has been updated, etc.
  • the first storage module on the first node and the second storage module on the second node are updated synchronously, so the values of P and Q are substantially the same.
  • the first node receives, through the first process, a first response message sent by the second node through the second process within a preset time period.
  • the second node sends the first response to the first node through the second process.
  • the first response message is used to indicate that the operation of writing the first data to the second storage module has been completed.
  • the first node sends a second response message to the first application through the first process.
  • the first node When the first node receives the first response message sent by the second node through the second process within the preset time period through the first process, the first node can determine that the second node has also completed the writing operation of the first data, Then the first node will send a second response message to the first application through the first process, and the second response message is used to instruct the operation of writing the first data to the first storage module and writing the first data to the second storage module. The manipulation of the data has been completed.
  • both the first storage module and the second storage module may be a hard disk or an in-memory database, which is not specifically limited here.
  • both the first storage module and the second storage module may be in-memory databases.
  • the first storage module is the first in-memory database
  • the second storage module is the second in-memory database.
  • the wheeled mobile device eg, self-driving vehicle, smart car, connected car, etc.
  • the first node (as the master node) sends the first node
  • the above-mentioned strong consistency that is, at any time, the data in the first node device and the second node device in the system
  • the data synchronization process is the same) to realize the real-time backup of data, that is, the reliability of the data is guaranteed.
  • the method of deploying 2 nodes in the present application reduces the cycle time.
  • the hardware cost of the wheeled mobile device is to achieve high reliability data storage in the hardware architecture of the low-cost wheeled mobile device.
  • FIG. 4 is a data storage method provided by this embodiment of the application.
  • FIG. 4 is a data storage method provided by this embodiment of the application.
  • the first process S1 includes the first database engine (ie the engine 1 in Figure 4) , and a control component HA1 is set on the engine 1, and the control component HA1 is used to realize the transfer process of the relevant information described in the above steps 301 to 306; similarly, the second node has a second storage module for storing data.
  • the second node also runs APP2 (actually can run multiple APPs at the same time, Figure 4 only takes one APP as an example for illustration) and the second process S2, and the second process S2 includes a second database engine (that is, Figure 4 Engine 2) in the engine 2, and a control component HA2 is set on the engine 2, and the control component HA2 is used to realize the transmission process of various information related to the above steps 301 to 306.
  • the process of writing data initiated by APP1 on the master node can be as follows:
  • Step 1 The APP1 on the first node sends a request a to the control component HA1 on the first process through the service interface C1, where the request a is used to instruct to write the data A to the storage module.
  • Step 2 The first node acts as the master node, and calls the engine 1 to write the data A to the first storage module, and records the current data version number M.
  • Step 3 After the data A is successfully written into the first storage module, the first request is sent by the control component HA1 to the control component HA2 of the second node, and the first request is generally included in the data sent by the first node to the second node.
  • the heartbeat it can also be sent separately, which is not limited here).
  • Step 4 The second node is used as the standby node, and the engine 2 is called to write the data A to the second storage module, and the current data version number M is also recorded.
  • Step 5 If the data A is successfully written into the second storage module, the second node will send a response message (ie, the above-mentioned first response message) to the control component HA1 of the first node through the control component HA2.
  • a response message ie, the above-mentioned first response message
  • Step 6 When the first node receives the first response message within a preset time period (for example, within 3s), indicating that data A has been successfully written into the second storage module, then the first node will then pass the control component HA1 to APP1. The execution result is returned, and the execution result is the above-mentioned second response message, which is used to inform APP1 that both the first node and the second node have completed the write operation to the data A.
  • a preset time period for example, within 3s
  • engine 1 and engine 2 are completely independent and do not strongly depend on each other, that is, engine 1 and engine 2 may be isomorphic database engines, or may be Heterogeneous data storage engines, which are not specifically limited here. Because an adaptation module is integrated in the control component HA1 and the control component HA2 for adapting to various database engines, that is, the differences of the database engines are shielded by the respective control components. Due to the different resource consumption of different database engines, the performance performance is also different.
  • the database engine can run on differentiated hardware (such as one high-profile, one low-profile). If engine 1 and engine 2 use heterogeneous database engines, engines with good performance generally run on the high-profile master node, and engines with weaker performance run on the low-profile standby node. Such differentiated deployment is also based on cost reduction considerations.
  • the APP that sends the first request to the first node and the first process are both running on the same OS, then the APP that initiates the request and the first process are both running on the same OS.
  • the data synchronization method of the first storage module on the first node and the second storage module on the second node is also different, which will be discussed in detail below. For details, please refer to the figure. 5.
  • FIG. 5 is another schematic flowchart of a data storage method provided by an embodiment of the present application, which may include the following steps:
  • the second node receives a second request sent by a second APP through a second process running on the second node.
  • the second node receives a second request sent by the second APP through a second process running on the second node, where the second process refers to a process used to process data storage, and the second process is used to process data storage.
  • the APP is used to obtain relevant information, which can be the data of the operation of each system of the wheeled mobile device, or the perception information of the surrounding environment of the wheeled mobile device (for example, through the camera, radar, etc. deployed on the wheeled mobile device. wait for the sensor to obtain sensing information), which is not specifically limited here.
  • the second APP can generate an operation instruction (ie, a second request) for the data based on the perception information, and the second request is sent to the first request corresponding to the second APP via the second APP.
  • the second process running on the second node sends.
  • the second request is used to instruct to write target data (which may be referred to as third data) to a storage module (eg, hard disk, memory, etc.).
  • the operation instruction of the APP on the data only includes the write operation of executing the data to the storage module, that is, the second request is used to instruct to write the third data to the storage module.
  • the write data is "add” data
  • the third data is the data that needs to be added to the storage module, which is included in the second request and sent by the second APP to the second node
  • the write data is "delete” data
  • the third data is the corresponding data to be deleted in the storage module, and the relevant identification information used to indicate the third data is included in the second request and sent to the second node by the second APP
  • the write data is "Modify” data
  • the third data is the corresponding data to be modified in the storage module, and the specific modification information and relevant identification information corresponding to the third data are also included in the second request from the second APP to the second node. send.
  • the second process refers to a process for processing data storage related processes.
  • other processes may also be running at the same time.
  • the objects to be processed are different.
  • only one process related to data storage is running on a node.
  • the first node receives, through the first process running on the first node, the second request forwarded by the second process running on the second node.
  • the second node since the second node is used as a standby node, the second node needs to forward the second request to the first node serving as the master node after receiving the second request sent by the corresponding second APP. deal with. Specifically, the first node receives, through the first process running on the first node, the second request forwarded by the second process running on the second node.
  • the first node In response to the second request and based on the first database engine on the first process, the first node writes third data to the first storage module of the first node through the first process.
  • the first node obtains the second request sent by the first APP, and according to the second request, the first process can call the first database engine on the first process to write the third data to the first storage module on the first node .
  • the second request is used to instruct the storage module to write the third data, and does not limit which storage module the third data is to be written to. For example, if the first node calls the first database engine through the first process, Then, the third data is written to the first storage module on the first node according to the second request. For another example, if the second node calls the second database engine through the second process, the third data is written to the second storage module on the second node according to the second request.
  • the The first process records (eg, updates) the current data version number, which may be referred to as the first data version number, which is used to represent the updated data version of the first storage module data (which may be referred to as the first data version). version), the updated data version is the data version when the operation of writing the third data to the first storage module has been completed.
  • the data version number before this data write operation is X
  • the data in the current first storage module is the latest version
  • the recordable data version number is X+1 , so that it is easy to find which version of the data updated each time, which data has been updated, etc.
  • the first node sends a third response message to the second node through the first process.
  • the first node When the operation of writing the third data to the first storage module has been completed, the first node will send a third response message to the second node through the first process, and the third response message is used to instruct the second node according to the first process.
  • the second process calls the second database engine on the second process to write the third data to the second storage module on the second node.
  • the second request may be included in the third response message and sent by the first process on the first node to the second process on the second node.
  • the advantage is that although the second node has received the second request sent by the second APP at the beginning, in the actual processing process, the second node may need to process many different requests on different processes at the same time, and each request does not It will not be kept on the node all the time. Therefore, for the convenience of management, the second request can be deleted by the second node after being forwarded to the first node.
  • the second request may be included in the third response message sent by the node, so that the second node also writes the third data to the second storage module on the second node according to the second request.
  • the third response message does not include the second request (provided that the second request also exists on the second node), in this case, the third response The message is only used to instruct the second node to call the second database engine through the second process to write the third data to the second storage module according to its existing second request.
  • the second node in response to the second request, writes third data to the second storage module on the second node by invoking the second database engine on the second process.
  • the second node After receiving the second request sent by the first node through the first process, the second node will write the third data to the second storage module on the second node by calling the second database engine on the second process, so that the second The updated data of the second storage module on the node and the first storage module can be kept completely consistent.
  • the second process also refers to a process for processing data storage.
  • the second node calls the second database engine through the second process to write the third data to the second storage module on the second node.
  • the operation is completed.
  • the current data version number can also be further recorded (eg, updated) through the second process, and the current data version number can be referred to as the second data version number, which is used to represent the updated data version of the second storage module data ( It may be referred to as the second data version), and the updated data version is the data version under the condition that the operation of writing the third data to the second storage module has been completed.
  • the data in the current second storage module is the latest version, and the recordable data version number is Y+1 , so that it is easy to find which version of the data updated each time, which data has been updated, etc. It should be noted here that because each time a node writes data, the first storage module on the first node and the second storage module on the second node are updated synchronously, so the values of X and Y are substantially the same.
  • the first node receives, through the first process, a fourth response message sent by the second node through the second process within a preset time period.
  • the second node When the second node completes writing the third data to the second storage module on the second node by calling the second database engine on the second process, it will send a fourth response message to the first node through the second process.
  • the fourth response message is used to indicate that the writing of the third data to the second storage module has been completed.
  • the first node sends a fifth response message to the second node through the first process.
  • the first node When the first node receives, through the first process, the fourth response message sent by the second node through the second process within a preset time period, the first node sends a fifth response message to the second node through the first process, and the fifth response message is sent to the second node through the first process.
  • the response message is used to indicate that the operation of writing the third data to the first storage module and the writing of the third data to the second storage module have been completed.
  • both the first storage module and the second storage module may be a hard disk or an in-memory database, which is not specifically limited here.
  • both the first storage module and the second storage module may be in-memory databases.
  • the first storage module is the first in-memory database
  • the second storage module is the second in-memory database.
  • the second node forwards the fifth response message to the second application through the second process.
  • the second node receives the fifth response message sent by the first node through the first process through the second process, and will forward the fifth response message to the second application, so that the second application can perceive the first Both the node and the second node have completed the write operation to the third data.
  • the first node since the first node is the master node, all response messages sent to the APP need to be sent by the first node. Therefore, the first node needs to send the fifth response message to the second node. , and then forwarded by the second node to the second application.
  • the first node whether it is the first request sent by the first APP or the second request sent by the second APP, the first node must first store the first request through the first process.
  • the module When the module writes data, if the write operation to the first storage module is successful, the second node writes data to the second storage module through the second process. This is to ensure strong consistency of data updates. For example, suppose an APP corresponding to the first process issues request A, and an APP corresponding to the second process issues request B at the same time.
  • both nodes write first Local, then the order of data update of the first storage module on the first node is A ⁇ B, and the order of data update of the second storage module on the second node is B ⁇ A, and the data of the two storage modules are inconsistent within a certain period of time. , and the data version number will not correspond.
  • FIG. 6 is a data storage method provided by this embodiment of the application.
  • FIG. 6 is a data storage method provided by this embodiment of the application.
  • the first process S1 includes the first database engine (ie the engine 1 in Figure 6) , and a control component HA1 is set on the engine 1, and the control component HA1 is used to realize the transmission process of the relevant information described in the above steps 501 to 508; similarly, there is a second storage module on the second node for storing data.
  • the first database engine ie the engine 1 in Figure 6
  • a control component HA1 is set on the engine 1
  • the control component HA1 is used to realize the transmission process of the relevant information described in the above steps 501 to 508; similarly, there is a second storage module on the second node for storing data.
  • the second node also runs APP2 (actually can run multiple APPs at the same time, Figure 6 only takes one APP as an example for illustration) and the second process S2, the second process S2 includes a second database engine (ie, Figure 6 The engine 2 in the engine 2), and a control component HA2 is set on the engine 2, and the control component HA2 is used to realize the transmission process of various information related to the above steps 501 to 508.
  • the process of writing data initiated by APP2 on the standby node can be as follows:
  • Step 1 The APP2 on the second node sends a request b to the control component HA2 on the second process through the service interface C2, where the request b is used to instruct to write the data B to the storage module.
  • Step 2 The second node acts as a standby node, and forwards the request b to the first node as the primary node through the control component HA2 for processing.
  • Step 3 The first node, as the master node, receives the request b sent by the second node through the control component HA1, calls the engine 1 to write the data B to the first storage module, and records the current data version number N.
  • Step 4 After the data B is successfully written into the first storage module, the control component HA1 sends a response message x (that is, the third response message described above) to the control component HA2 of the second node, and the response message x can Including the request b, the response message x is used to instruct the second node to call the engine 2 to write the data B to the second storage module according to the request b.
  • the response message x is generally included in the heartbeat sent by the first node to the second node (it may also be sent independently, which is not limited here).
  • Step 5 The second node is used as the standby node, and the engine 2 is invoked to perform the write operation of the data B on the second storage module, and the current data version number N is also recorded.
  • Step 6 If the data B is successfully written into the second storage module, the second node will send a response message y (ie, the above-mentioned fourth response message) to the control component HA1 of the first node through the control component HA2.
  • a response message y ie, the above-mentioned fourth response message
  • Step 7 When the first node receives the response message y within a preset time period (eg, within 3s), indicating that the data B has been successfully written into the second storage module, then the first node sends the second memory module through the control component HA1 to the second node. The node returns a response message z (that is, the above-mentioned fifth response message), which is used to inform the second node that both parties have completed the write operation to the data B.
  • a preset time period eg, within 3s
  • Step 8 The second node receives the response message z sent by the first node, and returns the execution result to APP2.
  • the execution result is the above-mentioned response message z, which is used to inform APP2 that both the first node and the second node are The write operation to data B is completed.
  • engine 1 and engine 2 can be homogeneous database engines, It may also be a heterogeneous data storage engine, which is not specifically limited here. Because an adaptation module is integrated in the control component HA1 and the control component HA2 for adapting to various database engines, that is, the differences of the database engines are shielded by the respective control components. Due to the different resource consumption of different database engines, the performance performance is also different.
  • the database engine can run on differentiated hardware (such as one high-profile, one low-profile). If engine 1 and engine 2 use heterogeneous database engines, engines with good performance generally run on the high-profile master node, and engines with weaker performance run on the low-profile standby node. Such differentiated deployment is also based on cost reduction considerations.
  • the reason why the primary node and the backup node are required to back up data is to avoid that if one node fails, the other node stores the exact same data, and continues based on the data. Provide services for wheeled mobile devices, so as to ensure availability and data will not be lost. Based on this, in some embodiments of the present application, if a node fails, other nodes will continue to run in stand-alone mode. In order to improve availability, In this embodiment of the present application, the internal and external detection results are combined to quickly perceive the fault. Since the faulty nodes are different, the processing methods are also different, which will be described separately below.
  • the second node will immediately replace the first node to continue to provide services and cache data, and the cached data is the second node.
  • the updated data in the second storage module and after the first node resumes normal use, send the cached data to the first node through the second process, so that the first node can call the first node by calling the first node.
  • the database engine updates the cached data to its own first storage module, thereby realizing data synchronization between the two nodes.
  • the judgment on the failure of the first node may specifically include the following methods:
  • the heartbeat sent by the first node to the second node is abnormal. For example, it is assumed that the frequency of the first node sending heartbeats to the second node is normally once every 300 milliseconds, because the purpose of sending heartbeats from the first node to the second node is to inform the peer It is running normally, but the second node does not receive the heartbeat sent by the first node within a preset period of time (for example, within 1 second), then the second node considers the first node to be faulty.
  • the first node actively sends a fault notification message to the second node.
  • the first node will actively send a fault notification message to the second node;
  • the third process that monitors the first process on the first node monitors that the first process is abnormal
  • the first node will also actively send a failure notification message to the second node; for example, if the communication of the first node is abnormal, Then, the first node will also actively send a failure notification message to the second node, which will not be illustrated by an example this time.
  • method b can make the second node perceive the failure of the first node more quickly, without having to wait for the second node to not receive the heartbeat of the first node for a preset period of time to determine the failure of the first node, thereby further shortening the fault detection delay.
  • the second node to replace the first node to continue to provide services may specifically include the following replacement methods:
  • the second node replaces the first node to provide services until the first node returns to normal use, that is, when the first node returns to normal use, the second node will not be replaced, and the second node will still work as a standby node.
  • the first node is a high configuration
  • the second node is a low configuration, that is, the hardware configuration of the first node and the second node is different, and the main database engine runs on the high configuration first node.
  • the second node with low configuration can replace the first node to provide services in a short period of time, and after the first node returns to normal use, the first node can continue to provide services as the master node. This method of setting high and low configuration nodes is also for the purpose of save costs.
  • the second node always replaces the first node to provide services, that is, after the second node replaces the first node to provide services as the master node, regardless of whether the first node resumes normal use, the second node will always serve as the master node. Service, if the first node resumes normal use, it will always be used as a standby node. Generally, the way of always replacing is when there is no difference in the hardware configuration of the first node and the second node.
  • the first node can set the peer to an abnormal state, while the first node acts as the master node to continue to provide services and cache data (For example, the second data described above in FIG. 3 and FIG. 4 , and the fourth data described in FIG. 5 and FIG. 6 ), the cached data is written by the first node to the first storage module through the first process data (eg, the first data or the third data), and after the second node resumes normal use, the first process sends the cached data to the second node, so that the second node can call the second node by calling the second node.
  • the database engine updates the cached data to its own second storage module, thereby realizing data synchronization between the two nodes.
  • the amount of cached data may be subject to an upper limit, and when the cached data reaches the upper limit, the data with the earliest cache time will be discarded in sequence, because the storage module capacity Limited, in order to ensure that the latest data can be stored in the storage module, only the earliest cached data can be deleted.
  • the judgment on the fault of the second node may specifically include the following methods:
  • the heartbeat sent by the second node to the first node is abnormal. For example, suppose that the frequency of the second node sending heartbeats to the first node is normally once every 300 milliseconds, because the purpose of sending heartbeats to the first node by the second node is to inform the peer It is running normally, but the first node does not receive the heartbeat sent by the second node within a preset time period (for example, within 1 second), then the first node considers that the second node is faulty.
  • the second node actively sends a fault notification message to the first node. For example, when the software on the second process running on the second node exits normally or abnormally, the second node will actively send a fault notification message to the first node; For another example, when the fourth process monitoring the second process on the second node monitors that the second process is abnormal, the second node will also actively send a failure notification message to the first node; for example, the communication of the second node is abnormal, Then, the second node will also actively send a failure notification message to the first node, which will not be illustrated by an example this time.
  • method b enables the first node to perceive the failure of the second node more quickly, without having to wait for the first node to not receive the heartbeat of the second node for a preset period of time to determine the failure of the second node, thereby further shortening the fault detection delay.
  • both the first node and the second node need to perform data synchronization.
  • incremental synchronization and full synchronization according to the data version number, priority is given to incremental synchronization (that is, only the cached data is sent to the peer), and if the cached data is insufficient, full synchronization is performed (for example, if the failure time is very long, At this time, most of the data may be updated, so directly synchronize all data in the storage module).
  • FIG. 7 is the A schematic structural diagram of a data storage service system 700 provided by an embodiment of the application, the data storage service system 700 is deployed on a wheeled mobile device (eg, an autonomous vehicle), and the data storage service system 700 includes a first node 701 and a second Node 702, the first node 701 is the master node, and the second node 702 is the standby node.
  • a wheeled mobile device eg, an autonomous vehicle
  • the first node 701 is configured to receive data corresponding to the first process through the first process running on the first node 701
  • the first request sent by the first application program of the The first database engine on the process writes the first data to the first storage module on the first node 701; the first node 701 is also used in the case where the writing of the first data to the first storage module has been completed Next, send the first request to the second node 702 through the first process; the second node 702 is configured to send the second node 702 to the second node by calling the second database engine on the second process according to the first request
  • the second storage module on 702 writes the first data, and the second process runs on the second node 702; the first node 701 is also used when the first node 701 passes the first process for a preset duration
  • the first response message sent by the second node 702 through the second process is received, and a second response message is sent to the first application through the
  • the data storage service system 700 deployed on wheeled mobile devices (eg, self-driving vehicles, smart cars, connected cars, etc.)
  • wheeled mobile devices eg, self-driving vehicles, smart cars, connected cars, etc.
  • the APP that the node 702 (as the standby node) sends the second request and the first process on the first node 701 are running on different OS
  • the real-time backup of the data is realized through the above-mentioned strong consistency data synchronization process, that is, the guarantee is ensured.
  • the way of deploying 2 nodes in the present application reduces the hardware cost of the wheeled mobile device, that is, the hardware cost of the low-cost wheeled mobile device is reduced. High-reliability data storage in the architecture.
  • the second node 702 is further configured to: when the first node 701 fails, replace the first node 701 to provide services and cache data, and the cached data is replaced by the second node 702
  • the first node 701 acts as the master node, the updated data in the second storage module, and after the first node 701 resumes normal use, the cached data is sent to the first node 701 through the second process, so that the first node 701 passes
  • the first database engine is invoked to update the cached data to its own first storage module, thereby realizing data synchronization between the two nodes.
  • the second node 702 if the first node 701 fails (the first node 701 acts as the master node, and the second node 702 serves as the backup node), the second node 702 is immediately promoted to the master node, replacing the first node 701 for service , so as to ensure the availability of the data storage service system and prevent data loss.
  • the second node 702 is specifically configured to: replace the first node 701 to provide services until the first node 701 returns to normal use, or to always replace the first node 701 to provide services. That is to say, the first replacement method is to wait for the first node 701 to resume normal use, then the second node 702 will not be replaced, and the second node 702 will still work as a standby node.
  • the first node 701 is a high-end node.
  • the second node 702 is of low configuration, that is, the hardware configuration of the first node 701 and the second node 702 are different, and the main database engine runs on the first node 701 of high configuration.
  • the low-configured second node 702 replaces the first node 701 to provide services, and after the first node 701 resumes normal use, the first node 701 continues to provide services as the master node. This way of setting high and low configuration nodes is also for saving cost.
  • the second replacement method is that after the second node 702 replaces the first node 701 as the master node to provide services, regardless of whether the first node 701 resumes normal use, the second node 702 will continue to provide services as the master node. After the node 701 is restored to normal use, it is always used as a standby node. Generally, the way of always replacing is when there is no difference in the hardware configuration of the first node 701 and the second node 702 .
  • the failure of the first node 701 includes: the heartbeat sent by the first node 701 to the second node 702 is abnormal; or, the first node 701 sends a failure notification message to the second node 702 .
  • the frequency of the first node 701 sending a heartbeat to the second node 702 is normally once every 300 milliseconds, since the purpose of sending the heartbeat to the second node 702 by the first node 701 is to inform the peer that it is running normally, but in the preset If the second node 702 does not receive the heartbeat sent by the first node 701 within a period of time (eg, within 1 second), the second node 702 considers that the first node 701 is faulty.
  • the first node 701 when the software on the first process running on the first node 701 exits normally or abnormally, the first node 701 will actively send a fault notification message to the second node 702; for another example, when the first node 701 monitors the first When the third process of the process monitors that the first process is abnormal, the first node 701 will also actively send a fault notification message to the second node 702; for example, if the communication of the first node 701 is abnormal, then the first node 701 will also actively Sending a fault notification message to the second node 702, which will not be illustrated by an example this time.
  • the above two fault sensing methods are both to enable the second node 702 to quickly sense that the first node 701 has a fault, thereby shortening the fault detection delay.
  • the second fault sensing method enables the second node 702 to sense that the first node 701 has a fault more quickly, without having to wait for a preset period of time when the second node 702 does not receive The heartbeat of the first node 701 determines that the first node 701 is faulty, thereby further shortening the fault detection delay.
  • the first node 701 is further configured to set the peer to an abnormal state when the second node 702 fails, and cache the second data, and after the second node 702 resumes normal use , send the second data to the second node 702 through the first process;
  • the second node 702 is also configured to update the second data to the second storage module by calling the second database engine, the first The second data is the data obtained after the first node 701 writes the first data to the first storage module through the first process.
  • the first node 701 will continue to provide services as the master node, and after the second node 702 resumes normal use, the first process will use the cached
  • the second data is sent to the second node 702, so that the second node 702 updates the cached second data to its own second storage module by invoking the second database engine, so as to realize data synchronization between the two nodes, and has the ability to Realization.
  • the failure of the second node 702 includes: the heartbeat sent by the second node 702 to the first node 701 is abnormal; or, the second node 702 sends a failure notification message to the first node 701 .
  • the frequency of the heartbeat sent by the second node 702 to the first node 701 is normally once every 300 milliseconds, because the purpose of the second node 702 sending the heartbeat to the first node 701 is to inform the peer that it is running normally, but in the preset If the first node 701 does not receive the heartbeat sent by the second node 702 within a period of time (eg, within 1 second), the first node 701 considers that the second node 702 is faulty.
  • the second node 702 when the software on the second process running on the second node 702 exits normally or abnormally, the second node 702 will actively send a failure notification message to the first node 701; for another example, when the second node 702 monitors the second node 702 When the fourth process of the process monitors that the second process is abnormal, the second node 702 will also actively send a failure notification message to the first node 701; for example, if the communication of the second node 702 is abnormal, then the second node 702 will also actively Send a fault notification message to the first node 701, and this time, no example is given.
  • the above two fault sensing methods are both to enable the first node 701 to quickly perceive that the second node 702 has a fault, thereby shortening the fault detection delay.
  • the second fault sensing method enables the first node 701 to sense that the second node 702 has a fault more quickly, without having to wait for a preset period of time when the first node 701 does not receive The heartbeat of the second node 702 determines that the second node 702 is faulty, thereby further shortening the fault detection delay.
  • the first storage module includes a first in-memory database; and/or the second storage module includes a second in-memory database.
  • the first storage module and the second storage module may be in-memory databases.
  • the reading speed of in-memory databases is faster, especially in the field of intelligent driving, due to the operation of the vehicle.
  • Data such as status data, perception data, intermediate calculation results, and SOA information need to be read frequently. If the first storage module and the second storage module are hard disks, the repeated and frequent reading/writing of data will shorten the life of the hard disks, which will lead to automotive use. The replacement cost of the device increases.
  • the first node 701 is further configured to: record a first data version number through the first process, where the first data version number is used to represent the updated data version of the first storage module data (may be referred to as the first data version), the updated data version is the data version under the condition that the operation of writing the first data to the first storage module has been completed.
  • the version number of the updated data is also recorded, so that it is easy to find out which version of the updated data is and which data has been updated. Wait.
  • the second node 702 is further configured to: record a second data version number through the second process, where the second data version number is used to represent the updated data version of the second storage module data (may be referred to as the second data version), the updated data version is the data version under the condition that the operation of writing the first data to the second storage module has been completed.
  • the version number of the updated data is also recorded, so that it is convenient to find out which version of the updated data is and which data has been updated. Wait.
  • the wheeled mobile device on which the data storage service system 700 is deployed may be an autonomous vehicle, and the autonomous vehicle may be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, Lawn mowers, recreational vehicles, playground vehicles, construction equipment, trams, golf carts, trains, and carts, etc., are not particularly limited in the embodiments of the present application.
  • the wheeled mobile device may be an autonomous driving vehicle, which is feasible.
  • the second node 702 is configured to receive and communicate with the second node 702 through a second process running on the second node 702 .
  • the second request sent by the second application program corresponding to the process, the second request is used to instruct to write the third data to the storage module;
  • the first node 701 is used to receive the third data through the first process running on the first node 701
  • the first node 701 is further configured to, in response to the second request, call the first database engine on the first process through the first process to send to the first node
  • the first storage module of a node 701 writes the third data;
  • the first node 701 is further configured to write the third data to the first storage module through the first process when the operation of writing the third data to the first storage module has been completed.
  • the second node 702 sends a third response message, where the third response message is used to instruct the second node 702 to call the second database engine on the second process to the second node through the second process according to the second request
  • the second storage module 702 writes the third data
  • the second node 702 is further configured to call the second database engine to the second storage module through the second process according to the third response message and the second request Write the third data
  • the first node 701 is further configured to, when the first node 701 receives the fourth response message sent by the second node 702 through the second process within a preset period of time through the first process, A fifth response message is sent to the second node 702 through the first process, the fourth response message is used to indicate that the operation of writing the third data to the second storage module has been performed, and the fifth response message is used to indicate Both the operation of writing the third data to the first storage module and the operation of writing the third data to the second storage module have been completed; the second node 702 is also used for writing to the second storage module
  • the data storage service system 700 deployed on a wheeled mobile device sends data to the second node 702 (as a standby node) on the premise that only two nodes are deployed.
  • a wheeled mobile device eg, an autonomous vehicle
  • the APP of the second request and the first process on the first node 701 are running on different OS
  • real-time data backup is realized through the above-mentioned strong consistency data synchronization process, which not only ensures the reliability of the data, but also reduces the The hardware cost of the wheeled mobile device is reduced, that is, high-reliability data storage is realized in the hardware architecture of the low-cost wheeled mobile device.
  • the second node 702 is further configured to: when the first node 701 fails, replace the first node 701 to provide services and cache data, and the cached data is replaced by the second node 702
  • the first node 701 acts as the master node, the updated data in the second storage module, and after the first node 701 resumes normal use, the cached data is sent to the first node 701 through the second process, so that the first node 701 passes
  • the first database engine is invoked to update the cached data to its own first storage module, thereby realizing data synchronization between the two nodes.
  • the second node 702 if the first node 701 fails (the first node 701 acts as the master node, and the second node 702 serves as the backup node), the second node 702 is immediately promoted to the master node, replacing the first node 701 for service , so as to ensure the availability of the data storage service system and prevent data loss.
  • the second node 702 is specifically configured to: replace the first node 701 to provide services until the first node 701 resumes normal use; or, always replace the first node 701 to provide services. That is to say, the first replacement method is to wait for the first node 701 to resume normal use, then the second node 702 will not be replaced, and the second node 702 will still work as a standby node.
  • the first node 701 is a high-end node.
  • the second node 702 is of low configuration, that is, the hardware configuration of the first node 701 and the second node 702 are different, and the main database engine runs on the first node 701 of high configuration.
  • the low-configured second node 702 replaces the first node 701 to provide services, and after the first node 701 resumes normal use, the first node 701 continues to provide services as the master node. This way of setting high and low configuration nodes is also for saving cost.
  • the second replacement method is that after the second node 702 replaces the first node 701 as the master node to provide services, regardless of whether the first node 701 resumes normal use, the second node 702 will continue to provide services as the master node. After the node 701 is restored to normal use, it is always used as a standby node. Generally, the way of always replacing is when there is no difference in the hardware configuration of the first node 701 and the second node 702 .
  • the failure of the first node 701 includes: the heartbeat sent by the first node 701 to the second node 702 is abnormal; or, the first node 701 sends a failure notification message to the second node 702 .
  • the frequency of the first node 701 sending a heartbeat to the second node 702 is normally once every 300 milliseconds, since the purpose of sending the heartbeat to the second node 702 by the first node 701 is to inform the peer that it is running normally, but in the preset If the second node 702 does not receive the heartbeat sent by the first node 701 within a period of time (eg, within 1 second), the second node 702 considers that the first node 701 is faulty.
  • the first node 701 when the software on the first process running on the first node 701 exits normally or abnormally, the first node 701 will actively send a fault notification message to the second node 702; for another example, when the first node 701 monitors the first When the third process of the process monitors that the first process is abnormal, the first node 701 will also actively send a fault notification message to the second node 702; for example, if the communication of the first node 701 is abnormal, then the first node 701 will also actively Sending a fault notification message to the second node 702, which will not be illustrated by an example this time.
  • the above two fault sensing methods are both to enable the second node 702 to quickly sense that the first node 701 has a fault, thereby shortening the fault detection delay.
  • the second fault sensing method enables the second node 702 to sense that the first node 701 has a fault more quickly, without having to wait for a preset time period when the second node 702 does not receive The heartbeat of the first node 701 determines that the first node 701 is faulty, thereby further shortening the fault detection delay.
  • the first node 701 is also used to set the peer to an abnormal state when the second node 702 fails, and cache the fourth data, and after the second node 702 resumes normal use , send the fourth data to the second node 702 through the first process;
  • the second node 702 is also configured to update the fourth data to the second storage module by calling the second database engine, the first The fourth data is the data obtained after the first node 701 writes the third data to the first storage module through the first process.
  • the first node 701 will continue to provide services as the master node, and after the second node 702 resumes normal use, the first process will use the cached
  • the fourth data is sent to the second node 702, so that the second node 702 updates the cached fourth data to its own second storage module by invoking the second database engine, thereby realizing data synchronization between the two nodes, with the ability to Realization.
  • the failure of the second node 702 includes: the heartbeat sent by the second node 702 to the first node 701 is abnormal; or, the second node 702 sends a failure notification message to the first node 701 .
  • the frequency of the heartbeat sent by the second node 702 to the first node 701 is normally once every 300 milliseconds, because the purpose of the second node 702 sending the heartbeat to the first node 701 is to inform the peer that it is running normally, but in the preset If the first node 701 does not receive the heartbeat sent by the second node 702 within a period of time (eg, within 1 second), the first node 701 considers that the second node 702 is faulty.
  • the second node 702 when the software on the second process running on the second node 702 exits normally or abnormally, the second node 702 will actively send a failure notification message to the first node 701; for another example, when the second node 702 monitors the second node 702 When the fourth process of the process monitors that the second process is abnormal, the second node 702 will also actively send a failure notification message to the first node 701; for example, if the communication of the second node 702 is abnormal, then the second node 702 will also actively Send a fault notification message to the first node 701, and this time, no example is given.
  • the above two fault sensing methods are both to enable the first node 701 to quickly perceive that the second node 702 has a fault, thereby shortening the fault detection delay.
  • the second fault sensing method enables the first node 701 to sense that the second node 702 has a fault more quickly, without having to wait for a preset period of time when the first node 701 does not receive The heartbeat of the second node 702 determines that the second node 702 is faulty, thereby further shortening the fault detection delay.
  • the first storage module includes a first in-memory database; and/or the second storage module includes a second in-memory database.
  • the first storage module and the second storage module may be in-memory databases.
  • the reading speed of in-memory databases is faster, especially in the field of intelligent driving, due to the operation of the vehicle.
  • Data such as status data, perception data, intermediate calculation results, and SOA information need to be read frequently. If the first storage module and the second storage module are hard disks, the repeated and frequent reading/writing of data will shorten the life of the hard disks, which will lead to automotive use. The replacement cost of the device increases.
  • the first node 701 is further configured to: record a first data version number through the first process, where the first data version number is used to represent the updated data version of the first storage module data (may be referred to as the first data version), the updated data version is the data version when the operation of writing the third data to the first storage module has been completed.
  • the version number of the updated data is also recorded, so that it is easy to find out which version of the updated data is and which data has been updated. Wait.
  • the second node 702 is further configured to: record a second data version number through the second process, where the second data version number is used to represent the updated data version of the second storage module data (may be referred to as the second data version), the updated data version is the data version under the condition that the operation of writing the third data to the second storage module has been completed.
  • the version number of the updated data is also recorded, so that it is convenient to find out which version of the updated data is and which data has been updated. Wait.
  • the wheeled mobile device on which the data storage service system 700 is deployed may be an autonomous vehicle, and the autonomous vehicle may be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, Lawn mowers, recreational vehicles, playground vehicles, construction equipment, trams, golf carts, trains, and carts, etc., are not particularly limited in the embodiments of the present application.
  • the wheeled mobile device may be an autonomous driving vehicle, which is feasible.
  • FIG. 8 8 is a schematic structural diagram of a first node 800 provided by an embodiment of the present application.
  • the first node 800 may specifically include: a receiving module 801 , configured to receive and receive data from a first process running on the first node 800 .
  • the first request sent by the first application program corresponding to the first process, the first request is used to instruct to write the first data to the storage module;
  • the calling module 802 is configured to, according to the first request, pass the first request to the storage module.
  • a process calls the first database engine on the first process to write the first data to the first storage module on the first node 800; the first sending module 803 is configured to write the first data to the first storage module In the case that the operation of writing the first data has been completed, the first process is used to send the first request to the second node, so that the second node according to the first request, by calling The second database engine on the second process writes the first data to the second storage module on the second node, and the second process runs on the second node; the second sending module 804 is used for when receiving, by the first process, a first response message sent by the second node through the second process within a preset time period, and sending a second response message to the first application by the first process; The first response message is used to indicate that the operation of writing the first data to the second storage module has been completed, and the second response message is used to indicate that the operation of writing the first data to the first storage module has been completed. Both the operation and the writing of the first data to the second storage module are completed.
  • the first sending module 803 is further configured to: when the second node fails, cache the second data, and after the second node resumes normal use, pass the first The process sends the second data to the second node, so that the second node updates the second data to the second storage module by calling the second database engine, the second data It is the data obtained by the first node 800 after the operation of writing the first data to the first storage module through the first process.
  • the failure of the second node includes: the heartbeat sent by the second node to the first node 800 is abnormal; or, the second node sends a failure notification to the first node 800 information.
  • the first storage module includes a first in-memory database; and/or the second storage module includes a second in-memory database.
  • the calling module 802 is further configured to: record a first data version number through the first process, where the first data version number is used to indicate that the data of the first storage module has been updated version of the data.
  • FIG. 9 is a schematic structural diagram of a first node 900 provided by an embodiment of the present application.
  • the first node 900 may specifically include: a receiving module 901 , configured to receive a running process through a first process running on the first node 900 A second request forwarded by a second process on the second node, the second request being sent to the second node by a second application corresponding to the second process, the second request being used for Instruct to write the third data to the storage module; the calling module 902 is configured to call the first database engine on the first process to the first database engine of the first node 900 through the first process according to the second request The storage module writes the third data; the first sending module 903 is configured to send the third data to the first storage module through the first process when the operation of writing the third data to the first storage module is completed.
  • a receiving module 901 configured to receive a running process through a first process running on the first node 900 A second request forwarded by a second process on the second node, the second request being sent to the second node by a second application corresponding to the second process, the second request being used for Instruct to write
  • the second node sends a third response message, where the third response message is used to instruct the second node to call the second database engine on the second process to the second node through the second process according to the second request.
  • the second storage module of the second node writes the third data;
  • the second sending module 904 is configured to receive, through the first process, the data sent by the second node through the second process within a preset time period a fourth response message, sending a fifth response message to the second node through the first process, so that the second node forwards the fifth response message to the second application through the second process , the fourth response message is used to indicate that the operation of writing the third data to the second storage module has been completed, and the fifth response message is used to indicate that the third data is written to the first storage module. and the operation of writing the third data to the second storage module has been completed.
  • the first sending module 903 is further configured to: when the second node fails, cache the fourth data, and after the second node resumes normal use, pass the first The process sends the fourth data to the second node, so that the second node updates the fourth data to the second storage module by calling the second database engine, and the fourth data Data obtained after the first node 900 writes the third data to the first storage module through the first process.
  • the failure of the second node includes: the heartbeat sent by the second node to the first node 900 is abnormal; or, the second node sends a failure notification to the first node 900 information.
  • the first storage module includes a first in-memory database; and/or the second storage module includes a second in-memory database.
  • the calling module 902 is further configured to: record a first data version number through the first process, where the first data version number is used to indicate that the data of the first storage module has been updated version of the data.
  • An embodiment of the present application also provides a data storage service system, where the data storage service system is deployed on a wheeled mobile device (eg, an autonomous vehicle).
  • a wheeled mobile device eg, an autonomous vehicle
  • FIG. 10 is the data storage provided by the embodiment of the present application.
  • a schematic structural diagram of the service system 1000 for the convenience of description, only shows the part related to the embodiment of the present application, and the specific technical details are not disclosed, please refer to the method part of the embodiment of the present application.
  • the data storage service system 1000 can be deployed with the modules of the data storage service system described in the corresponding embodiment of FIG. 7 to implement the functions of the corresponding embodiments of FIG. 3 to FIG. 6 .
  • the data storage service system 1000 consists of one or more When implemented with multiple servers, the data storage service system 1000 may vary greatly due to different configurations or performance, and may include one or more central processing units (CPU) 1022 (eg, one or more) and memory 1032, one or more storage media 1030 (eg, one or more mass storage devices) storing the application 1042 or data 1044.
  • the memory 1032 and the storage medium 1030 may be short-term storage or persistent storage.
  • the program stored in the storage medium 1030 may include one or more modules (not shown in the figure), and each module may include a series of instructions to operate on the training device.
  • the central processing unit 1022 may be configured to communicate with the storage medium 1030 to execute a series of instruction operations in the storage medium 1030 on the data storage service system 1000 .
  • the data storage service system 1000 may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input and output interfaces 1058, and/or, one or more operating systems 1041, such as Windows ServerTM , Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM and many more.
  • operating systems 1041 such as Windows ServerTM , Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM and many more.
  • the steps performed by the first node and the second node in the embodiments corresponding to FIG. 3 to FIG. 6 can be implemented based on the structure shown in FIG. 10 , and details are not repeated here.
  • the device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be A physical unit, which can be located in one place or distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
  • U disk U disk
  • mobile hard disk ROM
  • RAM random access memory
  • disk or CD etc.
  • a computer device which can be a personal computer, training equipment, or network equipment, etc. to execute the methods described in the various embodiments of the present application.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be retrieved from a website, computer, training device, or data
  • the center transmits to another website site, computer, training equipment, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line) or wireless (eg, infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a training device, a data center, or the like that includes an integration of one or more available media.
  • the available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, high-density digital video discs (DVDs)), or semiconductor media (eg, solid state disks) , SSD)) etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种数据的存储方法及设备,应用于数据存储技术领域,该方法包括:在轮式移动设备(如智能汽车、自动驾驶汽车、网联汽车等)上部署双节点构成的数据存储服务系统,第一节点为主节点,第二节点为备节点,第一节点和第二节点上分别运行第一进程和第二进程,当第一节点接收到写数据请求时,第一节点先通过第一进程调用第一数据库引擎向第一节点上的第一存储模块写数据,再阻塞式使得第二节点通过第二进程调用第二数据库引擎向第二节点上的第二存储模块进行同样的写数据操作,从而在轮式移动设备上部署两个节点前提下,通过强一致性的数据同步流程实现数据实时备份,保障了数据可靠性,同时降低了轮式移动设备的硬件成本。

Description

一种数据的存储方法及设备 技术领域
本申请涉及数据存储技术领域,尤其涉及一种数据的存储方法及设备。
背景技术
随着汽车智能化的演进,越来越多的数据被生成和利用,尤其是在智能驾驶领域,由于车辆的运行状态数据、感知数据、中间计算结果、面向服务架构(service-oriented architecture,SOA)信息等数据需要频繁被读取,因此其对数据读取的实时性、数据存储的可靠性以及数据的可用性等要求高。在此背景下,对数据处理能力、数据存储可靠性等要求不断提升,从而使得对数据存储的需求更加多样化。
现有技术中,为了提升数据存储的可靠性,一般采用分布式集群部署的方式,利用轻量级的数据库引擎,结合分布式一致性算法,构建出集群化的数据存储服务系统,该数据存储服务系统的部署方式有两种:1)如图1所示,图1以部署3个节点设备(也可简称为节点)为例进行示意,作为客户端的各个应用程序(application,APP)的读/写数据的请求集中由主节点(leader)上运行的进程S2处理,再由进程S2通过向集群内的其他节点的进程(如,进程S1和进程S3)发送同步命令请求(synchronization,SYNC),对各个节点上存储模块的数据进行同步(由节点上的数据库引擎实现,也可简称为引擎);2)如图2所示,图2依然以部署3个节点为例进行示意,作为客户端的各个APP就近访问节点,若某节点有更新数据,再由该节点上运行的进程向其他节点上运行的进程发送SYNC,实现各个节点上存储模块内的数据同步。
上述部署方式都依赖于分布式一致性算法(如,paxos算法及其演变算法、raft算法等),这些算法都具有一个关键约束:所有的选举过程(如,选举leader的过程)、数据的写操作等都要求集群内所有节点中过半数的进程同意才认为是达成共识。由于过半数的硬性要求,一般都是采用3个以上的奇数个节点的部署方式。但在汽车领域中,车用器件成本控制严格,数据可靠性要求高,上述这种部署3个节点的方式数据冗余备份多、资源消耗大,导致硬件成本高,使得其无法同时满足数据可靠性高和成本低等要求,尤其是很多中低端车型无法承受这些硬件成本的增加。
发明内容
本申请实施例提供了一种数据的存储方法及设备,可以用于在轮式移动设备(如,自动驾驶车辆)上部署2个节点设备的前提下,通过强一致性(即在任意时刻下,系统里第一节点设备和第二节点设备中的数据都是一样的)的数据同步流程实现数据的实时备份,保障了数据的可靠性,同时相比于现有技术中至少部署3个节点设备的方式,本申请部署2个节点设备的方式降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
基于此,本申请实施例提供以下技术方案,其中,在本申请以下各实施方式中,第一 节点设备简称为第一节点,第二节点设备简称为第二节点。
第一方面,本申请实施例首先提供一种数据的存储方法,该方法包括:首先,第一节点作为主节点,通过运行在第一节点上的第一进程接收第一APP发送的第一请求,该第一APP可以是与第一进程对应的APP,其中,该第一进程是指用于处理数据存储相关的进程,该第一APP用于获取轮式移动设备周围环境的感知信息(如,可通过部署在轮式移动设备上的摄像头、雷达等传感器获得感知信息),并基于感知信息生成对数据的操作指令(即第一请求),该第一请求经由第一APP向与该第一APP对应的第一节点上运行的第一进程发送。在本申请实施例中,该第一请求是用于指示向存储模块(如,硬盘、内存等)写目标数据(可称为第一数据)。第一节点获取到第一APP发送的第一请求,响应于该第一请求,第一进程就可以基于第一进程上的第一数据库引擎,向第一节点上的第一存储模块写第一数据。这里需要注意的是,第一请求是用于指示向存储模块写第一数据,并没有限定是对哪个存储模块写第一数据,例如,若是第一节点通过第一进程调用第一数据库引擎,那么就是根据该第一请求向第一节点上的第一存储模块写该第一数据。又例如,若是第二节点通过第二进程调用第二数据库引擎,那么就是根据该第一请求向第二节点上的第二存储模块写该第一数据。在向第一存储模块写第一数据的操作已完成的情况下,第一节点才会通过第一进程将第一请求向所述第二节点发送,将第一请求向第二节点发送的目的在于:使得第二节点根据该第二请求向第二节点上的第二存储模块也写入该第一数据。第二节点接收到第一节点通过第一进程发送的第一请求后,将通过调用第二进程(运行在第二节点)上的第二数据库引擎向第二节点上的第二存储模块写该第一数据,这样第二节点上的第二存储模块与第一存储模块更新后的数据就能保持完全一致。其中,与第一进程类似,该第二进程也是指用于处理数据存储相关的进程。第二节点通过调用第二进程上的第二数据库引擎向第二节点上的第二存储模块写第一数据的操作完成的情况下,就会通过第二进程向第一节点发送第一响应消息,该第一响应消息就用于指示向第二存储模块写第一数据的操作已完成。当第一节点通过第一进程在预设时长内接收到第二节点通过第二进程发送的第一响应消息,那么第一节点就可确定第二节点也完成了对第一数据的写操作,那么第一节点将会通过第一进程向第一应用程序发送第二响应消息,该第二响应消息就用于指示向第一存储模块写第一数据的操作和向第二存储模块写第一数据的操作均已完成。在一些实施方式中,该方法可以应用于轮式移动设备,该轮式移动设备就可以包括上述所述的第一节点和第二节点。
在本申请上述实施方式中,在轮式移动设备(如,自动驾驶车辆、智能汽车、网联汽车等)上仅部署2个节点的前提下,且向第一节点(作为主节点)发送第一请求的APP与第一节点上的第一进程均为运行在同一个OS的情况时,通过上述强一致性(即在任意时刻下,系统里第一节点设备和第二节点设备中的数据都是一样的)的数据同步流程实现数据的实时备份,即保障了数据的可靠性,同时相比于现有技术中至少部署3个节点设备的方式,本申请部署2个节点设备的方式降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
在第一方面的一种可能的设计中,如果第一节点故障,那么第二节点会立即取代第一 节点继续提供服务,并缓存数据,该缓存的数据就为第二节点取代第一节点作为主节点后第二存储模块内更新的数据,并且在第一节点恢复正常使用后,通过第二进程将该缓存的数据向第一节点发送,使得第一节点通过调用第一数据库引擎将该缓存的数据更新至自身的第一存储模块,从而实现两个节点之间的数据同步。
在本申请上述实施方式中,若第一节点故障(第一节点作为主节点,第二节点作为备节点),则第二节点立即升为主节点,取代第一节点进行服务,从而保障数据存储服务系统的可用性,数据不丢失。
在第一方面的一种可能的设计中,第二节点取代第一节点继续提供服务具体可以包括以下几种取代方式:a、第二节点取代第一节点提供服务,直至第一节点恢复正常使用,也就是说,等第一节点恢复了正常使用,第二节点就不再取代,第二节点依然作为备节点工作,一般这种情况是第一节点是高配,第二节点是低配,即第一节点与第二节点的硬件配置不同,主数据库引擎运行在高配的第一节点上,当高配的第一节点故障,则可以短期内由低配的第二节点取代第一节点提供服务,等第一节点恢复正常使用后,再由第一节点继续作为主节点提供服务,这种设置高低配置节点的方式也是为了节约成本。b、第二节点始终取代第一节点提供服务,也就是说,第二节点取代第一节点作为主节点提供服务后,不管后续第一节点是否恢复正常使用,第二节点就一直作为主节点提供服务,若第一节点恢复正常使用后,就一直作为备节点使用,一般这种一直取代的方式是第一节点和第二节点的硬件配置没有差别的情况。
在本申请上述实施方式中,具体阐述了当第一节点故障,第二节点取代第一节点提供服务的两种方式,具备灵活性和可选择性。
在第一方面的一种可能的设计中,对第一节点故障的判断具体可以包括如下几种方式:a、第一节点向第二节点发送的心跳异常,例如,假设第一节点向第二节点发送心跳的频率正常是300毫秒一次,由于第一节点向第二节点发送心跳的目的就是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第二节点都没有收到第一节点发送的心跳,那么第二节点就认为第一节点故障。b、第一节点向第二节点主动发送故障通知消息,例如,当第一节点上运行的第一进程上的软件正常或异常退出,则第一节点会主动向第二节点发送故障通知消息;又例如,当第一节点上监控第一进程的第三进程监控到第一进程异常时,则第一节点也会向第二节点主动发送故障通知消息;还例如,第一节点的通信异常,那么第一节点也会主动向第二节点发送故障通知消息,具体此次不做举例示意。
在本申请上述实施方式中,上述两种感知故障的方式都是为了使得第二节点能够快速感知到第一节点出现了故障,缩短了故障检测时延。其中,第二种故障感知的方式相比第一种故障感知的方式,能够让第二节点更快感知到第一节点出现了故障,而不必等待预设时长第二节点没有收到第一节点的心跳才确定第一节点故障,从而进一步缩短了故障检测时延。
在第一方面的一种可能的设计中,如果第二节点故障,那么第一节点可以将对端设置为异常状态,而第一节点作为主节点继续提供服务,并缓存数据(即第二数据),该缓存的数据就为第一节点通过第一进程向第一存储模块写第一数据的操作后得到的数据,并且在 第二节点恢复正常使用后,通过第一进程将该缓存的数据向第二节点发送,使得第二节点通过调用第二数据库引擎将该缓存的数据更新至自身的第二存储模块,从而实现两个节点之间的数据同步。
在本申请上述实施方式中,阐述了如果是第二节点故障,那么第一节点将作为主节点继续提供服务,并且在第二节点恢复正常使用后,通过第一进程将该缓存的第二数据向第二节点发送,使得第二节点通过调用第二数据库引擎将该缓存的第二数据更新至自身的第二存储模块,从而实现两个节点之间的数据同步,具备可实现性。
在第一方面的一种可能的设计中,对第二节点故障的判断具体也可以包括如下几种方式:a、第二节点向第一节点发送的心跳异常,例如,假设第二节点向第一节点发送心跳的频率正常是300毫秒一次,由于第二节点向第一节点发送心跳的目的也是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第一节点都没有收到第二节点发送的心跳,那么第一节点就认为第二节点故障。b、第二节点向第一节点主动发送故障通知消息,例如,当第二节点上运行的第二进程上的软件正常或异常退出,则第二节点会主动向第一节点发送故障通知消息;又例如,当第二节点上监控第二进程的第四进程监控到第二进程异常时,则第二节点也会向第一节点主动发送故障通知消息;还例如,第二节点的通信异常,那么第二节点也会主动向第一节点发送故障通知消息,具体此次不做举例示意。
在本申请上述实施方式中,上述两种感知故障的方式都是为了使得第一节点能够快速感知到第二节点出现了故障,缩短了故障检测时延。其中,第二种故障感知的方式相比第一种故障感知的方式,能够让第一节点更快感知到第二节点出现了故障,而不必等待预设时长第一节点没有收到第二节点的心跳才确定第二节点故障,从而进一步缩短了故障检测时延。
在第一方面的一种可能的设计中,在一种可能的设计中,该第一存储模块包括第一内存数据库;和/或,该第二存储模块包括第二内存数据库。
在本申请上述实施方式中,阐述了第一存储模块和第二存储模块可以是内存数据库,相对于传统的硬盘,内存数据库的读取速度更快,尤其是在智能驾驶领域,由于车辆的运行状态数据、感知数据、中间计算结果、SOA信息等数据需频繁被读取,若第一存储模块和第二存储模块为硬盘,那么反复频繁的读/写数据会缩短硬盘寿命,从而导致车用器件的更换成本提高。
在第一方面的一种可能的设计中,第一节点通过第一进程调用第一数据库引擎向第一节点上的第一存储模块写入第一数据的操作执行完成之后,还可以进一步通过第一进程记录(如,更新)当前的数据版本号,该当前数据版本号可称为第一数据版本号,其用于表示第一存储模块数据更新后的数据版本(可称为第一数据版本),该更新后的数据版本为向第一存储模块写第一数据的操作已完成的情况下的数据版本。例如,假设此次数据写操作之前的数据版本号为P,那么当前对第一数据写操作完成后,当前第一存储模块内的数据是最新版本的,则可记录数据版本号为P+1。
在本申请上述实施方式中,通过第一进程执行完第一数据的写操作后,还要记录此次更新后的数据版本号,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
在第一方面的一种可能的设计中,第二节点通过第二进程调用第二数据库引擎向第二节点上的第二存储模块写入第一数据的操作执行完成之后,也可以进一步通过第二进程记录(如,更新)当前的数据版本号,该当前数据版本号可称为第二数据版本号,其用于表示第二存储模块数据更新后的数据版本(可称为第二数据版本),所述更新后的数据版本为向第二存储模块写第一数据的操作已完成的情况下的数据版本。例如,假设此次数据写操作之前的数据版本号为Q,那么当前对第一数据写操作完成后,当前第二存储模块内的数据是最新版本的,则可记录数据版本号为Q+1,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。这里需要注意的是,由于每次节点对数据的写操作,都会同步更新第一节点上的第一存储模块和第二节点上的第二存储模块,因此实质上P和Q的取值相同。
在本申请上述实施方式中,通过第二进程执行完第一数据的写操作后,也要记录此次更新后的数据版本号,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
在一种可能的设计中,轮式移动设备可以是自动驾驶车辆,自动驾驶车辆可以为轿车、卡车、摩托车、公共汽车、船、飞机、直升飞机、割草机、娱乐车、游乐场车辆、施工设备、电车、高尔夫球车、火车、和手推车等,本申请实施例不做特别的限定。
在本申请上述实施方式中,阐述了轮式移动设备可以是自动驾驶车辆,具备可实现性。
本申请实施例第二方面还提供一种数据的存储方法,应用于轮式移动设备,该轮式移动设备包括第一节点和第二节点,该方法包括:首先,第二节点作为备节点,通过运行在第二节点上的第二进程接收与第二进程对应的第二APP发送的第二请求,其中,该第二进程是指用于处理数据存储相关的进程,该第二APP用于获取轮式移动设备周围环境的感知信息(如,可通过部署在轮式移动设备上的摄像头、雷达等传感器获得感知信息),并基于感知信息生成对数据的操作指令(即第二请求),该第二请求经由第二APP向与该第二APP对应的第二节点上运行的第二进程发送。在本申请实施例中,该第二请求是用于指示向存储模块(如,硬盘、内存等)写目标数据(可称为第三数据)。在本申请实施例中,由于第二节点是作为备节点使用,因此,第二节点接收到对应的第二APP发送的第二请求,需要把第二请求转发给作为主节点的第一节点进行处理。具体地,第一节点通过运行在第一节点上的第一进程接收运行在第二节点上的第二进程转发的第二请求。第一节点获取到第一APP发送的第二请求,根据该第二请求,就可通过第一进程调用第一进程上的第一数据库引擎向第一节点上的第一存储模块写第三数据。这里需要注意的是,第二请求是用于指示向存储模块写第三数据,并没有限定是向哪个存储模块写第三数据,例如,若是第一节点通过第一进程调用第一数据库引擎,那么就是根据该第二请求向第一节点上的第一存储模块写该第三数据。又例如,若是第二节点通过第二进程调用第二数据库引擎,那么就是根据该第二请求向第二节点上的第二存储模块写该第三数据。在向第一存储模块写第三数据的操作执行已完成的情况下,第一节点会通过第一进程向第二节点发送第三响应消息,该第三响应消息就用于指示第二节点根据第二请求,通过第二进程调用第二进程上的第二数据库引擎向第二节点上的第二存储模块写第三数据。第二节点接收到第一节点通过第一进程发送的第二请求后,将通过调用第二进程上的第二数据库引擎对第二节点上的第二存储模块执行该第三数据的写操作,这样第二节点上的第二存储模块与第一存储模块更新后的 数据就能保持完全一致。其中,与第一进程类似,该第二进程也是指用于处理数据存储相关的进程。第二节点通过调用第二进程上的第二数据库引擎向第二节点上的第二存储模块写第三数据的操作完成的情况下,就会通过第二进程向第一节点发送第四响应消息,该第四响应消息就用于指示向第二存储模块写第三数据的操作已完成。当第一节点通过第一进程在预设时长内接收到第二节点通过第二进程发送的第四响应消息,那么第一节点通过第一进程向第二节点发送第五响应消息,该第五响应消息就用于指示向第一存储模块写第三数据的操作和向第二存储模块写第三数据的操作均已完成。第二节点通过第二进程接收到第一节点通过第一进程发送的第五响应消息,将会将该第五响应消息向第二应用程序转发,这样第二应用程序也就可以感知到第一节点和第二节点均完成了对第三数据的写操作。
在本申请上述实施方式中,在轮式移动设备(如,自动驾驶车辆、智能汽车、网联汽车等)上仅部署2个节点的前提下,且向第二节点(作为备节点)发送第二请求的APP与第一节点上的第一进程为运行在不同OS的情况时,通过上述强一致性(即在任意时刻下,系统里第一节点设备和第二节点设备中的数据都是一样的)的数据同步流程实现数据的实时备份,即保障了数据的可靠性,同时相比于现有技术中至少部署3个节点的方式,本申请部署2个节点的方式又降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
在第一方面的一种可能的设计中,如果第一节点故障,那么第二节点会立即取代第一节点继续提供服务,并缓存数据,该缓存的数据就为第二节点取代第一节点作为主节点后第二存储模块内更新的数据,并且在第一节点恢复正常使用后,通过第二进程将该缓存的数据向第一节点发送,使得第一节点通过调用第一数据库引擎将该缓存的数据更新至自身的第一存储模块,从而实现两个节点之间的数据同步。
在本申请上述实施方式中,若第一节点故障(第一节点作为主节点,第二节点作为备节点),则第二节点立即升为主节点,取代第一节点进行服务,从而保障数据存储服务系统的可用性,数据不丢失。
在第一方面的一种可能的设计中,第二节点取代第一节点继续提供服务具体可以包括以下几种取代方式:a、第二节点取代第一节点提供服务,直至第一节点恢复正常使用,也就是说,等第一节点恢复了正常使用,第二节点就不再取代,第二节点依然作为备节点工作,一般这种情况是第一节点是高配,第二节点是低配,即第一节点与第二节点的硬件配置不同,主数据库引擎运行在高配的第一节点上,当高配的第一节点故障,则可以短期内由低配的第二节点取代第一节点提供服务,等第一节点恢复正常使用后,再由第一节点继续作为主节点提供服务,这种设置高低配置节点的方式也是为了节约成本。b、第二节点始终取代第一节点提供服务,也就是说,第二节点取代第一节点作为主节点提供服务后,不管后续第一节点是否恢复正常使用,第二节点就一直作为主节点提供服务,若第一节点恢复正常使用后,就一直作为备节点使用,一般这种一直取代的方式是第一节点和第二节点的硬件配置没有差别的情况。
在本申请上述实施方式中,具体阐述了当第一节点故障,第二节点取代第一节点提供服务的两种方式,具备灵活性和可选择性。
在第一方面的一种可能的设计中,对第一节点故障的判断具体可以包括如下几种方式:a、第一节点向第二节点发送的心跳异常,例如,假设第一节点向第二节点发送心跳的频率正常是300毫秒一次,由于第一节点向第二节点发送心跳的目的就是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第二节点都没有收到第一节点发送的心跳,那么第二节点就认为第一节点故障。b、第一节点向第二节点主动发送故障通知消息,例如,当第一节点上运行的第一进程上的软件正常或异常退出,则第一节点会主动向第二节点发送故障通知消息;又例如,当第一节点上监控第一进程的第三进程监控到第一进程异常时,则第一节点也会向第二节点主动发送故障通知消息;还例如,第一节点的通信异常,那么第一节点也会主动向第二节点发送故障通知消息,具体此次不做举例示意。
在本申请上述实施方式中,上述两种感知故障的方式都是为了使得第二节点能够快速感知到第一节点出现了故障,缩短了故障检测时延。其中,第二种故障感知的方式相比第一种故障感知的方式,能够让第二节点更快感知到第一节点出现了故障,而不必等待预设时长第二节点没有收到第一节点的心跳才确定第一节点故障,从而进一步缩短了故障检测时延。
在第一方面的一种可能的设计中,如果第二节点故障,那么第一节点可以将对端设置为异常状态,而第一节点作为主节点继续提供服务,并缓存数据(即第二数据),该缓存的数据就为第一节点通过第一进程向第一存储模块写第一数据的操作后得到的数据,并且在第二节点恢复正常使用后,通过第一进程将该缓存的数据向第二节点发送,使得第二节点通过调用第二数据库引擎将该缓存的数据更新至自身的第二存储模块,从而实现两个节点之间的数据同步。
在本申请上述实施方式中,阐述了如果是第二节点故障,那么第一节点将作为主节点继续提供服务,并且在第二节点恢复正常使用后,通过第一进程将该缓存的第二数据向第二节点发送,使得第二节点通过调用第二数据库引擎将该缓存的第二数据更新至自身的第二存储模块,从而实现两个节点之间的数据同步,具备可实现性。
在第一方面的一种可能的设计中,对第二节点故障的判断具体也可以包括如下几种方式:a、第二节点向第一节点发送的心跳异常,例如,假设第二节点向第一节点发送心跳的频率正常是300毫秒一次,由于第二节点向第一节点发送心跳的目的也是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第一节点都没有收到第二节点发送的心跳,那么第一节点就认为第二节点故障。b、第二节点向第一节点主动发送故障通知消息,例如,当第二节点上运行的第二进程上的软件正常或异常退出,则第二节点会主动向第一节点发送故障通知消息;又例如,当第二节点上监控第二进程的第四进程监控到第二进程异常时,则第二节点也会向第一节点主动发送故障通知消息;还例如,第二节点的通信异常,那么第二节点也会主动向第一节点发送故障通知消息,具体此次不做举例示意。
在本申请上述实施方式中,上述两种感知故障的方式都是为了使得第一节点能够快速感知到第二节点出现了故障,缩短了故障检测时延。其中,第二种故障感知的方式相比第一种故障感知的方式,能够让第一节点更快感知到第二节点出现了故障,而不必等待预设时长第一节点没有收到第二节点的心跳才确定第二节点故障,从而进一步缩短了故障检测 时延。
在第二方面的一种可能的设计中,在一种可能的设计中,该第一存储模块包括第一内存数据库;和/或,该第二存储模块包括第二内存数据库。
在本申请上述实施方式中,阐述了第一存储模块和第二存储模块可以是内存数据库,相对于传统的硬盘,内存数据库的读取速度更快,尤其是在智能驾驶领域,由于车辆的运行状态数据、感知数据、中间计算结果、SOA信息等数据需频繁被读取,若第一存储模块和第二存储模块为硬盘,那么反复频繁的读/写数据会缩短硬盘寿命,从而导致车用器件的更换成本提高。
在第二方面的一种可能的设计中,第一节点通过第一进程调用第一数据库引擎向第一节点上的第一存储模块写第三数据的操作执行完成之后,还可以进一步通过第一进程记录(如,更新)当前的数据版本号,该当前数据版本号可称为第一数据版本号,其用于表示第一存储模块数据更新后的数据版本(可称为第一数据版本),所述更新后的数据版本为向第一存储模块写第三数据的操作已完成的情况下的数据版本。例如,假设此次数据写操作之前的数据版本号为X,那么当前对第三数据写操作完成后,当前第一存储模块内的数据是最新版本的,则可记录数据版本号为X+1。
在本申请上述实施方式中,通过第一进程执行完第三数据的写操作后,还要记录此次更新后的数据版本号,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
在第二方面的一种可能的设计中,第二节点通过第二进程调用第二数据库引擎向第二节点上的第二存储模块写第三数据的操作执行完成之后,也可以进一步通过第二进程记录(如,更新)当前的数据版本号,该当前数据版本号可称为第二数据版本号,其用于表示第二存储模块数据更新后的数据版本(可称为第二数据版本),所述更新后的数据版本为向第二存储模块写第三数据的操作已完成的情况下的数据版本。例如,假设此次数据写操作之前的数据版本号为Y,那么当前对第三数据写操作完成后,当前第二存储模块内的数据是最新版本的,则可记录数据版本号为Y+1,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。这里需要注意的是,由于每次节点对数据的写操作,都会同步更新第一节点上的第一存储模块和第二节点上的第二存储模块,因此实质上X和Y的取值相同。
在本申请上述实施方式中,通过第二进程执行完第三数据的写操作后,也要记录此次更新后的数据版本号,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
在一种可能的设计中,轮式移动设备可以是自动驾驶车辆,自动驾驶车辆可以为轿车、卡车、摩托车、公共汽车、船、飞机、直升飞机、割草机、娱乐车、游乐场车辆、施工设备、电车、高尔夫球车、火车、和手推车等,本申请实施例不做特别的限定。
在本申请上述实施方式中,阐述了轮式移动设备可以是自动驾驶车辆,具备可实现性。
本申请实施例第三方面提供一种数据存储服务系统,该系统部署于轮式移动设备,该系统包括第一节点和第二节点,所述第一节点和第二节点具有实现上述第一方面或第一方面任意一种可能实现方式的方法的功能。
本申请实施例第四方面还提供一种数据存储服务系统,该系统部署于轮式移动设备,该系统包括第一节点和第二节点,所述第一节点和第二节点具有实现上述第二方面或第二 方面任意一种可能实现方式的方法的功能。
本申请实施例第五方面还提供一种节点,该节点应用于数据存储服务系统,所述系统部署于轮式移动设备,所述系统包括第一节点和第二节点,所述节点作为第一节点,具有实现上述第一方面或第一方面任意一种可能实现方式的方法中第一节点的功能。该功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。该硬件或软件包括一个或多个与上述功能相对应的模块。
本申请实施例第六方面还提供一种节点,该节点应用于数据存储服务系统,所述系统部署于轮式移动设备,所述系统包括第一节点和第二节点,所述节点作为第一节点,具有实现上述第二方面或第二方面任意一种可能实现方式的方法中第一节点的功能。该功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。该硬件或软件包括一个或多个与上述功能相对应的模块。
本申请实施例第七方面提供一种数据存储服务系统,该系统部署于轮式移动设备,该系统包括第一节点和第二节点,第一节点包括第一处理器和第一存储器,第一处理器与第一存储器耦合,第二节点包括第二处理器和第二存储器,第二处理器与第二存储器耦合;其中,第一存储器和第二存储器分别用于存储第一程序和第二程序,第一处理器用于调用该第一存储器中存储的第一程序以执行本申请实施例第一方面或第一方面任意一种可能实现方式中由第一节点所执行的步骤,或,第一处理器用于调用该第一存储器中存储的第一程序以执行本申请实施例第二方面或第二方面任意一种可能实现方式中由第一节点所执行的步骤;第二处理器用于调用该第二存储器中存储的第二程序以执行本申请实施例第一方面或第一方面任意一种可能实现方式中由第二节点所执行的步骤,或,第二处理器用于调用该第二存储器中存储的第二程序以执行本申请实施例第二方面或第二方面任意一种可能实现方式中由第二节点所执行的步骤。
本申请第八方面提供一种计算机可读存储介质,该计算机可读存储介质中存储有指令,当其在计算机上运行时,使得计算机可以执行上述第一方面或第一方面任意一种可能实现方式的方法,或,使得计算机可以执行上述第二方面或第二方面任意一种可能实现方式的方法。
本申请实施例第九方面提供了一种计算机程序产品,当其在计算机上运行时,使得计算机可以执行上述第一方面或第一方面任意一种可能实现方式的方法,或,使得计算机可以执行上述第二方面或第二方面任意一种可能实现方式的方法。
附图说明
图1为本申请实施例提供的数据存储服务系统的部署方式的一种示意图;
图2为本申请实施例提供的数据存储服务系统的部署方式的另一示意图;
图3为本申请实施例提供的数据的存储方法的一种流程示意图;
图4为本申请实施例提供的数据的存储方法的另一流程示意图;
图5为本申请实施例提供的数据的存储方法的另一流程示意图;
图6为本申请实施例提供的数据的存储方法的另一流程示意图;
图7为本申请实施例提供的数据存储服务系统的一种结构示意图;
图8为本申请实施例提的第一节点的一种结构示意图;
图9为本申请实施例提的第一节点的另一结构示意图;
图10为本申请实施例提供的数据存储服务系统的一种结构示意图。
具体实施方式
本申请实施例提供了一种数据的存储方法及设备,可以用于在轮式移动设备(如,自动驾驶车辆)上部署2个节点设备的前提下,通过强一致性(即在任意时刻下,系统里第一节点设备和第二节点设备中的数据都是一样的)的数据同步流程实现数据的实时备份,保障了数据的可靠性,同时相比于现有技术中至少部署3个节点设备的方式,本申请部署2个节点设备的方式降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
本申请实施例涉及了许多关于数据存储相关知识,为了更好地理解本申请实施例的方案,下面先对本申请实施例可能涉及的相关术语和概念进行介绍。应理解的是,相关的概念解释可能会因为本申请实施例的具体情况有所限制,但并不代表本申请仅能局限于该具体情况,在不同实施例的具体情况可能也会存在差异,具体此处不做限定。
(1)轮式移动设备
是集环境感知、动态决策与规划、行为控制与执行等多功能于一体的综合系统,也可称为轮式移动机器人或轮式智能体,例如,可以是轮式施工设备、自动驾驶车辆、辅助驾驶车辆等,只要是具备轮式可移动的设备就都称为本申请所述的轮式移动设备。为便于理解,在本申请以下实施例中,均以轮式移动设备为自动驾驶车辆为例进行说明,自动驾驶车辆可以为轿车、卡车、摩托车、公共汽车、船、飞机、直升飞机、割草机、娱乐车、游乐场车辆、施工设备、电车、高尔夫球车、火车、和手推车等,本申请实施例不做特别的限定。
(2)节点
节点也可称为计算机网络节点或物理节点,在数据通信中,一个物理节点可以是数据电路端接设备,如调制解调器、集线器、桥接器或交换机;也可以是一个数据终端设备,如数字手机,打印机或主机(例如路由器、工作站或服务器)。在本申请实施例中,该节点特指一个物理的硬件单元,例如,可以是主机。节点也称为节点设备,在本申请实施例中,第一节点设备和第二节点设备均分别简称为第一节点和第二节点。
(3)进程(process)
进程是计算机中的程序关于某数据集合上的运行活动,是系统进行资源分配和调度的基本单位,是操作系统(operating system,OS)结构的基础。在早期面向进程设计的计算机结构中,进程是程序的基本执行实体;在当代面向线程设计的计算机结构中,进程是线程的容器。程序是指令、数据及其组织形式的描述,进程是程序的实体。它可以申请和拥有系统资源,是一个活动的实体。它不只是程序的代码,还包括当前的活动,通过程序计数器的值和处理寄存器的内容来表示。进程的概念主要有两点:第一,进程是一个实体。 每一个进程都有它自己的地址空间,一般情况下,包括文本区域(text region)、数据区域(data region)和堆栈(stack region)。文本区域存储处理器执行的代码;数据区域存储变量和进程执行期间使用的动态分配的内存;堆栈区域存储着活动过程调用的指令和本地变量。第二,进程是一个“执行中的程序”。程序是一个没有生命的实体,只有处理器赋予程序生命时(操作系统执行之),它才能成为一个活动的实体,才被称为进程。
(4)数据库引擎
也可称为数据引擎,数据库引擎是用于存储、处理和保护数据的核心服务。利用数据库引擎可控制访问权限并快速处理事务,从而满足大多数需要处理大量数据的应用程序的要求。使用数据库引擎可创建用于联机事务处理或联机分析处理数据的关系数据库,包括创建用于存储数据的表和用于查看、管理和保护数据安全的数据库对象(如索引、视图和存储过程)。在本申请实施例中,每个进程上都运行有一个轻量级的数据库引擎。
(5)分布式一致性算法
一致性是指数据保持一致,在分布式系统中,可以理解为多个节点中存储模块的数据是一致的。分布性一致性算法是为了解决以下两个问题而存在:1)数据不能只存在单个节点上,否则可能出现单点故障,导致数据丢失;2)多个节点需要保证具有相同的数据,旨在保证数据可靠性。
分布式一致性算法大体可分为两类,一类称为强一致性算法,即系统保证改变提交以后立即改变集群中各节点内数据的状态,代表算法有paxos算法、raft算法、ZAB算法等;另一类称为弱一致性算法,也可称为最终一致性算法,即系统不保证改变提交以后立即改变集群中各节点内数据的状态,但随着时间的推移各节点内数据的最终状态是一致的,代表算法有Gossip协议。
其中,所谓强一致性,也可称为原子一致性或线性一致性,其具备两个要求:1)任何一次读数据都能读到某个数据的最近一次写的数据;2)系统中各节点所有进程的操作顺序,都和全局时钟下的顺序一致。在本申请实施例中,简言之,就是在任意时刻下,系统里第一节点和第二节点中的数据都是一样的。
下面以raft算法为例,且以图1所示的部署方式为例,介绍如何在分布式系统(即集群)中的多个节点中实现数据同步,这里需要说明的是,前提是每个节点上都运行一个对应的进程,所有的过程均通过各自节点运行的进程实现。
raft算法是对paxos算法的简化和优化,该算法将分布式系统内的各个节点区分为主节点(leader)和备节点(也可称为追随者节点follower),其中,主节点负责发出提案,提案实质为分布式系统的读/写数据的请求,该请求一般由客户端提出,该提案可表示为[提案编号n,提案内容value],备节点负责为接收到的主节点发出的提案进行表决,表决的原则是:不同意比自己以前接收过的提案编号要小的提案,其他提案都同意,例如备节点a以前给m号提案表决过,那么再收到小于等于m号的提案时就直接投反对票。当主节点出现故障,此时所有的备节点自动成为主节点候选者(candidate),来争夺成为新的主节点。
下面简单介绍该raft算法的基本逻辑:首先是主节点选举过程,每个备节点都持有一个定时器,当定时器时间到了而集群中仍然没有主节点,备节点将声明自己是主节点候选 者,并参与主节点选举,同时将选举消息发给其他备节点来争取它们的投票,若其他节点长时间没有响应该主节点候选者,则该主节点候选者将重新发送选举信息,当集群中其他节点收到该选举消息,将给该主节点候选者进行表决,获得过半数节点同意的主节点候选者将成为第M任主节点(M任是最新的任期,前提是第M-1任的主节点故障或失去主节点资格),在任期内的主节点会不断发送心跳给其他节点(即M任的备节点)证明自己还活着,其他节点收到心跳以后就清空自己的计时器并回复主节点的心跳,该机制用于保证其他节点不会在主节点任期内参加主节点选举。当主节点出现故障而导致主节点失联,没有接收到心跳的其他节点将准备成为主节点候选者进入下一轮主节点选举。若出现两个主节点候选者同时向其他节点发送了选举消息并获得了相同的支持票数,那么这两个主节点候选者将随机推迟一段时间后再向其他节点发出选举消息,这保证了再次发送选取消息以后不冲突。其次,是各个节点数据状态的复制过程,在图1的这种部署方式中,主节点负责接收来自应用程序的提案(也就是读/写数据的请求),提案内容将包含在主节点发出的下一个心跳中,备节点接收到主节点的心跳以后,若备节点同意该提案,则回复主节点的心跳,主节点接收到过半数的备节点的回复以后确认该提案通过,之后将该提案写入自己的存储空间,并回复应用程序,同时,主节点通知备节点确认提案并写入各自的存储空间,最终分布式系统内的所有节点都拥有相同的数据。
上述仅是以raft算法为例对分布式一致性算法的基本逻辑进行示意,其他算法的逻辑是类似的,即要实现分布式系统内各个节点的数据同步,均需要超过半数的节点同意才行,也就是说,每个分布式系统中至少需要部署3个节点才行(超半数同意的原则需要奇数个节点)。
下面结合附图,对本申请的实施例进行描述。本领域普通技术人员可知,随着技术的发展和新场景的出现,本申请实施例提供的技术方案对于类似的技术问题,同样适用。
本申请实施例提供了一种数据的存储方法,该方法可以应用于轮式移动设备(如,自动驾驶车辆、智能汽车、网联汽车等),该轮式移动设备包括两个节点,分别为第一节点和第二节点,在本申请实施例中,第一APP与第一进程运行在同一个OS上,一般是指运行在同一个节点(即第一节点)上,其中,第一节点作为主节点,第二节点作为备节点,具体请参阅图3,图3为本申请实施例提供的数据的存储方法的一种流程示意图,可以包括如下步骤:
301、第一节点通过运行在第一节点上的第一进程接收第一APP发送的第一请求。
首先,第一节点作为主节点,通过运行在第一节点上的第一进程接收第一APP发送的第一请求,其中,该第一进程是指用于处理数据存储相关的进程,该第一APP用于获取轮式移动设备周围环境的感知信息(如,可通过部署在轮式移动设备上的摄像头、雷达等传感器获得感知信息),并基于感知信息生成对数据的操作指令(即第一请求),该第一请求经由第一APP向与该第一APP对应的第一节点上运行的第一进程发送。在本申请实施例中,该第一请求是用于指示向存储模块(如,硬盘、内存等)写目标数据(可称为第一数据)。
需要说明的是,由于对数据的操作一般包括两种方式:读数据和写数据,其中,读数据是指读取存储模块中的数据(即“查”),APP读取存储模块的数据均通过主节点进行读 取,由于读数据不会改变存储模块中的数据,因此本申请实施例方案不涉及对数据的读操作;而写数据是指对存储模块中的数据进行增加、删除或修改等操作(即“增”、“删”、“改”),每个节点上运行的APP都可要求对存储模块中的数据进行增加、删除或修改等操作,这些操作都会使得存储模块内的数据被改变,因此,本申请实施例方案中,APP对数据的操作指令只包括对存储模块执行数据的写操作,即第一请求是用于指示向存储模块写第一数据。其中,若写数据为“增”数据,那么该第一数据就为需要增加进存储模块的数据,其包含在第一请求中由第一APP向第一节点发送;若写数据为“删”数据,那么该第一数据就为存储模块内待删除的对应数据,用于指示该第一数据的相关标识信息就包含在第一请求中由第一APP向第一节点发送;若写数据为“改”数据,那么该第一数据就为存储模块内待修改的对应数据,该第一数据对应的具体修改信息和相关标识信息就也包含在第一请求中由第一APP向第一节点发送。
还需要说明的是,在本申请的一些实施方式中,该第一APP与第一进程对应,具体地,与第一进程对应的第一APP一般是指第一进程与该第一APP运行在同一个OS上,或者,第一进程与第一APP均运行在第一节点上,具体此处不做限定。
还需要说明的是,在本申请的一些实施方式中,该第一进程是指用于处理数据存储相关的进程,在该第一节点上,可能还会同时运行其他的进程,不同的进程所处理的对象不同,一般来说,一个节点上只会运行一个用于处理数据存储相关的进程。
302、响应于第一请求,并基于第一进程上的第一数据库引擎,第一节点通过第一进程向第一节点上的第一存储模块写第一数据。
第一节点获取到第一APP发送的第一请求,根据该第一请求,就可通过第一进程调用第一进程上的第一数据库引擎,向第一节点上的第一存储模块写入该第一数据。
这里需要注意的是,第一请求是用于指示向存储模块写第一数据,并没有限定是向哪个存储模块写第一数据,例如,若是第一节点通过第一进程调用第一数据库引擎,那么就是根据该第一请求向第一节点上的第一存储模块写该第一数据。又例如,若是第二节点通过第二进程调用第二数据库引擎,那么就是根据该第一请求向第二节点上的第二存储模块写该第一数据。
需要说明的是,在本申请的一些实施方式中,第一节点通过第一进程调用第一数据库引擎向第一节点上的第一存储模块写第一数据的操作执行完成之后,还可以进一步通过第一进程记录(如,更新)当前的数据版本号,该当前数据版本号可称为第一数据版本号,其用于表示第一存储模块数据更新后的数据版本(可称为第一数据版本),该更新后的数据版本为向第一存储模块写第一数据的操作已完成的情况下的数据版本。例如,假设此次数据写操作之前的数据版本号为P,那么当前对第一数据写操作完成后,当前第一存储模块内的数据是最新版本的,则可记录数据版本号为P+1,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
303、在向第一存储模块写第一数据的操作已完成的情况下,第一节点通过第一进程向第二节点发送第一请求。
在向第一存储模块写第一数据的操作已完成的情况下,第一节点才会通过第一进程将 第一请求向所述第二节点发送,将第一请求向第二节点发送的目的在于:使得第二节点根据该第二请求向第二节点上的第二存储模块也写入该第一数据。
304、第二节点响应于接收到的第一请求,基于第二进程上的第二数据库引擎,通过第二进程向第二节点上的第二存储模块写该第一数据。
第二节点接收到第一节点通过第一进程发送的第一请求后,将通过调用第二进程(运行在第二节点)上的第二数据库引擎,向第二节点上的第二存储模块写该第一数据,这样第二节点上的第二存储模块与第一存储模块更新后的数据就能保持完全一致。其中,与第一进程类似,该第二进程也是指用于处理数据存储相关的进程。
需要说明的是,在本申请的一些实施方式中,与第一节点类似,第二节点通过第二进程调用第二数据库引擎向第二节点上的第二存储模块写第一数据的操作执行完成之后,也可以进一步通过第二进程记录(如,更新)当前的数据版本号,该当前数据版本号可称为第二数据版本号,其用于表示第二存储模块数据更新后的数据版本(可称为第二数据版本),所述更新后的数据版本为向第二存储模块写第一数据的操作已完成的情况下的数据版本。例如,假设此次数据写操作之前的数据版本号为Q,那么当前对第一数据写操作完成后,当前第二存储模块内的数据是最新版本的,则可记录数据版本号为Q+1,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。这里需要注意的是,由于每次节点对数据的写操作,都会同步更新第一节点上的第一存储模块和第二节点上的第二存储模块,因此实质上P和Q的取值相同。
305、第一节点通过第一进程在预设时长内接收到第二节点通过第二进程发送的第一响应消息。
第二节点通过调用第二进程上的第二数据库引擎向第二节点上的第二存储模块写第一数据的操作执行完成的情况下,就会通过第二进程向第一节点发送第一响应消息,该第一响应消息就用于指示向第二存储模块写第一数据的操作已完成。
306、第一节点通过第一进程向第一应用程序发送第二响应消息。
当第一节点通过第一进程在预设时长内接收到第二节点通过第二进程发送的第一响应消息,那么第一节点就可确定第二节点也完成了对第一数据的写操作,那么第一节点将会通过第一进程向第一应用程序发送第二响应消息,该第二响应消息就用于指示向第一存储模块写第一数据的操作和向第二存储模块写第一数据的操作均已完成。
需要说明的是,在本申请的一些实施方式中,第一存储模块和第二存储模块均可以是硬盘,也可以是内存数据库,具体此处不做限定。优选地,在本申请实施例中,第一存储模块和第二存储模块都可以是内存数据库,如,第一存储模块为第一内存数据库,第二存储模块为第二内存数据库,这是因为:相对于传统的硬盘,内存数据库的读取速度更快,尤其是在智能驾驶领域,由于车辆的运行状态数据、感知数据、中间计算结果、SOA信息等数据需频繁被读取,若第一存储模块和第二存储模块为硬盘,那么反复频繁的读/写数据会缩短硬盘寿命,从而导致车用器件的更换成本提高。
在本申请上述实施方式中,在轮式移动设备(如,自动驾驶车辆、智能汽车、网联汽车等)上仅部署2个节点的前提下,且向第一节点(作为主节点)发送第一请求的APP与 第一节点上的第一进程均为运行在同一个OS的情况时,通过上述强一致性(即在任意时刻下,系统里第一节点设备和第二节点设备中的数据都是一样的)的数据同步流程实现数据的实时备份,即保障了数据的可靠性,同时相比于现有技术中至少部署3个节点的方式,本申请部署2个节点的方式降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
为便于理解上述图3所对应的实施方式,下面从信息流向的角度,对上述步骤301至步骤306的过程进行示意,具体请参阅图4,图4为本申请实施例提供的数据的存储方法的另一流程示意图,其中,第一节点作为主节点,第二节点作为备节点,均部署在轮式移动设备上,第一节点上具有第一存储模块,用于存储数据,第一节点上还运行有APP1(实际可以同时运行多个APP,图4仅以一个APP为例进行示意)和第一进程S1,第一进程S1上包括有第一数据库引擎(即图4中的引擎1),且在引擎1上设置有控制组件HA1,控制组件HA1就用于实现上述步骤301至步骤306所述相关信息的传递过程;类似的,第二节点上具有第二存储模块,用于存储数据,第二节点上还运行有APP2(实际可以同时运行多个APP,图4仅以一个APP为例进行示意)和第二进程S2,第二进程S2上包括有第二数据库引擎(即图4中的引擎2),且在引擎2上设置有控制组件HA2,控制组件HA2就用于实现上述步骤301至步骤306相关各种信息的传递过程。具体地,主节点上的APP1发起的写数据的过程可以如下所示:
步骤①:第一节点上的APP1通过服务接口C1向第一进程上的控制组件HA1发送请求a,该请求a用于指示向存储模块写数据A。
步骤②:第一节点作为主节点,调用引擎1向第一存储模块写数据A,并记录当前的数据版本号M。
步骤③:当数据A被成功写入第一存储模块后,由控制组件HA1向第二节点的控制组件HA2发送该第一请求,该第一请求一般包含在第一节点向第二节点发送的心跳内(也可以单独发送,此处不做限定)。
步骤④:第二节点作为备节点,调用引擎2向第二存储模块写数据A,同样记录当前的数据版本号M。
步骤⑤:若数据A被成功写入第二存储模块,则第二节点会通过控制组件HA2向第一节点的控制组件HA1发送一个响应消息(即上述所述的第一响应消息)。
步骤⑥:当第一节点在预设时长内(如,3s内)接收到该第一响应消息,说明数据A被成功写入第二存储模块,那么第一节点就再通过控制组件HA1向APP1返回执行结果,该执行结果即为上述所述的第二响应消息,用于告知APP1第一节点和第二节点均对数据A完成了写操作。
需要说明的是,在本申请的一些实施方式中,引擎1和引擎2的选择完全独立,互相不强依赖,也就是说,引擎1和引擎2即可以是同构的数据库引擎,也可以是异构的数据存储引擎,具体此处不做限定。因为控制组件HA1和控制组件HA2中集成有一个适配模块,用于适配各种数据库引擎,即通过各自的控制组件屏蔽掉数据库引擎的差异。由于不同数据库引擎的资源消耗不同,性能表现也不同。数据库引擎可以运行在差异化的硬件上 (比如一个高配,一个低配)。若引擎1和引擎2是采用异构的数据库引擎,那么,性能表现好的引擎一般运行在高配的主节点上,性能表现弱一些的引擎则运行在低配的备节点上。这样差异化的部署也是基于降低成本考虑。
在本申请上述实施方式中,在轮式移动设备(如,自动驾驶车辆)上仅部署2个节点的前提下,且向第一节点(作为主节点)发送请求a的APP1与第一节点上的第一进程S1均为运行在同一个OS的情况时,通过上述强一致性的数据同步流程实现数据的实时备份,即保障了数据的可靠性,同时又降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
需要说明的是,在上述图3和图4对应的实施例中,向第一节点发送第一请求的APP与第一进程为均运行在同一个OS上的情况,那么在发起请求的APP与第一进程不运行在同一个OS上的情况时,第一节点上的第一存储模块与第二节点上的第二存储模块数据同步的方式也不一样,下面详细进行论述,具体请参阅图5,同样地,该方法应用于轮式移动设备(如,自动驾驶车辆),该轮式移动设备包括两个节点,分别为第一节点(即主节点)和第二节点(即备节点),在本申请实施例中,第二APP与第一进程运行在不同的OS上,一般是指运行在不同节点(即第一节点)上,在本申请实施例中,第二APP运行在第二节点上,图5为本申请实施例提供的数据的存储方法的另一流程示意图,可以包括如下步骤:
501、第二节点通过运行在第二节点上的第二进程接收第二APP发送的第二请求。
首先,第二节点作为备节点,通过运行在第二节点上的第二进程接收第二APP发送的第二请求,其中,该第二进程是指用于处理数据存储相关的进程,该第二APP用于获取相关信息,该相关信息可以是轮式移动设备各个系统运行的数据,也可以是轮式移动设备周围环境的感知信息(如,可通过部署在轮式移动设备上的摄像头、雷达等传感器获得感知信息),具体此处不做限定。假设该第二APP获取的是感知信息,那么该第二APP可基于感知信息生成对数据的操作指令(即第二请求),该第二请求经由第二APP向与该第二APP对应的第二节点上运行的第二进程发送。在本申请实施例中,该第二请求是用于指示向存储模块(如,硬盘、内存等)写目标数据(可称为第三数据)。
需要说明的是,本申请实施例方案中,由于APP对数据的操作指令只包括对存储模块执行数据的写操作,即第二请求是用于指示向存储模块写第三数据。其中,若写数据为“增”数据,那么该第三数据就为需要增加进存储模块的数据,其包含在第二请求中由第二APP向第二节点发送;若写数据为“删”数据,那么该第三数据就为存储模块内待删除的对应数据,用于指示该第三数据的相关标识信息就包含在第二请求中由第二APP向第二节点发送;若写数据为“改”数据,那么该第三数据就为存储模块内待修改的对应数据,该第三数据对应的具体修改信息和相关标识信息就也包含在第二请求中由第二APP向第二节点发送。
还需要说明的是,在本申请的一些实施方式中,该第二进程是指用于处理数据存储相关的进程,在该第二节点上,可能还会同时运行其他的进程,不同的进程所处理的对象不同,一般来说,一个节点上只会运行一个用于处理数据存储相关的进程。
502、第一节点通过运行在第一节点上的第一进程接收运行在第二节点上的第二进程转 发的第二请求。
在本申请实施例中,由于第二节点是作为备节点使用,因此,第二节点接收到对应的第二APP发送的第二请求,需要把第二请求转发给作为主节点的第一节点进行处理。具体地,第一节点通过运行在第一节点上的第一进程接收运行在第二节点上的第二进程转发的第二请求。
503、响应于第二请求,并基于第一进程上的第一数据库引擎,第一节点通过第一进程向第一节点的第一存储模块写第三数据。
第一节点获取到第一APP发送的第二请求,根据该第二请求,就可通过第一进程调用第一进程上的第一数据库引擎向第一节点上的第一存储模块写第三数据。
这里需要注意的是,第二请求是用于指示向存储模块写第三数据,并没有限定是向哪个存储模块写第三数据,例如,若是第一节点通过第一进程调用第一数据库引擎,那么就是根据该第二请求向第一节点上的第一存储模块写该第三数据。又例如,若是第二节点通过第二进程调用第二数据库引擎,那么就是根据该第二请求向第二节点上的第二存储模块写该第三数据。
需要说明的是,在本申请的一些实施方式中,第一节点通过第一进程调用第一数据库引擎向第一节点上的第一存储模块写第三数据的操作执行完成之后,还可以进一步通过第一进程记录(如,更新)当前的数据版本号,该当前数据版本号可称为第一数据版本号,其用于表示第一存储模块数据更新后的数据版本(可称为第一数据版本),该更新后的数据版本为向第一存储模块写第三数据的操作已完成的情况下的数据版本。例如,假设此次数据写操作之前的数据版本号为X,那么当前对第三数据写操作完成后,当前第一存储模块内的数据是最新版本的,则可记录数据版本号为X+1,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
504、在向第一存储模块写第三数据的操作已完成的情况下,第一节点通过第一进程向第二节点发送第三响应消息。
在向第一存储模块写第三数据的操作已完成的情况下,第一节点会通过第一进程向第二节点发送第三响应消息,该第三响应消息就用于指示第二节点根据第二请求,通过第二进程调用第二进程上的第二数据库引擎向第二节点上的第二存储模块写第三数据。
需要说明的是,在本申请的一些实施方式中,可以将第二请求包含在该第三响应消息中由第一节点上的第一进程向第二节点上的第二进程发送,这样做的好处在于:虽然第二节点上一开始已接收到第二APP发送的第二请求,但在实际处理过程中,第二节点在同一时间可能需要处理很多不同进程上的不同请求,每个请求并不会一直保留在节点上,因此为方便管理,第二请求可以在转发给第一节点后,第二节点就删除该第二请求,这种情况下,第一节点通过第一进程向第二节点发送的第三响应消息中就可包含该第二请求,使得第二节点根据该第二请求向第二节点上的第二存储模块也写入第三数据。
还需要说明的是,在本申请的一些实施方式中,该第三响应消息不包括第二请求(前提是第二节点上还存在有该第二请求),这种情况下,该第三响应消息仅用于指示第二节点根据自身已有的第二请求,通过第二进程调用第二数据库引擎向第二存储模块写第三数据。
505、根据第三响应消息,第二节点响应于第二请求,通过调用第二进程上的第二数据库引擎向第二节点上的第二存储模块写第三数据。
第二节点接收到第一节点通过第一进程发送的第二请求后,将通过调用第二进程上的第二数据库引擎向第二节点上的第二存储模块写该第三数据,这样第二节点上的第二存储模块与第一存储模块更新后的数据就能保持完全一致。其中,与第一进程类似,该第二进程也是指用于处理数据存储相关的进程。
需要说明的是,在本申请的一些实施方式中,与第一节点类似,第二节点通过第二进程调用第二数据库引擎向第二节点上的第二存储模块写第三数据的操作执行完成之后,也可以进一步通过第二进程记录(如,更新)当前的数据版本号,该当前数据版本号可称为第二数据版本号,其用于表示第二存储模块数据更新后的数据版本(可称为第二数据版本),所述更新后的数据版本为向第二存储模块写第三数据的操作已完成的情况下的数据版本。例如,假设此次数据写操作之前的数据版本号为Y,那么当前对第三数据写操作完成后,当前第二存储模块内的数据是最新版本的,则可记录数据版本号为Y+1,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。这里需要注意的是,由于每次节点对数据的写操作,都会同步更新第一节点上的第一存储模块和第二节点上的第二存储模块,因此实质上X和Y的取值相同。
506、第一节点通过第一进程在预设时长内接收到第二节点通过第二进程发送的第四响应消息。
第二节点通过调用第二进程上的第二数据库引擎向第二节点上的第二存储模块写第三数据完成的情况下,就会通过第二进程向第一节点发送第四响应消息,该第四响应消息就用于指示向第二存储模块写第三数据已完成。
507、第一节点通过第一进程向第二节点发送第五响应消息。
当第一节点通过第一进程在预设时长内接收到第二节点通过第二进程发送的第四响应消息,那么第一节点通过第一进程向第二节点发送第五响应消息,该第五响应消息就用于指示向第一存储模块写第三数据的操作和向第二存储模块写第三数据均已完成。
需要说明的是,在本申请的一些实施方式中,第一存储模块和第二存储模块均可以是硬盘,也可以是内存数据库,具体此处不做限定。优选地,在本申请实施例中,第一存储模块和第二存储模块都可以是内存数据库,如,第一存储模块为第一内存数据库,第二存储模块为第二内存数据库,这是因为:相对于传统的硬盘,内存数据库的读取速度更快,尤其是在智能驾驶领域,由于车辆的运行状态数据、感知数据、中间计算结果、SOA信息等数据需频繁被读取,若第一存储模块和第二存储模块为硬盘,那么反复频繁的读/写数据会缩短硬盘寿命,从而导致车用器件的更换成本提高。
508、第二节点通过第二进程向第二应用程序转发第五响应消息。
第二节点通过第二进程接收到第一节点通过第一进程发送的第五响应消息,将会将该第五响应消息向第二应用程序转发,这样第二应用程序也就可以感知到第一节点和第二节点均完成了对第三数据的写操作。
需要注意的是,在本申请实施例中,由于第一节点是主节点,向APP发送的响应消息 均需由第一节点发出,因此,需要由第一节点向第二节点发送第五响应消息,再由第二节点向第二应用程序转发。
还需要注意的是,在本申请实施例中,不管是由第一APP发出的第一请求,还是第二APP发出的第二请求,均需先由第一节点通过第一进程向第一存储模块写数据,在对第一存储模块写操作成功的情况下,再由第二节点通过第二进程向第二存储模块写数据。这是为了保证数据更新的强一致性,例如,假设与第一进程对应的某APP发出请求A,同时又有与第二进程对应的某APP发出请求B,如果两个节点各自都先写入本地,那么第一节点上第一存储模块数据更新的顺序是A→B,而第二节点上第二存储模块数据更新的顺序是B→A,两边存储模块在某时间段内数据是不一致的,且数据版本号也会对应不上。
在本申请上述实施方式中,在轮式移动设备(如,自动驾驶车辆)上仅部署2个节点的前提下,且向第二节点(作为备节点)发送第二请求的APP与第一节点上的第一进程为运行在不同OS的情况时,通过上述强一致性的数据同步流程实现数据的实时备份,即保障了数据的可靠性,同时又降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
为便于理解上述图5所对应的实施方式,下面从信息流向的角度,对上述步骤501至步骤508的过程进行示意,具体请参阅图6,图6为本申请实施例提供的数据的存储方法的另一流程示意图,其中,第一节点作为主节点,第二节点作为备节点,均部署在轮式移动设备上,第一节点上具有第一存储模块,用于存储数据,第一节点上还运行有APP1(实际可以同时运行多个APP,图6仅以一个APP为例进行示意)和第一进程S1,第一进程S1上包括有第一数据库引擎(即图6中的引擎1),且在引擎1上设置有控制组件HA1,控制组件HA1就用于实现上述步骤501至步骤508所述相关信息的传递过程;类似的,第二节点上具有第二存储模块,用于存储数据,第二节点上还运行有APP2(实际可以同时运行多个APP,图6仅以一个APP为例进行示意)和第二进程S2,第二进程S2上包括有第二数据库引擎(即图6中的引擎2),且在引擎2上设置有控制组件HA2,控制组件HA2就用于实现上述步骤501至步骤508相关各种信息的传递过程。具体地,备节点上的APP2发起的写数据的过程可以如下所示:
步骤①:第二节点上的APP2通过服务接口C2向第二进程上的控制组件HA2发送请求b,该请求b用于指示向存储模块写数据B。
步骤②:第二节点作为备节点,通过控制组件HA2将该请求b转发至作为主节点的第一节点进行处理。
步骤③:第一节点作为主节点,通过控制组件HA1接收到第二节点发送的请求b,调用引擎1向第一存储模块写数据B,并记录当前的数据版本号N。
步骤④:当数据B被成功写入第一存储模块后,由控制组件HA1向第二节点的控制组件HA2发送响应消息x(即上述所述的第三响应消息),该响应消息x中可以包含请求b,该响应消息x就用于指示第二节点根据请求b,调用引擎2向第二存储模块写数据B。该响应消息x一般包含在第一节点向第二节点发送的心跳内(也可以单独发送,此处不做限定)。
步骤⑤:第二节点作为备节点,调用引擎2对第二存储模块执行数据B的写操作,同 样记录当前的数据版本号N。
步骤⑥:若数据B被成功写入第二存储模块,则第二节点会通过控制组件HA2向第一节点的控制组件HA1发送响应消息y(即上述所述的第四响应消息)。
步骤⑦:当第一节点在预设时长内(如,3s内)接收到该响应消息y,说明数据B被成功写入第二存储模块,那么第一节点就再通过控制组件HA1向第二节点返回一个响应消息z(即上述所述的第五响应消息),用于告知第二节点双方均对数据B完成了写操作。
步骤⑧:第二节点接收到第一节点发送的响应消息z,就向APP2返回执行结果,该执行结果即为上述所述的响应消息z,用于告知APP2,第一节点和第二节点均对数据B完成了写操作。
需要说明的是,在本申请的一些实施方式中,类似地,引擎1和引擎2的选择完全独立,互相不强依赖,也就是说,引擎1和引擎2即可以是同构的数据库引擎,也可以是异构的数据存储引擎,具体此处不做限定。因为控制组件HA1和控制组件HA2中集成有一个适配模块,用于适配各种数据库引擎,即通过各自的控制组件屏蔽掉数据库引擎的差异。由于不同数据库引擎的资源消耗不同,性能表现也不同。数据库引擎可以运行在差异化的硬件上(比如一个高配,一个低配)。若引擎1和引擎2是采用异构的数据库引擎,那么,性能表现好的引擎一般运行在高配的主节点上,性能表现弱一些的引擎则运行在低配的备节点上。这样差异化的部署也是基于降低成本考虑。
在本申请上述实施方式中,在轮式移动设备(如,自动驾驶车辆、智能汽车、网联汽车等)上仅部署2个节点的前提下,且向第二节点(作为备节点)发送请求b的APP2与第一节点上的第一进程S1为运行在不同OS的情况时,通过上述强一致性(即在任意时刻下,系统里第一节点设备和第二节点设备中的数据都是一样的)的数据同步流程实现数据的实时备份,即保障了数据的可靠性,同时相比于现有技术中至少部署3个节点的方式,本申请部署2个节点的方式降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
需要说明的是,在本申请实施例中,之所以需要主节点和备节点对数据进行备份,就是为了避免一个节点出现故障的话,另一个节点中存储有完全相同的数据,并基于该数据继续为轮式移动设备提供服务,从而保障可用性,数据也不会丢失,基于此,在本申请一些实施方式中,若某个节点发生故障,那么其他节点会以单机模式继续运行,为了提升可用性,本申请实施例融合内外部的检测结果,对故障进行快速感知,由于故障的节点不同,处理的方式也不同,下面分别进行说明。
(1)第一节点(主节点)故障
在上述图3至图6中任一项对应的实施方式中,如果第一节点故障,那么第二节点会立即取代第一节点继续提供服务,并缓存数据,该缓存的数据就为第二节点取代第一节点作为主节点后第二存储模块内更新的数据,并且在第一节点恢复正常使用后,通过第二进程将该缓存的数据向第一节点发送,使得第一节点通过调用第一数据库引擎将该缓存的数据更新至自身的第一存储模块,从而实现两个节点之间的数据同步。
需要说明的是,在本申请的一些实施方式中,对第一节点故障的判断具体可以包括如 下几种方式:
a、第一节点向第二节点发送的心跳异常,例如,假设第一节点向第二节点发送心跳的频率正常是300毫秒一次,由于第一节点向第二节点发送心跳的目的就是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第二节点都没有收到第一节点发送的心跳,那么第二节点就认为第一节点故障。
b、第一节点向第二节点主动发送故障通知消息,例如,当第一节点上运行的第一进程上的软件正常或异常退出,则第一节点会主动向第二节点发送故障通知消息;又例如,当第一节点上监控第一进程的第三进程监控到第一进程异常时,则第一节点也会向第二节点主动发送故障通知消息;还例如,第一节点的通信异常,那么第一节点也会主动向第二节点发送故障通知消息,具体此次不做举例示意。方式b相比方式a,能够让第二节点更快感知到第一节点出现了故障,而不必等待预设时长第二节点没有收到第一节点的心跳才确定第一节点故障,从而进一步缩短了故障检测时延。
还需要说明的是,在本申请的一些实施方式中,第二节点取代第一节点继续提供服务具体可以包括以下几种取代方式:
a、第二节点取代第一节点提供服务,直至第一节点恢复正常使用,也就是说,等第一节点恢复了正常使用,第二节点就不再取代,第二节点依然作为备节点工作,一般这种情况是第一节点是高配,第二节点是低配,即第一节点与第二节点的硬件配置不同,主数据库引擎运行在高配的第一节点上,当高配的第一节点故障,则可以短期内由低配的第二节点取代第一节点提供服务,等第一节点恢复正常使用后,再由第一节点继续作为主节点提供服务,这种设置高低配置节点的方式也是为了节约成本。
b、第二节点始终取代第一节点提供服务,也就是说,第二节点取代第一节点作为主节点提供服务后,不管后续第一节点是否恢复正常使用,第二节点就一直作为主节点提供服务,若第一节点恢复正常使用后,就一直作为备节点使用,一般这种一直取代的方式是第一节点和第二节点的硬件配置没有差别的情况。
(2)第二节点(备节点)故障
在上述图3至图6中任一项对应的实施方式中,如果第二节点故障,那么第一节点可以将对端设置为异常状态,而第一节点作为主节点继续提供服务,并缓存数据(如,上述图3和图4中所述的第二数据、图5和图6中所述的第四数据),该缓存的数据就为第一节点通过第一进程向第一存储模块写数据(如,第一数据或第三数据)后得到的数据,并且在第二节点恢复正常使用后,通过第一进程将该缓存的数据向第二节点发送,使得第二节点通过调用第二数据库引擎将该缓存的数据更新至自身的第二存储模块,从而实现两个节点之间的数据同步。
需要说明的是,在本申请的一些实施方式中,缓存的数据量可能有上限约束,当缓存的数据达到该上限时,则按照先后顺序将缓存时间最早的数据丢弃,这是因为存储模块容量有限,为保证最新的数据能存储进存储模块,只能是删除最早缓存进来的数据。
类似地,在本申请的一些实施方式中,对第二节点故障的判断具体也可以包括如下几种方式:
a、第二节点向第一节点发送的心跳异常,例如,假设第二节点向第一节点发送心跳的频率正常是300毫秒一次,由于第二节点向第一节点发送心跳的目的也是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第一节点都没有收到第二节点发送的心跳,那么第一节点就认为第二节点故障。
b、第二节点向第一节点主动发送故障通知消息,例如,当第二节点上运行的第二进程上的软件正常或异常退出,则第二节点会主动向第一节点发送故障通知消息;又例如,当第二节点上监控第二进程的第四进程监控到第二进程异常时,则第二节点也会向第一节点主动发送故障通知消息;还例如,第二节点的通信异常,那么第二节点也会主动向第一节点发送故障通知消息,具体此次不做举例示意。方式b相比方式a,能够让第一节点更快感知到第二节点出现了故障,而不必等待预设时长第一节点没有收到第二节点的心跳才确定第二节点故障,从而进一步缩短了故障检测时延。
还需要说明的是,在本申请上述实施方式中,不管是第一节点故障,还是第二节点故障,当故障端重启恢复后,第一节点和第二节点都需要进行数据同步,数据同步有增量同步和全量同步两种方式:根据数据版本号,优先尝试增量同步(即只将缓存的数据发给对端),如果缓存数据不足,则进行全量同步(如,故障时间很长,此时可能大部分数据都是更新后的,因此直接将存储模块内的所有数据全部同步)。
在图3至图6所对应的实施例的基础上,为了更好的实施本申请实施例的上述方案,下面还提供用于实施上述方案的相关设备,具体请参阅图7,图7为本申请实施例提供的数据存储服务系统700的一种结构示意图,该数据存储服务系统700部署于轮式移动设备(如,自动驾驶车辆),该数据存储服务系统700包括第一节点701和第二节点702,第一节点701为主节点,第二节点702为备节点。
当该数据存储服务系统700用于实现上述图3至图4所对应的实施例时,第一节点701,用于通过运行在该第一节点701上的第一进程接收与该第一进程对应的第一应用程序发送的第一请求,该第一请求用于指示向存储模块写第一数据;该第一节点701,还用于响应于第一请求,通过该第一进程调用该第一进程上的第一数据库引擎向该第一节点701上的第一存储模块写该第一数据;该第一节点701,还用于在向该第一存储模块写该第一数据已完成的情况下,通过该第一进程将该第一请求向该第二节点702发送;该第二节点702,用于根据该第一请求,通过调用第二进程上的第二数据库引擎向该第二节点702上的第二存储模块写该第一数据,该第二进程运行在该第二节点702上;该第一节点701,还用于当该第一节点701通过该第一进程在预设时长内接收到该第二节点702通过该第二进程发送的第一响应消息,通过该第一进程向该第一应用程序发送第二响应消息,该第一响应消息用于指示向该第二存储模块写该第一数据的操作已执行完成,该第二响应消息用于指示向该第一存储模块写该第一数据和对该第二存储模块执行该第一数据的写操作均已完成。
在本申请上述实施方式中,部署于轮式移动设备(如,自动驾驶车辆、智能汽车、网联汽车等)上的数据存储服务系统700在仅部署2个节点的前提下,且向第二节点702(作为备节点)发送第二请求的APP与第一节点701上的第一进程为运行在不同OS的情况时,通过上述强一致性的数据同步流程实现数据的实时备份,即保障了数据的可靠性,同时相 比于现有技术中至少部署3个节点的方式,本申请部署2个节点的方式降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
在一种可能的设计中,该第二节点702,还用于:当该第一节点701故障,取代该第一节点701提供服务,并缓存数据,该缓存的数据就为第二节点702取代第一节点701作为主节点后第二存储模块内更新的数据,并且在第一节点701恢复正常使用后,通过第二进程将该缓存的数据向第一节点701发送,使得第一节点701通过调用第一数据库引擎将该缓存的数据更新至自身的第一存储模块,从而实现两个节点之间的数据同步。
在本申请上述实施方式中,若第一节点701故障(第一节点701作为主节点,第二节点702作为备节点),则第二节点702立即升为主节点,取代第一节点701进行服务,从而保障数据存储服务系统的可用性,数据不丢失。
在一种可能的设计中,该第二节点702,具体用于:取代该第一节点701提供服务,直至该第一节点701恢复正常使用,或,始终取代该第一节点701提供服务。也就是说,第一种取代方式是等第一节点701恢复了正常使用,第二节点702就不再取代,第二节点702依然作为备节点工作,一般这种情况是第一节点701是高配,第二节点702是低配,即第一节点701与第二节点702的硬件配置不同,主数据库引擎运行在高配的第一节点701上,当高配的第一节点701故障,则可以短期内由低配的第二节点702取代第一节点701提供服务,等第一节点701恢复正常使用后,再由第一节点701继续作为主节点提供服务,这种设置高低配置节点的方式也是为了节约成本。而第二种取代方式是第二节702点取代第一节点701作为主节点提供服务后,不管后续第一节点701是否恢复正常使用,第二节点702就一直作为主节点提供服务,若第一节点701恢复正常使用后,就一直作为备节点使用,一般这种一直取代的方式是第一节点701和第二节点702的硬件配置没有差别的情况。
在本申请上述实施方式中,具体阐述了当第一节点701故障,第二节点702取代第一节点701提供服务的两种方式,具备灵活性和可选择性。
在一种可能的设计中,该第一节点701故障包括:该第一节点701向该第二节点702发送的心跳异常;或,该第一节点701向该第二节点702发送故障通知消息。例如,假设第一节点701向第二节点702发送心跳的频率正常是300毫秒一次,由于第一节点701向第二节点702发送心跳的目的就是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第二节点702都没有收到第一节点701发送的心跳,那么第二节点702就认为第一节点701故障。例如,当第一节点701上运行的第一进程上的软件正常或异常退出,则第一节点701会主动向第二节点702发送故障通知消息;又例如,当第一节点701上监控第一进程的第三进程监控到第一进程异常时,则第一节点701也会向第二节点702主动发送故障通知消息;还例如,第一节点701的通信异常,那么第一节点701也会主动向第二节点702发送故障通知消息,具体此次不做举例示意。
在本申请上述实施方式中,上述两种感知故障的方式都是为了使得第二节点702能够快速感知到第一节点701出现了故障,缩短了故障检测时延。其中,第二种故障感知的方式相比第一种故障感知的方式,能够让第二节点702更快感知到第一节点701出现了故障, 而不必等待预设时长第二节点702没有收到第一节点701的心跳才确定第一节点701故障,从而进一步缩短了故障检测时延。
在一种可能的设计中,该第一节点701,还用于当该第二节点702故障,将对端设置为异常状态,并缓存第二数据,并在该第二节点702恢复正常使用后,通过该第一进程将该第二数据向该第二节点702发送;该第二节点702,还用于通过调用该第二数据库引擎将该第二数据更新至该第二存储模块,该第二数据为该第一节点701通过该第一进程向该第一存储模块写该第一数据后得到的数据。
在本申请上述实施方式中,阐述了如果是第二节点702故障,那么第一节点701将作为主节点继续提供服务,并且在第二节点702恢复正常使用后,通过第一进程将该缓存的第二数据向第二节点702发送,使得第二节点702通过调用第二数据库引擎将该缓存的第二数据更新至自身的第二存储模块,从而实现两个节点之间的数据同步,具备可实现性。
在一种可能的设计中,该第二节点702故障包括:该第二节点702向该第一节点701发送的心跳异常;或,该第二节点702向该第一节点701发送故障通知消息。例如,假设第二节点702向第一节点701发送心跳的频率正常是300毫秒一次,由于第二节点702向第一节点701发送心跳的目的是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第一节点701都没有收到第二节点702发送的心跳,那么第一节点701就认为第二节点702故障。例如,当第二节点702上运行的第二进程上的软件正常或异常退出,则第二节点702会主动向第一节点701发送故障通知消息;又例如,当第二节点702上监控第二进程的第四进程监控到第二进程异常时,则第二节点702也会向第一节点701主动发送故障通知消息;还例如,第二节点702的通信异常,那么第二节点702也会主动向第一节点701发送故障通知消息,具体此次不做举例示意。
在本申请上述实施方式中,上述两种感知故障的方式都是为了使得第一节点701能够快速感知到第二节点702出现了故障,缩短了故障检测时延。其中,第二种故障感知的方式相比第一种故障感知的方式,能够让第一节点701更快感知到第二节点702出现了故障,而不必等待预设时长第一节点701没有收到第二节点702的心跳才确定第二节点702故障,从而进一步缩短了故障检测时延。
在一种可能的设计中,该第一存储模块包括第一内存数据库;和/或,该第二存储模块包括第二内存数据库。
在本申请上述实施方式中,阐述了第一存储模块和第二存储模块可以是内存数据库,相对于传统的硬盘,内存数据库的读取速度更快,尤其是在智能驾驶领域,由于车辆的运行状态数据、感知数据、中间计算结果、SOA信息等数据需频繁被读取,若第一存储模块和第二存储模块为硬盘,那么反复频繁的读/写数据会缩短硬盘寿命,从而导致车用器件的更换成本提高。
在一种可能的设计中,该第一节点701,还用于:通过该第一进程记录第一数据版本号,该第一数据版本号用于表示该第一存储模块数据更新后的数据版本(可称为第一数据版本),该更新后的数据版本为向第一存储模块写第一数据的操作已完成的情况下的数据版本。
在本申请上述实施方式中,通过第一进程执行完第一数据的写操作后,还要记录此次更新后的数据版本号,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
在一种可能的设计中,该第二节点702,还用于:通过该第二进程记录第二数据版本号,该第二数据版本号用于表示该第二存储模块数据更新后的数据版本(可称为第二数据版本),所述更新后的数据版本为向第二存储模块写第一数据的操作已完成的情况下的数据版本。
在本申请上述实施方式中,通过第二进程执行完第一数据的写操作后,也要记录此次更新后的数据版本号,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
在一种可能的设计中,部署有该数据存储服务系统700的轮式移动设备可以是自动驾驶车辆,自动驾驶车辆可以为轿车、卡车、摩托车、公共汽车、船、飞机、直升飞机、割草机、娱乐车、游乐场车辆、施工设备、电车、高尔夫球车、火车、和手推车等,本申请实施例不做特别的限定。
在本申请上述实施方式中,阐述了轮式移动设备可以是自动驾驶车辆,具备可实现性。
此外,当该数据存储服务系统700用于实现上述图5至图6所对应的实施例时,第二节点702,用于通过运行在该第二节点702上的第二进程接收与该第二进程对应的第二应用程序发送的第二请求,该第二请求用于指示向存储模块写第三数据;第一节点701,用于通过运行在该第一节点701上的第一进程接收该第二节点702通过该第二进程转发的第二请求;该第一节点701,还用于响应于该第二请求,通过该第一进程调用该第一进程上的第一数据库引擎向该第一节点701的第一存储模块写该第三数据;该第一节点701,还用于在向该第一存储模块写该第三数据的操作已完成的情况下,通过该第一进程向该第二节点702发送第三响应消息,该第三响应消息用于指示该第二节点702根据该第二请求,通过该第二进程调用该第二进程上的第二数据库引擎向该第二节点702的第二存储模块写该第三数据;该第二节点702,还用于根据该第三响应消息和该第二请求,通过该第二进程调用该第二数据库引擎向该第二存储模块写该第三数据;该第一节点701,还用于当该第一节点701通过该第一进程在预设时长内接收到该第二节点702通过该第二进程发送的第四响应消息,通过该第一进程向该第二节点702发送第五响应消息,该第四响应消息用于指示向该第二存储模块写该第三数据的操作已执行完成,该第五响应消息用于指示向该第一存储模块写该第三数据的操作和向该第二存储模块写该第三数据的操作均已执行完成;该第二节点702,还用于通过该第二进程向该第二应用程序转发该第五响应消息。
在本申请上述实施方式中,部署于轮式移动设备(如,自动驾驶车辆)上的数据存储服务系统700在仅部署2个节点的前提下,且向第二节点702(作为备节点)发送第二请求的APP与第一节点701上的第一进程为运行在不同OS的情况时,通过上述强一致性的数据同步流程实现数据的实时备份,即保障了数据的可靠性,同时又降低了轮式移动设备的硬件成本,即在低成本的轮式移动设备的硬件架构中实现高可靠性的数据存储。
在一种可能的设计中,该第二节点702,还用于:当该第一节点701故障,取代该第一节点701提供服务,并缓存数据,该缓存的数据就为第二节点702取代第一节点701作为主节点后第二存储模块内更新的数据,并且在第一节点701恢复正常使用后,通过第二 进程将该缓存的数据向第一节点701发送,使得第一节点701通过调用第一数据库引擎将该缓存的数据更新至自身的第一存储模块,从而实现两个节点之间的数据同步。
在本申请上述实施方式中,若第一节点701故障(第一节点701作为主节点,第二节点702作为备节点),则第二节点702立即升为主节点,取代第一节点701进行服务,从而保障数据存储服务系统的可用性,数据不丢失。
在一种可能的设计中,该第二节点702,具体用于:取代该第一节点701提供服务,直至该第一节点701恢复正常使用;或,始终取代该第一节点701提供服务。也就是说,第一种取代方式是等第一节点701恢复了正常使用,第二节点702就不再取代,第二节点702依然作为备节点工作,一般这种情况是第一节点701是高配,第二节点702是低配,即第一节点701与第二节点702的硬件配置不同,主数据库引擎运行在高配的第一节点701上,当高配的第一节点701故障,则可以短期内由低配的第二节点702取代第一节点701提供服务,等第一节点701恢复正常使用后,再由第一节点701继续作为主节点提供服务,这种设置高低配置节点的方式也是为了节约成本。而第二种取代方式是第二节702点取代第一节点701作为主节点提供服务后,不管后续第一节点701是否恢复正常使用,第二节点702就一直作为主节点提供服务,若第一节点701恢复正常使用后,就一直作为备节点使用,一般这种一直取代的方式是第一节点701和第二节点702的硬件配置没有差别的情况。
在本申请上述实施方式中,具体阐述了当第一节点701故障,第二节点702取代第一节点701提供服务的两种方式,具备灵活性和可选择性。
在一种可能的设计中,该第一节点701故障包括:该第一节点701向该第二节点702发送的心跳异常;或,该第一节点701向该第二节点702发送故障通知消息。例如,假设第一节点701向第二节点702发送心跳的频率正常是300毫秒一次,由于第一节点701向第二节点702发送心跳的目的就是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第二节点702都没有收到第一节点701发送的心跳,那么第二节点702就认为第一节点701故障。例如,当第一节点701上运行的第一进程上的软件正常或异常退出,则第一节点701会主动向第二节点702发送故障通知消息;又例如,当第一节点701上监控第一进程的第三进程监控到第一进程异常时,则第一节点701也会向第二节点702主动发送故障通知消息;还例如,第一节点701的通信异常,那么第一节点701也会主动向第二节点702发送故障通知消息,具体此次不做举例示意。
在本申请上述实施方式中,上述两种感知故障的方式都是为了使得第二节点702能够快速感知到第一节点701出现了故障,缩短了故障检测时延。其中,第二种故障感知的方式相比第一种故障感知的方式,能够让第二节点702更快感知到第一节点701出现了故障,而不必等待预设时长第二节点702没有收到第一节点701的心跳才确定第一节点701故障,从而进一步缩短了故障检测时延。
在一种可能的设计中,该第一节点701,还用于当该第二节点702故障,将对端设置为异常状态,并缓存第四数据,并在该第二节点702恢复正常使用后,通过该第一进程将该第四数据向该第二节点702发送;该第二节点702,还用于通过调用该第二数据库引擎 将该第四数据更新至该第二存储模块,该第四数据为该第一节点701通过该第一进程向该第一存储模块写入该第三数据后得到的数据。
在本申请上述实施方式中,阐述了如果是第二节点702故障,那么第一节点701将作为主节点继续提供服务,并且在第二节点702恢复正常使用后,通过第一进程将该缓存的第四数据向第二节点702发送,使得第二节点702通过调用第二数据库引擎将该缓存的第四数据更新至自身的第二存储模块,从而实现两个节点之间的数据同步,具备可实现性。
在一种可能的设计中,该第二节点702故障包括:该第二节点702向该第一节点701发送的心跳异常;或,该第二节点702向该第一节点701发送故障通知消息。例如,假设第二节点702向第一节点701发送心跳的频率正常是300毫秒一次,由于第二节点702向第一节点701发送心跳的目的是告知对端自身是在正常运行,但在预设时长内(如,1秒内)第一节点701都没有收到第二节点702发送的心跳,那么第一节点701就认为第二节点702故障。例如,当第二节点702上运行的第二进程上的软件正常或异常退出,则第二节点702会主动向第一节点701发送故障通知消息;又例如,当第二节点702上监控第二进程的第四进程监控到第二进程异常时,则第二节点702也会向第一节点701主动发送故障通知消息;还例如,第二节点702的通信异常,那么第二节点702也会主动向第一节点701发送故障通知消息,具体此次不做举例示意。
在本申请上述实施方式中,上述两种感知故障的方式都是为了使得第一节点701能够快速感知到第二节点702出现了故障,缩短了故障检测时延。其中,第二种故障感知的方式相比第一种故障感知的方式,能够让第一节点701更快感知到第二节点702出现了故障,而不必等待预设时长第一节点701没有收到第二节点702的心跳才确定第二节点702故障,从而进一步缩短了故障检测时延。
在一种可能的设计中,该第一存储模块包括第一内存数据库;和/或,该第二存储模块包括第二内存数据库。
在本申请上述实施方式中,阐述了第一存储模块和第二存储模块可以是内存数据库,相对于传统的硬盘,内存数据库的读取速度更快,尤其是在智能驾驶领域,由于车辆的运行状态数据、感知数据、中间计算结果、SOA信息等数据需频繁被读取,若第一存储模块和第二存储模块为硬盘,那么反复频繁的读/写数据会缩短硬盘寿命,从而导致车用器件的更换成本提高。
在一种可能的设计中,该第一节点701,还用于:通过该第一进程记录第一数据版本号,该第一数据版本号用于表示该第一存储模块数据更新后的数据版本(可称为第一数据版本),所述更新后的数据版本为向第一存储模块写第三数据的操作已完成的情况下的数据版本。
在本申请上述实施方式中,通过第一进程执行完第一数据的写操作后,还要记录此次更新后的数据版本号,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
在一种可能的设计中,该第二节点702,还用于:通过该第二进程记录第二数据版本号,该第二数据版本号用于表示该第二存储模块数据更新后的数据版本(可称为第二数据版本),所述更新后的数据版本为向第二存储模块写第三数据的操作已完成的情况下的数据 版本。
在本申请上述实施方式中,通过第二进程执行完第一数据的写操作后,也要记录此次更新后的数据版本号,这样便于查找每次更新的数据是哪个版本,更新了哪些数据等。
在一种可能的设计中,部署有该数据存储服务系统700的轮式移动设备可以是自动驾驶车辆,自动驾驶车辆可以为轿车、卡车、摩托车、公共汽车、船、飞机、直升飞机、割草机、娱乐车、游乐场车辆、施工设备、电车、高尔夫球车、火车、和手推车等,本申请实施例不做特别的限定。
在本申请上述实施方式中,阐述了轮式移动设备可以是自动驾驶车辆,具备可实现性。
需要说明的是,图7对应实施例所述的各模块/单元之间的信息交互、执行过程等内容,与本申请中图3至图6对应的方法实施例基于同一构思,具体内容可参见本申请前述所示的方法实施例中的叙述,此处不再赘述。
在图3至图4所对应的实施例的基础上,为了更好的实施本申请实施例的上述方案,下面还提供用于实施上述方案的节点,该节点作为第一节点,具体参阅图8,图8是本申请实施例提供的第一节点800的一种结构示意图,第一节点800具体可以包括:接收模块801,用于通过运行在所述第一节点800上的第一进程接收与所述第一进程对应的第一应用程序发送的第一请求,所述第一请求用于指示向存储模块写第一数据;调用模块802,用于根据所述第一请求,通过所述第一进程调用所述第一进程上的第一数据库引擎向所述第一节点800上的第一存储模块写所述第一数据;第一发送模块803,用于在向所述第一存储模块写所述第一数据的操作已完成的情况下,通过所述第一进程将所述第一请求向所述第二节点发送,以使得所述第二节点根据所述第一请求,通过调用第二进程上的第二数据库引擎向所述第二节点上的第二存储模块写该第一数据,所述第二进程运行在所述第二节点上;第二发送模块804,用于当通过所述第一进程在预设时长内接收到所述第二节点通过所述第二进程发送的第一响应消息,通过所述第一进程向所述第一应用程序发送第二响应消息,所述第一响应消息用于指示向所述第二存储模块写所述第一数据的操作已完成,所述第二响应消息用于指示向所述第一存储模块写所述第一数据的操作和向所述第二存储模块写所述第一数据的操作均已完成。
在一种可能的设计中,所述第一发送模块803,还用于:当所述第二节点故障,缓存第二数据,并在所述第二节点恢复正常使用后,通过所述第一进程将所述第二数据向所述第二节点发送,以使得所述第二节点通过调用所述第二数据库引擎将所述第二数据更新至所述第二存储模块,所述第二数据为所述第一节点800通过所述第一进程向所述第一存储模块写所述第一数据的操作后得到的数据。
在一种可能的设计中,所述第二节点故障包括:所述第二节点向所述第一节点800发送的心跳异常;或,所述第二节点向所述第一节点800发送故障通知消息。
在一种可能的设计中,所述第一存储模块包括第一内存数据库;和/或,所述第二存储模块包括第二内存数据库。
在一种可能的设计中,所述调用模块802,还用于:通过所述第一进程记录第一数据版本号,所述第一数据版本号用于表示所述第一存储模块数据更新后的数据版本。
需要说明的是,图3至图4对应实施例所述的第一节点上各模块/单元之间的信息交互、执行过程等内容,与本申请中图8对应的方法实施例基于同一构思,具体内容可参见本申请前述所示的方法实施例中的叙述,此处不再赘述。
在图5至图6所对应的实施例的基础上,为了更好的实施本申请实施例的上述方案,下面还提供用于实施上述方案的节点,该节点作为第一节点,具体参阅图9,图9是本申请实施例提供的第一节点900的一种结构示意图,第一节点900具体可以包括:接收模块901,用于通过运行在所述第一节点900上的第一进程接收运行在所述第二节点上的第二进程转发的第二请求,所述第二请求由与所述第二进程对应的第二应用程序向所述第二节点发送,所述第二请求用于指示向存储模块写第三数据;调用模块902,用于根据所述第二请求,通过所述第一进程调用所述第一进程上的第一数据库引擎向所述第一节点900的第一存储模块写所述第三数据;第一发送模块903,用于在向所述第一存储模块写所述第三数据的操作已执行完成的情况下,通过所述第一进程向所述第二节点发送第三响应消息,所述第三响应消息用于指示所述第二节点根据所述第二请求,通过所述第二进程调用所述第二进程上的第二数据库引擎向所述第二节点的第二存储模块写所述第三数据;第二发送模块904,用于当通过所述第一进程在预设时长内接收到所述第二节点通过所述第二进程发送的第四响应消息,通过所述第一进程向所述第二节点发送第五响应消息,以使得所述第二节点通过所述第二进程向所述第二应用程序转发所述第五响应消息,所述第四响应消息用于指示向所述第二存储模块写所述第三数据的操作已完成,所述第五响应消息用于指示向所述第一存储模块写所述第三数据的操作和向所述第二存储模块写所述第三数据的操作均已完成。
在一种可能的设计中,所述第一发送模块903,还用于:当所述第二节点故障,缓存第四数据,并在所述第二节点恢复正常使用后,通过所述第一进程将所述第四数据向所述第二节点发送,以使得所述第二节点通过调用所述第二数据库引擎将所述第四数据更新至所述第二存储模块,所述第四数据为所述第一节点900通过所述第一进程向所述第一存储模块写所述第三数据后得到的数据。
在一种可能的设计中,所述第二节点故障包括:所述第二节点向所述第一节点900发送的心跳异常;或,所述第二节点向所述第一节点900发送故障通知消息。
在一种可能的设计中,所述第一存储模块包括第一内存数据库;和/或,所述第二存储模块包括第二内存数据库。
在一种可能的设计中,所述调用模块902,还用于:通过所述第一进程记录第一数据版本号,所述第一数据版本号用于表示所述第一存储模块数据更新后的数据版本。
需要说明的是,图5至图6对应实施例所述的第一节点上各模块/单元之间的信息交互、执行过程等内容,与本申请中图9对应的方法实施例基于同一构思,具体内容可参见本申请前述所示的方法实施例中的叙述,此处不再赘述。
本申请实施例还提供了一种数据存储服务系统,该数据存储服务系统部署于轮式移动设备(如,自动驾驶车辆)上,请参阅图10,图10是本申请实施例提供的数据存储服务系统1000的一种结构示意图,为便于说明,仅示出了与本申请实施例相关的部分,具体技 术细节未揭示的,请参照本申请实施例方法部分。数据存储服务系统1000上可以部署有图7对应实施例中所描述的数据存储服务系统的模块,用于实现图3至图6对应实施例的功能,具体的,数据存储服务系统1000由一个或多个服务器实现,数据存储服务系统1000可因配置或性能不同而产生比较大的差异,可以包括一个或一个以上中央处理器(central processing units,CPU)1022(例如,一个或一个以上)和存储器1032,一个或一个以上存储应用程序1042或数据1044的存储介质1030(例如一个或一个以上海量存储设备)。其中,存储器1032和存储介质1030可以是短暂存储或持久存储。存储在存储介质1030的程序可以包括一个或一个以上模块(图示没标出),每个模块可以包括对训练设备中的一系列指令操作。更进一步地,中央处理器1022可以设置为与存储介质1030通信,在数据存储服务系统1000上执行存储介质1030中的一系列指令操作。
数据存储服务系统1000还可以包括一个或一个以上电源1026,一个或一个以上有线或无线网络接口1050,一个或一个以上输入输出接口1058,和/或,一个或一个以上操作系统1041,例如Windows ServerTM,Mac OS XTM,UnixTM,LinuxTM,FreeBSDTM等等。
在本申请实施例中,上述图3至图6对应的实施例中由第一节点和第二节点所执行的步骤可以基于该图10所示的结构实现,具体此处不予赘述。
另外需说明的是,以上所描述的装置实施例仅仅是示意性的,其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外,本申请提供的装置实施例附图中,模块之间的连接关系表示它们之间具有通信连接,具体可以实现为一条或多条通信总线或信号线。
通过以上的实施方式的描述,所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现,当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下,凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现,而且,用来实现同一功能的具体硬件结构也可以是多种多样的,例如模拟电路、数字电路或专用电路等。但是,对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在可读取的存储介质中,如计算机的软盘、U盘、移动硬盘、只读存储器(read only memory,ROM)、随机存取存储器(random access memory,RAM)、磁碟或者光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,训练设备,或者网络设备等)执行本申请各个实施例所述的方法。
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。
所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时,全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是 通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、训练设备或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、训练设备或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的训练设备、数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,高密度数字视频光盘(digital video disc,DVD))、或者半导体介质(例如,固态硬盘(solid state disk,SSD))等。

Claims (38)

  1. 一种数据的存储方法,其特征在于,所述方法包括:
    第一节点设备通过运行在所述第一节点设备上的第一进程接收第一应用程序发送的第一请求;
    响应于所述第一请求,并基于所述第一进程上的第一数据库引擎,所述第一节点设备通过所述第一进程向所述第一节点设备上的第一存储模块写第一数据;
    在向所述第一存储模块写所述第一数据的操作已完成的情况下,所述第一节点设备通过所述第一进程将所述第一请求向第二节点设备发送,以使得所述第二节点设备响应于所述第一请求,并使得所述第二节点设备基于第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第一数据,所述第二进程运行在所述第二节点设备上;
    当所述第一节点设备通过所述第一进程在预设时长内接收到所述第二节点设备通过所述第二进程发送的第一响应消息,所述第一节点设备通过所述第一进程向所述第一应用程序发送第二响应消息,所述第一响应消息用于指示向所述第二存储模块写所述第一数据的操作已完成,所述第二响应消息用于指示向所述第一存储模块写所述第一数据的操作和向所述第二存储模块写所述第一数据的操作均已完成。
  2. 根据权利要求1所述的方法,其特征在于,所述方法还包括:
    当所述第一节点设备故障,所述第二节点设备取代所述第一节点设备提供服务。
  3. 根据权利要求2所述的方法,其特征在于,所述第二节点设备取代所述第一节点设备提供服务包括:
    所述第二节点设备取代所述第一节点设备提供服务,直至所述第一节点设备恢复正常使用;
    或,
    所述第二节点设备始终取代所述第一节点设备提供服务。
  4. 根据权利要求2-3中任一项所述的方法,其特征在于,所述第一节点设备故障包括:
    所述第一节点设备向所述第二节点设备发送的心跳异常;
    或,
    所述第一节点设备向所述第二节点设备发送故障通知消息。
  5. 根据权利要求1所述的方法,其特征在于,在所述第一节点设备通过所述第一进程向所述第一应用程序发送第二响应消息之前,所述方法还包括:
    当所述第二节点设备故障,所述第一节点设备缓存第二数据,并在所述第二节点设备恢复正常使用后,通过所述第一进程将所述第二数据向所述第二节点设备发送,以使得所述第二节点设备基于所述第二数据库引擎将所述第二数据更新至所述第二存储模块,所述第二数据为所述第一节点设备通过所述第一进程向所述第一存储模块写所述第一数据后得到的数据。
  6. 根据权利要求5所述的方法,其特征在于,所述第二节点设备故障包括:
    所述第二节点设备向所述第一节点设备发送的心跳异常;
    或,
    所述第二节点设备向所述第一节点设备发送故障通知消息。
  7. 根据权利要求1-6中任一项所述的方法,其特征在于,
    所述第一存储模块包括第一内存数据库;
    和/或,
    所述第二存储模块包括第二内存数据库。
  8. 根据权利要求1-7中任一项所述的方法,其特征在于,在所述响应于所述第一请求,并基于所述第一进程上的第一数据库引擎,所述第一节点设备通过所述第一进程向所述第一节点设备上的第一存储模块写所述第一数据之后,所述方法还包括:
    所述第一节点设备通过所述第一进程记录第一数据版本号,所述第一数据版本号用于表示所述第一存储模块数据更新后的第一数据版本,所述更新后的第一数据版本为向所述第一存储模块写所述第一数据的操作已完成的情况下的数据版本。
  9. 根据权利要求1-8中任一项所述的方法,其特征在于,所述第一节点设备通过所述第一进程将所述第一请求向所述第二节点设备发送,以使得所述第二节点设备响应于所述第一请求,并使得所述第二节点设备基于第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第一数据包括:
    所述第一节点设备通过所述第一进程将所述第一请求向所述第二节点设备发送,以使得所述第二节点设备响应于所述第一请求,并使得所述第二节点设备基于第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第一数据,且还使得所述第二节点设备通过所述第二进程记录第二数据版本号,所述第二数据版本号用于表示所述第二存储模块数据更新后的第二数据版本,所述更新后的第二数据版本为向所述第二存储模块写所述第一数据的操作已完成的情况下的数据版本。
  10. 一种数据的存储方法,其特征在于,所述方法包括:
    第一节点设备通过运行在所述第一节点设备上的第一进程接收运行在第二节点设备上的第二进程转发的第二请求,所述第二请求由第二应用程序向所述第二节点设备发送;
    响应于所述第二请求,并基于所述第一进程上的第一数据库引擎,所述第一节点设备通过所述第一进程向所述第一节点设备上的第一存储模块写第三数据;
    在向所述第一存储模块写所述第三数据的操作已完成的情况下,所述第一节点设备通过所述第一进程向所述第二节点设备发送第三响应消息,所述第三响应消息用于指示所述第二节点设备响应于所述第二请求,并用于指示所述第二节点设备基于所述第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第三数据;
    当所述第一节点设备通过所述第一进程在预设时长内接收到所述第二节点设备通过所述第二进程发送的第四响应消息,所述第一节点设备通过所述第一进程向所述第二节点设备发送第五响应消息,以使得所述第二节点设备通过所述第二进程向所述第二应用程序转发所述第五响应消息,所述第四响应消息用于指示向所述第二存储模块写所述第三数据的操作已完成,所述第五响应消息用于指示向所述第一存储模块写所述第三数据的操作和向 所述第二存储模块写所述第三数据的操作均已完成。
  11. 根据权利要求10所述的方法,其特征在于,所述方法还包括:
    当所述第一节点设备故障,所述第二节点设备取代所述第一节点设备提供服务。
  12. 根据权利要求11所述的方法,其特征在于,所述第二节点设备取代所述第一节点设备提供服务包括:
    所述第二节点设备取代所述第一节点设备提供服务,直至所述第一节点设备恢复正常使用;
    或,
    所述第二节点设备始终取代所述第一节点设备提供服务。
  13. 根据权利要求11-12中任一项所述的方法,其特征在于,所述第一节点设备故障包括:
    所述第一节点设备向所述第二节点设备发送的心跳异常;
    或,
    所述第一节点设备向所述第二节点设备发送故障通知消息。
  14. 根据权利要求10所述的方法,其特征在于,在所述第一节点设备通过所述第一进程向所述第二节点设备发送第五响应消息之前,所述方法还包括:
    当所述第二节点设备故障,所述第一节点设备缓存第四数据,并在所述第二节点设备恢复正常使用后,通过所述第一进程将所述第四数据向所述第二节点设备发送,以使得所述第二节点设备基于所述第二数据库引擎将所述第四数据更新至所述第二存储模块,所述第四数据为所述第一节点设备通过所述第一进程向所述第一存储模块写所述第三数据后得到的数据。
  15. 根据权利要求14所述的方法,其特征在于,所述第二节点设备故障包括:
    所述第二节点设备向所述第一节点设备发送的心跳异常;
    或,
    所述第二节点设备向所述第一节点设备发送故障通知消息。
  16. 根据权利要求10-15中任一项所述的方法,其特征在于,
    所述第一存储模块包括第一内存数据库;
    和/或,
    所述第二存储模块包括第二内存数据库。
  17. 根据权利要求10-16中任一项所述的方法,其特征在于,在所述响应于所述第二请求,并基于所述第一进程上的第一数据库引擎,所述第一节点设备通过所述第一进程向所述第一节点设备上的第一存储模块写第三数据之后,所述方法还包括:
    所述第一节点设备通过所述第一进程记录第一数据版本号,所述第一数据版本号用于表示所述第一存储模块数据更新后的第一数据版本,所述更新后的第一数据版本为向所述第一存储模块写所述第三数据的操作已完成的情况下的数据版本。
  18. 根据权利要求10-17中任一项所述的方法,其特征在于,所述第三响应消息用于指示所述第二节点设备响应于所述第二请求,并用于指示所述第二节点设备基于所述第二 进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第三数据包括:
    所述第三响应消息用于指示所述第一节点设备响应于所述第二请求,并用于指示所述第二节点设备基于所述第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第三数据,且还用于指示所述第二节点设备通过所述第二进程记录第二数据版本号,所述第二数据版本号用于表示所述第二存储模块数据更新后的第二数据版本,所述更新后的第二数据版本为向所述第二存储模块写所述第三数据的操作已完成的情况下的数据版本。
  19. 一种数据存储服务系统,其特征在于,所述系统部署于轮式移动设备,所述系统包括:
    第一节点设备,用于通过运行在所述第一节点设备上的第一进程接收第一应用程序发送的第一请求;
    所述第一节点设备,还用于响应于所述第一请求,并基于所述第一进程上的第一数据库引擎,所述第一节点设备通过所述第一进程向所述第一节点设备上的第一存储模块写第一数据;
    所述第一节点设备,还用于在向所述第一存储模块写所述第一数据的操作已完成的情况下,通过所述第一进程将所述第一请求向所述第二节点设备发送;
    所述第二节点设备,用于响应于所述第一请求,并基于第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第一数据,所述第二进程运行在所述第二节点设备上;
    所述第一节点设备,还用于当所述第一节点设备通过所述第一进程在预设时长内接收到所述第二节点设备通过所述第二进程发送的第一响应消息,通过所述第一进程向所述第一应用程序发送第二响应消息,所述第一响应消息用于指示向所述第二存储模块写所述第一数据的操作已完成,所述第二响应消息用于指示向所述第一存储模块写所述第一数据的操作和向所述第二存储模块写所述第一数据的操作均已完成。
  20. 根据权利要求19所述的系统,其特征在于,所述第二节点设备,还用于:
    当所述第一节点设备故障,取代所述第一节点设备提供服务。
  21. 根据权利要求20所述的系统,其特征在于,所述第二节点设备,具体用于:
    取代所述第一节点设备提供服务,直至所述第一节点设备恢复正常使用;
    或,
    始终取代所述第一节点设备提供服务。
  22. 根据权利要求20-21中任一项所述的系统,其特征在于,所述第一节点设备故障包括:
    所述第一节点设备向所述第二节点设备发送的心跳异常;
    或,
    所述第一节点设备向所述第二节点设备发送故障通知消息。
  23. 根据权利要求19所述的系统,其特征在于,
    所述第一节点设备,还用于当所述第二节点设备故障,缓存第二数据,并在所述第二节点设备恢复正常使用后,通过所述第一进程将所述第二数据向所述第二节点设备发送;
    所述第二节点设备,还用于基于所述第二数据库引擎将所述第二数据更新至所述第二存储模块,所述第二数据为所述第一节点设备通过所述第一进程向所述第一存储模块写所述第一数据后得到的数据。
  24. 根据权利要求23所述的系统,其特征在于,所述第二节点设备故障包括:
    所述第二节点设备向所述第一节点设备发送的心跳异常;
    或,
    所述第二节点设备向所述第一节点设备发送故障通知消息。
  25. 根据权利要求19-24中任一项所述的系统,其特征在于,
    所述第一存储模块包括第一内存数据库;
    和/或,
    所述第二存储模块包括第二内存数据库。
  26. 根据权利要求19-25中任一项所述的系统,其特征在于,所述第一节点设备,还用于:
    通过所述第一进程记录第一数据版本号,所述第一数据版本号用于表示所述第一存储模块数据更新后的第一数据版本,所述更新后的第一数据版本为向所述第一存储模块写所述第一数据的操作已完成的情况下的数据版本。
  27. 根据权利要求19-26中任一项所述的系统,其特征在于,所述第二节点设备,还用于:
    通过所述第二进程记录第二数据版本号,所述第二数据版本号用于表示所述第二存储模块数据更新后的第二数据版本,所述更新后的第二数据版本为向所述第二存储模块写所述第一数据的操作已完成的情况下的数据版本。
  28. 一种数据存储服务系统,其特征在于,所述系统部署于轮式移动设备,所述系统包括:
    第二节点设备,用于通过运行在所述第二节点设备上的第二进程接收第二应用程序发送的第二请求;
    第一节点设备,用于通过运行在所述第一节点设备上的第一进程接收所述第二节点设备通过所述第二进程转发的所述第二请求;
    所述第一节点设备,还用于响应于所述第二请求,并基于所述第一进程上的第一数据库引擎,通过所述第一进程向所述第一节点设备上的第一存储模块写第三数据;
    所述第一节点设备,还用于在向所述第一存储模块写所述第三数据的操作已完成的情况下,通过所述第一进程向所述第二节点设备发送第三响应消息,所述第三响应消息用于指示所述第二节点设备响应于所述第二请求,并用于指示所述第二节点设备基于所述第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第三数据;
    所述第二节点设备,还用于响应于所述第三响应消息和所述第二请求,基于所述第二 数据库引擎,通过所述第二进程向所述第二存储模块写所述第三数据;
    所述第一节点设备,还用于当所述第一节点设备通过所述第一进程在预设时长内接收到所述第二节点设备通过所述第二进程发送的第四响应消息,通过所述第一进程向所述第二节点设备发送第五响应消息,所述第四响应消息用于指示向所述第二存储模块写所述第三数据的操作已完成,所述第五响应消息用于指示向所述第一存储模块写所述第三数据的操作和向所述第二存储模块写所述第三数据的操作均已完成;
    所述第二节点设备,还用于通过所述第二进程向所述第二应用程序转发所述第五响应消息。
  29. 根据权利要求28所述的系统,其特征在于,所述第二节点设备,还用于:
    当所述第一节点设备故障,取代所述第一节点设备提供服务。
  30. 根据权利要求29所述的系统,其特征在于,所述第二节点设备,具体用于:
    取代所述第一节点设备提供服务,直至所述第一节点设备恢复正常使用;
    或,
    始终取代所述第一节点设备提供服务。
  31. 根据权利要求28所述的系统,其特征在于,
    所述第一节点设备,还用于当所述第二节点设备故障,缓存第四数据,并在所述第二节点设备恢复正常使用后,通过所述第一进程将所述第四数据向所述第二节点设备发送;
    所述第二节点设备,还用于基于所述第二数据库引擎将所述第四数据更新至所述第二存储模块,所述第四数据为所述第一节点设备通过所述第一进程向所述第一存储模块写所述第三数据后得到的数据。
  32. 一种节点设备,其特征在于,所述节点设备应用于数据存储服务系统,所述系统部署于轮式移动设备,所述系统包括第一节点设备和第二节点设备,所述节点设备作为第一节点设备,包括:
    接收模块,用于通过运行在所述第一节点设备上的第一进程接收第一应用程序发送的第一请求;
    调用模块,用于响应于所述第一请求,并基于所述第一进程上的第一数据库引擎,通过所述第一进程向所述第一节点设备上的第一存储模块写第一数据;
    第一发送模块,用于在向所述第一存储模块写所述第一数据的操作已完成的情况下,通过所述第一进程将所述第一请求向所述第二节点设备发送,以使得所述第二节点设备响应于所述第一请求,并使得所述第二节点设备基于第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第一数据,所述第二进程运行在所述第二节点设备上;
    第二发送模块,用于当通过所述第一进程在预设时长内接收到所述第二节点设备通过所述第二进程发送的第一响应消息,通过所述第一进程向所述第一应用程序发送第二响应消息,所述第一响应消息用于指示向所述第二存储模块写所述第一数据的操作已完成,所述第二响应消息用于指示向所述第一存储模块写所述第一数据的操作和向所述第二存储模块写所述第一数据的操作均已完成。
  33. 根据权利要求32所述的第一节点设备,其特征在于,所述第一发送模块,还用于:
    当所述第二节点设备故障,缓存第二数据,并在所述第二节点设备恢复正常使用后,通过所述第一进程将所述第二数据向所述第二节点设备发送,以使得所述第二节点设备基于所述第二数据库引擎将所述第二数据更新至所述第二存储模块,所述第二数据为所述第一节点设备通过所述第一进程向所述第一存储模块写所述第一数据后得到的数据。
  34. 根据权利要求32-33中任一项所述的第一节点设备,其特征在于,所述调用模块,还用于:
    通过所述第一进程记录第一数据版本号,所述第一数据版本号用于表示所述第一存储模块数据更新后的第一数据版本,所述更新后的第一数据版本为向所述第一存储模块写所述第一数据的操作已完成的情况下的数据版本。
  35. 一种节点设备,其特征在于,所述节点设备应用于数据存储服务系统,所述系统部署于轮式移动设备,所述系统包括第一节点设备和第二节点设备,所述节点设备作为第一节点设备,包括:
    接收模块,用于通过运行在所述第一节点设备上的第一进程接收运行在所述第二节点设备上的第二进程转发的第二请求,所述第二请求由第二应用程序向所述第二节点设备发送;
    调用模块,用于响应于所述第二请求,并基于所述第一进程上的第一数据库引擎,通过所述第一进程向所述第一节点设备上的第一存储模块写第三数据;
    第一发送模块,用于在向所述第一存储模块写所述第三数据的操作已完成的情况下,通过所述第一进程向所述第二节点设备发送第三响应消息,所述第三响应消息用于指示所述第二节点设备响应于所述第二请求,并用于指示所述第二节点设备基于所述第二进程上的第二数据库引擎,通过所述第二进程向所述第二节点设备上的第二存储模块写所述第三数据;
    第二发送模块,用于当通过所述第一进程在预设时长内接收到所述第二节点设备通过所述第二进程发送的第四响应消息,通过所述第一进程向所述第二节点设备发送第五响应消息,以使得所述第二节点设备通过所述第二进程向所述第二应用程序转发所述第五响应消息,所述第四响应消息用于指示向所述第二存储模块写所述第三数据的操作已完成,所述第五响应消息用于指示向所述第一存储模块写所述第三数据的操作和向所述第二存储模块写所述第三数据的操作均已完成。
  36. 根据权利要求35所述的第一节点设备,其特征在于,所述第一发送模块,还用于:
    当所述第二节点设备故障,缓存第四数据,并在所述第二节点设备恢复正常使用后,通过所述第一进程将所述第四数据向所述第二节点设备发送,以使得所述第二节点设备基于所述第二数据库引擎将所述第四数据更新至所述第二存储模块,所述第四数据为所述第一节点设备通过所述第一进程向所述第一存储模块写所述第三数据后得到的数据。
  37. 根据权利要求35-36中任一项所述的第一节点设备,其特征在于,所述调用模块,还用于:
    通过所述第一进程记录第一数据版本号,所述第一数据版本号用于表示所述第一存储 模块数据更新后的第一数据版本,所述更新后的第一数据版本为向所述第一存储模块写所述第三数据的操作已完成的情况下的数据版本。
  38. 一种数据存储服务系统,其特征在于,所述系统部署于轮式移动设备,所述系统包括第一节点和第二节点,所述第一节点包括第一处理器和第一存储器,所述第一处理器与所述第一存储器耦合,所述第二节点包括第二处理器和第二存储器,所述第二处理器与所述第二存储器耦合,
    所述第一存储器,用于存储第一程序;
    所述第一处理器,用于执行所述第一存储器中的第一程序,使得所述数据存储服务系统执行如权利要求1-9中任一项所述方法中由所述第一节点设备所执行的步骤,或,使得所述数据存储服务系统执行如权利要求10-18中任一项所述方法中由所述第一节点所执行的步骤;
    所述第二存储器,用于存储第二程序;
    所述第二处理器,用于执行所述第二存储器中的第二程序,使得所述数据存储服务系统执行如权利要求1-9中任一项所述方法中由所述第二节点设备所执行的步骤,或,使得所述数据存储服务系统执行如权利要求10-18中任一项所述方法中由所述第二节点所执行的步骤。
PCT/CN2020/118086 2020-09-27 2020-09-27 一种数据的存储方法及设备 Ceased WO2022061807A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN202080004307.2A CN112513832B (zh) 2020-09-27 2020-09-27 一种数据的存储方法及设备
PCT/CN2020/118086 WO2022061807A1 (zh) 2020-09-27 2020-09-27 一种数据的存储方法及设备
EP20954658.9A EP4213036A4 (en) 2020-09-27 2020-09-27 DATA STORAGE METHOD AND APPARATUS
US18/190,323 US12229025B2 (en) 2020-09-27 2023-03-27 Data storage method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/118086 WO2022061807A1 (zh) 2020-09-27 2020-09-27 一种数据的存储方法及设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/190,323 Continuation US12229025B2 (en) 2020-09-27 2023-03-27 Data storage method and device

Publications (1)

Publication Number Publication Date
WO2022061807A1 true WO2022061807A1 (zh) 2022-03-31

Family

ID=74953151

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/118086 Ceased WO2022061807A1 (zh) 2020-09-27 2020-09-27 一种数据的存储方法及设备

Country Status (4)

Country Link
US (1) US12229025B2 (zh)
EP (1) EP4213036A4 (zh)
CN (1) CN112513832B (zh)
WO (1) WO2022061807A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024021746A1 (zh) * 2022-07-29 2024-02-01 华为技术有限公司 数据处理方法、通信系统和相关设备
EP4435604A1 (de) 2023-03-21 2024-09-25 Volkswagen Ag Verfahren und kompensationsvorrichtung für ein fahrzeug zur kompensation einer auftretenden leistungsminderung einer recheneinrichtung eines fahrzeugs
US20250363129A1 (en) * 2022-07-08 2025-11-27 Beijing Oceanbase Technology Co., Ltd. Database system, and primary database election methods and apparatuses

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114461144B (zh) * 2022-01-19 2024-04-19 清华大学 协同驾驶的数据存储装置、数据处理方法及路侧设备
CN114896703B (zh) * 2022-05-25 2025-01-10 重庆长安汽车股份有限公司 一种基于数据响应实现汽车控制的方法
CN115408394A (zh) * 2022-08-30 2022-11-29 中国民航信息网络股份有限公司 一种数据存储方法和相关装置
CN116383472A (zh) * 2022-12-23 2023-07-04 北京字跳网络技术有限公司 一种展示方法、装置、计算机设备和存储介质
CN117251294B (zh) * 2023-11-14 2024-03-08 广东仁达科技有限公司 一种基于数据库副本技术的数据实时分发多通道并行系统
CN119676257A (zh) * 2024-11-29 2025-03-21 天翼云科技有限公司 分布式存储系统以及数据处理方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107451172A (zh) * 2016-03-31 2017-12-08 阿里巴巴集团控股有限公司 用于版本管理系统的数据同步方法及设备
CN108234641A (zh) * 2017-12-29 2018-06-29 北京奇虎科技有限公司 基于分布式一致性协议实现的数据读写方法及装置
US20190087104A1 (en) * 2017-09-19 2019-03-21 Robin Systems, Inc. Storage Scheme For A Distributed Storage System
US20190227888A1 (en) * 2018-01-23 2019-07-25 International Business Machines Corporation Handling node failure in multi-node data storage systems
CN110502460A (zh) * 2018-05-16 2019-11-26 华为技术有限公司 数据处理的方法和节点
CN111291062A (zh) * 2020-01-21 2020-06-16 腾讯科技(深圳)有限公司 数据同步写入方法、装置、计算机设备及存储介质

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6574749B1 (en) * 1999-10-29 2003-06-03 Nortel Networks Limited Reliable distributed shared memory
CN100512037C (zh) 2002-05-28 2009-07-08 中兴通讯股份有限公司 一种主备冗余系统中数据同步的方法
US8892509B2 (en) * 2006-03-28 2014-11-18 Oracle America, Inc. Systems and methods for a distributed in-memory database
US7613947B1 (en) * 2006-11-30 2009-11-03 Netapp, Inc. System and method for storage takeover
CN102855163B (zh) 2011-06-27 2016-03-30 华为软件技术有限公司 一种内存数据库双机热备方法及主机
US9116862B1 (en) * 2012-01-17 2015-08-25 Amazon Technologies, Inc. System and method for data replication using a single master failover protocol
CN102945195B (zh) 2012-11-26 2015-11-18 国电南瑞科技股份有限公司 一种基于SQLite数据库的主备冗余复制方法
CN103995759B (zh) 2014-05-21 2015-04-29 中国人民解放军国防科学技术大学 基于核内外协同的高可用计算机系统故障处理方法及装置
US20180285201A1 (en) * 2017-03-28 2018-10-04 Commvault Systems, Inc. Backup operations for large databases using live synchronization
US11221939B2 (en) * 2017-03-31 2022-01-11 Commvault Systems, Inc. Managing data from internet of things devices in a vehicle
US11055264B2 (en) * 2017-05-10 2021-07-06 Dropbox, Inc. Automatically coordinating application schema changes in a distributed data storage system
US11138061B2 (en) * 2019-02-28 2021-10-05 Netapp Inc. Method and apparatus to neutralize replication error and retain primary and secondary synchronization during synchronous replication
CN109871298A (zh) 2019-03-14 2019-06-11 大连海大船舶导航国家工程研究中心有限责任公司 一种linux系统船舶交通管理数据实时双机内存热备方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107451172A (zh) * 2016-03-31 2017-12-08 阿里巴巴集团控股有限公司 用于版本管理系统的数据同步方法及设备
US20190087104A1 (en) * 2017-09-19 2019-03-21 Robin Systems, Inc. Storage Scheme For A Distributed Storage System
CN108234641A (zh) * 2017-12-29 2018-06-29 北京奇虎科技有限公司 基于分布式一致性协议实现的数据读写方法及装置
US20190227888A1 (en) * 2018-01-23 2019-07-25 International Business Machines Corporation Handling node failure in multi-node data storage systems
CN110502460A (zh) * 2018-05-16 2019-11-26 华为技术有限公司 数据处理的方法和节点
CN111291062A (zh) * 2020-01-21 2020-06-16 腾讯科技(深圳)有限公司 数据同步写入方法、装置、计算机设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4213036A4 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20250363129A1 (en) * 2022-07-08 2025-11-27 Beijing Oceanbase Technology Co., Ltd. Database system, and primary database election methods and apparatuses
WO2024021746A1 (zh) * 2022-07-29 2024-02-01 华为技术有限公司 数据处理方法、通信系统和相关设备
EP4435604A1 (de) 2023-03-21 2024-09-25 Volkswagen Ag Verfahren und kompensationsvorrichtung für ein fahrzeug zur kompensation einer auftretenden leistungsminderung einer recheneinrichtung eines fahrzeugs
DE102023202522A1 (de) 2023-03-21 2024-09-26 Volkswagen Aktiengesellschaft Verfahren und Kompensationsvorrichtung für ein Fahrzeug zur Kompensation einer auftretenden Leistungsminderung einer Recheneinrichtung eines Fahrzeugs

Also Published As

Publication number Publication date
EP4213036A4 (en) 2023-09-27
US20230229571A1 (en) 2023-07-20
CN112513832A (zh) 2021-03-16
CN112513832B (zh) 2022-08-19
US12229025B2 (en) 2025-02-18
EP4213036A1 (en) 2023-07-19

Similar Documents

Publication Publication Date Title
WO2022061807A1 (zh) 一种数据的存储方法及设备
US11687555B2 (en) Conditional master election in distributed databases
US8055735B2 (en) Method and system for forming a cluster of networked nodes
CN102945139B (zh) 存储设备驱动器和集群参与
CN102402395B (zh) 基于仲裁磁盘的高可用系统不间断运行方法
US7900085B2 (en) Backup coordinator for distributed transactions
US20080281959A1 (en) Managing addition and removal of nodes in a network
JP2002522845A (ja) フォールトトレラント・コンピュータシステム
CN116055563B (zh) 基于Raft协议的任务调度方法、系统、电子设备和介质
WO2021043124A1 (zh) 一种kbroker分布式操作系统、存储介质和电子设备
US7499987B2 (en) Deterministically electing an active node
CN114756299A (zh) 车辆故障处理方法及装置、电子装置和存储介质
CN104618466A (zh) 基于消息传递的负载均衡和过负荷控制系统及其控制方法
US8732346B2 (en) Coordination of direct I/O with a filter
CN111221587A (zh) 配置中心的管理调度方法及装置
CN110502460B (zh) 数据处理的方法和节点
EP1107119A2 (en) Extending cluster membership and quorum determinations to intelligent storage systems
CN111813334B (zh) 一种Ceph的写性能优化和双控节点组合方法
CN115202803A (zh) 一种故障处理方法及装置
CN114490759B (zh) 一种设备影子的管理方法及应用其进行设备监控的方法
CN114416405A (zh) 一种基于消息中间件的语义方法及装置
CN115437752A (zh) 一种互联网银行分布式核心业务系统事务管理方法
RU2714602C1 (ru) Способ и система для обработки данных
CN117950709A (zh) 应用接口管理方法、装置、设备、存储介质
CN118981498A (zh) 分布式数据库的实时数据处理方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20954658

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202317026044

Country of ref document: IN

ENP Entry into the national phase

Ref document number: 2020954658

Country of ref document: EP

Effective date: 20230412

NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Ref document number: 2020954658

Country of ref document: EP