WO2017190518A1 - 智能冰箱及其控制方法和控制系统 - Google Patents

智能冰箱及其控制方法和控制系统 Download PDF

Info

Publication number
WO2017190518A1
WO2017190518A1 PCT/CN2016/113927 CN2016113927W WO2017190518A1 WO 2017190518 A1 WO2017190518 A1 WO 2017190518A1 CN 2016113927 W CN2016113927 W CN 2016113927W WO 2017190518 A1 WO2017190518 A1 WO 2017190518A1
Authority
WO
WIPO (PCT)
Prior art keywords
refrigerator
item
identified
information
image information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/113927
Other languages
English (en)
French (fr)
Inventor
吴勇
党广明
赵庆海
唐鹰
卓路琪
王洁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Haier Co Ltd
Original Assignee
Qingdao Haier Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Haier Co Ltd filed Critical Qingdao Haier Co Ltd
Priority to EP16901039.4A priority Critical patent/EP3438577B1/en
Publication of WO2017190518A1 publication Critical patent/WO2017190518A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D29/00—Arrangement or mounting of control or safety devices
    • F25D29/005—Mounting of control devices
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D29/00—Arrangement or mounting of control or safety devices
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/951—Indexing; Web crawling techniques
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D2400/00—General features of, or devices for refrigerators, cold rooms, ice-boxes, or for cooling or freezing apparatus not covered by any other subclass
    • F25D2400/36—Visual displays
    • F25D2400/361—Interactive visual displays
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D2500/00—Problems to be solved
    • F25D2500/06—Stock management

Definitions

  • the invention relates to the field of household appliances, in particular to a smart refrigerator and a control method and control system thereof.
  • the Internet of Things is another wave of global informationization following computer, Internet and mobile communication. It is a new stage of global information development. With the rise of the concept of Internet of Things, the products of the home appliance industry are accelerating and intelligent, and a series of smart refrigerators are launched.
  • the existing smart refrigerator usually adopts a camera to collect pictures of the articles therein, classifies the articles according to the article identifiers corresponding to the image of the articles, and then controls the temperature of the refrigerator to keep the articles fresh.
  • the database is usually established by the following methods:
  • the tester takes several photos, defines and marks a large number of ingredients, and then uploads the obtained data to the database, and further implants the database into the refrigerator of the client, in order to ensure the accuracy of the later identification,
  • the information of the items that need to be stored in the database is very large. Therefore, a large amount of manpower and material resources are needed to build the database. Further, after the refrigerator is shipped, the database remains unchanged. Thus, with the advancement of technology, more and more When more exotic items appear, the accuracy of the camera to identify the item will be greatly reduced.
  • an embodiment of the present invention provides a control method for a smart refrigerator, including: receiving image information of an item to be identified sent by a refrigerator through a network;
  • the identification information is sent to the refrigerator through the network to be displayed through the display interface of the refrigerator.
  • the method further includes:
  • the confirmation information is that the matching result of the image information of the item to be identified and the identification information output to the refrigerator is correct;
  • the confirmation information is that the matching result of the image information of the item to be identified and the identification information output to the refrigerator is incorrect;
  • the method further includes: matching the image information and the identification information of the item to be identified one by one, and synchronizing Send to the refrigerator to display through the display interface of the refrigerator.
  • the method further includes: performing reverse auditing on the image information of the to-be-identified item displayed on the display interface of the refrigerator and the corresponding identification information by using the network database, and updating the network if database.
  • the present invention also provides a control system for a smart refrigerator, comprising:
  • a data acquisition module configured to receive, by using a network, image information of an item to be identified sent by the refrigerator
  • a query processing module configured to query a network database by using image information of the received item to be identified, and obtain identification information matched thereto;
  • an output module configured to send the identification information to the refrigerator through the network to be displayed through a display interface of the refrigerator.
  • the query processing module is further configured to: if the confirmation information is received, the confirmation information is that the matching result of the image information of the item to be identified and the identification information output to the refrigerator is correct; and The image information and the identification information of the identified item are correspondingly updated to the network database.
  • the query processing module is further configured to:
  • the confirmation information is that the matching result of the image information of the item to be identified and the identification information output to the refrigerator is incorrect;
  • the output module is further configured to: match the image information and the identification information of the item to be identified one by one, and synchronously send the same to the refrigerator to display through the display interface of the refrigerator.
  • the query processing module is further configured to: display the to-be-displayed on a display interface of the refrigerator
  • the image information of the identified item and the corresponding identification information are reversely audited by means of the network database, and if approved, the network database is updated.
  • the present invention also provides a smart refrigerator configured to employ any of the control methods of the present invention; and/or configured to include any of the control systems of the present invention.
  • the control method and control system for a smart refrigerator of the present invention identifies and matches image information of an item to be identified based on a network; further, updates a network database based on data generated during use of the refrigerator, when a new one is generated again When the image information is used, it can be processed according to the updated network database.
  • the data stored in the network database is increased, the accuracy of the background food identification is ensured, and the user is further improved.
  • FIG. 1 is a flow chart showing a control method for a smart refrigerator according to an embodiment of the present invention
  • FIG. 2 is a block diagram of a control system for a smart refrigerator in accordance with an embodiment of the present invention.
  • the present invention provides a smart refrigerator control method, the method comprising:
  • the image information of the item to be identified is image information of the item stored in the refrigerator, which is usually acquired by a collecting device provided in the refrigerator, for example, a camera.
  • a plurality of control buttons are usually disposed on the outside of the refrigerator.
  • the corresponding control button When the corresponding control button is activated, it is regarded as issuing an identification signal, and starting to activate the image information of the stored article in the refrigerator.
  • a device such as a sensor may be disposed on the refrigerator for issuing an identification signal.
  • the refrigerator door when opened, it is regarded as an identification signal, which will not be described in detail herein.
  • the number and color of the acquired items may be different according to the installation position and the change of the brightness of the light, for example, an item may exist on the image of the obtained item. There may also be many different items, which will not be described in detail here.
  • connection mode of the network is not specifically limited, for example, RFID, Bluetooth, WIFI, Internet, LAN, and the like.
  • the method further includes:
  • the network database includes: attribute information of the item and corresponding identification information.
  • the network database can be stored in a cloud or a separate server, and the refrigerator of the client side is connected to and exchanged with the network database through a network.
  • the data stored in the network database can be updated according to the operation of the user's refrigerator. Therefore, the initial data stored in the network database is not required to be very large, thereby saving manpower and material cost of building a database. Introduction.
  • the identification information is usually the item name.
  • the step S2 specifically includes:
  • the image information is preprocessed by using contour detection technology and background segmentation technology to obtain a feature vector group of image information, and the feature vector group includes: shape, color, texture, and the like of the article.
  • the identification information corresponding to the feature vector class having the largest correlation is assigned to the image information corresponding to the object to be tested.
  • each feature vector class includes multiple sets of feature vectors.
  • the similarity may be represented by a numerical value; it may be an average value, a median value, a weighted average value, and the like of the similarities of the respective feature vectors in the feature vector group.
  • the similarity is expressed by the average of the similarities of the individual feature vectors in the feature vector group.
  • the method further includes:
  • the image information is directly sent to the network database for analysis, and simultaneously sent to the display interface of the refrigerator. Further, only the identification information matched by the network database is returned to the refrigerator, and further displayed on the display interface of the refrigerator.
  • the image information and the identification information of the item to be identified are matched one by one, and are synchronously sent to the refrigerator to be displayed through the display interface of the refrigerator.
  • the identification information is synchronously displayed in the display area of the image information; or in the form of a list, the image information and the corresponding identification information are displayed one by one, and no detailed information is provided here. Narration.
  • the image information of the item to be identified matches the identification information
  • the image information of the item to be identified and the matching identification information are matched one by one and returned to the refrigerator at the same time, and displayed through the refrigerator.
  • the interface is displayed.
  • the method further includes:
  • the confirmation information is that the matching result of the image information of the item to be identified and the identification information outputted to the refrigerator is correct; and the image information and the identification information of the item to be identified are correspondingly updated to the Network database.
  • the confirmation information is that the matching result of the image information of the item to be identified and the identification information outputted to the refrigerator is incorrect; and the correct identification information is configured for the image information of the item to be identified, and Image information of the item to be identified and corresponding correct identification
  • the information is updated to the network database.
  • a similarity threshold is set, and when the similarity between the feature vector group of the image information of the item to be identified and the feature vector class in the network database is greater than the similarity threshold, the output is judged.
  • the matching result of the image information and the identification information of the item to be identified to the display interface is correct; otherwise, it is an error.
  • the similarity threshold is a fixed value, and the range of values is usually between 50% and 100%.
  • the similarity threshold is 70%
  • the similarity of the item to be identified 1 is greater than the similarity threshold preset by the system, it is determined that the matching result of the image information and the identification information of the item to be identified output to the display interface is correct. If the similarity of the item to be identified 2 is less than the similarity threshold preset by the system, it is determined that the matching result of the image information and the identification information of the item to be identified output to the display interface is an error.
  • the data stored in the network database is actively expanded by the operation of the refrigerator at the user end, thereby saving the labor and material cost of building the database.
  • the method may further include:
  • the reverse auditing process querying the network database with the identifier information, acquiring a unique feature vector class corresponding to the identifier information in the network database; and image information of the item to be identified and the feature The vector class is compared, and the item to be identified is determined by the comparison result Whether the image information and the corresponding identification information can be reverse audited by the network database.
  • the image information of the item to be identified may be preprocessed by using the contour detection technology and the background segmentation technology to acquire the feature vector group of the image information of the item to be identified.
  • the feature vector group includes: an object shape, a color, a texture, and the like.
  • an audit similarity threshold is set, when the similarity between the feature vector group of the image information of the item to be identified and the known feature vector class in the network database is greater than the similarity threshold, determining to return to the network database. The image information and the identification information of the item to be identified are reviewed, so that the user's misoperation can be avoided to disturb the network database.
  • the audit similarity threshold is also a fixed value, and the range of values is usually between 50% and 100%.
  • the identification information corresponding to the item is identified as “orange”
  • further "orange” is The picture information and the matching identification information are sent to the refrigerator for display through the display interface of the refrigerator.
  • the picture information displayed on the display interface and the matching identification information are not changed.
  • the network database is reversely queried with the identification information as "orange”. After the comparison, the confirmation is passed, and the current acquisition is confirmed.
  • the picture information and identification information of the identified item are updated to the network database.
  • the identification information corresponding to the item is identified as “orange”
  • further "orange” is The picture information and the matching identification information are sent to the refrigerator for display through the display interface of the refrigerator.
  • the picture information displayed on the display interface and the matching identification information are changed, for example, the user misuses, and the identification information of the picture information corresponding to the "orange” is changed to "banana”; and the identification information is further reversed as "banana”
  • the network database After querying the network database, after re-compare, it can be confirmed that the changed data fails to pass the audit. At this time, in order to simplify the program and prevent entry into an infinite loop, the corresponding data can be directly discarded.
  • the image information of "orange” is received through the network. After querying the network database, due to factors such as light and dark changes in light, the collected picture information is queried after querying the database, and the identification information corresponding to the item is mistakenly considered as “orange”, and the image of "orange” is further The information and the matching identification information are sent to the refrigerator for display through the display interface of the refrigerator.
  • Modifying the picture information displayed on the display interface and the matching identification information for example, changing the identification information of the picture information corresponding to "orange” from “orange” to “orange”; and further reversing the identification information as "orange”
  • the modified data is confirmed to pass the audit, and then the picture information and the identification information of the item to be identified acquired at the time are updated to the network database.
  • the number of times threshold may be set. After the user end modifies the identification information given by the network database N times in a time period, if the N is greater than or equal to the threshold number, the user end Set to maliciously operate the client and restrict it from making changes to the data for a certain period of time, so I won't go into details here.
  • the update period of the network database may also be set, for example, 1 hour, 1 day, one week, etc., or the administrator of the network database may periodically update according to the demand, when the network After receiving the corresponding update information, the database first stores the corresponding update information, and then updates the data uniformly under the set update period. Thus, the calculation amount is reduced, and details are not described herein.
  • the administrator of the network database may also detect the validity of the uploaded data, and then perform a corresponding update operation. This will not be described in detail.
  • the image information may be processed according to the updated network database, so that the data stored in the network database is increased by the data fed back by the user terminal to ensure background food identification. The accuracy.
  • the network database of the present invention is uniformly managed through the network, so that a better service can be provided according to the needs of the customer, and at the same time, the push content can be added to the refrigerator of the user terminal through the network database, thereby facilitating unified management. .
  • a control system for a smart refrigerator includes: a user end and a server end connected to the network; the user end includes: an image acquisition module 100, configured to collect the to-be-identified Image information of the item.
  • the server side includes: a data acquisition module 200, query processing module 300, output module 400, network database 500.
  • the data obtaining module 200 is configured to receive, by using a network, image information of an item to be identified sent by the refrigerator;
  • the image information of the item to be identified is image information of the item stored in the refrigerator, which is usually acquired by the image collecting module 100 provided in the refrigerator, for example, a camera.
  • a plurality of control buttons are usually disposed on the outside of the refrigerator.
  • the corresponding control button When the corresponding control button is activated, it is regarded as issuing an identification signal, and the image acquisition module 100 is started to recognize the image information of the stored articles in the refrigerator.
  • a device such as a sensor may be disposed on the refrigerator for issuing an identification signal.
  • the refrigerator door when opened, it is regarded as an identification signal, which will not be described in detail herein.
  • the number and color of the acquired items may be different according to the installation position and the change of the brightness of the light, for example, an item may exist on the image of the obtained item. There may also be many different items, which will not be described in detail here.
  • connection mode of the network is not specifically limited, for example, RFID, Bluetooth, WIFI, Internet, LAN, and the like.
  • the query processing module 300 is configured to: query the network database 500 by using the received image information of the item to be identified, and obtain the identification information matched thereto;
  • the network database 500 includes attribute information of the item and corresponding identification information.
  • the network database 500 can be stored in a cloud or on a separate server, and the refrigerator of the client is connected to and exchanged with the network database 500 through a network.
  • the data stored in the network database 500 can be updated according to the operation of the user's client refrigerator.
  • the network database 500 may not store data, or only store a small amount of data, thereby saving manpower and material cost of building a database. The details will be described below.
  • the identification information is usually the item name.
  • the query processing module 300 is specifically configured to:
  • the image information is preprocessed by using contour detection technology and background segmentation technology to obtain a feature vector group of image information, and the feature vector group includes: shape, color, texture, and the like of the article.
  • the identification information corresponding to the feature vector class having the largest correlation is assigned to the image information corresponding to the object to be tested.
  • each feature vector class includes multiple sets of feature vectors.
  • the similarity may be represented by a numerical value; it may be an average value, a median value, a weighted average value, and the like of the similarities of the respective feature vectors in the feature vector group.
  • the similarity is expressed by the average of the similarities of the individual feature vectors in the feature vector group.
  • the similarity of the corresponding item 1 is 90%, and the shape similarity is 96%.
  • the output module 400 is configured to: send the identification information to a refrigerator through a network to display through a display interface of the refrigerator.
  • the image information of the item is recognized, the image information is directly sent to the network database 500 for analysis, and simultaneously sent to The display interface of the refrigerator is displayed; further, only the identification information that has been analyzed after being parsed by the network database 500 is returned to the refrigerator, and further displayed on the display interface of the refrigerator.
  • the output module 400 is configured to match the image information and the identification information of the item to be identified one by one, and simultaneously send it to the refrigerator to be displayed through the display interface of the refrigerator.
  • the identification information is synchronously displayed in the display area of the image information; or in the form of a list, the image information and the corresponding identification information are displayed one by one, and no detailed information is provided here. Narration.
  • the output module 400 matches the image information of the item to be identified and the identification information matched thereto, and returns to the refrigerator at the same time.
  • the display interface of the refrigerator is displayed.
  • the query processing module 300 is further configured to: if the confirmation information is received, the confirmation information is that the matching result of the image information of the item to be identified and the identification information output to the refrigerator is correct; The image information and the identification information of the item to be identified are correspondingly updated to the network database 500.
  • the query processing module 300 is further configured to: if the confirmation information is received, the confirmation information is that the matching result of the image information of the item to be identified and the identification information output to the refrigerator is incorrect; and the image information of the item to be identified is correctly configured
  • the identification information is updated, and the image information of the item to be identified and the corresponding correct identification information are updated to the network database 500.
  • a similarity threshold is set, and the query processing module 300 determines that the similarity between the feature vector group of the image information of the item to be identified and the feature vector class in the network database 500 is greater than the similarity.
  • the threshold it is determined that the matching result of the image information and the identification information of the item to be identified output to the display interface is correct; otherwise, it is an error.
  • the similarity threshold is a fixed value, and the range of values is usually between 50% and 100%.
  • the similarity threshold is 70%
  • the similarity of the item to be identified 1 is greater than the similarity threshold preset by the system, it is determined that the matching result of the image information and the identification information of the item to be identified output to the display interface is correct.
  • the similarity of the item to be identified 2 is less than the similarity preset by the system If the degree threshold is used, it is determined that the matching result of the image information and the identification information of the item to be identified output to the display interface is an error.
  • the data stored in the network database 500 is actively expanded by the operation of the refrigerator at the user end, thereby saving the labor and material cost of building the database.
  • the query processing module 300 is further configured to: perform reverse auditing on the image information of the item to be identified and the corresponding identification information displayed on the display interface of the refrigerator through the network database 500, if The network database 500 is updated upon review.
  • the query processing module 300 queries the network database 500 with the identification information, and acquires a unique feature vector class corresponding to the identifier information in the network database 500; and image information of the item to be identified Comparing with the feature vector class, and determining, by the comparison result, whether the image information of the item to be identified and the corresponding identification information can pass the reverse review of the network database.
  • the query processing module 300 may also perform pre-processing on the image information of the item to be identified by using the contour detection technology and the background segmentation technology to acquire a feature vector group of the image information of the item to be identified.
  • the feature vector group includes: the shape, color, texture, and the like of the item.
  • the audit similarity threshold is also a fixed value, and the range of values is usually between 50% and 100%.
  • the identification information corresponding to the item is identified as “orange”
  • further "orange” is The picture information and the matching identification information are sent to the refrigerator for display through the display interface of the refrigerator.
  • the picture information displayed on the display interface and the matching identification information are not changed.
  • the network database is reversely queried with the identification information as "orange”. After the comparison, the confirmation is passed, and the current acquisition is confirmed.
  • the picture information and identification information of the identified item are updated to the network database.
  • the identification information corresponding to the item is identified as “orange”
  • further "orange” is The picture information and the matching identification information are sent to the refrigerator for display through the display interface of the refrigerator.
  • the picture information displayed on the display interface and the matching identification information are changed, for example, the user misuses, and the identification information of the picture information corresponding to the "orange” is changed to "banana”; and the identification information is further reversed as "banana”
  • the network database After querying the network database, after re-compare, it can be confirmed that the changed data fails to pass the audit. At this time, in order to simplify the program and prevent entry into an infinite loop, the corresponding data can be directly discarded.
  • the item to be identified is “orange”
  • the collected picture information is queried after querying the database due to factors such as light and dark changes of light.
  • the identification information corresponding to the item is mistaken as “orange”
  • the picture information of the "orange” and the matching identification information are further sent to the refrigerator for display through the display interface of the refrigerator.
  • Modifying the picture information displayed on the display interface and the matching identification information for example, changing the identification information of the picture information corresponding to "orange” from “orange” to “orange”; and further reversing the identification information as "orange”
  • the modified data is confirmed to pass the audit, and then the picture information and the identification information of the item to be identified acquired at the time are updated to the network database.
  • the number of times threshold may be set. After the user end modifies the identification information given by the network database N times in a time period, if the N is greater than or equal to the threshold number, the user end Set to maliciously operate the client and restrict it from making changes to the data for a certain period of time, so I won't go into details here.
  • an update period of the network database 500 may also be set, for example, 1 hour, 1 day, one week, etc., or the administrator of the network database 500 may periodically update according to requirements. After the network database 500 receives the corresponding update information, the corresponding update information is stored first, and then the data is uniformly updated under the set update period. Thus, the calculation amount is reduced, and details are not described herein.
  • the administrator of the network database 500 may also detect the validity of the uploaded data, and then perform a corresponding update operation. I will not go into details here.
  • the image information may be processed according to the updated network database 500.
  • the data stored in the network database 500 is increased to ensure the background. The accuracy of food identification.
  • the network database 500 of the present invention is uniformly managed through the network, so that a better service can be provided according to the needs of the customer, and at the same time, the push content can be added to the refrigerator of the user terminal through the network database 500, which is convenient. Unified management.
  • the smart refrigerator control method and control system of the present invention identifies and matches image information of an item to be identified based on a network; further, updates a network database based on data generated during use of the refrigerator, when regenerated When the new image information is processed, it can be processed according to the updated network database, so that the data stored in the network database is increased by the data fed back by the user-side refrigerator to ensure the accuracy of the background food identification, and further Improve user experience.
  • a smart refrigerator configured to employ a control method in accordance with any of the embodiments of the present invention; and/or configured to include a control system in accordance with any of the embodiments of the present invention.
  • the modules described as separate components may or may not be physically separated.
  • the components displayed as modules may or may not be physical modules, that is, may be located in one place, or may be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
  • each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated in one In the module.
  • the above integrated modules can be implemented in the form of hardware or in the form of hardware plus software function modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Mechanical Engineering (AREA)
  • Thermal Sciences (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)

Abstract

一种智能冰箱及其控制方法和控制系统。该控制方法包括：通过网络接收冰箱发送的待识别物品的图像信息；以接收的待识别物品的图像信息查询网络数据库，获取与其匹配的标识信息；将标识信息通过网络发送至冰箱，以通过冰箱的显示界面显示。该用于智能冰箱的控制方法及控制系统可借助用户端冰箱反馈的数据，增大网络数据库中存储的数据，保证后台食物识别的准确性，并进一步提升用户体验感。

Description

智能冰箱及其控制方法和控制系统 技术领域
本发明涉及家用电器领域，尤其涉及智能冰箱及其控制方法和控制系统。
背景技术
物联网是继计算机、互联网与移动通信之后又一次全球信息化浪潮，是全球信息化发展的新阶段，随着物联网概念的兴起，家电行业产品加速智能化，推出一系列智能冰箱。
现有的智能冰箱，通常采用摄像头采集其内的物品图片，根据物品图片对应的物品标识，对物品进行分类，进而控制冰箱的温度对物品进行保鲜等操作。
通过摄像头对物品进行识别过程中，需要预先设置一数据库对物品的图片以及相应标识进行保持。现有冰箱识别系统中，数据库的建立通常采用下述方法：
在冰箱出厂之前，测试人员对大量食材进行数次拍照、定义、标记，之后将获得的数据上传到数据库，并进一步将该数据库植入到客户端的冰箱内，为了保证后期识别的准确性，该数据库中需要存储的物品信息十分庞大，如此，需要大量的人力、物力建设该数据库，进一步的，当冰箱出厂之后，该数据库即始终保持不变，如此，随着科技的进步，当越来越多的异类物品出现时，摄像头对物品识别的准确率也会大大降低。
发明内容
本发明的目的在于提供一种智能冰箱及其控制方法和控制系统。
为实现上述发明目的之一，本发明一实施方式提供了一种用于智能冰箱的控制方法，其包括：通过网络接收冰箱发送的待识别物品的图像信息；
以接收的待识别物品的图像信息查询网络数据库，获取与其匹配的标识信息；
将所述标识信息通过网络发送至冰箱，以通过冰箱的显示界面显示。
可选地，所述方法还包括：
若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果正确；
将所述待识别物品的图像信息及标识信息对应更新至所述网络数据库。
可选地，若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果错误；
为所述待识别物品的图像信息配置正确的标识信息，并将所述待识别物品的图像信息以及对应的正确的标识信息更新至网络数据库。
可选地，以接收的待识别物品的图像信息查询网络数据库，获取与其匹配的标识信息后，所述方法还包括：将所述待识别物品的图像信息和标识信息一一进行匹配，并同步发送至冰箱，以通过冰箱的显示界面显示。
可选地，所述方法还包括：对在冰箱的显示界面显示的所述待识别物品的图像信息以及对应的标识信息借助所述网络数据库进行反向审核，若通过审核，则更新所述网络数据库。
为实现上述发明目的之一，本发明还提供了一种用于智能冰箱的控制系统，其包括：
数据获取模块，用于通过网络接收冰箱发送的待识别物品的图像信息；
查询处理模块，用于以接收的待识别物品的图像信息查询网络数据库，获取与其匹配的标识信息；
输出模块，用于将所述标识信息通过网络发送至冰箱，以通过冰箱的显示界面显示。
可选地，所述查询处理模块还用于：若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果正确；以及，将所述待识别物品的图像信息及标识信息对应更新至所述网络数据库。
可选地，所述查询处理模块还用于：
若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果错误；
为所述待识别物品的图像信息配置正确的标识信息，并将所述待识别物品的图像信息以及对应的正确的标识信息更新至网络数据库。
可选地，所述输出模块还用于：将所述待识别物品的图像信息和标识信息一一进行匹配，并同步发送至冰箱，以通过冰箱的显示界面显示。
可选地，所述查询处理模块还用于：对在冰箱的显示界面显示的所述待 识别物品的图像信息以及对应的标识信息借助所述网络数据库进行反向审核，若通过审核，则更新所述网络数据库。
本发明还提供了智能冰箱，其配置成采用本发明的任一控制方法；和/或配置成包括本发明的任一控制系统。
本发明的用于智能冰箱的控制方法及控制系统，基于网络识别待识别物品的图像信息，并对其进行匹配；进一步地，基于冰箱使用过程中产生的数据更新网络数据库，当再次产生新的图像信息时，即可以依据更新后的网络数据库对其进行处理，如此，借助用户端冰箱反馈的数据，增大所述网络数据库中存储的数据，保证后台食物识别的准确性，并进一步提升用户体验感。
根据下文结合附图对本发明具体实施例的详细描述，本领域技术人员将会更加明了本发明的上述以及其他目的、优点和特征。
附图说明
后文将参照附图以示例性而非限制性的方式详细描述本发明的一些具体实施例。附图中相同的附图标记标示了相同或类似的部件或部分。本领域技术人员应该理解，这些附图未必是按比例绘制的。附图中：
图1是根据本发明一实施例的用于智能冰箱的控制方法的流程示意图；
图2是根据本发明一实施例的用于智能冰箱的控制系统的模块示意图。
具体实施方式
以下将结合附图所示的具体实施方式对本发明进行详细描述。但这些实施方式并不限制本发明，本领域的普通技术人员根据这些实施方式所做出的结构、方法、或功能上的变换均包含在本发明的保护范围内。
结合图1所示，本发明提供一种智能冰箱控制方法，所述方法包括：
S1、通过网络接收冰箱发送的待识别物品的图像信息；
所述待识别物品的图像信息，为冰箱中存储物品的图像信息，其通常由冰箱中设置的采集装置进行获取，例如：摄像头。
为了防止冰箱误操作，所述冰箱外侧通常还设置若干控制按钮，相应控制按钮启动时，视为发出识别信号，开始启动摄像头识别冰箱中保存物品的图像信息。当然，在本发明的其他实施方式中，还可以在冰箱上设置传感器等设备，用于发出识别信号。在本发明一优选实施方式中，当打开冰箱门后，即视为发出识别信号，在此不做详细赘述。
可以理解的是，每次图像采集装置工作时，根据其安装位置以及光线的明暗度的变化，其获取物品的数量及颜色均可能不同，例如：其获取的物品图片上可能存在一种物品，也可能存在多种不同的物品，在此不做详细赘述。
所述网络的连接方式不做具体限定，例如：RFID、蓝牙、WIFI、Internet、局域网等连接方式。
进一步的，本发明一实施方式中，所述方法还包括：
S2、以接收的待识别物品的图像信息查询网络数据库，获取与其匹配的标识信息；
所述网络数据库包括：物品的属性信息以及对应的标识信息。
所述网络数据库可存储于云端或单独的服务器上，用户端的冰箱通过网络与该网络数据库进行连接以及进行交换。
所述网络数据库存储的数据可根据用户对其客户端冰箱的操作进行更新，如此，所述网络数据库存储的初始数据并不要求十分庞大，进而节约建库的人力、物力成本，以下将会详细介绍。
所述标识信息通常为物品名称。
本发明一优选实施方式中，所述步骤S2具体包括：
采用轮廓检测技术及背景分割技术对图像信息进行预处理，以获取图像信息的特征向量组，所述特征向量组包括：物品的形状、颜色、纹理等。
查找所述网络数据库，从分别对应不同物品的多个特征向量类中，选择与所述待识别物品的图像信息所对应的特征向量之间具有最大相关度的特征向量类；
将所述具有最大相关度的特征向量类所对应的标识信息赋予所述待测物品对应的图像信息。
通常情况下，每个特征向量类包括多组特征向量。
所述相似度可以用数值表示；其可为特征向量组中的各个特征向量的相似度的平均值，中值、加权平均值等。
例如：以相似度采用特征向量组中的各个特征向量的相似度的平均值表示，
假设将待识别物品的图像信息的特征向量组与所述网络数据库400中的特征向量类进行比对搜索后，待识别物品1，其对应的颜色的相似度为90％、形状相似度为96％、纹理相似度为93％，那么所述相似度的值为 (90％+96％+93％)/3＝93％；待识别物品2，其对应的颜色的相似度为50％、形状相似度为40％、纹理相似度为60％，那么所述相似度的值为(50％+40％+60％)/3＝50％，在此不做详细赘述。
进一步的，本发明一实施方式中，所述方法还包括：
S3、将所述标识信息通过网络发送至冰箱，以通过冰箱的显示界面显示。
本实施方式中，为了避免冰箱与网络数据库交互过程中产生较多的数据，当识别到物品的图像信息后，将所述图像信息直接发送给网络数据库进行解析，同时，发送到冰箱的显示界面进行显示；进一步的，仅将经过网络数据库解析后匹配的标识信息返回给冰箱，并进一步的在冰箱的显示界面进行显示。
本发明另一实施方式中，将所述待识别物品的图像信息和标识信息一一进行匹配，并同步发送至冰箱，以通过冰箱的显示界面显示。
相应的，冰箱的显示界面上，在图像信息的显示区域内，同步显示标识信息；或是以列表的形式，一一进行匹配显示所述图像信息以及其对应的标识信息，在此不做详细赘述。
该实施方式中，当所述待识别物品的图像信息匹配到标识信息后，将所述待识别物品的图像信息和与之匹配的标识信息一一进行匹配并同时返回给冰箱，通过冰箱的显示界面进行显示，如此，当接收的图像信息为多个物品的图像信息时，用户可以更易观察图像信息与标识信息的匹配关系。
当然，显示方式有多种，例如：分区域显示、列表显示等，在此不做详细赘述。
进一步的，本发明一实施方式中，所述方法还包括：
S41、若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果正确；将所述待识别物品的图像信息及标识信息对应更新至所述网络数据库。
该实施方式中，判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果是否正确；若匹配结果正确，将获得的待识别物品的图像信息及标识信息更新至所述网络数据库。
S42、若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果错误；为所述待识别物品的图像信息配置正确的标识信息，并将所述待识别物品的图像信息以及对应的正确的标识 信息更新至网络数据库。
该实施方式中，判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果是否错误；若匹配结果错误，为所述待识别物品的图像信息配置正确的标识信息，并将所述待识别物品的图像信息以及对应的标识信息更新至所述网络数据库。
上述步骤S41、S42为选择关系，在此不做详细赘述。
判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果是否正确有多种方式；
本发明一种实施方式中，借助用户辅助识别输出至冰箱的显示界面的物品图像信息和标识信息的匹配结果是否正确，并由用户操作显示界面进而更新相应数据。
本发明一种实施方式中，设置一相似度阈值，当所述待识别物品的图像信息的特征向量组与所述网络数据库中的特征向量类的相似度大于所述相似度阈值时，判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果为正确；否则为错误。
所述相似度阈值为一固定数值，其取值范围通常在50％至100％之间。
为了方便理解，以下将描述一具体示例用于理解本发明。
假设相似度阈值为70％，那么接续上述示例，待识别物品1的相似度大于系统预设的相似度阈值，则判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果为正确；待识别物品2的相似度小于系统预设的相似度阈值，则判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果为错误。
如此，通过用户端的冰箱操作，主动扩充所述网络数据库中存储的数据，进而节约建库的人力、物力成本。
本发明一优选实施方式中，所述方法还可包括：
S5、对在冰箱的显示界面显示的所述待识别物品的图像信息以及对应的标识信息通过所述网络数据库进行反向审核，若通过审核，则更新所述网络数据库。
所述反向审核过程中，以所述标识信息查询所述网络数据库，获取所述网络数据库中与所述标识信息对应的唯一特征向量类；将所述待识别物品的图像信息与所述特征向量类进行比对，并通过比对结果确定所述待识别物品 的图像信息以及对应的标识信息是否可以通过所述网络数据库的反向审核。
本发明具体示例中，结合步骤S2所述内容，可以同样采用轮廓检测技术及背景分割技术对所述待识别物品的图像信息进行预处理，以获取所述待识别物品的图像信息的特征向量组，所述特征向量组包括：物品的形状、颜色、纹理等。
同样设置一审核相似度阈值，当所述待识别物品的图像信息的特征向量组与所述网络数据库中已知的特征向量类的相似度大于所述相似度阈值时，判断返回至网络数据库的待识别物品的图像信息和标识信息通过审核，如此，可以避免用户误操作而扰乱网络数据库。
所述审核相似度阈值同样为一固定数值，其取值范围通常在50％至100％之间。
场景1下：
例如：待识别物品为“橘子”，在通过网络接收到“橘子”的图片信息时，经过查询网络数据库后，识别出该物品对应的标识信息为“橘子”，并进一步的将“橘子”的图片信息和匹配的标识信息发送至冰箱，以通过冰箱的显示界面显示。
显示界面显示的图片信息和匹配的标识信息未做任何改动；同时，以标识信息为“橙子”反向查询所述网络数据库，经过再次比对后，确认通过审核，并将当次获取的待识别物品的图片信息和标识信息更新至网络数据库。
场景2下：
例如：待识别物品为“橘子”，在通过网络接收到“橘子”的图片信息时，经过查询网络数据库后，识别出该物品对应的标识信息为“橘子”，并进一步的将“橘子”的图片信息和匹配的标识信息发送至冰箱，以通过冰箱的显示界面显示。
对显示界面显示的图片信息和匹配的标识信息进行改动，例如：用户误操作，将对应“橙子”的图片信息的标识信息更改为“香蕉”；并进一步的以标识信息为“香蕉”反向查询所述网络数据库，经过再次比对后，可确认改动后的数据未能通过审核，此时，为了简化程序，防止进入死循环，可直接对该相应数据做丢弃处理。
场景3下：
例如：待识别物品为“橘子”，在通过网络接收到“橘子”的图片信息 时，经过查询网络数据库后，由于光线的明暗变化等因素，采集出的图片信息经过查询数据库后，识别出该物品对应的标识信息被误认为“橙子”，并进一步的将“橙子”的图片信息和匹配的标识信息发送至冰箱，以通过冰箱的显示界面显示。
对显示界面显示的图片信息和匹配的标识信息进行改动，例如：将对应“橙子”的图片信息的标识信息由“橙子”更改为“橘子”；并进一步的以标识信息为“橘子”反向查询所述网络数据库，经过再次比对后，改动后的数据确认通过审核，如此后，将当次获取的待识别物品的图片信息和标识信息更新至网络数据库。
当然，在本发明的其他实施方式中，还可以设置次数阈值，用户端在一个时间段内，对网络数据库给出的标识信息N次进行修改后，N大于等于次数阈值，则将该用户端设置为恶意操作用户端，并限制其在一定时间内禁止对数据进行更改，在此不做详细赘述。
当然，在本发明的技术领域中，通过图像比对，判断数据的准确性还具有多种方式，在此不在一一列举。在本发明的其他优选实施方式中，为了缩减计算量，还可以设置网络数据库的更新周期，例如：1小时、1天、一周等，也可以有网络数据库的管理者按照需求定期更新，当网络数据库接收到相应的更新信息后，先对相应的更新信息进行存储，之后在设定的更新周期下，统一对数据进行更新，如此，缩减计算量，在此不做详细赘述。
另外，本发明一实施方式中，当所述网络数据库接收到用户端上传的数据后，也可以由所述网络数据库的管理者辅助检测上传数据的有效性，之后再进行相应的更新操作，在此不做详细赘述。
进一步的，当再次通过网络接收到图像信息后，即可以依据更新后的网络数据库对其进行处理，如此，借助用户端反馈的数据，增大所述网络数据库中存储的数据，保证后台食物识别的准确性。
可以理解的是，本发明的网络数据库，通过网络统一进行管理，如此，可以根据客户的需求，给与更好的服务，同时，还可以通过网络数据库为用户端的冰箱增加推送内容，便于统一管理。
结合图2所示，根据本发明一实施例的用于智能冰箱的控制系统包括：用户端以及与之进行网络连接的服务器端；所述用户端包括：图像采集模块100，用于采集待识别物品的图像信息。所述服务器端包括：数据获取模块 200，查询处理模块300，输出模块400、网络数据库500。
数据获取模块200用于通过网络接收冰箱发送的待识别物品的图像信息；
所述待识别物品的图像信息，为冰箱中存储物品的图像信息，其通常由冰箱中设置的图像采集模块100进行获取，例如：摄像头。
为了防止冰箱误操作，所述冰箱外侧通常还设置若干控制按钮，相应控制按钮启动时，视为发出识别信号，开始启动图像采集模块100识别冰箱中保存物品的图像信息。当然，在本发明的其他实施方式中，还可以在冰箱上设置传感器等设备，用于发出识别信号。在本发明一优选实施方式中，当打开冰箱门后，即视为发出识别信号，在此不做详细赘述。
可以理解的是，每次图像采集模块100工作时，根据其安装位置以及光线的明暗度的变化，其获取物品的数量及颜色均可能不同，例如：其获取的物品图片上可能存在一种物品，也可能存在多种不同的物品，在此不做详细赘述。
所述网络的连接方式不做具体限定，例如：RFID、蓝牙、WIFI、Internet、局域网等连接方式。
进一步的，本发明一实施方式中，查询处理模块300用于：以接收的待识别物品的图像信息查询网络数据库500，获取与其匹配的标识信息；
网络数据库500包括：物品的属性信息以及对应的标识信息。
网络数据库500可存储于云端或单独的服务器上，用户端的冰箱通过网络与该网络数据库500进行连接以及进行交换。
网络数据库500存储的数据可根据用户对其客户端冰箱的操作进行更新，如此，网络数据库500在初始状态下，可以不存储数据，或仅存储少量数据，进而节约建库的人力、物力成本，以下将会详细介绍。
所述标识信息通常为物品名称。
本发明一优选实施方式中，查询处理模块300具体用于：
采用轮廓检测技术及背景分割技术对图像信息进行预处理，以获取图像信息的特征向量组，所述特征向量组包括：物品的形状、颜色、纹理等。
查找所述网络数据库500，从分别对应不同物品的多个特征向量类中，选择与所述待识别物品的图像信息所对应的特征向量之间具有最大相关度的特征向量类；
将所述具有最大相关度的特征向量类所对应的标识信息赋予所述待测物品对应的图像信息。
通常情况下，每个特征向量类包括多组特征向量。
所述相似度可以用数值表示；其可为特征向量组中的各个特征向量的相似度的平均值，中值、加权平均值等。
例如：以相似度采用特征向量组中的各个特征向量的相似度的平均值表示，
假设将待识别物品的图像信息的特征向量组与网络数据库500中的特征向量类进行比对搜索后，待识别物品1，其对应的颜色的相似度为90％、形状相似度为96％、纹理相似度为93％，那么所述相似度的值为(90％+96％+93％)/3＝93％；待识别物品2，其对应的颜色的相似度为50％、形状相似度为40％、纹理相似度为60％，那么所述相似度的值为(50％+40％+60％)/3＝50％，在此不做详细赘述。
进一步的，本发明一实施方式中，输出模块400用于：将所述标识信息通过网络发送至冰箱，以通过冰箱的显示界面显示。
本实施方式中，为了避免用户端的冰箱与网络数据库500交互过程中产生较多的数据，当识别到物品的图像信息后，将所述图像信息直接发送给网络数据库500进行解析，同时，发送到冰箱的显示界面进行显示；进一步的，仅将经过网络数据库500解析后匹配的标识信息返回给冰箱，并进一步的在冰箱的显示界面进行显示。
本发明另一实施方式中，输出模块400或用于将所述待识别物品的图像信息和标识信息一一进行匹配，并同步发送至冰箱，以通过冰箱的显示界面显示。
相应的，冰箱的显示界面上，在图像信息的显示区域内，同步显示标识信息；或是以列表的形式，一一进行匹配显示所述图像信息以及其对应的标识信息，在此不做详细赘述。
该实施方式中，当所述待识别物品的图像信息匹配到标识信息后，输出模块400将所述待识别物品的图像信息和与之匹配的标识信息一一进行匹配并同时返回给冰箱，通过冰箱的显示界面进行显示，如此，当接收的图像信息为多个物品的图像信息时，用户可以更易观察图像信息与标识信息的匹配关系。
当然，显示方式有多种，例如：分区域显示、列表显示等，在此不做详细赘述。
进一步的，本发明一实施方式中，查询处理模块300还用于：若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果正确；将所述待识别物品的图像信息及标识信息对应更新至所述网络数据库500。
该实施方式中，判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果是否正确；若匹配结果正确，将获得的待识别物品的图像信息及标识信息更新至所述网络数据库500。
查询处理模块300还用于：若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果错误；为所述待识别物品的图像信息配置正确的标识信息，并将所述待识别物品的图像信息以及对应的正确的标识信息更新至网络数据库500。
该实施方式中，判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果是否错误；若匹配结果错误，为所述待识别物品的图像信息配置正确的标识信息，并将所述待识别物品的图像信息以及对应的标识信息更新至所述网络数据库500。
判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果是否正确有多种方式；
本发明一种实施方式中，借助用户辅助识别输出至冰箱的显示界面的物品图像信息和标识信息的匹配结果是否正确，并由用户操作显示界面进而更新相应数据。
本发明一种实施方式中，设置一相似度阈值，当查询处理模块300判断所述待识别物品的图像信息的特征向量组与所述网络数据库500中的特征向量类的相似度大于所述相似度阈值时，判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果为正确；否则为错误。
所述相似度阈值为一固定数值，其取值范围通常在50％至100％之间。
为了方便理解，以下将描述一具体示例用于理解本发明。
假设相似度阈值为70％，那么接续上述示例，待识别物品1的相似度大于系统预设的相似度阈值，则判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果为正确；待识别物品2的相似度小于系统预设的相似 度阈值，则判断输出至显示界面的待识别物品的图像信息和标识信息的匹配结果为错误。
如此，通过用户端的冰箱操作，主动扩充所述网络数据库500中存储的数据，进而节约建库的人力、物力成本。
本发明一优选实施方式中，查询处理模块300还用于：对在冰箱的显示界面显示的所述待识别物品的图像信息以及对应的标识信息通过所述网络数据库500进行反向审核，若通过审核，则更新所述网络数据库500。
所述反向审核过程中，查询处理模块300以所述标识信息查询网络数据库500，获取所述网络数据库500中与所述标识信息对应的唯一特征向量类；将所述待识别物品的图像信息与所述特征向量类进行比对，并通过比对结果确定所述待识别物品的图像信息以及对应的标识信息是否可以通过所述网络数据库的反向审核。
本发明具体示例中，查询处理模块300可以同样采用轮廓检测技术及背景分割技术对所述待识别物品的图像信息进行预处理，以获取所述待识别物品的图像信息的特征向量组，所述特征向量组包括：物品的形状、颜色、纹理等。
同样设置一审核相似度阈值，当所述待识别物品的图像信息的特征向量组与所述网络数据库中的已知特征向量类的相似度大于所述相似度阈值时，判断返回至网络数据库的待识别物品的图像信息和标识信息通过审核，如此，可以避免用户误操作而扰乱网络数据库。
所述审核相似度阈值同样为一固定数值，其取值范围通常在50％至100％之间。
场景1下：
例如：待识别物品为“橘子”，在通过网络接收到“橘子”的图片信息时，经过查询网络数据库后，识别出该物品对应的标识信息为“橘子”，并进一步的将“橘子”的图片信息和匹配的标识信息发送至冰箱，以通过冰箱的显示界面显示。
显示界面显示的图片信息和匹配的标识信息未做任何改动；同时，以标识信息为“橙子”反向查询所述网络数据库，经过再次比对后，确认通过审核，并将当次获取的待识别物品的图片信息和标识信息更新至网络数据库。
场景2下：
例如：待识别物品为“橘子”，在通过网络接收到“橘子”的图片信息时，经过查询网络数据库后，识别出该物品对应的标识信息为“橘子”，并进一步的将“橘子”的图片信息和匹配的标识信息发送至冰箱，以通过冰箱的显示界面显示。
对显示界面显示的图片信息和匹配的标识信息进行改动，例如：用户误操作，将对应“橙子”的图片信息的标识信息更改为“香蕉”；并进一步的以标识信息为“香蕉”反向查询所述网络数据库，经过再次比对后，可确认改动后的数据未能通过审核，此时，为了简化程序，防止进入死循环，可直接对该相应数据做丢弃处理。
场景3下：
例如：待识别物品为“橘子”，在通过网络接收到“橘子”的图片信息时，经过查询网络数据库后，由于光线的明暗变化等因素，采集出的图片信息经过查询数据库后，识别出该物品对应的标识信息被误认为“橙子”，并进一步的将“橙子”的图片信息和匹配的标识信息发送至冰箱，以通过冰箱的显示界面显示。
对显示界面显示的图片信息和匹配的标识信息进行改动，例如：将对应“橙子”的图片信息的标识信息由“橙子”更改为“橘子”；并进一步的以标识信息为“橘子”反向查询所述网络数据库，经过再次比对后，改动后的数据确认通过审核，如此后，将当次获取的待识别物品的图片信息和标识信息更新至网络数据库。
当然，在本发明的其他实施方式中，还可以设置次数阈值，用户端在一个时间段内，对网络数据库给出的标识信息N次进行修改后，N大于等于次数阈值，则将该用户端设置为恶意操作用户端，并限制其在一定时间内禁止对数据进行更改，在此不做详细赘述。
当然，在本发明的技术领域中，通过图像比对，判断数据的准确性还具有多种方式，在此不在一一列举。
在本发明的其他优选实施方式中，为了缩减计算量，还可以设置网络数据库500的更新周期，例如：1小时、1天、一周等，也可以有网络数据库500的管理者按照需求定期更新，当网络数据库500接收到相应的更新信息后，先对相应的更新信息进行存储，之后在设定的更新周期下，统一对数据进行更新，如此，缩减计算量，在此不做详细赘述。
另外，本发明一实施方式中，当所述网络数据库500接收到用户端上传的数据后，也可以由所述网络数据库500的管理者辅助检测上传数据的有效性，之后再进行相应的更新操作，在此不做详细赘述。
进一步的，当再次通过网络接收到图像信息后，即可以依据更新后的网络数据库500对其进行处理，如此，借助用户端反馈的数据，增大所述网络数据库500中存储的数据，保证后台食物识别的准确性。
可以理解的是，本发明的网络数据库500，通过网络统一进行管理，如此，可以根据客户的需求，给与更好的服务，同时，还可以通过网络数据库500为用户端的冰箱增加推送内容，便于统一管理。
综上所述，本发明的智能冰箱控制方法及控制系统，基于网络识别待识别物品的图像信息，并对其进行匹配；进一步的，基于冰箱使用过程中产生的数据更新网络数据库，当再次产生新的图像信息时，即可以依据更新后的网络数据库对其进行处理，如此，借助用户端冰箱反馈的数据，增大所述网络数据库中存储的数据，保证后台食物识别的准确性，并进一步提升用户体验感。
而且，在本发明的一些实施例中还提供了一种智能冰箱，其配置成采用根据本发明任一实施例的控制方法；和/或配置成包括根据本发明任一实施例的控制系统。
在本发明所提供的几个实施方式中，应该理解到，以上所描述的结构、系统以及方法的实施方式仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个模块或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或模块的间接耦合或通信连接，可以是电性，机械或其他的形式。
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施方式方案的目的。
另外，在本发明各个实施方式中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以2个或2个以上模块集成在一 个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用硬件加软件功能模块的形式实现。
最后应说明的是：以上实施方式仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施方式对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施方式所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施方式技术方案的精神和范围。

Claims (12)

  1. 一种用于智能冰箱的控制方法，包括：
    通过网络接收冰箱发送的待识别物品的图像信息；
    以接收的待识别物品的图像信息查询网络数据库，获取与其匹配的标识信息；
    将所述标识信息通过网络发送至冰箱，以通过冰箱的显示界面显示。
  2. 根据权利要求1所述的控制方法，还包括
    若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果正确；
    将所述待识别物品的图像信息及标识信息对应更新至所述网络数据库。
  3. 根据权利要求1所述的控制方法，还包括：
    若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果错误；
    为所述待识别物品的图像信息配置正确的标识信息，并将所述待识别物品的图像信息以及对应的正确的标识信息更新至网络数据库。
  4. 根据权利要求1所述的控制方法，其中
    以接收的待识别物品的图像信息查询网络数据库，获取与其匹配的标识信息后，所述方法还包括：
    将所述待识别物品的图像信息和标识信息一一进行匹配，并同步发送至冰箱，以通过冰箱的显示界面显示。
  5. 根据权利要求2所述的控制方法，还包括：
    对在冰箱的显示界面显示的所述待识别物品的图像信息以及对应的标识信息借助所述网络数据库进行反向审核，若通过审核，则更新所述网络数据库。
  6. 一种用于智能冰箱的控制系统，包括：
    数据获取模块，用于通过网络接收冰箱发送的待识别物品的图像信息；
    查询处理模块，用于以接收的待识别物品的图像信息查询网络数据库，获取与其匹配的标识信息；
    输出模块，用于将所述标识信息通过网络发送至冰箱，以通过冰箱的显示界面显示。
  7. 根据权利要求6所述的控制系统，其中所述查询处理模块还用于：
    若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果正确；
    将所述待识别物品的图像信息及标识信息对应更新至所述网络数据库。
  8. 根据权利要求6所述的控制系统，其中所述查询处理模块还用于：
    若接收到确认信息，所述确认信息为所述待识别物品的图像信息和输出至冰箱的标识信息的匹配结果错误；
    为所述待识别物品的图像信息配置正确的标识信息，并将所述待识别物品的图像信息以及对应的正确的标识信息更新至网络数据库。
  9. 根据权利要求6所述的控制系统，其中所述输出模块还用于：
    将所述待识别物品的图像信息和标识信息一一进行匹配，并同步发送至冰箱，以通过冰箱的显示界面显示。
  10. 根据权利要求6所述的控制系统，其中所述查询处理模块还用于：
    对在冰箱的显示界面显示的所述待识别物品的图像信息以及对应的标识信息借助所述网络数据库进行反向审核，若通过审核，则更新所述网络数据库。
  11. 一种智能冰箱，其配置成采用权利要求1所述的控制方法。
  12. 一种智能冰箱，其配置成包括权利要求6所述的控制系统。
PCT/CN2016/113927 2016-05-06 2016-12-30 智能冰箱及其控制方法和控制系统 Ceased WO2017190518A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP16901039.4A EP3438577B1 (en) 2016-05-06 2016-12-30 Control method and control system for identifying items stored in an intelligent refrigerator

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610302282.X 2016-05-06
CN201610302282.XA CN105953520B (zh) 2016-05-06 2016-05-06 智能冰箱控制方法及其控制系统

Publications (1)

Publication Number Publication Date
WO2017190518A1 true WO2017190518A1 (zh) 2017-11-09

Family

ID=56914470

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/113927 Ceased WO2017190518A1 (zh) 2016-05-06 2016-12-30 智能冰箱及其控制方法和控制系统

Country Status (3)

Country Link
EP (1) EP3438577B1 (zh)
CN (1) CN105953520B (zh)
WO (1) WO2017190518A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112861684A (zh) * 2021-01-29 2021-05-28 北京电解智科技有限公司 一种物品展示方法和装置
CN114508895A (zh) * 2019-03-13 2022-05-17 青岛海尔电冰箱有限公司 用于冰箱的数据交互方法及交互系统
CN115923020A (zh) * 2022-11-17 2023-04-07 长虹美菱股份有限公司 一种冰箱柜箱体发泡防差错系统及方法
CN116629285A (zh) * 2023-07-24 2023-08-22 长沙智医云科技有限公司 一种rfid温度导入智能冰箱的管理方法

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105953520B (zh) * 2016-05-06 2018-08-10 青岛海尔股份有限公司 智能冰箱控制方法及其控制系统
CN107062794A (zh) * 2017-03-29 2017-08-18 青岛海尔股份有限公司 智能冰箱，其控制方法及控制系统
CN107665247B (zh) * 2017-09-19 2020-12-25 北京奇艺世纪科技有限公司 一种物品召回方法、装置及电子设备
CN110955791A (zh) * 2018-09-27 2020-04-03 上海小蚁科技有限公司 物品入库、物品信息检索方法及装置、存储介质、终端
CN109784193A (zh) * 2018-12-20 2019-05-21 上海扩博智能技术有限公司 用于智能冰箱的终端数据采集装置及系统
CN110013197A (zh) * 2019-04-16 2019-07-16 上海天诚通信技术股份有限公司 一种扫地机器人物体识别方法
CN112988765B (zh) * 2019-12-02 2023-11-03 青岛海尔电冰箱有限公司 冰箱保鲜模型数据更新方法、设备及存储介质
CN113465251B (zh) * 2020-05-28 2022-10-18 海信集团有限公司 智能冰箱及食材识别方法
CN113468936A (zh) * 2020-06-23 2021-10-01 青岛海信电子产业控股股份有限公司 一种食材识别方法、装置和设备
CN112270217A (zh) * 2020-10-13 2021-01-26 深圳拓邦股份有限公司 应用于智能家电设备的本地图像识别装置、系统及方法
CN113065394B (zh) * 2021-02-26 2022-12-06 青岛海尔科技有限公司 用于图像识别物品的方法、电子设备及存储介质
CN118168234B (zh) * 2024-05-11 2024-09-10 苏州零距云控人工智能科技有限公司 可精确分析光影数据的智能统计冰箱

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001349661A (ja) * 2000-06-05 2001-12-21 Sharp Corp 冷蔵庫
KR20120117464A (ko) * 2011-04-15 2012-10-24 엘지전자 주식회사 냉장고 및 냉장고의 제어방법
CN103604273A (zh) * 2013-11-13 2014-02-26 四川长虹电器股份有限公司 智能冰箱食品管理的方法
CN103604271A (zh) * 2013-11-07 2014-02-26 四川长虹电器股份有限公司 一种基于智能冰箱的食品识别方法
CN104197632A (zh) * 2014-09-19 2014-12-10 合肥美的电冰箱有限公司 一种智能调温冰箱
EP2843336A1 (en) * 2013-09-03 2015-03-04 Panasonic Corporation Refrigerator and refrigerator system for monitoring food
CN104896868A (zh) * 2015-06-18 2015-09-09 合肥美菱股份有限公司 一种根据远程图像识别管理冰箱食品列表的方法
CN105953520A (zh) * 2016-05-06 2016-09-21 青岛海尔股份有限公司 智能冰箱控制方法及其控制系统
CN106016892A (zh) * 2016-05-24 2016-10-12 青岛海尔股份有限公司 智能冰箱控制方法及其控制系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014031976A1 (en) * 2012-08-23 2014-02-27 Medchain Systems, Inc. Smart storage of temperature sensitive pharmaceuticals
WO2016036015A1 (en) * 2014-09-04 2016-03-10 Samsung Electronics Co., Ltd. Refrigerator and controlling method thereof

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001349661A (ja) * 2000-06-05 2001-12-21 Sharp Corp 冷蔵庫
KR20120117464A (ko) * 2011-04-15 2012-10-24 엘지전자 주식회사 냉장고 및 냉장고의 제어방법
EP2843336A1 (en) * 2013-09-03 2015-03-04 Panasonic Corporation Refrigerator and refrigerator system for monitoring food
CN103604271A (zh) * 2013-11-07 2014-02-26 四川长虹电器股份有限公司 一种基于智能冰箱的食品识别方法
CN103604273A (zh) * 2013-11-13 2014-02-26 四川长虹电器股份有限公司 智能冰箱食品管理的方法
CN104197632A (zh) * 2014-09-19 2014-12-10 合肥美的电冰箱有限公司 一种智能调温冰箱
CN104896868A (zh) * 2015-06-18 2015-09-09 合肥美菱股份有限公司 一种根据远程图像识别管理冰箱食品列表的方法
CN105953520A (zh) * 2016-05-06 2016-09-21 青岛海尔股份有限公司 智能冰箱控制方法及其控制系统
CN106016892A (zh) * 2016-05-24 2016-10-12 青岛海尔股份有限公司 智能冰箱控制方法及其控制系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3438577A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114508895A (zh) * 2019-03-13 2022-05-17 青岛海尔电冰箱有限公司 用于冰箱的数据交互方法及交互系统
CN112861684A (zh) * 2021-01-29 2021-05-28 北京电解智科技有限公司 一种物品展示方法和装置
CN115923020A (zh) * 2022-11-17 2023-04-07 长虹美菱股份有限公司 一种冰箱柜箱体发泡防差错系统及方法
CN116629285A (zh) * 2023-07-24 2023-08-22 长沙智医云科技有限公司 一种rfid温度导入智能冰箱的管理方法
CN116629285B (zh) * 2023-07-24 2023-10-13 长沙智医云科技有限公司 一种rfid温度导入智能冰箱的管理方法

Also Published As

Publication number Publication date
EP3438577B1 (en) 2025-11-05
CN105953520A (zh) 2016-09-21
EP3438577A4 (en) 2019-05-08
CN105953520B (zh) 2018-08-10
EP3438577A1 (en) 2019-02-06

Similar Documents

Publication Publication Date Title
WO2017190518A1 (zh) 智能冰箱及其控制方法和控制系统
CN106016892B (zh) 智能冰箱控制方法及其控制系统
US11120311B2 (en) Adjusting machine settings through multi-pass training of object detection models
US10559194B2 (en) System and method for providing customized connected device functionality and for operating a connected device via an alternate object
US10168675B2 (en) Industrial machine management system, industrial machine management device, industrial machine management method, and information storage medium
CN105243060B (zh) 一种检索图片的方法及装置
US20170206439A1 (en) Providing Image Search Templates
US20220012523A1 (en) Repair diagnostic system and method
WO2020026643A1 (ja) 情報処理装置、情報処理方法及び情報処理プログラム
CN105091491A (zh) 智能冰箱的食品管理方法及装置
CN110825973A (zh) 场景和联动的推荐方法及系统、存储介质和网络侧设备
WO2021028442A1 (en) An automated support system for connected devices
CN113269828A (zh) 物品查找方法、装置、空调设备和存储介质
CN107563467A (zh) 物品寻找方法和装置
CN107578018A (zh) 基于情感识别的试衣满意度处理方法、系统及智能试衣镜
CN112770126A (zh) 直播间的推送方法、装置、服务器及存储介质
CN114417988A (zh) 操作信息的确定方法和装置、存储介质及电子装置
CN112183914B (zh) 一种菜项的评价方法及装置
CN112458695B (zh) 洗涤方法、装置、电子设备及可读存储介质
CN105933744B (zh) 遥控终端配对方法、装置及终端
CN112269894B (zh) 物品池生成方法、图片搜索方法、装置、电子设备及介质
CN113296680A (zh) 菜品图片上传方法、装置、计算机设备及可读存储介质
CN111435511B (zh) 订单处理方法、装置、设备、系统及可读存储介质
CN114898842A (zh) 一种食物营养成分的处理方法、装置以及存储介质
CN114511380A (zh) 用于确定物品属性的方法和装置

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2016901039

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2016901039

Country of ref document: EP

Effective date: 20181102

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16901039

Country of ref document: EP

Kind code of ref document: A1

WWG Wipo information: grant in national office

Ref document number: 2016901039

Country of ref document: EP