EP4351392B1 - Kochphasenidentifizierung - Google Patents

Kochphasenidentifizierung Download PDF

Info

Publication number
EP4351392B1
EP4351392B1 EP23714637.8A EP23714637A EP4351392B1 EP 4351392 B1 EP4351392 B1 EP 4351392B1 EP 23714637 A EP23714637 A EP 23714637A EP 4351392 B1 EP4351392 B1 EP 4351392B1
Authority
EP
European Patent Office
Prior art keywords
cooking
color
food item
interest
phase
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP23714637.8A
Other languages
English (en)
French (fr)
Other versions
EP4351392A1 (de
Inventor
Jingwei Tan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Versuni Holding BV
Original Assignee
Versuni Holding BV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from EP22183149.8A external-priority patent/EP4272613A1/de
Application filed by Versuni Holding BV filed Critical Versuni Holding BV
Priority claimed from PCT/EP2023/056069 external-priority patent/WO2023217436A1/en
Publication of EP4351392A1 publication Critical patent/EP4351392A1/de
Application granted granted Critical
Publication of EP4351392B1 publication Critical patent/EP4351392B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47J—KITCHEN EQUIPMENT; COFFEE MILLS; SPICE MILLS; APPARATUS FOR MAKING BEVERAGES
    • A47J36/00—Parts, details or accessories of cooking-vessels
    • A47J36/32—Time-controlled igniting mechanisms or alarm devices
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24C—DOMESTIC STOVES OR RANGES ; DETAILS OF DOMESTIC STOVES OR RANGES, OF GENERAL APPLICATION
    • F24C7/00—Stoves or ranges heated by electric energy
    • F24C7/08—Arrangement or mounting of control or safety devices
    • F24C7/082—Arrangement or mounting of control or safety devices on ranges, e.g. control panels, illumination
    • F24C7/085—Arrangement or mounting of control or safety devices on ranges, e.g. control panels, illumination on baking ovens
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/68—Food, e.g. fruit or vegetables
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]

Definitions

  • the invention relates to a cooking method, a non-transitory machine-readable medium and a cooking apparatus for identifying a phase of a cooking process.
  • Cooking apparatus may implement cooking processes such as roasting, grilling, frying, air frying, etc., to cook food.
  • consumers may use a recipe which indicates the approximate required cooking time and heating temperature to determine when the food is likely to be completely cooked. For example, when baking a certain type of cake, the recipe may indicate that the cake needs to be cooked for 60 minutes with a heating temperature of 160°C. However, consumers cannot be sure whether the cake is undercooked or overcooked in this time. There are some tips like checking the cake 10 minutes prior to the time indicated in the recipe by inserting a probe into the cake to check whether there is some raw cake dough stuck to the probe after removing the probe. However, this may not be considered to be convenient. In some cases, the checking period may be in the range of more than 20 minutes, which means the cake might be done more than 10 minutes earlier or more than 10 minutes later than expected by the recipe.
  • a doneness level may be defined for the food item to indicate by how much food item has been cooked. Examples of doneness level may include: raw, undercooked, cooked, overcooked, burnt, etc. Each doneness levels may be achieved during a certain phase of the cooking process.
  • a scale may be used to detect a weight change during the cooking process where the weight change may indicate how well cooked the food item is (e.g., moisture lost due to cooking may reduce the weight of the food item). While the doneness of some food can be checked using weight change, some food types and/or cooking processes may not be compatible with this technology. For example, some cooking processes use water bathing, or use hot air such as in an air fryer. In these cases, any detected weight loss may not provide an accurate indications of whether the food item is cooked.
  • Computer vision technology may have utility in the food industry.
  • computer-vision may be used for food quality detection in a commercial setting e.g., to remove rotten fruits, maintain the final consistent food quality, like cookies, etc.
  • these technologies may not be used during the cooking process itself.
  • the food industry may carefully control parameters such as heating time, temperature, ingredients, recipe, etc., in order to obtain a consistent cooking result without needing to monitor the cooking process itself.
  • the food ingredients, temperature, cooking time, recipe used, cooking skill, type of cooking apparatus used, etc. may vary by consumer.
  • WO 2020/148418 A1 discloses a cooking simulation software for calculating and providing data to be used for later real-time cooking, such as temperature for the cooking, time.
  • the software receives input of several parameters (e.g. food type, weight, dimension information/size, consumers sensory preference, final cooking level, or nutrition, health target), the oven information (size, heating method, heating capacity, etc.), time to consume the food.
  • Said input is provided by user input and/or detection, and is used in several mathematical modellings which are generally about math-physics equations of heat, water, material (food, oven,any physical accessory, size the oven, etc.), which are the main function of the simulation software running on powerful computers or mobile devices.
  • An input to this system is the user preferred food quality which could be one of several images of cooking levels which consumers could select one among them. It could be a full image of the food or simply a zone of the food. The only purpose of said images is to be used for consumer selection or input for calculating data to be used for later real-time cooking.
  • Certain aspects or embodiments described herein relate to identifying a phase of a cooking process based on a color of a food item. Certain aspects or embodiments may reduce or obviate certain problems associated with checking how well cooked a food item is during a cooking process in various settings.
  • a computer-implemented cooking method is disclosed in claim 1 as a first aspect.
  • the method comprises receiving first image data corresponding to a view of a food item at a first time of a cooking process implemented by a cooking apparatus.
  • a perimeter of the food item visible in the view encloses a first area of the food item.
  • the method further comprises receiving second image data corresponding to the view of the food item at a second time of the cooking process.
  • the method further comprises selecting a region of interest in the view.
  • the region of interest maps to a second area of the food item. The second area is less than the first area.
  • the method further comprises identifying a phase of the cooking process based on a comparison of a first color of the food item at the first time with a second color of the food item at the second time.
  • the first color is derived from a part of the first image data that corresponds to the region of interest.
  • the second color is derived from a part of the second image data that corresponds to the region of interest.
  • the method comprises comprising determining whether or not the identified phase of the cooking process corresponds to an expected phase of the cooking process based on recipe instructions for the food item and/or historical cooking data for the food item.
  • the phase comprises a doneness phase in which the cooking process is completed and/or browning of the food item occurs as a result of the cooking process.
  • the first color is derived from pixel intensity data associated with the part of the first image data that corresponds to the region of interest
  • the second color is derived from pixel intensity data associated with the part of the second image data that corresponds to the region of interest.
  • the method comprises deriving the first color from the pixel intensity data for the part of the first image data that corresponds to the region of interest.
  • the method may further comprise deriving the second color from the pixel intensity data for the part of the second image data that corresponds to the region of interest.
  • the method may further comprise comparing the first color with the second color to determine a color change between the first and second times.
  • the method may further comprise identifying the phase by comparing the color change with a threshold indicative of the phase of the cooking process.
  • the first and second color is represented by a color channel of a color model for representing color of the food item.
  • the method comprises applying a function to: the pixel intensity data for the part of the first image data to derive the first color; and the pixel intensity data for the part of the second image data to derive the second color.
  • the function comprises an averaging function to compute an average pixel intensity value corresponding to the first color or second color.
  • the average pixel intensity value corresponding to the first color or second color may be computed from the pixel intensity data of a set of pixels corresponding to the part of the first image data or second image data, respectively.
  • the set of pixels may be associated with the color channel.
  • the method comprises using a segmentation model to segment the first image data to identify the perimeter and/or at least part of the first area of the food item.
  • the method may further comprise selecting the region of interest in the segmented first image data.
  • the method may further comprise using the same region of interest in the second image data.
  • a non-transitory machine readable medium according to the invention is described in claim 14.
  • the non-transitory machine readable medium stores instructions readable and executable by a processor to implement the cooking method of the first aspect or any one of the related embodiments.
  • a cooking apparatus for implementing a cooking process according to the invention is described in claim 15.
  • the cooking apparatus comprises a cooking chamber for receiving a food item.
  • the cooking apparatus further comprises a housing defining the cooking chamber.
  • the cooking apparatus further comprises an air circulation system for circulating air flow inside the cooking chamber.
  • the cooking apparatus further comprises a camera for capturing images during the cooking process.
  • the cooking apparatus further comprises a controller configured to implement the cooking apparatus of the first aspect or any one of the related embodiments.
  • Certain aspects or embodiments described herein may provide various technical benefits such as improving the accuracy of identifying a phase of a cooking process, reducing compute resources needed to identify a phase of the cooking process, providing a simple but reliable way to identify the phase of the cooking process without resorting to complex methods for checking doneness (e.g., such as machine learning methods), facilitating local processing of the identification of the phase of the cooking process to reduce the cost of the compute resources needed in the cooking apparatus and/or reduce the need to rely on a cloud-based system or a communication network in order to verify how well a food item has been cooked, etc.
  • complex methods for checking doneness e.g., such as machine learning methods
  • a "cooking process” refers to applying heat to a food item to cause a change in the food item. Such application of heat may result in a mere warming of the food item, or a more substantial change in the food item such as may be achieved by using cooking methods such as roasting, grilling, frying, air frying, etc.
  • a "cooking apparatus” refers to any device capable of applying heat to a food item, in order to complete the cooking process as referred to above. Heat may be applied to the food item by the cooking apparatus in one or multiple ways such as by conduction, convection or radiation. Examples of cooking apparatus include: an oven, microwave oven, hob, air fryer, etc.
  • Figure 1 refers to a cooking method 100 according to an embodiment.
  • the cooking method 100 is computer-implemented e.g., by a processor of a cooking apparatus, as described in more detail below.
  • Image data may be obtained by a camera during the cooking process.
  • the image data depicts at least part of a food item during the cooking process. Changes to the color of the food item during the cooking process are detected via the image data.
  • the method 100 is implemented to identify a phase of a cooking process based on a color of the food item.
  • the method 100 comprises, at block 102, receiving first image data corresponding to a view of a food item at a first time of a cooking process implemented by a cooking apparatus.
  • a perimeter of the food item visible in the view encloses a first area of the food item.
  • a camera may image at least part of the food item within its field of view.
  • a perimeter of the food item may be visible to the camera.
  • the perimeter may refer to an outer edge of the food item according to the view (i.e., a different perimeter may be visible if the food item is imaged from a different direction).
  • the perimeter of the food item visible to the camera may correspond to the angular size of the food item according to the view of the food item as observed by the camera.
  • the visible part of the food item imaged by the camera may map to part (i.e., a subset of pixels) of a detection area (i.e., the complete set of pixels for forming the image) of the camera.
  • part of the food item may be within the field of view but another part of the food item may be outside the field of view or the other part of the food item may be partially obscured by another object such as another food item.
  • a perimeter of the food item may be visible. This perimeter encloses a first area of the food item where the first area may correspond to the angular size of the food item.
  • the first area may map to a certain subset of pixels of the camera.
  • certain pixels may provide a record of the pixel intensity values indicative of the color of the food item at the time during the cooking process when the food item was imaged.
  • the method 100 further comprises, at block 104, receiving second image data corresponding to the view of the food item at a second time of the cooking process.
  • the second image data may be acquired by the same camera as used to acquire the first image data although, in some cases, it may be possible for a different camera to acquire the first and second image data though this may add complexity.
  • the method 100 further comprises, at block 108, identifying a phase of the cooking process.
  • the identifying is based on a comparison of a first color of the food item at the first time with a second color of the food item at the second time.
  • the first color is derived from a part of the first image data that corresponds to the region of interest.
  • the second color is derived from a part of the second image data that corresponds to the region of interest.
  • the first and second color may be derived from the pixel intensity values recorded for the pixels corresponding to the region of interest at the first and second times (i.e., from the first and second image data, respectively).
  • the first and second colors may be derived from the pixel intensity values for at least one of the RGB channels. Not all of the pixels may record the same pixel intensity values. Therefore, a function may be applied (as described in more detail below) in order to derive the first and second color.
  • a different color model may be used to represent color in the region of interest.
  • controller 206 Any one or combination of the controller 206, cloud computing service 210 and the user equipment 212 may be used to implement the cooking method 100 and other embodiments described herein.
  • the controller 206 of the cooking apparatus of the invention implements the cooking method 100 and related embodiments.
  • the controller 206 may comprise a processor (not shown) for implementing the cooking method 100 and related embodiments.
  • processing circuitry associated with the various devices and entities of the cooking ecosystem 200 may implement the cooking method 100 and related embodiments.
  • the cooking apparatus 300 comprises a cooking chamber 302 for receiving a food item 304.
  • the cooking apparatus 300 further comprises a housing 306 defining the cooking chamber 302.
  • the cooking apparatus 300 further comprises an air circulation system 308 for circulating air flow inside the cooking chamber 302. Therefore, in this regard, the cooking apparatus 300 may have a similar form to a fan oven or an air fryer.
  • the cooking apparatus 300 further comprises a camera 310 for capturing images (of the "view" associated with the food item 304) during the cooking process. The captured images may correspond to or be used to derive the first and second image data.
  • the cooking apparatus 300 further comprises a controller 312 such as corresponding to the controller 206 of Figure 2 .
  • the controller 312 is configured to implement the cooking method 100.
  • the controller 312 is configured to receive, at block 102, first image data corresponding to a view of a food item at a first time of a cooking process implemented by a cooking apparatus.
  • a perimeter of the food item visible in the view encloses a first area of the food item.
  • the controller 312 is further configured to receive, at block 104, second image data corresponding to the view of the food item at a second time of the cooking process.
  • the controller 312 is further configured to select, at block 106, a region of interest in the view.
  • the region of interest maps to a second area of the food item.
  • the second area is less than the first area.
  • the controller 312 is further configured to identify, at block 108, a phase of the cooking process based on a comparison of a first color of the food item at the first time with a second color of the food item at the second time.
  • the first color is derived from a part of the first image data that corresponds to the region of interest.
  • the second color is derived from a part of the second image data that corresponds to the region of interest.
  • Figure 3 describes that the controller 312 of the cooking apparatus 300 implements the cooking method 100, in some cases, other devices or entities (such as depicted by Figure 2 ) may implement at least some of the functionality of the cooking method 100 (and related embodiments).
  • Figures 4(a)-(b) schematically depict example images 400 of an example food item 402 during a cooking process (i.e., each image depicts a "view" of the food item 402) at the first and second times, respectively.
  • a region of interest 404 is selected. As shown by Figures 4(a)-(b) , this region of interest 404 is a box/window corresponding to a portion of the visible area of the food item 402 (although a different shape of region of interest could be used).
  • a perimeter 406 of the food item 402 is visible in the view of the food item 402.
  • the perimeter 406 defines a first area of the food item 402 (i.e., the angular size of the food item 402), which maps to a first subset of pixels of the camera (e.g., camera 310) used to image the food item 402.
  • the region of interest 404 maps to a second subset of pixels of the camera (in this case, the second subset of pixels includes pixels that are also within the first subset).
  • the example food item 402 has shrunk between the first and second times based on a comparison of Figures 4(a) and (b) .
  • Not all food types may shrink during a cooking process. Some food types may expand and some may stay the same size.
  • the nature of the changes to the food item e.g., color, size, texture, etc.
  • a region of interest is selected.
  • the region of interest may correspond to a subset of the pixels of the camera (at least one of which corresponds to a pixel that maps to a visible part of the food item).
  • a region of interest of size 60 by 60 pixels (which equals 3600 pixels) may provide suitable pixel intensity data to implement various embodiments described herein.
  • the subset of 3600 pixels represents a small proportion of the overall number of pixels.
  • Other camera pixel resolutions and region of interest sizes may be used to implement the embodiments described herein.
  • the selection of the region of interest may be performed based on a pre-defined protocol or rule (e.g., a "selection rule") for selecting the region of interest within an image.
  • a pre-defined protocol or rule e.g., a "selection rule”
  • the selection of the region of interest may be based on a geometrical consideration (that is, a coordinate defining a location of the region of interest within the area of the food item could be selected with respect to the coordinates of the perimeter).
  • the selection rule may specify the size/shape of the region of interest, as well as the location (e.g., based on a "positioning rule") with respect to the perimeter of the food item.
  • a location defining the region of interest could be a center pixel, top-left-hand pixel, etc., of the region of interest.
  • the size of the region of interest could be pre-set (e.g., a certain number of pixels in the horizontal and vertical directions with respect to the location).
  • the location could then be set with respect to the detected perimeter of the food item. For example, the uppermost and lowermost pixels of the perimeter may be identified (to determine the apparent (vertical) height of the food item in the image).
  • the vertical location (e.g., y coordinate) of the region of interest could then be set at some predefined point (e.g., one third from the bottom, half-way up, etc., along the height of the food item) along the vertical (y-axis) direction.
  • the most left-hand point and most righthand point of the perimeter may be identified (to determine the apparent (horizontal) width of the food item in the image).
  • the horizontal location (e.g., x coordinate) of the region of interest could then be set at some predefined point (e.g., half-way across the width of the food item, etc.) along the horizontal direction.
  • the location could be at some point on the food item that is considered useful for performing color analysis.
  • Some cooking apparatus may include an illumination system which may lead to saturated colors (e.g., white) on the part of the food item directly facing the illumination system. However, the side of the food item may be less saturated. The location of the region of interest may then be selected as being on the side of the food item. This location could be determined in a pre-defined manner based on knowledge of the illumination system and camera system position with respect to the location of the food item.
  • the camera is above and to the side of the food item in the cooking chamber (so that it images the top and side of the food item) and the illumination system provides illumination towards the top of the food item, then selecting the region of interest at a location that is in the lower-half of the area of the food item within the view of the camera may yield less-saturated colors.
  • This example arrangement is represented by Figures 4(a)-(b) where the region of interest 404 is in the lower half of the food item 402.
  • the shape of the food item may influence which location on the food item is appropriate for the region of interest.
  • the region of interest may be selected based on which pixels of the image data do not appear to be too saturated or too dark (e.g., based on a maximum and minimum pixel intensity threshold, respectively). In some cases, the region of interest could be selected at random from the identified area of the food item.
  • the above description refers to various ways to select the region of interest. However, various other possible ways to select the region of interest exist.
  • a mean color value is derived from the subset of pixels at each time interval (e.g., at the first time, second time, third time and so on).
  • the pixel intensity values for a single color channel (such as the green channel) in the region of interest may be used to determine the mean color value (e.g., by adding up all of the green pixel intensity values in the region of interest and divide by the total number of green pixels in the region of interest).
  • Using a single color channel may save compute resources compared with the case of using a plurality of color channels.
  • a plurality of color channels (e.g., selected from RGB, etc.) may be used to derive the color e.g., based on a function as described below.
  • example cooking phases include a food size growing phase (where the food item expands), a size stable phase (where the food item has stopped expanding) and a doneness phase (with different doneness levels, which might be indicated by browning level or cooking time).
  • FIG. 4(a)-(b) Another example implementation of a cooking method (e.g., based on the cooking method 100) according to an embodiment is now described with reference to Figures 4(a)-(b) .
  • This example implementation may include at least one element that corresponds to or is similar to at least one element of the example implementation described previously.
  • segmentation may be performed on the image data to segment food item(s) from the background. Segmentation may be performed initially as part of the identification of the region of interest.
  • a segmentation model may be used to implement the segmentation.
  • other methods such as user selection via a user interface may be used to indicate the position and/or size of the region of interest.
  • Image-based segmentation may be performed using various techniques, which may include machine learning-based and non-machine learning-based segmentation models. Examples of non-machine learning-based models include boundary detection e.g., by using a threshold to identify an abrupt transition in pixel intensity values within an image where such an abrupt transition may be indicative of a perimeter of a food item.
  • identification of the region of interest may be performed e.g., according to any of the examples described above.
  • a first phase initial cooking occurs.
  • the color may gradually change from red/yellow into white.
  • the size of the food item may expand or grow.
  • the detected color change may be large e.g., by more than a specified pixel intensity value (e.g., more than 2). For example, where pixel intensity values are on a scale of 0 to 255, a pixel intensity value change of more than 2 may indicate that the cooking process could be in the first phase.
  • the color and/or size of the food item may be stable.
  • the surface color may become and stay white.
  • the growing may stop during the second phase.
  • the color change may be small e.g., the pixel intensity value change may be less than the specified pixel intensity value (e.g., 2 or less than 2).
  • a third phase the food is cooked and its color might become browned depending on the doneness levels.
  • some food may develop into a darker color for different doneness level.
  • the color change in the third phase may be greater than about 4 pixel intensity values to have noticeable color difference.
  • Figure 5 refers to a cooking method 500 according to an embodiment.
  • the cooking method 500 is computer-implemented e.g., by a processor of a cooking apparatus, as described above.
  • the method 500 may implement the method 100 and/or various other embodiments described herein.
  • Figure 5 refers to an example of baking bread. Certain parameter values (e.g., time intervals mentioned below) may be varied depending on the type of food being cooked.
  • the blocks of the method 500 are now described with reference to the description of the method 100 and other embodiments described herein. Certain blocks of the method 500 may be omitted or performed in a different order to that depicted by Figure 5 , in accordance with various embodiments described herein.
  • the method 500 starts at block 502 (e.g., the cooking process may be started at this point in time).
  • the method 500 proceeds to block 504 where the food type is identified.
  • bread is being cooked.
  • the cooking method may be varied appropriately.
  • the method 500 proceeds to block 506 where the food item is segmented from the background (e.g., using a segmentation model as described previously).
  • the method 500 proceeds to block 508 where the region of interest is selected.
  • the method 500 proceeds to block 510 where the cooking time, t, is obtained (i.e., the elapsed time of the cooking process).
  • the method 500 proceeds to block 512 where a check is made as to whether the cooking time, t, is equal to or greater than a preset starting time (e.g., that depends on the type of food). For bread, this could be 5 minutes. If "no", the method 500 returns to block 510.
  • a preset starting time e.g., that depends on the type of food. For bread, this could be 5 minutes. If "no", the method 500 returns to block 510.
  • the method 500 proceeds to block 514 where the color information (e.g., pixel intensity values) is extracted from the region of interest of the first image data.
  • the color information e.g., pixel intensity values
  • the color information is extracted from the region of interest of the second image data.
  • the method 500 proceeds to block 516 where the first color and second color are derived from the average pixel intensity value for the color channel in the region of interest (for each of the first and second image data, respectively).
  • Use of a single color channel such as green has been found to provide sufficient information to implement the method 500 and may reduce usage of compute resources instead of using multiple color channels.
  • the mean color is calculated for the region of interest (for each of the first and second image data, respectively). That is, the mean color at the first time corresponds to the first color and the mean color at the second time corresponds to the second color.
  • the strategy is to stop cooking when the color change crosses a threshold (e.g., goes above or below the threshold). For example, if the food is browning, the amount of acceptable browning may be defined by a threshold. Thus, when this threshold is crossed, the cooking process may be stopped or at least the temperature reduced. In case there is some variation in the recipe, ingredients, food preparation by the consumer, etc., this strategy may increase the likelihood of a consistent result and/or prevent overcooking of the food item.
  • a threshold e.g., goes above or below the threshold.
  • phase 1 from 0 to 7 minutes
  • phase 2 from 9 to 15 minutes
  • the change in pixel intensity value after each time interval is small (e.g., of order 2) due to the size stable phase of the cooking process where the growing stops and the surface of the cookies begin to brown.
  • phase 3 from 17 to 29 minutes
  • the change in pixel intensity value after each time interval is large (e.g., of order 7 to 15) due to increased browning of the surface of the cookies.
  • Figure 7 refers to a cooking method 700 according to various embodiments.
  • the cooking method 700 is computer-implemented e.g., by a processor of a cooking apparatus, as described previously.
  • the method 700 implements the cooking method 100 as well as further embodiments described below (some of which are referred to previously). Certain blocks of the method 700 may be omitted or performed in a different order to that shown by Figure 7 according to the various embodiments.
  • the cooking method 700 comprises, at block 702, using a segmentation model to segment the first image data to identify the perimeter and/or at least part of the first area of the food item.
  • the cooking method 700 further comprises, at block 704, selecting the region of interest in the segmented first image data.
  • the cooking method 700 further comprises, at block 706, using the same region of interest in the second image data. Using the same region of interest in the images acquired subsequently to the first image data may avoid the need to re-segment the image data and perform any re-selection of the region of interest since the same pixels are used between images, which may save compute resource.
  • the phase comprises an expanding phase in which the food item expands (e.g., grows) as a result of the cooking process.
  • the phase comprises a shrinking phase in which the food item shrinks as a result of the cooking process.
  • the phase comprises a size stable phase in which the food item stays the same size. This may be approximate and there may be some size change, however the size change in the size stable may be less than in the size change phase.
  • the phase comprises a doneness phase in which the cooking process is completed and/or browning of the food item occurs as a result of the cooking process.
  • the first color is derived from pixel intensity data (i.e., pixel intensity value(s)) associated with the part of the first image data that corresponds to the region of interest.
  • the second color is derived from pixel intensity data associated with the part of the second image data that corresponds to the region of interest.
  • the cooking method 700 comprises, at block 708, deriving the first color from the pixel intensity data (i.e., pixel intensity value(s)) for the part of the first image data that corresponds to the region of interest.
  • the cooking method 700 further comprises, at block 710, deriving the second color from the pixel intensity data for the part of the second image data that corresponds to the region of interest.
  • the cooking method 700 further comprises, at block 712, comparing the first color with the second color to determine a color change between the first and second times.
  • the cooking method 700 further comprises, at block 714, identifying the phase by comparing the color change with a threshold indicative of the phase of the cooking process.
  • the first and second color is represented by a color channel of a color model for representing color of the food item.
  • the color model is a red-green-blue, RGB, color model.
  • the first and second color is represented by a single color channel of the RGB color model (e.g., green). Using a single color channel may reduce compute resources needed to implement certain embodiments described herein.
  • a plurality of color channels may be used (e.g., red plus blue, red plus green, green plus blue, red plus green plus blue, etc.). For example, a function may be applied as described below in order to derive the first and second color from the plurality of color channels.
  • the color model may be the HSV color model or any other appropriate color model that may be used to represent color in image data.
  • a function is applied to the pixel intensity data for the part of the first image data to derive the first color.
  • the function is further applied to the pixel intensity data for the part of the second image data to derive the second color
  • the function comprises an averaging function to compute an average (e.g., mean) pixel intensity value corresponding to the first color or second color.
  • the average pixel intensity value corresponding to the first color or second color is computed from the pixel intensity data of a set of pixels corresponding to the part of the first image data or second image data, respectively.
  • the set of pixels is associated with the color channel.
  • the function may add up the pixel intensity values for the color channel in the region of interest and then divide the total pixel intensity value by the number of pixels for the color channel in order to compute the mean pixel intensity value.
  • Some embodiments refer to use of a single color channel such as green. However, in some embodiments, the pixel intensity data from a plurality color channels could be used to compute the average pixel intensity value.
  • the function comprises a weighting function to compute a weighted pixel intensity value corresponding to the first color or second color.
  • the weighted pixel intensity value corresponding to the first color or second color is computed from a weighted combination of the pixel intensity data of a plurality of subsets of the set of pixels corresponding to the part of the first image data or second image data, respectively. Each subset is associated with a different color channel.
  • the average pixel intensity value for every channel in the RGB color model yields a value corresponding to the color grey. Therefore, the weighting function may be to bias towards at least one of the color channels such as green. For example, a greater weight may be applied to the green channel instead of the red and/or blue channel.
  • a first subset of the set of pixels comprises the red pixels
  • a second subset of the set of pixels comprises the green pixels
  • a third subset of the set of pixels comprises the blue pixels.
  • the cooking method 700 comprises, at block 716, determining whether or not the identified phase of the cooking process corresponds to an expected phase of the cooking process based on recipe instructions for the food item and/or historical cooking data for the food item.
  • the recipe instructions comprise a cooking temperature and a cooking time at the cooking temperature for each phase of the cooking process.
  • the historical cooking data comprises the cooking temperature and the cooking time at the cooking temperature used to cook the food item previously in the cooking process.
  • the user may have an expectation of how well cooked a food item is during a cooking process. This expectation may be based on recipe instructions and/or historical cooking data, which may indicate the expected color at an expected time of the cooking process for the cooking temperature.
  • the cooking method 700 comprises, at block 718, instructing the cooking apparatus to modify the cooking process.
  • the modification of the cooking process may be in response to the recipe instructions being indicative of a different cooking temperature and/or cooking time at the cooking temperature to use for the identified phase of the cooking process or a subsequent phase of the cooking process
  • the modification of the cooking process may be in response to the identified phase of the cooking process being indicative of a deviation from the expected phase of the cooking process.
  • selecting the region of interest is based on a selection rule that indicates a location of the region of interest to use for the food item.
  • selecting the region of interest is based on a selection rule that indicates a shape of the region of interest.
  • the cooking method 700 is implemented by a processor (e.g., a processor of the controller 206) of the cooking apparatus.
  • FIG 8 is a schematic drawing of a non-transitory machine-readable medium 800 for implementing various embodiments described herein.
  • the machine-readable medium 800 stores instructions 802 readable and executable by a processor 804 to implement the method of any of the embodiments described herein (e.g., cooking methods 100, 500, 700 and/or related embodiments).
  • the machine-readable medium 800 and/or the processor 804 may be implemented by any of the controller 206, cloud computing service 210, user equipment 212 and/or controller 312 of Figures 2 or 3 .
  • the apparatus 900 further comprises a memory 906 (e.g., non-transitory or otherwise) storing instructions 908 readable and executable by the processor 902 to implement various embodiments described herein (e.g., cooking method 100 or any of the associated embodiments).
  • a memory 906 e.g., non-transitory or otherwise
  • instructions 908 readable and executable by the processor 902 to implement various embodiments described herein (e.g., cooking method 100 or any of the associated embodiments).
  • Embodiments in the present disclosure can be provided as methods, systems or as a combination of machine-readable instructions and processing circuitry.
  • Such machine-readable instructions may be included on a non-transitory machine (for example, computer) readable storage medium (including but not limited to disc storage, CD-ROM, optical storage, flash storage, etc.) having computer readable program codes therein or thereon.
  • Such machine-readable instructions may also be stored in a computer readable storage that can guide the computer or other programmable data processing devices to operate in a specific mode.
  • Such machine-readable instructions may also be loaded onto a computer or other programmable data processing devices, so that the computer or other programmable data processing devices perform a series of operations to produce computer-implemented processing, thus the instructions executed on the computer or other programmable devices realize functions specified by block(s) in the flow charts and/or in the block diagrams.
  • teachings herein may be implemented in the form of a computer program product, the computer program product being stored in a storage medium and comprising a plurality of instructions for making a computer device implement the methods recited in the embodiments of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Food Science & Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Chemical & Material Sciences (AREA)
  • Mechanical Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Combustion & Propulsion (AREA)
  • General Preparation And Processing Of Foods (AREA)
  • Image Analysis (AREA)

Claims (15)

  1. Computerimplementiertes Garverfahren (100), umfassend:
    Empfangen (102) erster Bilddaten, die einer Ansicht eines Nahrungsmittels zu einem ersten Zeitpunkt eines von einer Gareinrichtung implementierten Garprozesses entsprechen, wobei ein in der Ansicht sichtbarer Umfang des Nahrungsmittels einen ersten Bereich des Nahrungsmittels umschließt, wobei das Verfahren gekennzeichnet ist durch
    Empfangen (104) zweiter Bilddaten, die der Ansicht des Nahrungsmittels zu einem zweiten Zeitpunkt des Garprozesses entsprechen;
    Auswählen (106) einer Region von Interesse in der Ansicht, wobei die Region von Interesse einem zweiten Bereich des Nahrungsmittels zugeordnet ist und wobei der zweite Bereich kleiner als der erste Bereich ist; und
    Identifizieren (108) einer Phase des Garprozesses basierend auf einem Vergleich einer ersten Farbe des Nahrungsmittels zum ersten Zeitpunkt mit einer zweiten Farbe des Nahrungsmittels zum zweiten Zeitpunkt, wobei die erste Farbe aus einem Teil der ersten Bilddaten abgeleitet wird, welcher der Region von Interesse entspricht, und wobei die zweite Farbe aus einem Teil der zweiten Bilddaten abgeleitet wird, welcher der Region von Interesse entspricht.
  2. Computerimplementiertes Garverfahren (700) nach Anspruch 1, das Bestimmen (716) umfasst, ob die identifizierte Phase des Garprozesses einer erwarteten Phase des Garprozesses basierend auf Rezeptanweisungen für das Nahrungsmittel und/oder historischen Gardaten für das Nahrungsmittel entspricht oder nicht.
  3. Computerimplementiertes Garverfahren nach Anspruch 2, wobei:
    die Rezeptanweisungen eine Gartemperatur und eine Garzeit bei der Gartemperatur für jede Phase des Garprozesses umfassen; und
    die historischen Gardaten die Gartemperatur und die Garzeit bei der Gartemperatur umfassen, die verwendet wird, um das Nahrungsmittel zuvor im Garprozess zu garen.
  4. Computerimplementierte Garverfahren nach einem der Ansprüche 2 bis 3, das Anweisen (718) der Gareinrichtung umfasst, den Garprozess als Reaktion darauf zu ändern, dass:
    die Rezeptanweisungen eine verschiedene Gartemperatur und/oder Garzeit bei der Gartemperatur angeben, die für die identifizierte Phase des Garprozesses oder eine nachfolgende Phase des Garprozesses zu verwenden sind; und/oder
    die identifizierte Phase des Garprozesses eine Abweichung von der erwarteten Phase des Garprozesses angibt.
  5. Computerimplementiertes Garverfahren nach einem der Ansprüche 1 bis 4, wobei die Phase umfasst:
    eine Ausdehnungsphase, in der sich das Nahrungsmittel infolge des Garprozesses ausdehnt;
    eine Schrumpfphase, in der das Nahrungsmittel infolge des Garprozesses schrumpft;
    eine größenstabile Phase, in der das Nahrungsmittel die gleiche Größe beibehält;
    eine Pigmentierungsänderungsphase, in der eine pigmentierte Komponente des Nahrungsmittels aus dem Nahrungsmittel entweicht oder infolge des Garprozesses eine chemische Änderung erfährt; und/oder
    eine Garstufenphase, in welcher der Garprozess abgeschlossen ist und/oder infolge des Garprozesses eine Bräunung des Nahrungsmittels erfolgt.
  6. Computerimplementiertes Garverfahren nach einem der Ansprüche 1 bis 5, wobei:
    die erste Farbe aus Pixelintensitätsdaten abgeleitet wird, die mit dem Teil der ersten Bilddaten verknüpft sind, der der Region von Interesse entspricht; und
    die zweite Farbe aus Pixelintensitätsdaten abgeleitet wird, die mit dem Teil der zweiten Bilddaten verknüpft sind, der der Region von Interesse entspricht.
  7. Computerimplementiertes Garverfahren (700) nach Anspruch 6, umfassend:
    Ableiten (708) der ersten Farbe aus den Pixelintensitätsdaten für den Teil der ersten Bilddaten, der der Region von Interesse entspricht;
    Ableiten (710) der zweiten Farbe aus den Pixelintensitätsdaten für den Teil der zweiten Bilddaten, der der Region von Interesse entspricht;
    Vergleichen (712) der ersten Farbe mit der zweiten Farbe, um eine Farbänderung zwischen dem ersten und dem zweiten Zeitpunkt zu bestimmen; und
    Identifizieren (714) der Phase durch Vergleichen der Farbänderung mit einem Schwellenwert, der die Phase des Garprozesses angibt.
  8. Computerimplementiertes Garverfahren nach Anspruch 7, wobei:
    die erste und zweite Farbe durch einen Farbkanal eines Farbmodells zur Darstellung der Farbe des Nahrungsmittels dargestellt werden.
  9. Computerimplementierte Garverfahren nach Anspruch 8, wobei das Farbmodell ein Rot-Grün-Blau-, RGB-, Farbmodell ist, und wobei die erste und zweite Farbe durch einen einzigen Farbkanal des RGB-Farbmodells dargestellt werden.
  10. Computerimplementiertes Garverfahren nach einem der Ansprüche 8 bis 9, welches das Anwenden einer Funktion umfasst auf:
    die Pixelintensitätsdaten für den Teil der ersten Bilddaten, um die erste Farbe abzuleiten; und
    die Pixelintensitätsdaten für den Teil der zweiten Bilddaten, um die zweite Farbe abzuleiten, wobei die Funktion eines umfasst von:
    einer Mittelungsfunktion, um einen Pixelintensitätsmittelwert zu berechnen, welcher der ersten Farbe oder zweiten Farbe entspricht, wobei der Pixelintensitätsmittelwert, welcher der ersten Farbe oder zweiten Farbe entspricht, aus den Pixelintensitätsdaten eines Pixelsatzes berechnet wird, der dem Teil der ersten Bilddaten bzw. zweiten Bilddaten entspricht, und wobei der Pixelsatz mit dem Farbkanal verknüpft ist; oder
    einer Gewichtungsfunktion, um einen gewichteten Pixelintensitätswert zu berechnen, welcher der ersten Farbe oder zweiten Farbe entspricht, wobei der gewichtete Pixelintensitätswert, welcher der ersten Farbe oder zweiten Farbe entspricht, aus einer gewichteten Kombination der Pixelintensitätsdaten einer Vielzahl von Teilmengen des Pixelsatzes berechnet wird, die dem Teil der ersten Bilddaten bzw. zweiten Bilddaten entsprechen, und wobei jede Teilmenge mit einem verschiedenen Farbkanal verknüpft ist.
  11. Computerimplementiertes Garverfahren nach einem der Ansprüche 1 bis 10, umfassend:
    Verwenden (702) eines Segmentierungsmodells, um die ersten Bilddaten zu segmentieren, um den Umfang und/oder zumindest ein Teil des ersten Bereichs des Nahrungsmittels zu identifizieren;
    Auswählen (704) der Region von Interesse in den segmentierten ersten Bilddaten; und
    Verwenden (706) der gleichen Region von Interesse in den zweiten Bilddaten.
  12. Computerimplementiertes Garverfahren nach einem der Ansprüche 1 bis 11, umfassend:
    Auswählen der Region von Interesse basierend auf einer Auswahlregel, die angibt:
    einen Standort der Region von Interesse, die für das Nahrungsmittel zu verwenden ist; und/oder
    eine Form der Region von Interesse, wobei die Auswahlregel den Standort und/oder Form angibt basierend auf:
    einer Positionierungsregel zur Positionierung der Region von Interesse in Bezug auf einen segmentierten Teil der ersten und zweiten Bilddaten;
    einer Art des Nahrungsmittels; und/oder
    einem Entwurf der Gareinrichtung.
  13. Computerimplementiertes Garverfahren nach einem der Ansprüche 1 bis 12, wobei das Verfahren von einem Prozessor der Gareinrichtung implementiert wird.
  14. Nichtflüchtiges maschinenlesbares Medium (800), das Anweisungen (802) speichert, die von einem Prozessor (804) gelesen und ausgeführt werden können, um das computerimplementierte Garverfahren nach einem der Ansprüche 1 bis 13 zu implementieren.
  15. Gareinrichtung (300) zum Implementieren eines Garprozesses, umfassend:
    eine Garkammer (302) zum Aufnehmen eines Nahrungsmittels (304);
    ein Gehäuse (306), welches die Garkammer definiert;
    ein Luftzirkulationssystem (308) zum Zirkulieren des Luftstroms innerhalb des Garraums;
    eine Kamera (310) zum Aufnehmen von Bildern während des Kochvorgangs; und
    eine Steuereinheit (312), die dazu konfiguriert ist, um:
    erste Bilddaten zu empfangen (102), die einer Ansicht eines Nahrungsmittels zu einem ersten Zeitpunkt eines von einer Gareinrichtung implementierten Garprozesses entsprechen, wobei ein in der Ansicht sichtbarer Umfang des Nahrungsmittels einen ersten Bereich des Nahrungsmittels umschließt, wobei die Einrichtung dadurch gekennzeichnet ist, dass die Steuereinheit weiter konfiguriert ist, um:
    zweite Bilddaten zu empfangen (104), die der Ansicht des Nahrungsmittels zu einem zweiten Zeitpunkt des Garprozesses entsprechen;
    eine Region von Interesse in der Ansicht auszuwählen (106), wobei die Region von Interesse einem zweiten Bereich des Nahrungsmittels zugeordnet ist und wobei der zweite Bereich kleiner als der erste Bereich ist; und
    eine Phase des Garprozesses basierend auf einem Vergleich einer ersten Farbe des Nahrungsmittels zum ersten Zeitpunkt mit einer zweiten Farbe des Nahrungsmittels zum zweiten Zeitpunkt zu identifizieren (108), wobei die erste Farbe aus einem Teil der ersten Bilddaten abgeleitet wird, welcher der Region von Interesse entspricht, und wobei die zweite Farbe aus einem Teil der zweiten Bilddaten abgeleitet wird, welcher der Region von Interesse entspricht.
EP23714637.8A 2022-05-07 2023-03-09 Kochphasenidentifizierung Active EP4351392B1 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN2022091564 2022-05-07
EP22183149.8A EP4272613A1 (de) 2022-05-07 2022-07-05 Kochphasenidentifizierung
PCT/EP2023/056069 WO2023217436A1 (en) 2022-05-07 2023-03-09 Cooking phase identification

Publications (2)

Publication Number Publication Date
EP4351392A1 EP4351392A1 (de) 2024-04-17
EP4351392B1 true EP4351392B1 (de) 2025-07-09

Family

ID=90231721

Family Applications (1)

Application Number Title Priority Date Filing Date
EP23714637.8A Active EP4351392B1 (de) 2022-05-07 2023-03-09 Kochphasenidentifizierung

Country Status (3)

Country Link
US (1) US12502028B2 (de)
EP (1) EP4351392B1 (de)
PL (1) PL4351392T3 (de)

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9538880B2 (en) * 2012-05-09 2017-01-10 Convotherm Elektrogeraete Gmbh Optical quality control system
KR20140089729A (ko) 2013-01-07 2014-07-16 재단법인 아산사회복지재단 음식사진을 이용한 칼로리 자동 계산 방법 및 이를 이용한 식습관 관리 시스템
EP3277096B1 (de) * 2015-03-27 2020-04-22 Prashant Choudhary Autonome kochvorrichtung zur speisenzubereitung aus einer rezeptdatei
WO2016179424A1 (en) * 2015-05-05 2016-11-10 June Life, Inc. Connected food preparation system and method of use
US11156366B2 (en) * 2015-09-10 2021-10-26 Brava Home, Inc. Dynamic heat adjustment of a spectral power distribution configurable cooking instrument
CN106446909A (zh) 2016-09-06 2017-02-22 广东顺德中山大学卡内基梅隆大学国际联合研究院 一种中餐食物图像特征提取方法
CN111527348B (zh) * 2017-08-11 2023-03-07 布拉瓦家居公司 可配置的烹饪系统和方法
CN109166054A (zh) * 2017-08-16 2019-01-08 嘉易烤株式会社 烹饪鱼肉类时碳化时点管理服务提供系统
FR3091814A1 (fr) 2019-01-18 2020-07-24 Institut National De La Recherche Agronomique Procédé de contrôle de la cuisson d’un produit alimentaire dans une enceinte à convection avec circulation de fluide
CN110123149B (zh) 2019-05-10 2021-08-03 九阳股份有限公司 一种烹饪设备的烹饪控制方法以及烹饪设备
US11048976B2 (en) * 2019-11-11 2021-06-29 Midea Group Co., Ltd. Method and system for controlling machines based on object recognition
CN111358306B (zh) 2020-04-30 2021-08-24 广东美的厨房电器制造有限公司 加热控制方法及烹饪设备
CN110956217A (zh) 2019-12-06 2020-04-03 广东美的白色家电技术创新中心有限公司 食物成熟度识别方法、装置以及计算机存储介质
JP2021131179A (ja) 2020-02-19 2021-09-09 三星電子株式会社Samsung Electronics Co., Ltd. 調理機器
CN112056945B (zh) 2020-08-12 2024-07-30 华帝股份有限公司 一种烹饪设备食物烹饪状态的提醒方法
KR102945821B1 (ko) 2020-10-05 2026-03-31 엘지전자 주식회사 조리 기기 및 그의 동작 방법
CN112741508A (zh) 2021-01-27 2021-05-04 九阳股份有限公司 一种烹饪设备的控制方法及烹饪设备
CN113838004B (zh) 2021-08-31 2025-06-03 青岛海尔科技有限公司 食物烘焙的控制方法和装置、存储介质及电子装置

Also Published As

Publication number Publication date
EP4351392A1 (de) 2024-04-17
PL4351392T3 (pl) 2025-09-08
US12502028B2 (en) 2025-12-23
US20250160565A1 (en) 2025-05-22

Similar Documents

Publication Publication Date Title
US11478108B2 (en) Intelligent identification cooking system for oven
CN110780628B (zh) 烹饪设备的控制方法、装置、烹饪设备及存储介质
WO2021098474A1 (zh) 烹饪设备及其控制方法、加热控制方法和服务器
CN110806699A (zh) 烹饪设备的控制方法、装置、烹饪设备及存储介质
CN116868241A (zh) 烹饪阶段标识
EP4218424A1 (de) Implementierung eines garverfahrens
CN113838004B (zh) 食物烘焙的控制方法和装置、存储介质及电子装置
CN107095590A (zh) 烹饪方法及装置
EP4272613A1 (de) Kochphasenidentifizierung
US20250185844A1 (en) Control Method and Device for Cooking Equipment, Cooking Equipment and Storage Medium
CN112515491A (zh) 一种烤箱食材烹饪方法、烤箱和烤盘
CN111990902A (zh) 一种烹饪控制方法及装置、电子设备及存储介质
WO2023217436A1 (en) Cooking phase identification
EP4468879B1 (de) Implementierung eines garverfahrens
CN119107610A (zh) 一种油炸品熟化控制方法及其系统
EP4351392B1 (de) Kochphasenidentifizierung
CN110486761A (zh) 一种油烟机控烟方法及油烟机
AU2024203960B2 (en) Food redistribution
EP4365493A1 (de) Kochprozesssteuerung
EP4328872A1 (de) Bestimmung des temperaturzustands von lebensmitteln
EP4612439A1 (de) Kochprozesssteuerung
WO2024042095A1 (en) Food temperature state determination
EP4540794B1 (de) Garstufenbestimmung
EP4293636A1 (de) Garstufenbestimmung
EP4455557A1 (de) Lebensmittelumverteilung

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20240122

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
INTG Intention to grant announced

Effective date: 20250507

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602023004728

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20251110

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1810993

Country of ref document: AT

Kind code of ref document: T

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20251109

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20251009

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20251010

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20251009

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20260326

Year of fee payment: 4

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250709

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20260323

Year of fee payment: 4

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20260225

Year of fee payment: 4