EP4202765A1 - Verfahren und system zum zählen von backbaren lebensmittelprodukten - Google Patents

Verfahren und system zum zählen von backbaren lebensmittelprodukten Download PDF

Info

Publication number
EP4202765A1
EP4202765A1 EP22215325.6A EP22215325A EP4202765A1 EP 4202765 A1 EP4202765 A1 EP 4202765A1 EP 22215325 A EP22215325 A EP 22215325A EP 4202765 A1 EP4202765 A1 EP 4202765A1
Authority
EP
European Patent Office
Prior art keywords
product
inspection area
food products
processor module
captured image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP22215325.6A
Other languages
English (en)
French (fr)
Other versions
EP4202765C0 (de
EP4202765B1 (de
Inventor
Rafael JUAN FERNÁNDEZ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dulcesa SL
Original Assignee
Dulcesa SL
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dulcesa SL filed Critical Dulcesa SL
Publication of EP4202765A1 publication Critical patent/EP4202765A1/de
Application granted granted Critical
Publication of EP4202765C0 publication Critical patent/EP4202765C0/de
Publication of EP4202765B1 publication Critical patent/EP4202765B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06M—COUNTING MECHANISMS; COUNTING OF OBJECTS NOT OTHERWISE PROVIDED FOR
    • G06M7/00—Counting of objects carried by a conveyor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06M—COUNTING MECHANISMS; COUNTING OF OBJECTS NOT OTHERWISE PROVIDED FOR
    • G06M7/00—Counting of objects carried by a conveyor
    • G06M7/02—Counting of objects carried by a conveyor wherein objects ahead of the sensing element are separated to produce a distinct gap between successive objects
    • G06M7/04—Counting of piece goods, e.g. of boxes

Definitions

  • the present invention relates to the technical field of quality control on food product production lines and, more specifically, to computer vision systems for food products to be baked for the automatic counting and production control thereof.
  • Mass production poses a challenge in terms of process supervision, control and management, among other problems. Although many stages of the production chain are automated, other tasks that are among the simplest are usually still carried out manually, which, in addition to the natural errors introduced by human intervention, entails an interruption in the information chain that feeds the control systems that are increasingly widespread in the industry, based on Big Data technologies and artificial intelligence.
  • mass-produced bakeable food products under very specific manufacturing parameters such as baking time, are continuously introduced by means of conveyor belts into a baking module.
  • This process usually requires control personnel to confirm correct operation, which is done visually (with the accuracy that the operator can guarantee) or, sometimes, it is directly omitted and quality control of the already baked product is postponed. In both cases, the efficiency and ability to respond to errors is low.
  • the present invention describes, in a first aspect, a method for counting bakeable food products that comprises the following steps:
  • applying, by the processor module, an HSV conversion to the captured image is envisaged. More specifically, one embodiment envisages applying, by the processor module, a conversion to black and white of the captured image. More specifically, one embodiment comprises applying, by the processor module, a colour mask on the captured image which eliminates any intermediate value between black and white.
  • the colour transformations allow the background to be removed, which results in a simpler and better-quality image processing.
  • the processing of the images further comprises applying, by the processor module, a blur filter on the captured image. More specifically, one of the embodiments envisages applying, by the processor module, an erosion filter on the captured image. More specifically, one of the embodiments envisages applying, by the processor module, a dilation filter on the captured image.
  • the filters applied to the captured images make it possible to eliminate small defects in the conversion or enhance the shape of the product for better definition.
  • the pre-established minimum threshold of identified pixels to determine the presence of the product is set to at least one third of the width of the captured image.
  • the presence of a product in the inspection area if the number of identified pixels is greater than a pre-established minimum threshold, it must also be verified that the number of identified pixels is greater than said threshold in a certain number of successive readings.
  • the presence of the product is correct and is not due to a false positive caused by some impurity or loose paste stain on the conveyor belt.
  • the present invention relates to a system for counting bakeable food products, which comprises the following elements:
  • an LED lighting module arranged to illuminate the inspection area is envisaged.
  • the system is independent of external lighting conditions and can work under more homogeneous conditions.
  • One specific embodiment of the invention further comprises a conveyor belt for conveying bakeable food products to an oven, wherein the optical sensor is arranged on the conveyor belt along an axis perpendicular to the conveying direction. More specifically, it is envisaged that the optical sensor comprises hinged fastening means that can be coupled to a support structure, to adjust the inspection area along a line perpendicular to the conveying direction.
  • the processor module is a Raspberry Pi computer board and the optical sensor is a Raspberry Pi camera module.
  • Figure 1 shows a perspective view of one of the preferred embodiments of the system of the invention for recognising and counting bakeable food products ( 1 ) arranged on a conveyor belt ( 2 ).
  • Figure 2 shows a perspective view of one of the preferred embodiments of the system of the invention for recognising and counting bakeable food products ( 5 ) arranged on a conveyor belt ( 6 ).
  • a hinged part ( 7 ) fastens the casing ( 3 ) to a support structure ( 8 ), so that the inclination of the casing can be oriented and thereby adjust the image captured by the optical sensor and, therefore, the inspection area.
  • Figure 3 shows a perspective view of one of the preferred embodiments of the system of the invention for recognising and counting bakeable food products arranged on a conveyor belt.
  • the casing also has an LED lighting device ( 9 ).
  • a guide system ( 10 ) can be used to allow longitudinal displacements of the casing along the guide ( 10 ) to vary the inspection area on different points of the conveyor belt.
  • the casing can be fastened by means of screws to a fixed element of the structure.
  • the casing can be coupled to a rod or bar, with a square or round section, as long as they allow the casing to be displaced transversely to the conveying direction of the conveyor belt.
  • the LED lighting module is powered by the GPIO bus at 3.3V or 5V, depending on the needs, or even by an external power supply. With the connections made, the casing is placed in the inspection area for its operation.
  • the operation of the present invention is generally based on the fact that, after performing an HSV conversion of the image by applying colour filters, blurring, eroding, dilation of the pixels and their conversion to black and white, the pixels in a certain area are counted and, after a minimum margin of pixels has been exceeded, the determination is made that there is a product in said area and a product counter is updated.
  • Figure 4 schematically shows the main functional blocks of the invention, according to the main process. Following the flow chart represented, this process begins with the capture stage ( 11 ), in which the optical sensor acquires the images of the products; this is followed by a processing stage ( 12 ) for processing the captured images; after processing, a search stage ( 13 ) for searching for the product in the images occurs; and finally, a last stage of information transmission ( 14 ) is carried out.
  • Figure 5 outlines the steps of the processing stage ( 12 ) according to one of the embodiments wherein, once there is an image capture ( 15 ), an inspection area ( 16 ) is defined in which the product is to be searched. In general, the inspection area should be wide enough to accommodate a product, or a large part of it, preferably defined perpendicular to the product conveying line.
  • a first transformation of the image from RGB to HSV is performed ( 17 )
  • a colour mask is applied ( 18 ) so that any colour that does not fall within the given spectrum tends to 0 (black) and the rest tends to 1 (white), which will give a better contrast between product and background
  • a second transformation is applied to the resulting greyscale image ( 19 ). These colour space transformations will allow the background of the image to be removed.
  • a first blur or distortion filter ( 20 ), which blurs the image and eliminates small gaps in the product, allowing the product's transformation to black and white ( 21 ) to be more homogeneous; a second erosion filter ( 22 ), which eliminates loose pixels that do not correspond to the product image, making it possible to eliminate colour impurities in the background; and a third dilation filter ( 23 ), which increases the size of the white areas, contributing to a better marking of the product and a greater number of white pixels. After this process, only white pixels remain in the processed image, which correspond to the food product.
  • An inspection area that is more adjusted than the initial area is defined ( 24 ) and finally, the white pixels in the image in this adjusted inspection area are counted ( 25 ) to determine whether or not they are greater than a pre-established minimum threshold, which will define in the next stage whether or not the presence of the product in the image is confirmed.
  • a minimum threshold of white pixels can be established to determine the presence of the product close to the width of the image.
  • a value of at least a third or half of the width of the image is typically used to ensure that there will always be a number of pixels that ensures the presence of the product, avoiding failures due to deformation or poor positioning of the product prior to the image capture.
  • Figure 6 outlines the search stage ( 13 ) according to one of the embodiments.
  • the search for the product basically consists of evaluating whether certain requirements are met to determine that there is a product ( 29 ), which in this case are: exceeding a pre-established minimum number of pixels ( 26 ), exceeding a pre-established minimum number of readings ( 27 ) and exceeding a minimum time between product rows ( 28 ).
  • the minimum number of readings is the number of successive times that the white pixel count must be greater than the threshold. It is established to avoid erroneous product detections caused by impurities or loose paste stains on the conveyor belt.
  • the requirement of establishing a minimum time between rows seeks to avoid gaps within one same product and to avoid counting the same product twice. Additionally, one more criterion can be added to verify that, prior to detecting the presence of the current product, a gap has been detected ( 30 ) and, if so, the product counter is updated by one unit ( 31 ) and the search is concluded ( 32 ). To detect gaps, the flow chart takes a path that diverges from product detection.
  • the last stage of information transmission relates to configuring the entire loop of the diagram to determine whether data is to be sent, when it is going to be sent and how it is going to be sent.
  • the area where the camera is located can be defined and it can be coded according to factory parameters. For example, selecting when to send the information can be configured, as long as there is a minimum time gap between products, designed, for example, for discontinuous product cans or if the number of counted units is greater than the minimum entered. It can also be configured to send the histogram (total number of white pixels during the entire product selection process in all readings) for an analysis of results and subsequent adjustment of values and the sending of results by UDP protocol or for execution by a web page.
  • Figure 7 represents, by way of example, one of the images processed in one of the embodiments of the present invention to detect the presence and subsequently count bakeable food products.
  • the bottom ( 36 ) is represented completely in black (although in the figure it has not been coloured to optimise ink in a possible reproduction on paper), offering the maximum possible contrast with the white pixels, which represent the areas of the image occupied by the products ( 37 ).
  • FIG. 8 schematically represents the hardware and software structure of said installation, wherein all the control devices ( 80 ) distributed throughout the factory are located at the base and they communicate through their corresponding communication controllers ( 81 ) with a central server ( 82 ).
  • the central server in communication with the cloud ( 83 ), allows access from different platforms ( 84, 85, 86 ) to use the information collected at the source by the control devices in tasks of supervising, display, data logging, etc.
  • a PLC programmable logic controller
  • a PLC programmable logic controller
  • programmable automaton is an industrial computer that processes all the data of a machine, such as sensors, buttons, timers and any input signal, to subsequently control the actuators (pistons, motors, valves, etc.) and thus be able to control any industrial process automatically.
  • control devices ( 80 ) deployed by a smart factory with a computer vision system such as the one described by the present invention can integrate scales for different types of weighing during the process; metal detectors to alert the user of any metal objects intercepted by the detector's electromagnetic field, which automatically ejects the product from the production line; X-ray systems for detecting dense contaminants such as glass, metal, mineral stone, calcified bone and high-density plastic, regardless of their size, shape or location inside the product, which is automatically ejected in the case of detection following parallel operation to the metal detector; high-precision dynamic weighers to verify the weight of the product on the conveyor belt, unaffected by the vibrations of said belt and reliable so as to properly discard incomplete products at high speed; expiry date printers for continuously printing an expiry date on products; or Raspberry Pi boards with various functions adapted to each situation, such as temperature probes, computer vision or integration of new devices.
  • the communication controllers ( 81 ) are necessary to establish communication between the different control devices and the central server ( 83 ), for example of the OPC Unified Architecture type. Specific communication drivers are used for each piece of equipment to be able to recognise which devices are connected and thus make optimal use of them. To use the data collected by the control devices in the form of variables or registers on different data logging or display and supervising platforms, all the devices send the data via these drivers to the OPC UA server, wherein each driver is given a different configuration to achieve successful communication on one same platform with all the connected devices.
  • the platforms from which a user can access and process the information initially collected by the control devices can comprise, for example, a SCADA system ( 84 ), which is an industrial automation and control tool used in production processes that can control, supervise, collect data, analyse data and generate reports remotely through a computer application. Its main function is to evaluate the data in order to correct possible errors.
  • SCADA system 84
  • an alarm system in which certain variables to be controlled are received, whether they come from a PLC, databases or other integrated systems, where scripts can be created with conditions that, when met, activate an alarm. Selected users can be notified of the alarms via e-mail or telephone, although a sound or light system that can more immediately capture the attention of operators can also be implemented.
  • a smart factory like the one described herein also needs a database ( 86 ) between the platforms.
  • These factories often have an enterprise resource planning (ERP) system to address the business needs from a process standpoint and to integrate company-wide information systems.
  • ERP enterprise resource planning
  • the control devices ( 80 ) directly communicate with the ERP, through TCP, sending data that will be stored in a table of said database ( 86 ).
  • a scale would send the line on which it is working, the ordinal of said scale, the moment of sending, the value of the weight, the established maximum and minimum, and all the information that the user needs to save.
  • Communication with this database ( 86 ) is two-way, such that variables can be created from a query to the database and these variables can be processed, for example, in a SCADA system as described above. Therefore, a log of all the necessary information is obtained to subsequently process it according to the needs.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Meat, Egg Or Seafood Products (AREA)
EP22215325.6A 2021-12-23 2022-12-21 Verfahren und system zum zählen von backbaren lebensmittelprodukten Active EP4202765B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
ES202131210A ES2944786A1 (es) 2021-12-23 2021-12-23 Metodo y sistema para contar productos de alimentacion horneables

Publications (3)

Publication Number Publication Date
EP4202765A1 true EP4202765A1 (de) 2023-06-28
EP4202765C0 EP4202765C0 (de) 2025-02-12
EP4202765B1 EP4202765B1 (de) 2025-02-12

Family

ID=84887648

Family Applications (1)

Application Number Title Priority Date Filing Date
EP22215325.6A Active EP4202765B1 (de) 2021-12-23 2022-12-21 Verfahren und system zum zählen von backbaren lebensmittelprodukten

Country Status (2)

Country Link
EP (1) EP4202765B1 (de)
ES (2) ES2944786A1 (de)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12258158B2 (en) 2023-04-20 2025-03-25 Grupo Bimbo S.A.B. De C.V. Food delivery system for packaging of food and method of delivering food to be packaged
US12534307B2 (en) 2023-04-20 2026-01-27 Grupo Bimbo, S.A.B. De C.V. Food delivery system for packaging of food and method of delivering food to be packaged

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6233966B1 (en) * 1997-03-03 2001-05-22 L'air Liquide, Societe Anonyme Pour L'etude Et Exploitation Des Procedes Georges Claude Freezing tunnel
US20210321820A1 (en) * 2020-04-15 2021-10-21 Air Products And Chemicals, Inc. Sensor Device for Providing Control for a Food Processing System

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2812086B1 (fr) * 2000-07-18 2003-01-24 Air Liquide Procede et dispositif de mesure du taux d'occupation sur un tapis convoyeur notamment d'un tunnel cryogenique de produits transportes par ce convoyeur
FR2812380A1 (fr) * 2000-07-25 2002-02-01 Air Liquide Tunnel cryogenique pour la refrigeration de produits notamment alimentaires, equipe de deflecteurs de gaz de refrigeration
FR2860068A1 (fr) * 2003-09-23 2005-03-25 Air Liquide Procede et installation de determination d'une quantite de produits alimentaires transportes
US20110265492A1 (en) * 2010-04-28 2011-11-03 Newman Michael D Freezer with cryogen injection control system
CN104537671B (zh) * 2015-01-04 2017-12-29 长沙理工大学 一种基于机器视觉的香烟滤棒在线计数和质量检测方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6233966B1 (en) * 1997-03-03 2001-05-22 L'air Liquide, Societe Anonyme Pour L'etude Et Exploitation Des Procedes Georges Claude Freezing tunnel
US20210321820A1 (en) * 2020-04-15 2021-10-21 Air Products And Chemicals, Inc. Sensor Device for Providing Control for a Food Processing System

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KHEKARE GANESH ET AL: "Design of Fruit Segregation and Packaging Machine", 2020 INTERNATIONAL CONFERENCE ON COMPUTATIONAL PERFORMANCE EVALUATION (COMPE), IEEE, 2 July 2020 (2020-07-02), pages 709 - 714, XP033828348, DOI: 10.1109/COMPE49325.2020.9199986 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12258158B2 (en) 2023-04-20 2025-03-25 Grupo Bimbo S.A.B. De C.V. Food delivery system for packaging of food and method of delivering food to be packaged
WO2024220655A3 (en) * 2023-04-20 2025-04-17 Grupo Bimbo, S.A.B. De C.V. Food delivery system for packaging of food and method of delivering food to be packaged
US12534307B2 (en) 2023-04-20 2026-01-27 Grupo Bimbo, S.A.B. De C.V. Food delivery system for packaging of food and method of delivering food to be packaged

Also Published As

Publication number Publication date
EP4202765C0 (de) 2025-02-12
EP4202765B1 (de) 2025-02-12
ES2944786A1 (es) 2023-06-23
ES3013589T3 (en) 2025-04-14

Similar Documents

Publication Publication Date Title
US12368824B2 (en) Machine-vision system and method for remote quality inspection of a product
Javaid et al. Exploring impact and features of machine vision for progressive industry 4.0 culture
EP4202765A1 (de) Verfahren und system zum zählen von backbaren lebensmittelprodukten
US12288390B2 (en) System and method of object detection using AI deep learning models
EP4088233A1 (de) Systeme und verfahren zur anomalieerkennung und -detektion unter verwendung von lebenslangen tiefenneuronalen netzwerken
CN117358615B (zh) 一种自动喷码印刷缺陷检测方法及系统
CN107133725B (zh) 一种电路板的自动化生产管理系统
CN115144399B (zh) 一种基于机器视觉的装配质量检测方法及装置
CN117575381A (zh) 热转印机生产管控方法、系统、装置及存储介质
CN117909892A (zh) 基于LOF算法和Prophet模型的数据异常实时预警方法及系统
Shaloo et al. Flexible automation of quality inspection in parts assembly using CNN-based machine learning
US12111643B2 (en) Inspection system, terminal device, inspection method, and non-transitory computer readable storage medium
KR102689889B1 (ko) 협동로봇 연동 비젼인식 시스템
CN119023577A (zh) 一种智能工厂生产线视觉检测系统及方法
US20230281536A1 (en) Work management device, work management method, and non-transitory computer-readable medium
CN108898576A (zh) 一种智能机器视觉筛选平台
KR100462121B1 (ko) 카메라를 이용한 피혁상태 검사장치
CN117479440B (zh) 分板机控制方法、装置、分板机及电路板生产线
ES1309030U (es) Sistema para contar productos de alimentacion horneables
CN121505301A (zh) 一种多工位协同的密封圈装配视觉防错检测方法及装置
Patel et al. Automated Optical Inspection Using AI for PCB Assembly Quality Assurance
Pierre AI-Driven Process Optimization in Automated Microsoldering for Fine-Pitch PCB Assembly
CN119246546A (zh) 一种基于机器视觉的pcb板质量检测系统及方法
KR20250177925A (ko) 스마트 팩토리 시스템
AKI et al. QUALITY CONTROL OF PARTS IN INDUSTRIAL PRODUCTION WITH IMAGE PROCESSING METHODS

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20231030

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20241014

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602022010475

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

U01 Request for unitary effect filed

Effective date: 20250217

U07 Unitary effect registered

Designated state(s): AT BE BG DE DK EE FI FR IT LT LU LV MT NL PT RO SE SI

Effective date: 20250221

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 3013589

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20250414

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250512

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250212

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250512

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250612

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250212

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250513

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250212

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250212

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250212

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

U20 Renewal fee for the european patent with unitary effect paid

Year of fee payment: 4

Effective date: 20251124

26N No opposition filed

Effective date: 20251113

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20260102

Year of fee payment: 4