Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/20—Analysis of motion
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/40—Scenes; Scene-specific elements in video content
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2200/00—Indexing scheme for image data processing or generation, in general
  • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10016—Video; Image sequence

Definitions

• the subject of the invention is a method to determine the selected properties of a selected object by means of analysis of the contents of an electronically available image, in the course of which the change of at least one selected parameter of the object is detected, and from the direction and rate of change, the changes of the selected property of the object is implicated, and such change is stored in a database, as the case may be.
• the method based on the invention provides such a camera-based natural scientific observation and measurement system, which allows the users to perform scientific measurements, observations by means of a simple webcam or any other image or video recording device. All functions of the method are based on image processing, where the image can be received from any external image recording device or a file.
• Object of the invention is to provide a method, by means of which it can be ensured, that several webcam-based technical-scientific applications are accessed and operated through a single and uniform interface, without the need for additional resources, and that the webcam, making up the core of the methods can be universally applied.
• Said object has been solved by the subject matter of the independent claim defining a method for defining a selected feature of a selected object by analyzing the content of a still image available in electronic form, comprising the steps of detecting the change of at least one selected parameter and drawing a conclusion regarding the change of the selected object based on the direction and amount of the detected change, and if appropriate, storing data regarding the change in a database, wherein in a kinematic monitoring mode, the at least one selected object based on color and/or form will be detected and the movement of the object will be observed; in a duration monitoring mode, the state of the at least one selected object will be recorded and the recorded states will be displayed at a serially accelerated speed; in a microscope mode, a known measuring value will be assigned to a part of the at least one selected object, and based on the already known measuring value the value of another selected part of the object will be determined; in a spy camera mode, a change exceeding a preset threshold in one or more selected parts of the at least one selected object will be
• Fig. 1 shows the flowchart of a possible kinematic observation mode of the method
• Fig. 2 shows the flowchart of a possible time span observation mode of the method
• Fig. 3 shows the flowchart of a possible spy-cam mode of the method
• Fig. 4 shows the flowchart of a possible microscopic mode of the method
• Fig. 5 shows the flowchart of a possible logging mode of the method
• Fig. 6 shows a the flowchart of a possible tracking mode of the method
• Figures 7 to 35 show examples of the possible operations and possible displays of the method modes presented in Figures 1 to 6.
• the following description discloses only a preferred and advantageous embodiment of the method according to the invention.
• the method comprises stacked and connected method modules, with one important aspect, namely, that they are to be performed in the same or a similar way through the modules, thus a user does not only receive a universal and versatile tool for technical and measurement technological tasks, he/she is also capable of acquiring the performance of each - and even future - module in a short time.
• Fig. 1 The basics of the example method presented on Fig. 1 is that it is capable of recognizing and following the object selected by the user (or selected by the method automatically) on the basis of either color or form and register, record and display the movement graphs of such object, or objects (as several objects can be recognized at once).
• Object recognition based on form allows us to record rotation, spin, etc. in addition to position, speed and acceleration.
• the kinematic function is basically a novelty in terms of real-time monitoring of objects and real-time display of movement graphs.
• the simultaneous display of camera image and real-time movement graph is a significant characteristic of the method; on the main screen the live video image display window and the window displaying the movement graphs of the objects recognized on the live image, i.e. the movement of objects and the movement graphs of objects are seen simultaneously, which is in favor of understanding the movements and a successful analysis of such later on.
• a further special feature of the module is that it is capable of ordering objects in a hierarchy, thus in addition to absolute, also relative movement characteristics may be recorded.
• Objects can be added in two different ways. We may either click on the object on the camera image, or have the program detect the object through automatic object recognition. In addition to recognition according to color, recognition according to form may also be selected. The program follows objects based on their colors in real-time. In case of a 3D movement analysis, the distance of the object is derived from changes in its dimensions. Events seen through the camera may also be recorded, thus the experiment may be displayed. Further functions are available through the icons on the left side.
• Movement graphs can be displayed directly on the live image, connected to the investigated object, based on x or y directions, thus change in position of the object and its relationship with the change in the respective movement value can be displayed simultaneously.
• the program After launching data recording, the program stores data on the graph, thus analysis of a longer experiment is also possible. Data can be saved, which allows further analysis using external applications (Matlab, SciLab, Excel, etc.).
• graphs can be cut, merged, compressed, and smoothened.
• breakpoints can be identified, and thus graph smoothing can be set to the characteristics of the given function.
• Graphs can be saved as an image or as an XLS file.
• a specific feature of the analysis window is that in the bottom of the screen, the video recording is also displayed in the form of a film strip, according to the timing of the graphs, thus changes seen on the graphs and real-time events can be investigated simultaneously.
• Time-span Observation (fast recording) Mode Using the method module presented as an example on Fig. 2, slow natural processes can be accelerated to normal speed, thus observation of changes of a vivid, ever-changing world is possible, which seems static to the human eye.
• the real-time camera image seen on screen, the recording timer, time setting and the list of recorded videos are displayed.
• the method module operates as follows: following a manual or automatic start of recording, the program records an image at a preset interval (e.g. each second), until recording is stopped manually or automatically.
• a video is created, which can be viewed.
• the method module provides the following functions: Real-time image
• the live image of the webcam or any other camera.
• the live image displays the following captions:
• captions can be turned on/off
• the video will only include the actual time caption, if it is enabled.
• a red flashing "REC" caption is displayed in the upper left corner of the screen.
• the video image is displayed in this same window, yet without the afore mentioned captions, only the ones, that have been recorded with the video.
• While recording While recording, previous video recordings can be viewed by selecting from the list to the right.
• a specific feature of the method module is that recordings can be launched both manually and according to timing, thus observation of elongated events, or events, which occur at a specific time is also possible automatically with timing.
• Recording of images can be started in two different ways. Either by pressing the record button on the key pad, or by ticking the checkbox above the clock on the upper right of the screen. In the latter case, recording starts and stops according to the set times automatically. Recording only starts, if a time is set, which has not yet passed.
• Recording Timer This is a slider setting button. It allows setting the frequency of saving image frames. Setting is possible from 0,2 and 3600 seconds.
• a specific feature of the method is that in addition to setting the speed of recording based on the time interval between image frames, small icons also indicate the approximate time interval setting required for the given type of phenomena. Along the slider a cloud, snail, icicle, flower, growing plant and sun icons are shown.
• the list of recorded videos is displayed on the bottom right side of the screen.
• the list is ordered by date, with the latest recordings on top. Videos recorded on different days are grouped separately for clarity.
• the list includes the serial number of videos, the start and end time of video recordings and the duration of the video.
• Another specific feature of the method module is that an image frame of the video is displayed on a thumbnail image next to the captions, clicking on which, allows a preview of the recording.
• This toolbar is only displayed during the playing of a recorded video.
• the video recording can be started, paused, fast forwarded, or rewinded.
• the video By clicking on the slider, the video is jumped to the given position.
• the video remains active until the user clicks on the red X; after which the real-time, live image shall be displayed in the left-side window.
• a specific feature of the method module is that it allows subsequent speeding up of already recorded videos.
• This method module allows the recording of rarely occurring events. Using this mode, the user need not wait hours long to capture an exciting event, such as the appearance of hiding animals, a spider catching its prey, etc. The user just prepares the tools, sets the program and waits for the event to occur.
• the method module presented as an example on Fig. 3 can be used in two distinct modes: In the basic mode 24 (or as many as set) previous image frames are stored continuously in the buffer, while the current and last image frame (or the one set) received from the camera are compared, analyzing the changes observed in the area selected by the user or in the entire image. If the rate of changes exceed a preset threshold value (i.e. there is movement in the image), video recording is started, during which analysis is further continued, and if the rate of changes remains below the set level of action for a preset period of time (movement is stopped), the video recording stops.
• a specific feature of the method module is that with the help of the recorded buffer, the programs is capable of recording the period prior to the occurrence of movement, and append it before the recording.
• the type of movements are continuously analyzed on the recorded video flow (direction, speed, dynamics of motion), a typical movement map is plotted from the camera image, and recording is started (as described previously), when a user determined parameter of movement changes in a rate higher, than the set threshold value (e.g. the direction, speed, etc. of movement changes).
• a user determined parameter of movement changes in a rate higher, than the set threshold value (e.g. the direction, speed, etc. of movement changes).
• Starting of a recording can be set by means of two slides. With the upper slider the size of the object, while with the bottom slider, the rate of movement, that starts recording is set. In the column shown in the left-side window, the user can verify if settings are appropriate.
• the list of recorded videos is shown in the bottom right side of the screen.
• the list is ordered by date, with the latest recordings on top. Videos recorded on different days are grouped separately for clarity.
• the list includes the serial number of videos, the start and end time of video recordings and the duration of the video.
• An image frame of the video is also displayed in a thumbnail image next to the captions.
• buttons By clicking on a given video in the list, two buttons are displayed, which allow playing or deleting the given video. By clicking delete, the video is removed both from the list and from the hard drive. By clicking play, the video is started in the left-side window, and the video player toolbar is displayed under it.
• This toolbar is only displayed during the playing of a recorded video.
• the video recording can be started, paused, fast forwarded, or rewinded. By clicking on the slider, the video is jumped to the given position.
• the video remains active until the user clicks on the red X; after which the real-time, live image shall be displayed in the left-side window.
• the method module presented as an example on Fig. 4 is only microscopic in its name. It is actually a universal measurement tool.
• the basis of its operation is that, if a known dimension is set on an image captured with a camera or received from an external file (e.g. Moon diameter), i.e. the image is calibrated, then based on this piece of information, the method is capable of displaying any other dimensions on the screen. If for example, we have a photographic image of the Moon, with a calibration dimension being the diameter of the Moon, then the diameter of any Moon crater can be measured.
• the user can measure distance, area and angle.
• a specific feature of the method module is that in addition to managing a single image, it is suitable for the simultaneous (layered) display of multiple images as well (which may be images loaded from file, images of a camera, or the selected frames of a video recording), and in the application the frames can be rotated or moved, the opacity of frames can be set, and pixel-level logical operations can be performed between the images, etc., as a result of which, measurements are not only possible within a single image, but also changes between two or more image frames can be measured.
• Another specific feature of the method module is that it is capable of immediate conversion of the measurement result, supported by an integrated program language.
• the basis of this function is that a given measurement result does not represent the final result of a measurement in all cases, but merely a piece of data for further computations.
• measurement results can be immediately converted according to a preset, or user-defined function. For example, if we wish to measure the height of buildings on a satellite image by measuring the length of shadows (knowing the angle of entry of sunbeams), then only a simple trigonometric computation needs to be performed. In this case the user can enter the conversion function manually, or in a definite case, select the appropriate function from among presets and only enter the angle of entry manually, as a parameter.
• analysis of an already recorded image or video is also possible in addition to analysis of the camera image.
• the user shall click the appropriate icon to load the image or video.
• the calibration window is displayed, where the user can set the reference value and the applied unit of measurement (um, mm, cm, m, km, inch, mile, universal). If universal measurement unit is selected for calibration, then no measurement unit is displayed next to the measured values.
• the live image is displayed on the screen. If pressing the image capture button on the right button bar, the image on which measurements can be performed will be frozen. After the calibration is done, three types of measurements can be performed by clicking on the buttons on the left side. Distance, angle and area can be measured. Besides the measured values the measurement itself will appear on the image in the color selected from the lower right corner. If you enlarge the image, the scaled downed version of the original images appears in the upper left corner of the image, and a white rectangle indicates the current part of the original image.
• Measurements can be performed possible not just on the freeze-frame image, but differences can be checked with help of a single interlaced image composed of images from single or multiple halt points.
• the data of recognized spots (smallest diameter, width, height) will be automatically drawn.
• guides can be placed to help with the measurements. These guides will not be visible on the saved image.
• the control buttons are located here. With the first icon on the right you can open the color settings window, where brightness, contrast and RGB (red, green, blue) values can be adjusted. You can move the image by pressing the next icon, if you zoomed in on the original image. The moving can be performed by dragging the image while continuously pressing the left mouse button. With the next Undo icon you can delete the previous measurement. The measurement can be performed again in the left window with the Measure icon, if the Move Image was active before. The first icon on the left can be used to record an image. If you press it again after recording, you will see the live image and the previous measurement will be deleted.
• RGB red, green, blue
• Measurement Data All measurement data can be found in this window. You can assign a name to the measurement or you can delete any of the measurements at any time. The data shown in this window are also included on the print and saved image.
• the process module shown as an example in Fig. 5 is fundamentally new, its principle is to utilize the fact that each instrument designed for human observations is made with a display, which to varying degrees, but generally is easily readable to the human eye.
• the process module can of course store the readings, can display them in graphic form and can even assign alarms to the recognized values (if a certain value is reached, or change rate exceeds a value).
• the diagram of readers is drawn onto the image. At the bottom of the image the names (A, B, C), associated color and the current values of the instruments appear.
• the graph assigned to the readers will have the same color in the chart window. You can delete the desired reader with the Recycle Bin icon.
• the binary factor threshold value can be set with the slider on the right side. If the reader cannot read the instrument, use the slider to set the value for the right instrument recognition. The reader can also be adjusted in size and position as well, furthermore you can set the number of digits to be recognized by the digital reader.
• the data graph measured by instruments is shown in real time in the window.
• the captured data can not only be saved, but also a measurement video can be recorded and played back later.
• An adjustable alarm threshold can also be set, and if the graph exceeds this value, an image / voice / e-mail alert is sent by the software.
• the automated graph trend detection can be used to identify the breakpoints and allows the adjustment of graph smoothing in accordance with the characteristics of the function. Above the graphs the images recorded during the experiment and belonging to the given point will appear in a film strip fashion, if video capture was chosen at recording. If the mouse pointer is moved over a graph, he associated image is also displayed together with the graph values.
• the graphs can be saved as an image or XLS file.
• the main feat of the process module shown as an example in Fig. 6 is the ability to detect and monitor movements. This is achieved by marking the image where movement is detected on the video image (or on pre-loaded video frame), and the event of movement is recorded. If the system detects motion in the same area, the image will be marked again, this time stronger, and the motion event will again be recorded, by increasing the movement value assigned to the given area. Thus, a gradually stronger movement map is plotted on the area of movement, which can be displayed as a live image or in the form of a separate static map. The sensitivity of motion detection (size of moving objects, strength of movement, etc.) can be adjusted.
• the process module is unique in that not only is it able to create a static map of the movement quantities, but it also can display the change of motion in the form of a video through a process of erosion. This is due to the fact that the value of movement quantity belonging to a particular area can not only increase, when motion is detected, but can also decrease if there was no movement in the particular area. Thus a map is created that can change in both positive and negative directions. This map can be recorded in the form of a video with the time interval solution already known from the time-span function (that is, a real-time video of the changes can be recorded, but also a time-span video can be recorded).
• the direction and speed of motion can also be recorded so that not just a simple movement map is created, but one where to each image area an average average speed and direction vector is assigned.
• the live image or the recorded video is provided here. Any movement detected there will be indicated on the image in red.
• the red color can be set to stay all along or to fade gradually (erosion).
• the speed of the erosion can be adjusted.
• the result of the recording can be either still image or video recording.
• image frames are saved real time, or at the specified frequency.
• the amount of movement is indicated on the frames.
• not only the changes can be shown but also the direction and speed of movement using arrows.
• the thickness of the vectors indicates the frequency at which that movement took place.
• the speed of motion can be shown.
• the average speed and the average direction will appear. Changing between the displays can be accomplished using the icons on the left side of the screen.
• One of the displays shows the movement on the video recording in red, while on the other one, changes are graphed on a homogeneous blue background.
• Recording can be started in two ways: either by pushing the record pushbutton in the button line, or by checking the checkbox on the upper right part of the screen, above the clock.
• the recording takes place at the specified time and stops fully automatically.
• the recording starts only, if the time set is in the future.
• the list of recorded videos is displayed on the bottom right side of the screen.
• the list is ordered by date, with the latest recordings on top. Videos recorded on different days are grouped separately for clarity.
• the list includes the serial number of videos, the start and end time of video recordings and the duration of the video.
• buttons are displayed, which allow playing or deleting the given video.
• delete the video is removed both from the list and from the hard drive.
• play the video is started in the left-side window, and the video player toolbar is displayed under it.
• buttons also appear: one deletes the image, the other allows the user to rename the image.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Image Analysis (AREA)

Priority Applications (1)

Applications Claiming Priority (1)

Publications (1)

Family

ID=47172822

Family Applications (1)

Country Status (1)

Cited By (2)

• 2012
  • 2012-07-12 WO PCT/IB2012/053564 patent/WO2014009776A1/fr not_active Ceased

Non-Patent Citations (5)

Similar Documents

Legal Events