WO2017143745A1 - Procédé et appareil pour la détermination d'information de mouvement d'un objet à détecter - Google Patents

Procédé et appareil pour la détermination d'information de mouvement d'un objet à détecter Download PDF

Info

Publication number
WO2017143745A1
WO2017143745A1 PCT/CN2016/096379 CN2016096379W WO2017143745A1 WO 2017143745 A1 WO2017143745 A1 WO 2017143745A1 CN 2016096379 W CN2016096379 W CN 2016096379W WO 2017143745 A1 WO2017143745 A1 WO 2017143745A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
point
tested
determining
nth frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/096379
Other languages
English (en)
Chinese (zh)
Inventor
陆真国
王金亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Lexiang Technology Co Ltd
Original Assignee
Shanghai Lexiang Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Lexiang Technology Co Ltd filed Critical Shanghai Lexiang Technology Co Ltd
Publication of WO2017143745A1 publication Critical patent/WO2017143745A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements

Definitions

  • the present invention relates to the field of virtual reality technologies, and in particular, to a method and apparatus for determining motion information of an object to be tested.
  • a virtual reality helmet refers to a helmet that guides a user to create a feeling in a virtual environment by using a helmet display to close a person's visual and auditory sense to the outside world.
  • virtual reality helmets have allowed users to control virtual images according to their viewpoints and locations in the virtual environment through a variety of advanced sensing methods, specifically, in the process of users using virtual reality helmets.
  • By presenting the motion state of the user's head the user is presented with different scenes.
  • An important experience of the virtual reality helmet is immersion. Therefore, whether the motion state of the user's head can be accurately and quickly perceived is an important indicator that affects the performance of the virtual reality helmet.
  • Embodiments of the present invention provide a method and apparatus for determining motion information of an object to be tested, so as to achieve accurate and rapid perception of a virtual reality helmet motion state.
  • an Nth frame image of the object to be tested collected by the camera device where the image of the Nth frame includes a point image of each physical marker point on the first side of the object to be tested;
  • the determining, according to the point image in the image of the Nth frame, the correspondence between the point image and the physical point including:
  • a reference mark point image in the image of the Nth frame is one of the mark point images in the image of the Nth frame
  • the number of the mark point image is obtained based on the same numbering rule; the physical mark points of the first side of the object to be tested are distributed in a convex polygon array.
  • determining, according to the positional relationship between the point image and the reference point image in the image of the Nth frame, the number of each point image in the image of the Nth frame including:
  • the marker point image is determined based on at least a pixel value of each pixel of the Nth frame image, a number of pixel points on each contour, and a number of pixel points included in the contour.
  • the image of the marker point is determined according to at least a pixel value of each pixel of the image of the Nth frame, a number of pixels on each contour, and a number of pixels included in the contour, including:
  • the object to be tested is determined in the Nth frame according to the correspondence between the point image and the physical point, and the position information of each physical point and the image of each point
  • the motion information of the image corresponding to the moment including:
  • Determining the camera device by using a PnP algorithm according to the correspondence between the point image and the physical point, and the position information of each of the physical point and the image of each point The amount of rotation and the amount of translation for the object to be tested;
  • the motion information corresponding to the time is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the time corresponding to the image of the Nth frame.
  • determining the motion information of the object to be tested at the time corresponding to the image of the Nth frame Also includes:
  • the amount of rotation and the amount of translation of the camera device relative to the object to be tested are optimized using an LM algorithm.
  • the physical mark point is an infrared point
  • the mark point image is an infrared point image
  • the method further includes:
  • the infrared point of the first side of the object to be tested is turned off, and the second side of the object to be tested is opened.
  • An infrared point; the second side is predicted according to motion information of the object to be tested at a time corresponding to the image of the Nth frame;
  • the N+1 frame image Determining, by the N+1 frame image, an infrared point image of each infrared point of the second side of the object to be tested, and if yes, determining, according to the (N+1)th frame image, the object to be tested is
  • the N+1 frame image corresponds to the motion information of the time; if not, the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image capturing device is acquired.
  • the N+2th frame image; the third side is obtained according to a preset cyclic sequence.
  • An embodiment of the present invention provides an apparatus for determining motion information of an object to be tested, including:
  • a first acquiring module configured to acquire an image of an Nth frame of the object to be tested collected by the camera, where the image of the Nth frame includes a point image of each physical point of the first side of the object to be tested;
  • a determining module configured to determine the mark point image according to the mark point image in the image of the Nth frame Corresponding relationship with the physical marker points;
  • a second acquiring module configured to acquire position information of each physical point of the first side of the object to be tested in a preset world coordinate system, and image points of the image of the Nth frame in preset image coordinates Location information in the system;
  • a processing module configured to determine, according to a correspondence between the mark point image and the physical mark point, and location information of each of the physical mark points and the image of each mark point, the object to be tested is at the Nth
  • the frame image corresponds to the motion information at the time.
  • the determining module is specifically configured to:
  • a reference mark point image in the image of the Nth frame is one of the mark point images in the image of the Nth frame
  • the number of the mark point image is obtained based on the same numbering rule; the physical mark points of the first side of the object to be tested are distributed in a convex polygon array.
  • the determining module is specifically configured to:
  • M is an integer greater than or equal to 2;
  • the determining module is further configured to:
  • At least a pixel value of each pixel of the image of the Nth frame, a pixel point of each contour The number of pixels and the number of pixels included in the contour determine the image of the marker point.
  • the determining module is specifically configured to:
  • the processing module is specifically configured to:
  • the motion information corresponding to the time is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the time corresponding to the image of the Nth frame.
  • the processing module is further configured to:
  • the amount of rotation and the amount of translation of the camera device relative to the object to be tested are optimized using an LM algorithm.
  • the physical mark point is an infrared point
  • the mark point image is an infrared point image
  • the processing module is further configured to:
  • the infrared point of the first side of the object to be tested is turned off, and the object to be tested is turned on.
  • An infrared point of the second side; the second side is predicted according to motion information of the object to be tested at a time corresponding to the image of the Nth frame;
  • the N+1 frame image Determining, by the N+1 frame image, an infrared point image of each infrared point of the second side of the object to be tested, and if yes, determining, according to the (N+1)th frame image, the object to be tested is
  • the N+1 frame image corresponds to the motion information of the time; if not, the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image capturing device is acquired.
  • the N+2th frame image; the third side is obtained according to a preset cyclic sequence.
  • the image of the Nth frame of the object to be tested collected by the camera device is acquired, and the image of the Nth frame includes the image of the point of each physical mark on the first side of the object to be tested;
  • the mark point image determines the correspondence relationship between the mark point image and the physical mark point; acquires the position information of each physical mark point of the first side of the object to be tested in the preset world coordinate system and the image of each mark point of the Nth frame image The position information in the preset image coordinate system; determining the object to be tested at the corresponding moment of the image of the Nth frame according to the correspondence between the point image and the physical point, and the position information of each physical point and each point image Sports information.
  • the motion information of the object to be tested is determined, which is compared with the prior art.
  • the method for obtaining a rotational posture by using a sensor such as a gyroscope can effectively determine the amount of translation of the object to be tested, thereby more accurately and quickly sensing the motion state of the object to be tested, and has high real-time performance, which can significantly improve the actual user. Experience.
  • FIG. 1 is a schematic structural diagram of a system applicable to an embodiment of the present invention
  • FIG. 2 is a schematic view showing the arrangement of infrared lamps on each side of the virtual reality helmet
  • FIG. 3 is a schematic flowchart of determining motion information of an object to be tested according to an embodiment of the present disclosure
  • FIG. 4 is a schematic flowchart of preprocessing an image according to an embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of determining a correspondence between a mark point image and a physical mark point according to an embodiment of the present invention
  • FIG. 6 is a schematic diagram of a marker point image number according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of a process of performing prediction processing according to motion information of an object to be tested
  • FIG. 8 is a schematic structural diagram of an apparatus for determining motion information of an object to be tested according to an embodiment of the present disclosure
  • FIG. 9 is a schematic structural diagram of an apparatus for determining motion information of an object to be tested according to an embodiment of the present invention.
  • FIG. 1 exemplarily shows a system architecture diagram applicable to the embodiment of the present invention.
  • the system architecture includes a server 101, an imaging device 102, and an object to be tested 103.
  • the server 101 and the camera 102 can perform wired or wireless communication, that is, the information can be transmitted between the server 101 and the camera 102 by wired transmission or wireless transmission.
  • the camera 102 can be transmitted by wire or wirelessly.
  • the method sends the captured image to the server 101; the server 101 and the object to be tested 103 can also perform wired or wireless communication.
  • the server 101 sends the scene rendering data to the object to be tested 103 by wired transmission or wireless transmission. .
  • the server 101 may be a PC host with data processing capability.
  • the object to be tested 103 may be a virtual reality helmet including first to sixth sides (assuming that the normal use condition of the virtual reality helmet is worn on the user's head, based on which the first to sixth helmets may be determined
  • the side faces are front, back, top, bottom, left, and right with respect to the camera.
  • Infrared lights also called infrared dots
  • the routing rule in the embodiment of the present invention may be that the infrared lamps on each side are arranged in a manner of a convex polygon array, and may be based on the above-mentioned layout rules for different sides.
  • FIG. 2 it is a schematic diagram of the arrangement of infrared lamps on each side of the virtual reality helmet. It should be noted that FIG. 2 is only an exemplary representation of the arrangement of the infrared lamps, and there may be a difference in the proportion of the specific objects.
  • the camera device 102 can be an infrared camera, which is mainly used for capturing state information of the infrared lamp disposed on the virtual reality helmet, and transmits the captured image to the server 101, so that the server determines the motion information of the virtual reality helmet through correlation calculation (rotation) Matrix R and translation vector T).
  • the virtual reality helmet and the infrared camera are connected to the PC host through a USB data line, and both USB2.0 and USB3.0, preferably USB3.0.
  • the virtual reality helmet is also connected to the PC host through the HDMI interface to facilitate capturing scene rendering data.
  • FIG. 3 is a schematic flowchart of determining motion information of an object to be tested according to an embodiment of the present invention.
  • Step 301 Acquire an image of an Nth frame of the object to be tested collected by the camera, where the image of the Nth frame includes a point image of each physical mark on the first side of the object to be tested;
  • Step 302 Determine, according to the point image in the image of the Nth frame, a correspondence between the point image and the physical point;
  • Step 303 Acquire location information of each physical marker point of the first side of the object to be tested in a preset world coordinate system, and position of each marker image of the Nth frame image in a preset image coordinate system. information;
  • Step 304 according to the correspondence between the mark point image and the physical mark point, and the And determining, by the physical location points and the location information of the respective marker point images, motion information of the object to be tested at a time corresponding to the image of the Nth frame.
  • the motion information of the object to be tested is determined, which is compared with the prior art.
  • the method for obtaining a rotational posture by using a sensor such as a gyroscope can effectively determine the amount of translation of the object to be tested, thereby more accurately and quickly sensing the motion state of the object to be tested, and has high real-time performance, which can significantly improve the actual user. Experience.
  • the physical marker point in the embodiment of the present invention may be an infrared point, and the marker point image is an infrared point image.
  • the optical tracking process of the motion information (space shift amount) of the virtual reality helmet only the side of the six sides of the helmet that can be completely photographed by the camera is performed on the upper, the lower, the left, the right, the front and the back. deal with. Therefore, the following is a detailed description of the process of determining the motion information of the virtual reality helmet by a single side.
  • the image capturing device may interfere with the ambient light and other factors when capturing the image
  • the image of the Nth frame should be preprocessed. And determining, according to at least the pixel value of each pixel of the image of the Nth frame, the number of pixels on each contour, and the number of pixels included in the contour, the interference image is excluded, and the marker image is determined.
  • the first candidate point image is obtained according to the pixel value of the pixel of each image; wherein the pixel value of the pixel of the first candidate point image is greater than or equal to the first threshold; according to each first candidate mark Obtaining a second candidate point image by the number of pixels on the contour of the image; the number of pixels on the contour of the second candidate point image is greater than or equal to a second threshold and less than or equal to a third threshold; The number of pixels included in the contour of each second candidate point image obtains a third candidate point image; the number of pixels included in the contour of the third candidate point image is greater than or equal to a fourth threshold Determining an ellipse parameter of the third candidate point image, and determining a third candidate point image that matches the ellipse parameter to the preset parameter range as the point image.
  • the first threshold, the second threshold, the third threshold, the fourth threshold, and the preset parameter range may all be set by experience by those skilled in the art.
  • FIG. 4 is a schematic flowchart of pre-processing an image according to an embodiment of the present invention, including steps 401 to 408, which are specifically described below in conjunction with FIG. 4.
  • Step 401 Acquire an image of an Nth frame.
  • Step 402 Binarization processing, to obtain a first candidate point image; specifically: determining a maximum pixel value max of each pixel in the image of the Nth frame, using a*max as a binarization threshold (first threshold) Each pixel is traversed. If the pixel value of the pixel is less than a*max, the pixel value is set to 0. If the pixel value of the pixel is greater than or equal to a*max, the pixel value is set to 255. Where a is a weight, the value of a can be set by a person skilled in the art according to experience, for example, can be set to 0.9;
  • Step 403 Obtain a number of pixel points on a contour of each first candidate point image, where the contour is a position of a pixel point whose pixel value changes from 0 to 255 or changes from 255 to 0.
  • the contour is a position of a pixel point whose pixel value changes from 0 to 255 or changes from 255 to 0.
  • 8 neighborhood pixels can be obtained (except for the edge region of the image, each pixel is adjacent to 8 pixels, and the first pixel on the contour is determined.
  • Step 404 For each first candidate point image, delete an image whose number of pixels on the contour is smaller than a second threshold or greater than a third threshold, to obtain a second candidate point image;
  • Step 405 Delete, for each second candidate point image, an image whose number of pixels included in the contour is less than a fourth threshold, to obtain a third candidate point image;
  • Step 406 based on a preset image coordinate system (including an x-axis and a y-axis), fitting an ellipse parameter (including an ellipse center, a long and short axis, a tilt angle, and the like) of the third candidate point image by a fitting algorithm;
  • a preset image coordinate system including an x-axis and a y-axis
  • an ellipse parameter including an ellipse center, a long and short axis, a tilt angle, and the like
  • Step 407 For each third candidate point image, delete an image whose ellipse parameter does not meet the preset parameter range, and determine a third candidate point image whose ellipse parameter meets the preset parameter range as the point image.
  • Step 408 Output an ellipse parameter of the obtained point image, and determine position information of the point according to the ellipse parameter.
  • the image of the Nth frame is preprocessed by the foregoing process, thereby quickly and accurately eliminating interference factors in the environment, determining the image of the marker point, and determining the image and label of the marker point for subsequent determination.
  • the correspondence between the points has laid a good foundation.
  • the correspondence between the point image in the image of the Nth frame and the physical point of the first side is determined by determining an image of the reference point in the image of the Nth frame based on the envelope method;
  • the reference mark point image is one of the mark point images in the Nth frame image;
  • the number of the mark point image in the Nth frame image is determined according to the positional relationship between the mark point image and the reference mark point image in the Nth frame image;
  • the physical marker point with the same number as the marker point image is determined as the physical marker point corresponding to the marker point image, and the correspondence relationship between the marker point image and the physical marker point is obtained; the number of the physical marker point and the number of the marker point image are based on the same Numbering rules are obtained.
  • the present invention when determining the number of each point image in the image of the Nth frame according to the positional relationship between the point image in the image of the Nth frame and the image of the reference point image, Based on the envelope method, layer by layer, from the outermost layer to the innermost layer, specifically: determining the marking point of the first layer according to the positional relationship between the marked point image and the reference point image in the image of the Nth frame The image and the sorting of the image of the first layer of the marker point; the reference marker image is the marker image of the first layer; and the marker points other than the marker image of the first layer to the M-1 layer according to the image of the Nth frame The positional relationship between the image and the reference point image determines the order of the image of the mark of the Mth layer and the image of the mark of the Mth layer; M is an integer greater than or equal to 2; according to the image of the mark of the first layer to the Mth layer Sort to determine the number of the marker image of the first layer to the third layer.
  • M can be set by the person skilled in the art according to the experience and the arrangement of the infrared light on the side. In general, the value of M can be set to 3.
  • FIG. 5 is a schematic flowchart of determining a correspondence between a point image and a physical point according to an embodiment of the present invention, which includes steps 501 to 504, which are specifically described below with reference to FIG. 5.
  • Step 501 Determine, according to the convex hull algorithm, the reference mark point image and the mark image of the first layer (the outermost layer), and according to the distance between each mark point image of the first layer and the reference mark point, according to the small to
  • the order of the reference point images is specifically as follows: the point images of the image of the Nth frame are arranged in ascending order according to the y-axis coordinate, and if the same y-axis coordinate corresponds to multiple Marking the point image, the plurality of point images are arranged in ascending order according to the x-axis coordinate, and the point image with the largest y-axis coordinate and the smallest x-axis coordinate (ie, the point image in the lower left corner) is determined as the reference point image;
  • Step 502 determining four image points of the second layer (secondary outer layer), according to the distance between the four point image and the reference point image, arranged in order from small to large;
  • Step 503 determining three image points of the third layer (the innermost layer), according to the distance between the three point image and the reference point image, arranged in order from small to large;
  • Step 504 according to the first layer to the third layer obtained above, sequentially numbered in order from outer to inner, and the first layer number is 1-8, the second layer number is 9-12, and the third layer number is 13-15, as shown in FIG. 6, is a schematic diagram of the number of points of the marker point.
  • the first side physical mark point is obtained based on the same numbering rule as described above, if the number of the mark point image determined by the above method is normal, the physical mark point having the same number as the mark point image can be determined to correspond. Physical point points, the correspondence between the point image and the physical point is obtained.
  • the convex bundle algorithm is used to determine the correspondence between the point image and the physical marker point, so that the determination of the correspondence relationship is more accurate and fast, and lays a good foundation for subsequently determining the motion information of the object to be tested.
  • step 304 based on the correspondence between the point image and the physical point obtained in step 302, and the position information of each physical point and each point image obtained in step 303, the PnP algorithm is used to determine the camera relative to the test.
  • the amount of rotation and the amount of translation of the object, and the LM algorithm is used to optimize the amount of rotation and the amount of translation of the camera relative to the object to be tested; according to the amount of rotation and the amount of translation of the camera device relative to the object to be measured,
  • the motion information of the object to be tested at the time corresponding to the image of the Nth frame; the motion information of the object to be tested at the time corresponding to the image of the Nth frame is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the corresponding time of the image of the Nth frame.
  • FIG. 7 is a schematic diagram of a process of performing prediction processing according to motion information of an object to be tested.
  • the embodiment of the present invention further determines the prediction processing procedure shown in FIG. 7 after the object to be measured is rotated and translated by the camera at the corresponding time of the image of the Nth frame, specifically:
  • Step 701 Determine, by using a PnP algorithm and an LM algorithm, a rotation amount and a translation amount of the object to be tested relative to the imaging device at a corresponding time of the image of the Nth frame;
  • Step 702 determining that the motion information of the object to be tested at the time corresponding to the image of the Nth frame conforms to the preset motion amount range, specifically: whether the rotation amount is within a preset rotation amount range, and whether the translation amount is within a preset translation amount range; if yes, Then, step 703 is performed; if not, step 701 is performed; wherein the preset rotation amount range and the preset translation amount range can be obtained by a person skilled in the art according to experience or according to a large number of experiments;
  • Step 703 predict a second side according to the motion information of the object to be tested at the corresponding time of the image of the Nth frame, close the infrared point of the first side of the object to be tested, and open the infrared point of the second side of the object to be tested;
  • Step 704 Acquire an image of an N+1th frame acquired by the camera.
  • Step 705 determining whether the infrared point image of each infrared point of the second side of the object to be tested is included in the image of the N+1th frame, and if yes, executing step 701, determining, according to the image of the (N+1)th frame Determining the motion information of the object to be measured at the time corresponding to the (N+1)th frame image; if not, executing step 706;
  • Step 706 the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image of the N+2 frame acquired by the camera device is acquired; wherein the third side is It is obtained according to the preset cycle order.
  • the preset cycle order can be front, left, right, top, bottom, and back.
  • Step 707 Determine whether the infrared point image of each infrared point of the third side of the object to be tested is included in the image of the N+2 frame, and if yes, execute step 701, and determine the image according to the image of the N+2 frame.
  • the embodiment of the present invention further provides an apparatus for determining motion information of an object to be tested, and the specific content of the apparatus may be implemented by referring to the foregoing method.
  • FIG. 8 is a schematic structural diagram of an apparatus for determining motion information of an object to be tested according to an embodiment of the present invention.
  • the first obtaining module 801 is configured to acquire an image of the Nth frame of the object to be tested collected by the camera,
  • the image of the Nth frame includes a mark point image of each physical mark point of the first side of the object to be tested;
  • a determining module 802 configured to determine, according to the point image in the image of the Nth frame, a correspondence between the point image and the physical point;
  • a second acquiring module 803 configured to acquire position information of each physical mark point of the first side of the object to be tested in a preset world coordinate system, and image of each point of the Nth frame image in a preset image Position information in the coordinate system;
  • the processing module 804 is configured to determine, according to the correspondence between the mark point image and the physical mark point, and the position information of each physical mark point and the image of each mark point, the object to be tested is in the first
  • the N frame image corresponds to the motion information of the time.
  • the determining module 802 is specifically configured to:
  • a reference mark point image in the image of the Nth frame is one of the mark point images in the image of the Nth frame
  • the number of the mark point image is obtained based on the same numbering rule; the physical mark points of the first side of the object to be tested are distributed in a convex polygon array.
  • the determining module 802 is specifically configured to:
  • M is an integer greater than or equal to 2;
  • Determining the first layer to the Mth according to the ordering of the image points of the first layer to the Mth layer The number of the point image of the layer.
  • the determining module 802 is further configured to:
  • the marker point image is determined based on at least a pixel value of each pixel of the Nth frame image, a number of pixel points on each contour, and a number of pixel points included in the contour.
  • the determining module 802 is specifically configured to:
  • processing module 804 is specifically configured to:
  • the motion information corresponding to the time is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the time corresponding to the image of the Nth frame.
  • processing module 804 is further configured to:
  • the amount of rotation and the amount of translation of the camera device relative to the object to be tested are optimized using an LM algorithm.
  • the physical mark point is an infrared point
  • the mark point image is an infrared point image
  • the processing module 804 is further configured to:
  • the infrared point of the first side of the object to be tested is turned off, and the second side of the object to be tested is opened.
  • An infrared point; the second side is predicted according to motion information of the object to be tested at a time corresponding to the image of the Nth frame;
  • the N+1 frame image Determining, by the N+1 frame image, an infrared point image of each infrared point of the second side of the object to be tested, and if yes, determining, according to the (N+1)th frame image, the object to be tested is
  • the N+1 frame image corresponds to the motion information of the time; if not, the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image capturing device is acquired.
  • the N+2th frame image; the third side is obtained according to a preset cyclic sequence.
  • the image of the Nth frame of the object to be tested collected by the camera device is acquired, and the image of the Nth frame includes the image of the point of each physical mark on the first side of the object to be tested; Marking the point image to determine the correspondence between the point image and the physical point; obtaining the position information of each physical point of the first side of the object to be tested in a preset world coordinate system and the image of each point of the Nth frame image Position information in a preset image coordinate system; determining the motion of the object to be tested at the corresponding moment of the image of the Nth frame according to the correspondence between the point image and the physical point, and the position information of each physical point and each point image information.
  • the motion information of the object to be tested is determined, which is compared with the prior art.
  • the method for obtaining a rotational posture by using a sensor such as a gyroscope can effectively determine the amount of translation of the object to be tested, thereby more accurately and quickly sensing the motion state of the object to be tested, and has high real-time performance, which can significantly improve the actual user. Experience.
  • the embodiment of the present application provides another apparatus for determining motion information of an object to be tested.
  • the device for determining the motion information of the object to be tested provided by the embodiment of the present application is as shown in FIG. 9.
  • the device for determining the motion information of the object to be tested includes: a communication interface 901, a processor 902, and a memory. 903 and bus system 904;
  • the memory 903 is used to store a program.
  • the program can include program code, the program code including computer operating instructions.
  • the memory 903 may be a random access memory (RAM) or a non-volatile memory such as at least one disk storage. Only one memory is shown in the figure, of course, the memory can also be set to a plurality as needed. Memory 903 can also be a memory in processor 902.
  • the memory 903 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof:
  • Operation instructions include various operation instructions for implementing various operations.
  • Operating system Includes a variety of system programs for implementing various basic services and handling hardware-based tasks.
  • the processor 902 controls the operation of the device that determines the motion information of the object to be tested, and the processor 902 may also be referred to as a CPU (Central Processing Unit).
  • the components of the device for determining the motion information of the object to be tested are coupled together by a bus system 904.
  • the bus system 904 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus.
  • various buses are labeled as bus system 904 in the figure. For ease of representation, only the schematic drawing is shown in FIG.
  • Processor 902 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 902 or an instruction in a form of software.
  • the processor 902 described above may be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or discrete hardware. Component.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly implemented by the hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a random access memory. Flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, etc. are well-known storage media in the field.
  • the storage medium is located in the memory 903, and the processor 902 reads the information in the memory 903 and performs the following steps in conjunction with its hardware:
  • an Nth frame image of the object to be tested collected by the camera device where the Nth frame image includes a mark point image of each physical mark point of the first side of the object to be tested; according to the mark in the image of the Nth frame a point image, determining a correspondence between the point image and the physical point; acquiring position information of each physical point of the first side of the object to be tested in a preset world coordinate system, and the Nth frame Position information of each point image of the image in a preset image coordinate system; according to the correspondence between the point image and the physical point, and the position of each physical point and the image of each point And determining information about the motion of the object to be tested at the time corresponding to the image of the Nth frame.
  • the processor 902 is specifically configured to:
  • a reference mark point image in the image of the Nth frame is one of the mark point images in the image of the Nth frame
  • the number of the mark point image is obtained based on the same numbering rule; the physical mark points of the first side of the object to be tested are distributed in a convex polygon array.
  • the processor 902 is specifically configured to:
  • M is an integer greater than or equal to 2;
  • the processor 902 is further configured to:
  • the marker point image is determined based on at least a pixel value of each pixel of the Nth frame image, a number of pixel points on each contour, and a number of pixel points included in the contour.
  • the processor 902 is specifically configured to:
  • the processor 902 is specifically configured to:
  • the motion information corresponding to the time is the amount of rotation and the amount of translation of the object to be tested relative to the camera at the time corresponding to the image of the Nth frame.
  • the processor 902 is further configured to:
  • the amount of rotation and the amount of translation of the camera device relative to the object to be tested are optimized using an LM algorithm.
  • the physical mark point is an infrared point
  • the mark point image is an infrared point image
  • the processor 902 is further configured to:
  • the infrared point of the first side of the object to be tested is turned off, and the second side of the object to be tested is opened.
  • An infrared point; the second side is predicted according to motion information of the object to be tested at a time corresponding to the image of the Nth frame;
  • the N+1 frame image Determining, by the N+1 frame image, an infrared point image of each infrared point of the second side of the object to be tested, and if yes, determining, according to the (N+1)th frame image, the object to be tested is
  • the N+1 frame image corresponds to the motion information of the time; if not, the infrared point of the second side of the object to be tested is turned off, the infrared point of the third side of the object to be tested is turned on, and the image capturing device is acquired.
  • the N+2th frame image; the third side is obtained according to a preset cyclic sequence.
  • the image of the Nth frame of the object to be tested collected by the camera device is acquired, and the image of the Nth frame includes the image of the point of each physical mark on the first side of the object to be tested; Marking the point image to determine the correspondence between the point image and the physical point; obtaining the position information of each physical point of the first side of the object to be tested in a preset world coordinate system and the image of each point of the Nth frame image Position information in a preset image coordinate system; determining the motion of the object to be tested at the corresponding moment of the image of the Nth frame according to the correspondence between the point image and the physical point, and the position information of each physical point and each point image information.
  • the motion information of the object to be tested is determined, which is compared with the prior art.
  • the method for obtaining a rotational posture by using a sensor such as a gyroscope can effectively determine the amount of translation of the object to be tested, thereby more accurately and quickly sensing the motion state of the object to be tested, and has high real-time performance, which can significantly improve the actual user. Experience.
  • embodiments of the present invention may be provided as a method, or a computer Program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)

Abstract

La présente invention concerne un procédé et un appareil pour la détermination d'information de mouvement d'un objet à détecter. Selon des modes de réalisation de la présente invention, la trame d'image de rang N<sp /> d'un objet à détecter acquise par un dispositif de photographie est obtenue; des relations correspondantes entre des fantômes de points de repère et des points de repère physiques sont déterminées en fonction de fantômes de points de repère dans la trame d'image de rang N<sp />; et une information de mouvement de l'objet à détecter à un moment correspondant à la trame d'image de rang N<sp /> est déterminée en fonction d'information de position des points de repère physiques et d'information de position des fantômes de points de repère. Selon les modes de réalisation de la présente invention, des relations correspondantes entre des fantômes de points de repère et des points de repère physiques sont déterminées, et une information de mouvement d'un objet à détecter est déterminée en fonction d'information de position des fantômes de points de repère et d'information de position des points de repère physiques. Par rapport au procédé dans l'état antérieur de la technique selon lequel une attitude de rotation est obtenue au moyen d'un capteur tel qu'un gyroscope, les modes de réalisation de la présente invention peuvent déterminer efficacement le mouvement de translation de l'objet à détecter, afin de détecter plus précisément et rapidement l'état de mouvement de l'objet devant être détecté, permettant d'obtenir une instantanéité élevée et d'améliorer significativement l'expérience réelle d'un utilisateur.
PCT/CN2016/096379 2016-02-22 2016-08-23 Procédé et appareil pour la détermination d'information de mouvement d'un objet à détecter Ceased WO2017143745A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610096765.9A CN105631901A (zh) 2016-02-22 2016-02-22 一种确定待测对象的运动信息的方法及装置
CN201610096765.9 2016-02-22

Publications (1)

Publication Number Publication Date
WO2017143745A1 true WO2017143745A1 (fr) 2017-08-31

Family

ID=56046788

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/096379 Ceased WO2017143745A1 (fr) 2016-02-22 2016-08-23 Procédé et appareil pour la détermination d'information de mouvement d'un objet à détecter

Country Status (2)

Country Link
CN (1) CN105631901A (fr)
WO (1) WO2017143745A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190304130A1 (en) * 2018-03-30 2019-10-03 Beijing Boe Optoelectronics Technology Co., Ltd. Spatial positioning method, spatial positioning device, spatial positioning system and computer readable storage medium
CN110378922A (zh) * 2019-06-10 2019-10-25 五邑大学 基于自适应阈值分割算法的平滑图像生成方法和装置
CN113111687A (zh) * 2020-01-13 2021-07-13 阿里巴巴集团控股有限公司 数据处理方法、系统及电子设备

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105631901A (zh) * 2016-02-22 2016-06-01 上海乐相科技有限公司 一种确定待测对象的运动信息的方法及装置
CN107528993B (zh) * 2016-06-22 2019-09-20 北京蚁视科技有限公司 用于虚拟现实中播放运动视频的虚拟播放器的显示方法
CN106028001B (zh) * 2016-07-20 2019-01-04 上海乐相科技有限公司 一种光学定位方法及装置
CN106780609B (zh) * 2016-11-28 2019-06-11 中国电子科技集团公司第三研究所 视觉定位方法和视觉定位装置
CN107293182A (zh) * 2017-07-19 2017-10-24 深圳国泰安教育技术股份有限公司 一种基于vr的汽车教学方法、系统及终端设备
CN108769668B (zh) * 2018-05-31 2019-08-20 歌尔股份有限公司 相机成像中像素点在vr显示屏的位置的确定方法及装置
CN111207747B (zh) * 2018-11-21 2021-09-28 中国科学院沈阳自动化研究所 基于HoloLens眼镜的空间定位方法
CN113989368B (zh) * 2021-10-13 2025-05-13 中山大学 一种物体表面高精度定位方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103315739A (zh) * 2013-05-22 2013-09-25 华东师范大学 基于动态跟踪技术免除运动伪影的磁共振影像方法和系统
CN104298345A (zh) * 2014-07-28 2015-01-21 浙江工业大学 一种人机交互系统的控制方法
CN104463108A (zh) * 2014-11-21 2015-03-25 山东大学 一种单目实时目标识别及位姿测量方法
CN105631901A (zh) * 2016-02-22 2016-06-01 上海乐相科技有限公司 一种确定待测对象的运动信息的方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010534316A (ja) * 2007-07-10 2010-11-04 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ 対象物の動きを捕捉するシステム及び方法
CN101894377B (zh) * 2010-06-07 2012-09-05 中国科学院计算技术研究所 三维标记点序列的跟踪方法及其系统
CN103198492A (zh) * 2013-03-28 2013-07-10 沈阳航空航天大学 一种人体运动捕获方法
CN103488291B (zh) * 2013-09-09 2017-05-24 北京诺亦腾科技有限公司 一种基于运动捕捉的浸入式虚拟现实系统
CN104616292B (zh) * 2015-01-19 2017-07-11 南开大学 基于全局单应矩阵的单目视觉测量方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103315739A (zh) * 2013-05-22 2013-09-25 华东师范大学 基于动态跟踪技术免除运动伪影的磁共振影像方法和系统
CN104298345A (zh) * 2014-07-28 2015-01-21 浙江工业大学 一种人机交互系统的控制方法
CN104463108A (zh) * 2014-11-21 2015-03-25 山东大学 一种单目实时目标识别及位姿测量方法
CN105631901A (zh) * 2016-02-22 2016-06-01 上海乐相科技有限公司 一种确定待测对象的运动信息的方法及装置

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190304130A1 (en) * 2018-03-30 2019-10-03 Beijing Boe Optoelectronics Technology Co., Ltd. Spatial positioning method, spatial positioning device, spatial positioning system and computer readable storage medium
US10872436B2 (en) * 2018-03-30 2020-12-22 Beijing Boe Optoelectronics Technology Co., Ltd. Spatial positioning method, spatial positioning device, spatial positioning system and computer readable storage medium
CN110378922A (zh) * 2019-06-10 2019-10-25 五邑大学 基于自适应阈值分割算法的平滑图像生成方法和装置
CN110378922B (zh) * 2019-06-10 2022-11-08 五邑大学 基于自适应阈值分割算法的平滑图像生成方法和装置
CN113111687A (zh) * 2020-01-13 2021-07-13 阿里巴巴集团控股有限公司 数据处理方法、系统及电子设备

Also Published As

Publication number Publication date
CN105631901A (zh) 2016-06-01

Similar Documents

Publication Publication Date Title
WO2017143745A1 (fr) Procédé et appareil pour la détermination d&#39;information de mouvement d&#39;un objet à détecter
US9519968B2 (en) Calibrating visual sensors using homography operators
US9679384B2 (en) Method of detecting and describing features from an intensity image
US9129435B2 (en) Method for creating 3-D models by stitching multiple partial 3-D models
US10311285B2 (en) Polarization imaging for facial recognition enhancement system and method
KR20220009393A (ko) 이미지 기반 로컬화
US9858680B2 (en) Image processing device and imaging apparatus
JP5785664B2 (ja) 深度画像における人の頭部の検出
CN109564432B (zh) 通过姿势与可移动设备通信/控制可移动设备的方法和相关系统
US10776657B2 (en) Template creation apparatus, object recognition processing apparatus, template creation method, and program
CN113474816B (zh) 弹性动态投影映射系统和方法
WO2022174594A1 (fr) Procédé et système de suivi et d&#39;affichage de main nue basés sur plusieurs caméras, et appareil
CN103150544A (zh) 用于对象姿态估计的方法和设备
CN104246793A (zh) 移动设备的三维脸部识别
CN113689578A (zh) 一种人体数据集生成方法及装置
JP6817742B2 (ja) 情報処理装置およびその制御方法
JP2022541100A (ja) 共同環境再構築およびカメラキャリブレーション
KR101652594B1 (ko) 증강 현실 컨텐츠 제공 장치 및 방법
WO2024169384A1 (fr) Procédé et appareil d&#39;estimation de regard, support de stockage lisible et dispositif électronique
KR102567379B1 (ko) 3d 모델링 자동 보정을 통한 실감 오브젝트 생성 시스템
CN113095347A (zh) 基于深度学习的标记识别方法和训练方法及其系统和电子设备
CN117412018A (zh) 追踪装置、方法及其非暂态电脑可读取存储介质
WO2019150431A1 (fr) Dispositif de traitement d&#39;informations
KR20210091033A (ko) 객체 정보 추정과 가상 객체 생성을 위한 전자 장치 및 전자 장치의 동작 방법
CN114972618A (zh) 三维关键点采集系统、方法、装置及存储介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16891193

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 07/12/2018)

122 Ep: pct application non-entry in european phase

Ref document number: 16891193

Country of ref document: EP

Kind code of ref document: A1