WO2017196251A1 - Système destiné à traduire des mouvements visuels en stimulations tactiles - Google Patents

Système destiné à traduire des mouvements visuels en stimulations tactiles Download PDF

Info

Publication number
WO2017196251A1
WO2017196251A1 PCT/SE2017/050493 SE2017050493W WO2017196251A1 WO 2017196251 A1 WO2017196251 A1 WO 2017196251A1 SE 2017050493 W SE2017050493 W SE 2017050493W WO 2017196251 A1 WO2017196251 A1 WO 2017196251A1
Authority
WO
WIPO (PCT)
Prior art keywords
light emitting
elements
camera
signals
emitting device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/SE2017/050493
Other languages
English (en)
Inventor
Runo ANDERSSON
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of WO2017196251A1 publication Critical patent/WO2017196251A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
    • G09B21/001—Teaching or communicating with blind persons
    • G09B21/003—Teaching or communicating with blind persons using tactile presentation of the information, e.g. Braille displays
    • G09B21/004—Details of particular tactile cells, e.g. electro-mechanical or mechanical layout
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
    • G09B21/001—Teaching or communicating with blind persons
    • G09B21/003—Teaching or communicating with blind persons using tactile presentation of the information, e.g. Braille displays
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user

Definitions

  • a system for translating visual movements into tactile stimulations A system for translating visual movements into tactile stimulations.
  • the present invention relates to a system for translating visual movements of an object into tactile stimulations of a user, comprising an IR camera adapted to capture a series of images, an image processing unit adapted to receive the series of images from the camera and to generate one or more signals based on the received images, and a receptor stimulating device comprising a two dimensional matrix having a plurality of elements adapted to, upon activation, stimulate at least one tactile receptor of a part of the user, and the receptor stimulating device is adapted to receive the signals from the image processing unit, and to activate one or more of the elements based on the received signals.
  • Braille display whereby patterns of raised dots pressed on paper represent Braille characters.
  • Tactile receptors under the skin of the fingers allow the user to feel the different characters.
  • Many computer based systems have been developed, whereby objects are translated into signals that can be received by tactile receptors.
  • DE 197 11 125 and DE 42 41 937 disclose systems that allow a blind person to experience the intensity of colors.
  • a grid pattern of elements is used to send signals to the skin of a person. These elements may be pins moving from and to the skin surface, or electrical elements sending a current to the skin. Colored images from a camera are processed in a computer device to produce signals for an element, whereby the signals have different intensities.
  • GB 2523355 discloses a system to aid a blind person to learn musical notations.
  • a camera records musical notes on paper and translates these notes to elements positioned under a sole of a foot.
  • the elements are transducers and positioned on specific spots under the foot, such that each note has its own position under the foot.
  • US 8,154,392 discloses a system comprising a touch inducer pad that is in contact with a sole of a foot.
  • the inducer pad has a grid pattern of electro vibrating inducers as elements that are wirelessly connected using a low radiofrequency for transmission.
  • the system is adapted for armed forces in combat, whereby each individual soldier has a gate code wearer's address selector with its own frequency.
  • the system is adapted to send simple signals, such as directions to the soldiers.
  • the source of the system may be equipped with a camera in combination with software manipulation to "visualize" outlines of an object placed in front of the camera.
  • US 7,352,356 discloses a system to tactile simulate a virtual displayed image using a high density set of points (e.g. movable pressure responsive pins) at a finger or foot of a blind person.
  • a high density set of points e.g. movable pressure responsive pins
  • the image is translated into signals of the pins in the matrix in order to visualize the scanned image.
  • a pattern of the image is caused to move across a surface of the skin by selective activation and deactivation of the pins in a matric of pins.
  • the pressure response is caused using a fluid in a microelectromechanical system (MEMS) comprising an array of microvalve actuators.
  • MEMS microelectromechanical system
  • Electro elements that give an electric impulse have a good response time, but are unpleasant for the user and the signal produced is not always clear or strong enough. Vibration elements sometimes suffer the same problem.
  • the use of pressure responsive pins as elements provide clear signaling to the tactile receptor.
  • the time needed to move the pins e.g. by using a fluid in MEMS comprising an array of micro valve actuators may introduce a latency that prohibits sensing the tactile signals in real time, i.e. without delay.
  • US2009/0326604 discloses a system for managing brain functions and improving sensory perception.
  • An electro-tactile screen of elements is placed in contact with the tongue of a person by using an individual molded retainer to hold the screen.
  • the elements are preferably wireless connected to a computer device or image processing unit, which sends output signals to the elements.
  • the image processing unit receives signals from e.g. an IR camera placed on the head of a blind person.
  • the system allows a blind person to visualize a ball and catch it.
  • the image processing unit creates a complex multidimensional electro tactile image similar to visual imagery.
  • the stimulation by the element may be an electro-tactile stimulation, a vibro- tactile simulation or the input may be audio input, visual input or a temperature depending input using heat and cold for signaling.
  • a blind m usician that wishes to play in an orchestra needs to read or sense musical notes or sense instructions from a conductor in order to be able to play together with the others. Especially for playing music together with others, significant delays in translation time from signa l source to tactile receptor are not acceptable. There is a need for the translating visual movements into tactile stimulations without delay in time.
  • the existing systems do not allow a blind person to play in a concert, because the present systems do not allow approximate real time translation from the movement of a baton of the conductor to a tactile signal.
  • the system comprises an I R camera adapted to capture a series of images and an image processing unit, such as a computer. This unit is adapted to receive the series of images from the camera and to generate one or more signals based on the received images.
  • the system also comprises a receptor stimulating device comprising a two dimensional matrix of a plurality of elements, which extend along an X-axis and a Y-axis. This device is adapted to, upon activation, stimulate at least one tactile receptor of a part of the user, and the receptor stimulating device is adapted to receive the signa ls from the image processing unit, and to activate one or more of the elements based on the received signals.
  • the system further comprises a light emitting device designed to be attached to the object and configured to emit light of a wavelength larger than 700 nm.
  • the IR camera is adapted to capture a series of images of the light emitting device.
  • the light emitting device may be a moving device.
  • the I R camera is disposed at a distance between 0.5 and 20 meters, or 1 to 10 meters, from the light emitting device.
  • the image processing unit is adapted to determine two dimensional positions of the light emitting device in a defined 2D coordinate system based on the received images, and to generate one or more signals representing the determined two dimensional positions of the light emitting device, a nd the receptor stim ulating device is ada pted to activate at least one of the elements of the matrix corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals, such that translation of the movement of the object to the elements is realize in real time.
  • the light emitting device may be attached to a moving object, such as a stick/baton of a conductor. The system allows for rapid recording of a movement by the IR camera.
  • the system according to the invention allows translation of visual movement into element activation almost without delay, i.e. practically in real time. This makes it for example possible for a blind musician to play together with other musicians in an orchestra.
  • the system may be used for many other applications, such as teaching.
  • the IR camera has a resolution of at least 1665 x 1088. In another embodiment, the IR camera has a resolution of at least 640 x 480. IR cameras may have a latency in the recoding. By using an IR camera having at least a resolution of at least 640 x 480, or a latency of 20 milliseconds or less the translation of the recoded image to the receptor stimulating device can be achieved without delay. In one embodiment, the IR camera has a latency between 1 and 20 milliseconds.
  • the IR camera has a pixel size between 3 and 7 micrometers.
  • the quality of the camera is important for the translation of the movement. The better the quality the quicker the translation of the movement of the object to the receptor stimulating device.
  • the IR camera operates at a wavelength between 1000 and 14000 nm.
  • the wavelength may be between 690 and 1050 nm when a diode is used as the light emitting device.
  • the wavelength may be between 1000 and 14000 nm if a temporized (heated or cooled) light emitting device is used.
  • the distance between the IR camera and the light emitting device is between 1 and 5 meters.
  • the IR camera is a thermal imaging camera.
  • a thermal imaging camera is believed to be less sensitive to light in a background of the light emitting device, which improves the accuracy of translation of the movement of the object.
  • the light emitting device comprises at least one light emitting diode.
  • Light emitting diodes LED
  • LEDs are small and can easily be attached to an object, such as the baton of a conductor. Further, LEDs are electrically efficient.
  • a ring of diodes may be used as a light emitting source.
  • the diodes can be stacked in a housing or casing. This way, the IR camera perceives the diodes as one and the same light spot. This increases the intensity of the light and improves the distinction by the camera between light from the light emitting device and light from the backgraound.
  • the light emitting device comprises at least one temporized element.
  • the temporized element may be any kind of element that can be heated or cooled.
  • the temporized element is a resistor.
  • the temporized element is a peltier element.
  • the receptor stimulating device is adapted to keep the elements activated for a defined time period after the position has been received.
  • the time period is between 0.1ms and Is.
  • several of the elements can be activated simultaneously and thereby forming a pattern, which can be recognized by the user.
  • the pattern of activated elements corresponds to the movements of the light emitting device and accordingly to the movements of the object.
  • the receptor stimulating device comprises a pushing member adapted to move each individual element to and from the tactile receptor, depending on the signal from the image processing unit.
  • the pushing member allows all elements to be signaled individually in sequence. This gives the user a tactile replication of the exact movement of the light emitting device.
  • the receptor stimulating device comprises two linear electromagnetic motors adapted to move the pushing member along an X-axis and a Y-axis of the matrix. The motors are advantageously used in the receptor stimulating device, because they can operate at high speed, such as at least 5 m/s. This allows movement of the writing ball along the X- or Y-axis at a speed of e.g. 5 m/s and thus prevents delay in signaling the movement to the elements.
  • the receptor stimulating device is preferably robust and comprises only a few parts.
  • the material is light such as aluminum.
  • the receptor stimulating device can be used by any individual, i.e. the device does not need to be fitted or molded before it can be used. This reduces cost and improves ease and flexibility in use of the device.
  • the receptor stimulating device comprises two linear ultrasonic piezo motors adapted to move the pushing member along an X-axis and a Y-axis of the matrix.
  • the pushing member is a writing ball in a holder.
  • the round surface of the ball allows for a fluid movement along pins.
  • the elements are pressure responsive pins movable along a Z-axis, which extends substantially perpendicular to the X- and Y-axis. When movable pressure responsive pins are used as elements, the pins will be pushed up by the writing ball one by one.
  • the receptor stimulating device ca n be provided in different ways depending on the application of the system and the technique available.
  • the present invention is not limited to a specific receptor stimulating device.
  • the elements in the matrix may be stimulated by a motor adapted to move the pushing member along an X-axis and a Y-axis of the matrix as defined above.
  • the elements in the matrix may be stimulated more directly by the image processing unit, i.e. without the use of a motor that stimulates the elements.
  • the elements are linear ultrasonic piezo motors adapted to move along a Z-axis upon receiving the one or more signals from the image processing unit.
  • the elements are electromagnets adapted to move along a Z-axis upon receiving the one or more signals from the image processing unit.
  • the elements are electroactive polymer actuators adapted to move along a Z-axis upon receiving the one or more signals from the image processing unit.
  • the more direct stimulation improves the speed of translation of the movement of the object into tactile stimulation of the receptor of the user. This also improves flexibi lity of the use of the system because writing in Braille is possible if each element is stimulated individually.
  • the system can thus be used for e.g. teaching.
  • the elements are embedded in a polymer.
  • the polymer may be polyurethane and the like. Embedding of the elements in the polymer makes the receptor stimulating device more robust and less sensitive to damage. This increases the life time of the receptor stimulating device and thus reduces costs for the user.
  • the image processing unit uses tracking software to register the movement of the light emitting device.
  • the tracking software improves the speed of transport of the signal from the light emitting device into the movement of the elements in the matrix of the receptor stimulating device.
  • the object of the invention is also achieved by a method for tra nslating visual movements of an object into tactile stimulations of a user, comprising:
  • n I R camera ca pturing a series of images from the emitted light by a n I R camera, wherein the camera is disposed at a distance between 0.5 and 20 meter from the light emitting device
  • the camera has a resolution of at least 1665 x 1088, and a latency between 1 and 20 milliseconds.
  • the camera and the receptor stimulating device may be defined according to any of the embodiments mentioned above.
  • a further object of the invention relates to a use of the system as defined above or the method defined above for visualizing the movement of an object for a user having impaired vision.
  • a further object of the invention relates to a use of the system as defined above or the method defined above for visualizing movements of a baton of a conductor for a user having impaired vision, wherein the light emitting device is attached to the baton.
  • the invention relates to a use of the system and the method by a visually impaired person to play music while following visual instructions of a conductor.
  • the invention relates to a use of the system as defined above or the method defined above for providing a tactile pattern redition.
  • the present invention also relates to a teaching system for translating visual movements of an input member into tactile stimulations of a user, comprising:
  • processing unit adapted to receive signals from the input member and to generate one or more signals based on the received signals
  • the system further comprises
  • a signaling device designed to be connected to the input member and configured to emit a signal to the processing unit, whereby the signaling device is positioned at a distance between 0.5 and 20 meters from the receptor stimulating device,
  • the processing unit is adapted to determine two dimensional positions of the signaling device in a defined 2D coordinate system based on the received images, and to generate one or more signals representing the determined two dimensional positions of the light emitting device, and
  • the receptor stimulating device is adapted to activate an element of the matrix corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals, such that translation of the movement of the input member to the elements is realize in real time.
  • the input member may be a stick or pen held by a teacher or a keyboard or a computer mouse.
  • the signaling device can be connected to a plurality of receptor stimulating devices.
  • a teacher using or holding the input member can teach a plurality of pupils at the same time and in real time. For example, the writing of a teacher on a board ortablet can immediately be translated into Braille forthe pupils.
  • the system can thus also be used by people that are deaf and visually impaired.
  • the signal may be a moving signal.
  • the camera and the receptor stimulating device may be defined according to any of the embodiments mentioned above.
  • One embodiment relates to a use of the teaching system as defined above for visualizing the input of the input member for a user having impaired vision.
  • the input member may be a keyboard used by the teacher for writing, or a data screen used to visualize an object or scheme.
  • the object on the screen can be directly translated to the receptor stimulating device.
  • Fig. 1 shows a block diagram of a system for translating visual movements of an object into tactile stimulations of the invention
  • F ig. 2 illustrates an example of a system of the invention.
  • F ig. 3 shows an example of a receptor stimulating device positioned under a foot
  • F ig. 4 shows an example of a baton of a conductor provided with a light emitting device.
  • Fig. 5 shows details of a receptor stimulating device according to one embodiment of the invention.
  • Figs. show two linear motors in a frame of the receptor stimulating device.
  • Fig. 7 shows an example of a matrix of elements and a position of a writing ball in a matrix.
  • Fig. 8 illustrates an example of a translation of a movement of a stick into signals of activated elements.
  • Fig. 9 shows a flow diagram of an example of a method of the invention.
  • Fig. 10 shows a block diagram of another system for translating visual movements of an object into tactile stimulations of the invention.
  • Fig. 11 shows a flow diagram of an example of a method of the other system of the invention.
  • Figure 1 shows a block diagram of a system for translating visual movements of an object into tactile stimulations of a user according to an embodiment of the invention.
  • the system comprises a light emitting device 1 designed to be attached to the object, which movements are to be detected, and configured to emit light at a wavelength larger than 700 nm, i.e. in the IR spectrum.
  • the system further comprises an IR camera 2 adapted to register the IR light emitted from the light emitting device 1 and to capture a series of images of the light emitting device 1.
  • the IR camera 2 may operate at a wavelength from 700 nm, or between 690 and 14000 nm.
  • the IR camera may operate at a wavelength between 690 and 1050 nm. If the IR camera 2 is a thermal imaging camera, the wavelength may be between 1000 and 5000 nm (short wave camera) or between 5000 and 14000 nm. Examples of thermal imaging camera may be a FLIR E 60 or Testo865.
  • the IR camera 2 must have a high resolution that prevents latency in recording of images from the light emitting device 1.
  • the IR camera has a resolution of at least 640 x 480, or 1280 x 1024, or 1665 x 1088, or 2048 x 2048.
  • the latency may be below 10, 9, 8, 7, 6, 5, 4, 3,
  • the pixel size may be between
  • the IR camera and/or the image processing unit 4 may use a tracking application that allows tracking of the movement of the light emitting device 5.
  • the IR camera is a 2D camera and the images are 2D images.
  • Any light emitting device 1 may be used in the system that emits light at a wavelength larger than 700 nm, or between 700 and 1050 nm, or between 700 and 1000 nm, or between 800 and 900 nm, or between 825 and 875 nm. If the light emitting device is a temporized element, it may emit light at a wavelength between 1000 and 14000 nm, or between 100 and 5000 nm, or between 5000 and 14000 nm. Suitably, the light emitting device 1 is an IR Light Emitting Diode (LED).
  • the light emitting device 1 may be one or more temporized element. Examples of the temporized element may be a resistor. Any type of resistor may be used that can be attached to a moving object.
  • Examples may be a metal film resistor.
  • the temporized element may also be a peltier element.
  • Peltier elements can be heated or cooled, which increases flexibility of use of the light emitting device.
  • An example of a peltier element may be a Quickcool QC 32-0.6-1.2.
  • the dimensions of the light emitting device is preferably about 10 x 10 x 5 mm, or 8 x 8 x 3 mm or smaller.
  • a plurality of light emitting devices may be organized in a casing attached or connected to the object (not shown).
  • the IR camera is disposed at a distance from the light emitting device 1.
  • the I R camera is disposed at a distance of between 0.5m and 20m or between lm and 5, or 1 and 3m from the light emitting device 1 so that the view of the camera and accordingly the images will contain the light emitting device 1 even though the light emitting device is moved around, without moving the camera.
  • the camera 2 has a fixed position in the surroundings of the moving object.
  • the system also comprises an image processing unit 4 connected to the IR camera 2 and adapted to receive the series of images from the IR camera 2, to determine two dimensional positions of the light emitting device in a defined plane based on the received images, and to generate one or more signals representing the determined two dimensional positions of the light emitting device based on the received images.
  • the positions of the light emitting device are determined in a 2D coordinate system defined in relation to the view of the camera.
  • the camera comprises a lens, i.e. wide angle which defines the camera's image capture zone.
  • the 2D coordinate system has two coordinate axes, such X and Y axes.
  • the image processing unit 4 is, for example, configured to generate one signal corresponding to an X coordinate of the determined position and another signal corresponding to a Y coordinate of the determined position.
  • the signal is a data signal comprising data packages including information about the position, such that coordinates of the position.
  • the image processing unit 4 comprises input and output means, processing means, such as a CPU, a FPGA or similar hardware, and memory means, such as ROM, RAM or similar hardware.
  • the image processing unit 4 also comprises software for providing image processing of the images.
  • the image processing unit is, for example, a computer, such as, a PC. Appropriate data processing means and software for carrying out image processing as such are known by the skilled person and will thus not be explained in detail.
  • the system further comprises a receptor stimulating device 5 comprising a matrix 16 having a plurality of elements 6 adapted to, upon activation, stimulate at least one tactile receptor of a part of the user.
  • the elements are elongated and are moved in a longitudinal direction upon activation.
  • the tactile receptors of the user are stimulated by applying pressure on the receptor.
  • the elements are caused to vibrate upon activation, and the tactile receptors of a part of the user are stimulated by the vibrations.
  • the matrix defines a two dimensional plane corresponding to the defined plane of the 2D positions of the light emitting device. Each element of the matrix corresponds to a position of the light emitting device.
  • the receptor stimulating device 5 is connected to the image processing unit 4 and adapted to receive the signals from the image processing unit 4, and to activate an element of the matrix 16 corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals.
  • the element 6 of the matrix having a position in the matrix corresponding to the received two dimensional position, is active upon receiving the position que. A series of positions is received.
  • the receptor stimulating device 5 is adapted to keep the elements 6 activated for a defined time period of between 0.1ms and Is after they have been activated. Thus, several of the elements can be activated simultaneously or sequentially and thereby form a pattern, as shown in figure 8, which pattern can be recognized by the user.
  • the pattern of activated elements corresponds to the movements of the light emitting device and accordingly to the movements of the object.
  • Figure 2 illustrates a practical example of the system.
  • the light emitting device 1 is attached to a baton moved by a conductor. Movement of the light emitting device is recorded by the IR camera 2.
  • the IR camera captures a series of images from the emitted light device. The series of images are received using a wire 3 or wireless by the image processing unit 4.
  • the image processing unit 4 translates the input from the IR camera into a signal using software.
  • the signal is sent to the receptor stimulating device 5.
  • the receptor stimulating device causes activation of the elements 6 comprised in the receptor stimulating device.
  • the receptor stimulating device is positioned under a sole of a foot 7 of a musician 8 playing a flute, while seated on a chair 9.
  • all the parts of the system can be held together and easily transported in a suitcase in order to be used at different locations.
  • the receptor stimulating device is a device capable of receiving signals from the image processing unit 4 and translating these signals into element activity. Such a device may use fluid or air to activate microvalves in electro-tactile element 6, or the device may use electricity to provide an electrical impulse or a vibration. Different types of elements may be used, such as electronic or electrical elements, vibration elements, heat elements, pressure elements, and the like.
  • Figure 3 shows a foot 7 of a user positioned on the elements of the receptor stimulating device.
  • the receptor stimulating device 5 comprises a matrix 16 of a plurality of elements 6.
  • the matrix extends along an X-axis and a Y-axis as shown in figure 7, 8.
  • the matrix 16 of elements 6 may cover a portion of the area under the foot or the entire area under the foot.
  • the receptor stimulating device comprises two linear electromagnetic motors positioned in a frame 18 under the matrix 16.
  • One motor 19a can be moved along an X-axis X and one motor 19b can be moved along a Y-axis Y.
  • the motors may be positioned on top of each other as shown in figures 6a, 6b.
  • the motors receive the output signal from the image processing unit 4, or an ⁇ , ⁇ -signal. Depending on that signal, the motors will move along the ⁇ , ⁇ -axes of the matrix of the elements 6.
  • Figure 6d shows that the area of the matrix 16 may be less than the area of the receptor stimulating device. The area may be between 20 x 20 cm or 10 x 10 cm 4 x 4 cm, or 10 x 5 cm, and so on. The invention is not limited to any specific area and can be varied depending on the application and user. A smaller area reduces costs.
  • the motors may be two linear electromagnetic motors 19a, 19b (figures 6a, 6b) or linear ultrasonic piezo motors 19a, 19b as shown in figures 6c.
  • a reverse piezoelectric effect can be explained as an internal generation of a mechanical strain resulting from an applied electrical field.
  • materials may be ceramic, tendon, silk, wood, enamel, dentin, DNA, viral proteins, quartz-analogous crystal, such as langasite (La3Ga5Si014) or gallium orthophosphate (GaP04), lithium niobate (LiNb03), lithium tantalate (LiTa03), synthetic ceramics such as tetragonal unit cell of lead titanate, non-ferroelectric piezoelectric materials, such as AIN and ZnO, tungsten-bronze, barium titanate (BaTi03), lead zirconate titanate (Pb[ZrxTil-x]03 with 0 ⁇ x ⁇ 1) more commonly known as PZT, potassium niobate (KNb03), Sodium tungstate (Na2W03), Ba2NaNb505, Pb2KNb
  • a pushing member 20 Attached to the motors is a pushing member 20.
  • This pushing member is in contact with the elements that are positioned above the pushing member in the direction of a Z axis Z. As shown in figure 5, the pushing member pushes the elements up.
  • the sensors move down by gravity.
  • the pushing member 20 may be a writing ball 20b attached to a holder 20a. The member pushes each individual element up, when the motors 19a, 19b move along the X- and Y-axes.
  • the elements 6 may be movable pins 17.
  • the extent by which the pins are pushed upwards toward the foot must be such that the tactile receptors in the skin are stimulated by the pressure of the pins.
  • the length of the element can be increased in upward extension. This allows to effectively demark an additional piece of information valuable to a user, such as for musical interpretation. Solenoid magnets can me employed for the purpose.
  • the elements 6 may also be moved individually upon receiving a signal from the image processing unit. i.e. without the use of a motor.
  • the elements 6 may be individual linear ultrasonic piezo motors that can move along a Z-axis Z upon receiving the one or more signals from the image processing unit 4.
  • An example of such a piezo motor may be an MPO-05 Piezo, such as the MPO-050015.
  • the elements 6 may be individual electromagnets that can move along a Z-axis upon receiving the one or more signals from the image processing unit. Solenoid magnets can me employed for the purpose.
  • the elements 6 may be individual electroactive polymer actuators that can move along a Z- axis upon receiving the one or more signals from the image processing unit.
  • Electroactive polymers are polymers that exhibit a change in size or shape when stimulated by an electric field. The most common applications of this type of material are in actuators and sensors. A typical characteristic property of an EAP is that they will undergo a large amount of deformation while sustaining large forces. The majority of historic actuators are made of ceramic piezoelectric materials. In the context of this application, EAPs are defined as electroactive polymer actuators not including piezoelectric materials, which are mentioned separately. Examples of EAPs may be electrostrictive polymers and dielectric elastomers.
  • ferroelectric polymers examples may be ferroelectric polymers, electrostrictive graft polymers, liquid crystalline polymers, ionic EAPs, in which actuation is caused by the displacement of ions inside the polymer, electrorheological fluid, ionic polymer-metal composite and stimuli-responsive gels.
  • EAP materials allow to pack many actuators in a small area such as the matrix 16 without interferences.
  • Individual stimulated elements allow for display of an active, full-page and refreshable Braille dots to a user.
  • the elements 6 may be supported by a cloth or canvas on a bottom side of the elements.
  • the elements may be embedded in a polymer 13, preferably a biodegradable polymer as shown in figure 6e.
  • suitable polymers may be polyurethane.
  • Figure 9 shows a chart illustration of a method for translating visual movements into tactile stimulations. It will be understood that some of the blocks of the flow chart can be implemented by computer program instructions.
  • the method may comprise the steps outlined below.
  • Emitting light of a wavelength larger than 700 nm by a light emitting device 1 attached to the object Any light emitting device as mentioned above can be used for this purpose.
  • the object may be moving.
  • the camera is disposed at a distance between 0.5 and 20 meter from the light emitting device. Any of the cameras mentioned above may be used.
  • a receptor stimulating device 5 comprising a two dimensional matrix of a plurality of elements 6.
  • a receptor stimulating device Stimulating at least one tactile receptor of a part of the user by a receptor stimulating device. Any of the receptor stimulating devices outlined above may be used for this purpose.
  • the translation of the movement of the object to the elements is realize in real time.
  • the system or method may be used for visualizing an object for a user being blind or having impaired vision.
  • the system and the method are especially useful for musicians using wind instruments, which do not allow stimulation of tactile receptors in a mouth of the user.
  • the receptor may be located anywhere on the body of the user.
  • the receptor may be located at the sole of the foot of the user.
  • the user may be a musician and the moving object may be a baton 10 of a conductor 11.
  • the elements 6 may be pins 17 as described above. Alternatively, the individual elements 6 can be stimulated in the same manner, such that the same pattern or line is felt by the user.
  • Figure 7 shows a top side of the matrix 16.
  • a cursor 21 is provided on the matrix.
  • a "line” Ix can be felt on the sole of the foot 7.
  • a "line” ly can be felt on the sole of the foot 7.
  • the "line” exists of a sequence of individual upward and downward moving pins 17 along the area of the matrix 16.
  • Figure 8 symbolizes a translation on the matrix of the movement of the light emitting device, which has been moved up, down and then to the right. These types of movements are commonly used with a baton of a conductor 11 to assign a task to an orchestra. Because the elements 6 react at the same time as the light emitting device is moved, i.e. without delay, the system of the present invention can be used by a blind person playing in an orchestra. Examples of further application can be for educational purpose, such as teaching dance or yoga: where tactile stimulation is the preferred means of receiving instruction. The deaf-blind can be especially appreciative of this form of communication and instruction.
  • the invention also relates to a teaching system for translating visual movements of an input member 12 into tactile stimulations of a user as shown in figure 10 and 11.
  • the input member may be a pin or a keyboard.
  • This system comprises a processing unit 4 adapted to receive signals from the input member and to generate one or more signals based on the received signals, and a receptor stimulating device 5.
  • the receptor stimulating device 5 comprises a two dimensional matrix 16 having a plurality of elements 6 adapted to, upon activation, stimulate at least one tactile receptor of a part of the user.
  • the receptor stimulating device is adapted to receive the signals from the processing unit, and to activate one or more of the elements 6 based on the received signals. Any of the receptor stimulating device mentioned above can be used for this purpose.
  • the system further comprises a signaling device la designed to be connected to the input member and configured to emit a signal to the processing unit 4.
  • the signaling device la may be attached to or connected to the input member.
  • the signaling device may be a data screen with images or moving images.
  • the signaling device la is positioned at a distance between 0.5 and 20 meters, or 1 and 5 meters, or about 3 meters, from the receptor stimulating device 5.
  • the processing unit 4 is adapted to determine two dimensional positions of the signaling device in a defined 2D coordinate system based on the received images, and to generate one or more signals representing the determined two dimensional positions of the light emitting device, and the receptor stimulating device 5 is adapted to activate an element of the matrix 16 corresponding to the received two dimensional position of the light emitting device upon receiving the one or more signals, such that translation of the movement of the input member to the elements is realize in real time.
  • the systems of the invention are especially useful for communication with visually impaired persons as well as persons that are both deaf and visually impaired.
  • the elements may be vibrating or heat elements. Different methods may be used to move the pins, or different types of pushing member 20 may be applied.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un système destiné à traduire des mouvements visuels d'un objet en stimulations tactiles d'un utilisateur, comprenant un appareil photo IR (2), une unité de traitement d'image (4) recevant des images et générant des signaux, et un dispositif de stimulation de récepteur (5) comprenant une matrice à double dimension (16) comportant des éléments (6) recevant des signaux à partir de l'unité et des éléments d'activation (6). Le système comprend en outre un dispositif d'émission de lumière (1) fixé à l'objet. L'appareil photo IR capture des images du dispositif mobile (1) à une distance comprise entre 0,5 et 20 mètres. L'unité détermine deux positions dimensionnelles du dispositif dans un système de coordonnées 2D à partir des images et génère des signaux représentant deux positions dimensionnelles du dispositif (1). Le dispositif de stimulation récepteur active un élément correspondant à la position du dispositif (1) lors de la réception des signaux. La traduction du mouvement de l'objet vers les éléments est réalisée en temps réel.
PCT/SE2017/050493 2016-05-13 2017-05-15 Système destiné à traduire des mouvements visuels en stimulations tactiles Ceased WO2017196251A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
SE1650649A SE540186C2 (en) 2016-05-13 2016-05-13 A system for translating visual movements into tactile stimulations.
SE1650649-5 2016-05-13

Publications (1)

Publication Number Publication Date
WO2017196251A1 true WO2017196251A1 (fr) 2017-11-16

Family

ID=60266559

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SE2017/050493 Ceased WO2017196251A1 (fr) 2016-05-13 2017-05-15 Système destiné à traduire des mouvements visuels en stimulations tactiles

Country Status (2)

Country Link
SE (1) SE540186C2 (fr)
WO (1) WO2017196251A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07261751A (ja) * 1994-03-25 1995-10-13 Nippon Telegr & Teleph Corp <Ntt> 指揮者指揮棒動作盲人用表示装置
WO2010119397A2 (fr) * 2009-04-15 2010-10-21 Koninklijke Philips Electronics N.V. Unité d'affichage tactile pliable
WO2010142689A2 (fr) * 2009-06-08 2010-12-16 Kieran O'callaghan Dispositif de détection d'objet
WO2015054789A1 (fr) * 2013-10-18 2015-04-23 Hagedorn Douglas Systèmes et procédés d'interfaçage spatial non visuel avec un ordinateur

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07261751A (ja) * 1994-03-25 1995-10-13 Nippon Telegr & Teleph Corp <Ntt> 指揮者指揮棒動作盲人用表示装置
WO2010119397A2 (fr) * 2009-04-15 2010-10-21 Koninklijke Philips Electronics N.V. Unité d'affichage tactile pliable
WO2010142689A2 (fr) * 2009-06-08 2010-12-16 Kieran O'callaghan Dispositif de détection d'objet
WO2015054789A1 (fr) * 2013-10-18 2015-04-23 Hagedorn Douglas Systèmes et procédés d'interfaçage spatial non visuel avec un ordinateur

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HOSSNY M ET AL.: "Haptically-Enabled Dance Visualisation Framework for Deafblind-Folded Audience and Artists", IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS, 9 October 2015 (2015-10-09), XP032847175 *

Also Published As

Publication number Publication date
SE1650649A1 (en) 2017-11-14
SE540186C2 (en) 2018-04-24

Similar Documents

Publication Publication Date Title
TWI710934B (zh) 觸覺資訊轉換裝置、觸覺資訊轉換方法、及觸覺資訊轉換程式與元件配置構造體
Pacchierotti et al. Cutaneous/tactile haptic feedback in robotic teleoperation: Motivation, survey, and perspectives
CN1121790C (zh) 触觉通信装置和方法
US5736978A (en) Tactile graphics display
US20110316798A1 (en) Tactile Display for Providing Touch Feedback
JP6877688B2 (ja) プログラミング装置及びその制御プログラム、プログラミング方法
US6693516B1 (en) Electro-mechanical transducer suitable for tactile display and article conveyance
US20110155044A1 (en) Kinesthetically concordant optical, haptic image sensing device
US6445284B1 (en) Electro-mechanical transducer suitable for tactile display and article conveyance
KR20180112847A (ko) 촉각 정보 변환 장치, 촉각 정보 변환 방법, 및, 촉각 정보 변환 프로그램
Strohmeier et al. From Pulse Trains to" Coloring with Vibrations" Motion Mappings for Mid-Air Haptic Textures
Horvath et al. FingerSight: Fingertip haptic sensing of the visual environment
WO2015121970A1 (fr) Dispositif tactile éducatif et système
CN109213317A (zh) 主动矩阵触觉反馈
CN113311946A (zh) 一种面向移动终端应用的多模式指套式装置
Tirado et al. Electroar: Distributed electro-tactile stimulation for tactile transfer
CN110728886B (zh) 盲文学习系统、指尖传感器及其形成方法
Stetten et al. Fingersight: Fingertip visual haptic sensing and control
Abad Fine-grained haptics: Sensing and actuating haptic primary colours (force, vibration, and temperature)
SE540186C2 (en) A system for translating visual movements into tactile stimulations.
US20260016899A1 (en) Display terminal, virtual tactile device, and virtual tactile system including thereof
Ray et al. An overview on tactile display, haptic investigation towards beneficial for blind person
Strohmeier et al. To improve haptic experiences in VR and AR, we must first improve tracking.
CN117348756B (zh) 一种基于感觉增强的信息交互系统及方法
Howe The shape of things to come: pin-based tactile shape displays

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17796494

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27/02/2019)

122 Ep: pct application non-entry in european phase

Ref document number: 17796494

Country of ref document: EP

Kind code of ref document: A1