JP2000207485A - Information processing apparatus and method, and providing medium - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To enable even a person unfamiliar with the operation to quickly and reliably start a predetermined application program. SOLUTION: For example, the name of a word processor program is "Santaro", and then separated by "." And opened by Santaro (takes data to start processing)
The CCD video camera 23 picks up an image of the card-like object 111 on which the name of a predetermined application program and its arguments are printed, such as a card-like object 111 on which "patent specification" is printed as a file name. The personal computer 1 performs a character recognition process on the image data captured by the CCD video camera 23, activates an application program having the name of the character obtained as a result of the recognition, and then loads the data of the file name recognized as an argument. Automatically perform operations such as setting options.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus and method, and a providing medium, and more particularly to an information processing apparatus capable of recognizing characters from data of a captured image and performing predetermined processing. And a method, and a providing medium.

[0002]

2. Description of the Related Art A system that encodes alphanumeric characters indicating the type and state of an article and affixes it to an article, for example, and reads it later to obtain information (such as the type and state of the article); So-called barcode systems are now widespread in many industrial fields.

FIG. 1 shows an example of a bar code. In this example, the alphanumeric characters displayed on the character display portion 502 below the code portion 501 are displayed on the barcode label 500 by the width of a bar (black portion) arranged in the code portion 501 and the width of each bar. It is coded based on the width between the bars (white parts). Thus, a barcode that encodes information with one parameter (bar or width between bars) is called a one-dimensional barcode.

As a method of reading a barcode from a barcode label, a method of reading a barcode is known. For example, when the arrangement of bars increases with the information to be coded, the video camera uses the method. Barcode reading becomes difficult.

Therefore, in order to enable reading by a video camera even when the information to be coded becomes large, a cell (black square) as shown in FIG. However, a two-dimensional barcode arranged two-dimensionally has been proposed. According to this, since the information is coded two-dimensionally, a larger amount of information can be coded, and even if more information is coded than a bar of a one-dimensional barcode, ,
Reading by a video camera becomes easy.

[0006]

However, such a one-dimensional bar code and a two-dimensional bar code must be prepared with such a code pattern in advance, so that a general user can use, for example, various types of personal computers. There was a problem that could not be used to control the processing.

[0007] Japanese Patent Application Laid-Open No. 9-289624 discloses an OCR (Optical Character Reade) for a digital camera.
It has been proposed to combine r) to recognize characters captured by a camera.

However, this proposal merely adds a function as an OCR device to the digital camera, and does not contribute to controlling the operation of the digital camera itself.

The present invention has been made in view of such a situation, and is intended to enable a general user to easily and reliably control the processing of an information processing device such as a personal computer.

[0010]

According to a first aspect of the present invention, there is provided an information processing apparatus, comprising: a specific processing unit for executing a predetermined process; an imaging unit for imaging a subject; and an image captured by the imaging unit. Character recognition means for performing character recognition processing based on the data of
It is characterized by comprising a comparing means for comparing a character string associated with a specific processing means registered in advance, and a control means for controlling the specific processing means based on a comparison result of the comparing means.

According to a fourth aspect of the present invention, there is provided an information processing method comprising: a specific processing step of executing a predetermined process; a capturing step of capturing an image of a subject; and data of an image captured in the capturing step. A character recognition processing step of performing character recognition processing; a comparison processing step of comparing a recognition result in the character recognition processing step with a character string associated with a specific processing step registered in advance; Based on the result,
And a control processing step for controlling the processing of the specific processing step.

According to a fifth aspect of the present invention, there is provided a providing medium based on a specific processing step of executing a predetermined process, an imaging processing step of imaging a subject, and data of an image captured in the imaging processing step. A character recognition processing step of performing character recognition processing, a comparison processing step of comparing a recognition result in the character recognition processing step with a character string associated with a specific processing step registered in advance, and a comparison result in the comparison processing step And a computer-readable program that causes the information processing apparatus to execute a process including a control process step of controlling the process of the specific process step based on the program.

[0013] The information processing apparatus according to claim 1,
In the information processing method described in the above, and the providing medium described in the claim 5, predetermined processing is controlled based on a comparison result between a character recognition result of a captured image and a character string registered in advance. You.

[0014]

3 to 8 show examples of the configuration of a portable personal computer to which the present invention is applied. The personal computer 1 is a mini-notebook type personal computer.
The display unit 3 is configured to be openable and closable with respect to the main body 2. 3 is an external perspective view showing a state in which the display unit 3 is opened with respect to the main body 2, FIG. 4 is a plan view in a state in which the display unit 3 is opened with respect to the main body 2, and FIG. Body 2
6 is a right side view showing the display unit 3 opened 180 degrees with respect to the main body 2, and FIG. 7 is a view showing the display unit 3 closed with respect to the main body 2. FIG. 8 is a bottom view in a state where the display unit 3 is opened with respect to the main body 2.

The main body 2 is provided on its upper surface with a keyboard 4 operated when inputting various characters and symbols, and a track point (trademark) 5 operated when moving a mouse cursor. . Further, on the upper surface of the main body 2, there are further provided a speaker 8 for outputting sound and a shutter button 10 operated when capturing an image with the CCD video camera 23 provided on the display unit 3.

A claw 13 is provided at an upper end portion of the display unit 3. As shown in FIG. 5, when the display unit 3 is closed with respect to the main body 2, Is provided with a hole 6 into which the claw 13 is fitted. A slide lever 7 is provided on the front surface of the main body 2 so as to be movable in parallel with the front surface. The slide lever 7 engages with a claw 13 fitted in the hole 6 to lock and unlock. I can do it. By releasing the lock, the display unit 3 can be rotated with respect to the main body 2. Microphone 2 next to claw 13
4 is attached. As shown in FIG. 8, the microphone 24 can also collect sound from the back.

On the front of the main body 2, a programmable power key (PPK) 9 is provided. As shown in FIG. 6, an exhaust hole 11 is provided on the right side surface of the main body 2, and an intake hole 14 is provided on the lower front surface of the main body 2 as shown in FIG. 7. In addition, a PCMCIA (Personal Computer Memory Card Interna
nation association) card (PC card) is provided.

An LCD for displaying an image is provided on the front of the display unit 3.
(Liquid Crystal Display) 21 is provided, and an imaging unit 22 is provided at the upper end thereof so as to be rotatable with respect to the display unit 3. That is, this imaging unit 22
It can rotate to any position within a range of 180 degrees between the same direction as D21 and the opposite direction (backward direction). A CCD video camera 23 is attached to the imaging unit 22.

A power lamp is provided on the lower body side of the display unit 3.
PL, battery lamp BL, message lamp ML, other LEDs
Is provided. Reference numeral 40 shown in FIG. 5 is a power switch provided on the left side surface of the main body 2, and reference numeral 25 shown in FIG. 7 is an adjustment ring for adjusting the focus of the CCD video camera 23. Further, FIG.
Reference numeral 26 denotes a lid for covering an opening for mounting an additional memory in the main body 2, and reference numeral 41 denotes a lid 26.
This is a small hole for inserting a pin for removing the lock claw.

FIG. 9 shows an example of the internal configuration of the personal computer 1. The internal bus 51 includes a CPU (Centr
al Processing Unit) 52, PC inserted as needed
A card 53, a RAM (Random Access Memory) 54, and a graphic chip 81 are connected. The internal bus 51 is connected to an external bus 55. The external bus 55 has a hard disk drive (HDD) 56, an I / O
(Input / output) controller 57, keyboard controller 58, track point controller 59, sound chip 60, LCD controller 83, modem 50 and the like are connected.

The CPU 52 is a controller that controls each function, and the PC card 53 is appropriately mounted when an optional function is added.

Image data captured by the CCD video camera 23 is input to the graphic chip 81 after being processed by the processing unit 82. The graphic chip 81 stores video data input from the CCD video camera 23 via the processing unit 82 into the built-in VRAM 8.
1A, read it out as appropriate, and output it to the LCD controller 83. The LCD controller 83 converts the image data supplied from the graphic chip 81 into the LCD 21
And display it. The backlight 84 is connected to the LCD 21
To be illuminated from behind.

In the RAM 54, at the time of completion of the activation, predetermined information such as an OS (Operating System) (basic program) 54A, an autopilot program (application program) 54B, and a finder application (application program) 54C are stored. Things are transferred from the HDD 56 and stored.

OS (basic program software) 54A
Is for controlling the basic operation of a computer typified by Windows 98 (trademark).

The autopilot program 54B is a program for sequentially activating and processing a plurality of preset processes (or programs) in a preset order.

The finder application 54C has a CC
The image captured by the D video camera 23 is received in real time, and the image is sequentially subjected to binarization processing and character area extraction processing, and the binarized image subjected to the character area extraction processing is sent to the character recognition engine 89. , CCD video camera 2
3 captures images, character extraction areas, character recognition results, etc.
It can be displayed effectively using the GUI, or according to the recognition result,
Performs interlocking processing with other application programs.

On the other hand, a hard disk drive (HDD) 56 on the external bus 55 stores an OS (basic program software) 56A, an autopilot program 56B, and a finder application 56C.

The hard disk drive 56 also has an electronic mail program 56D, an Internet browser program 56E, a word processor program 5
Application programs such as 6F and a translation program 56G are stored.

The e-mail program 56D is a program for exchanging messages via a network from a communication line such as a telephone line. E-mail program 56D
Has an incoming mail acquisition function as a specific function. This incoming mail acquisition function executes processing for checking with the mail server 93 whether mail addressed to the user (user) has arrived in the mailbox 93A, and acquiring mail if there is any mail addressed to the user. I do.

Internet browser program 56E
Executes a function of communicating with another information processing apparatus via the Internet 92, for example, like Netscape (trademark) of Netscape Corporation (trade name).

The word processor program 56F includes, for example, Ichitaro (trademark) and Word (trademark).
Perform word processing in Japanese or English. The translation program 56G performs a process of translating English into Japanese or vice versa.

The I / O controller 57 has a microcontroller 61.
An I / O interface 62 is provided. This microcontroller 61 includes an I / O interface 62, a CPU 6
3, a RAM 64 and a ROM 69 are connected to each other. The RAM 64 stores the key input status register 6
5. It has an LED (light emitting diode) control register 66, a set time register 67, and a register 68. The set time register 67 is used to start the operation of the start-up sequence control unit 76 at a time (start-up condition) set by the user in advance. The register 68 stores a correspondence between a preset operation key combination (start condition) and an application program to be started. When the stored operation key combination is input by the user, the register 68 stores the correspondence. The application program (for example, the e-mail program 56D) is activated.

The key input status register 65 stores an operation key flag when the programmable power key (PPK) 9 for one-touch operation is pressed. The LED control register 66 is a message lamp ML for displaying a start-up state of the application program (e-mail program 56D) stored in the register 68.
This controls the lighting of. Setting time register 67
Can set a predetermined time arbitrarily.

A backup battery 74 is connected to the microcontroller 61 so that the values of the registers 65, 66, and 67 are retained even when the power of the main body 2 is turned off. It has become.

The ROM 69 in the microcontroller 61 stores a wakeup program 70, a key input monitoring program 71, and an LED control program 72 in advance. The ROM 69 is, for example, an EEPROM (electricall
y erasable and programmable read only memory). This EEPROM is also called a flash memory. Further, an RTC (Real-Time Clock) 75 that constantly counts the current time is connected to the microcontroller 61.

The wake-up program 70 in the ROM 69 checks, based on the current time data supplied from the RTC 75, whether or not the time set in the set time register 67 has been reached. , A program for starting a predetermined process (or program) or the like. The key input monitoring program 71
9 is a program for constantly monitoring whether or not 9 has been pressed by the user. The LED control program 72 is a program for controlling lighting of the message lamp ML.

The ROM 69 further includes a BIOS (Basic Input / O
utput System) 73 is written. The BIOS refers to a basic input / output system, and is a software program that controls data transfer (input / output) between an OS or application software and peripheral devices (display, keyboard, hard disk drive, etc.).

A keyboard controller 58 connected to the external bus 55 controls input from the keyboard 4. The track point controller 59 controls input of the track point 5.

The sound chip 60 includes the microphone 24
Or an audio signal is supplied to the built-in speaker 8.

The modem 50 can be connected to a communication network 92 such as the Internet, a mail server 93, and the like via a public telephone line 90 and an Internet service provider 91.

The power switch 40 is operated when the power is turned on or off. The half-press switch 85 is turned on when the shutter button 10 is half-pressed, and the full-press switch 86 is turned on when the shutter button 10 is fully pressed. The reversing switch 87 is connected to the imaging unit 2
2 is rotated 180 degrees (CCD video camera 23
It is turned on when it is rotated in the direction of imaging the opposite side of the CD 21).

Next, the personal computer 1 of the present invention
A description will now be given of a process performed when the finder application 54C is executed. When the finder application 54C is activated, the user:
As shown in FIG. 10, the operation of the personal computer 1 can be controlled by capturing an image of the card-shaped object 111 made of cardboard or the like on which predetermined characters are printed by the CCD video camera 23. 11 and 1 for the processing of the finder application 54C.
This will be described with reference to the flowchart of FIG.

First, in step S1, the finder application 54C controls the graphic chip 81 and creates a GUI (Graphical User Interface).
Display. The graphic chip 81 creates GUI image data in response to a command from the finder application 54C and draws the GUI image data on the VRAM 81A. VRAM81A
Is read out of it, output to the LCD 21 via the LCD controller 83, and displayed. Thereby, for example, the GUI of the finder application as shown in FIG. In this GUI, a target image display unit 241 for displaying an image captured by the CCD video camera 23 and a recognition history display unit 242 for displaying a history of results of character recognition performed by the character recognition engine 89 as described later. Is provided.

Next, in step S2, the finder application 54C determines whether image data has been received from the CCD video camera 23 (more specifically, its driver). Wait until you do. If image data is received, step S3
The finder application 54C displays the image on the GUI. That is, the image of the object 111 captured by the CCD video camera 23 is
After a predetermined process is performed by 2, it is supplied to the graphic chip 81. The graphic chip 81 transmits the image data to the target image display unit 241 of the GUI of the VRAM 81A.
Draw on the part corresponding to. The image data drawn in the VRAM 81A is read out and the LCD controller 83
, The image captured by the CCD video camera 23 is displayed as a moving image on the target image display unit 241 as shown in FIG. The data of the image captured by the CCD video camera 23 is also stored in the RAM 54 in order to extract an area (character area) for character recognition.

In step S4, the finder application 54C determines whether a character area has been designated by the user. If the character area has not been specified, the process proceeds to step S5, and the finder application 54C extracts a predetermined range from the one-frame image stored in the RAM 54 as a character area. Then, in step S6, the finder application 54C transfers the extracted pixel data of the character area to the character recognition engine 89, and causes the character recognition engine 89 to perform character recognition. The character recognition engine 89 stores various character patterns in advance, compares the pattern with pixel data in the transferred character area range, and recognizes whether or not a character is included. Then, the character recognition engine 89 outputs the recognition result (character) to the finder application 54C.

As described above, normally, the finder application 54C continuously executes the character recognition processing on the image of each frame, but the user operates the track point 5 or the like to specify the character area. Is also possible. When the character area is designated by the user, the processing in step S5 is skipped, and the character recognition processing is executed by the character recognition engine 89 in the range of the designated character area.

In step S7, the finder application 54C determines whether or not the character has been recognized based on the output of the character recognition engine 89. If not, the process returns to step S2, and the subsequent processing is performed. Is repeatedly executed.

If it is determined in step S7 that the character has been recognized, the process proceeds to step S8, where the finder application 54C displays a frame indicating the range of the character area in which the character has been detected on the GUI. That is, at this time, the finder application 54C controls the graphic chip 81 to display a character area in the target image display unit 241 of the GUI where the character is recognized in the extraction frame 251. Thus, the user can know from which image of the region the character has been recognized. Further, in step S9, the finder application 54C
Displays the character recognition result (character) 252 supplied from the character recognition engine 82 below the extraction frame 251. At this time, if a plurality of candidates exist as a character recognition result, the candidates are also displayed at the same time. By operating the track point 5 or the like, the user selects a correct character from the character recognition results 252 or, if one correct character is already displayed, an input for confirming the recognition result. Perform the operation. Character recognition result 25
2, an extraction frame 251 indicating the range in which it is recognized.
Is displayed, the user can quickly and reliably determine whether or not the recognition result is correct.

Therefore, in step S10, the finder application 54C determines whether or not a confirmation input has been made that the recognition result is correct. If the confirmation input has not been made, the process returns to step S2, and the subsequent processing is performed. Repeat and execute.

If the correct input of the recognition result is made, the finder application 54C enters the edit box and the history box in the RAM 54 or the hard disk drive 56 in step S11.
The determined character recognition result is copied. The character recognition result registered in the edit box can be read and used later as appropriate.

Next, in step S12, the finder application 54C reads the contents in the history box and displays them on the recognition history display section 242 of the GUI. As a result, a history of character recognition results is displayed on the recognition history display section 242 as shown in FIG. 13 (recognized characters are displayed in order).

In step S13, the finder application 54C further checks the recognition history display section 242.
It is determined whether or not a predetermined one is selected from the character recognition results displayed in. That is, the user can select a predetermined character from the characters displayed on the recognition history display unit 242 by operating the track point 5 or the like. If at least one character recognition result has been selected, the process proceeds to step S14, and the finder application 54C determines whether or not the character string is the name of an application program stored in the hard disk drive 56 or the RAM 54 in advance. judge. If it is determined that the selected character string is the name of a predetermined application program, the process proceeds to step S15, where the finder application 54C
Execute message display processing.

For example, the character recognition result of "Santaro" is selected, and this character string is
If the name is 6F, "Do you want to start Santaro?"
Is displayed together with a YES button and a NO button. The user selects the YES button by operating the track point 5 or the like when starting the application program, and selects the NO button when not starting the application program.

Therefore, in step S16, the finder application 54C determines whether the YES button has been selected. If the YES button has been selected, the process proceeds to step S17 to activate the application. That is, in the case of the present example, the finder application 54 </ b> C
6F is transferred from the hard disk drive 56 to the RAM 54 and activated. Therefore, the user can cause the personal computer 1 to start the program simply by causing the CCD video camera 23 to image the object 111 on which the name of the program to be started is printed. Further, in step S6 described above, after the character string of "Santarou" of the object 111, a character string of "patent specification" is recognized as a file name to be opened by "Santaro" separated by "." , Step S1
In step 7, the data fetch operation of the file name recognized as the argument is automatically executed.

In step S13, when it is determined that the character recognition result has not been selected, when it is determined in step S14 that the character string is not the name of the application, or in step S16, the NO button is selected. If it is determined that the application program has been executed, the process is terminated without starting the application program.

In the above description, the word processor program is started, but it goes without saying that other programs may be started.

In the above, the case where the present invention is applied to a portable personal computer has been described as an example.
The present invention provides a desktop personal computer,
It can be applied to other information processing devices.

Note that as a providing medium for providing a user with a computer program for performing the above-described processing,
In addition to recording media such as magnetic disks, CD-ROMs, and solid-state memories, communication media such as networks and satellites can be used.

[0059]

As described above, the information processing apparatus according to the first aspect, the information processing method according to the fourth aspect, and the fifth aspect.
According to the providing medium described in the above, based on the data of the captured image, the recognition result of the character recognition processing, based on the comparison result of the character string registered in advance, for example, start a word processor program As a result, a predetermined process such as opening a desired file is executed, so that even a user unfamiliar with the operation can quickly and surely execute various processes.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a one-dimensional barcode.

FIG. 2 is a diagram illustrating an example of a two-dimensional barcode.

FIG. 3 is a perspective view showing a configuration example of a portable personal computer to which the present invention is applied.

4 is a plan view of the portable personal computer of FIG. 3 in a state where a display unit is opened.

5 is a left side view of the portable personal computer of FIG. 3 in a state where a display unit is closed.

6 is a right side view of the portable personal computer of FIG. 3 with a display unit opened.

7 is a front view of the portable personal computer of FIG. 3 in a state in which a display unit is closed.

8 is a bottom view of the portable personal computer of FIG. 3 in a state where a display unit is opened.

9 is a block diagram showing an example of the internal configuration of the portable personal computer of FIG.

FIG. 10 is a perspective view showing a use state of the portable personal computer of FIG. 3;

FIG. 11 is a flowchart illustrating the operation of the portable personal computer of FIG. 3;

FIG. 12 is a flowchart illustrating an operation of the portable personal computer of FIG. 3;

FIG. 13 is a diagram illustrating a display example of a GUI.

[Explanation of symbols]

1 portable personal computer, 2 body, 3
Display, 21 LCD, 23 CCD video camera,
52 CPU, 54 RAM, 54C finder application, 56D e-mail program, 56
E Internet browser program, 56F word processor program, 56G translation program,
89 character recognition engine

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Takahiko Sueyoshi 6-35 Kita Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F-term (reference) 5B064 AA07 BA01 CA08 EA05 FA05 FA06 FA13

Claims (5)

[Claims] 1. A specific processing means for executing a predetermined process, an imaging means for imaging a subject, and a character recognition means for performing a character recognition process based on data of an image taken by the imaging means. Comparing means for comparing the result of recognition by the character recognizing means with a character string associated with the specific processing means registered in advance; controlling the specific processing means based on a comparison result of the comparing means An information processing apparatus comprising: 2. The information processing apparatus according to claim 1, wherein the specific processing unit is an application program, and the control unit activates the application program based on a comparison result of the comparison unit. . 3. The information processing apparatus according to claim 2, wherein the control unit executes an operation on the application program after activating the application program based on a comparison result of the comparison unit. . 4. A specific processing step for executing a predetermined process, a capturing step for capturing an image of a subject, and a character for performing a character recognition process based on data of the image captured in the capturing step. A recognition processing step, a comparison processing step of comparing the recognition result in the character recognition processing step with a character string associated with the specific processing step registered in advance, based on the comparison result in the comparison processing step, A control processing step of controlling the processing of the specific processing step. 5. A specific processing step of executing a predetermined process, an imaging processing step of imaging a subject, and a character performing a character recognition processing based on data of an image captured in the imaging processing step. A recognition processing step, a comparison processing step of comparing the recognition result in the character recognition processing step with a character string associated with the specific processing step registered in advance, based on the comparison result in the comparison processing step, A providing medium for providing a computer-readable program that causes an information processing apparatus to execute processing including a control processing step of controlling the processing of the specific processing step.