WO2021102632A1 - 字符获取、页面处理与知识图谱构建方法及装置、介质 - Google Patents

字符获取、页面处理与知识图谱构建方法及装置、介质 Download PDF

Info

Publication number
WO2021102632A1
WO2021102632A1 PCT/CN2019/120634 CN2019120634W WO2021102632A1 WO 2021102632 A1 WO2021102632 A1 WO 2021102632A1 CN 2019120634 W CN2019120634 W CN 2019120634W WO 2021102632 A1 WO2021102632 A1 WO 2021102632A1
Authority
WO
WIPO (PCT)
Prior art keywords
entity
verified
entities
candidate
level
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/120634
Other languages
English (en)
French (fr)
Inventor
刘楠
王川
苗新宇
雷一鸣
王洪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BOE Technology Group Co Ltd
Original Assignee
BOE Technology Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BOE Technology Group Co Ltd filed Critical BOE Technology Group Co Ltd
Priority to PCT/CN2019/120634 priority Critical patent/WO2021102632A1/zh
Priority to JP2022504256A priority patent/JP2023510667A/ja
Priority to EP19945445.5A priority patent/EP4068121B1/en
Priority to CN201980002570.5A priority patent/CN113260988B/zh
Priority to US16/982,185 priority patent/US12131809B2/en
Publication of WO2021102632A1 publication Critical patent/WO2021102632A1/zh
Anticipated expiration legal-status Critical
Priority to US18/885,106 priority patent/US20250006320A1/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/40—Document-oriented image-based pattern recognition
    • G06V30/41—Analysis of document content
    • G06V30/412—Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/40—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for data related to laboratory analysis, e.g. patient specimen analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/26—Techniques for post-processing, e.g. correcting the recognition result

Definitions

  • the embodiments of the present disclosure relate to a method and device for obtaining character information in a picture, a storage medium, a page processing method, and a knowledge graph construction method.
  • the patient (or physical examiner) will do a lot of medical examinations to confirm whether the body is healthy, and the medical examination results are used as auxiliary information for the doctor to make a diagnosis.
  • the medical examination results are used as auxiliary information for the doctor to make a diagnosis.
  • paper documents test forms, prescriptions, medical records, medical reports, etc.
  • patients may throw away the paper documents or store the paper documents in the corner of the home. Therefore, the preservation rate of the patient's historical medical examination test data is low, and it is difficult to reuse it.
  • paper documents and their images are unstructured data, which is not easy to view and manage.
  • OCR Optical Character Recognition
  • At least one embodiment of the present disclosure provides a method for obtaining character information in a picture, which includes: obtaining a picture and extracting at least one item of character information included in the picture; and performing processing on the at least one item of character information based on a knowledge graph. check.
  • the verifying at least one item of character information based on the knowledge graph includes: confirming that there is wrong character information in the at least one item of character information based on the knowledge graph And correcting the character information with errors based on the knowledge graph.
  • the at least one item of character information includes multiple items of character information.
  • the confirming that there is an error in the at least one item of character information based on the knowledge graph includes: obtaining multiple entities respectively according to multiple pieces of character information of the picture, and selecting from the multiple entities to be verified The entity is used to determine whether the character information corresponding to the entity to be verified has errors; and according to the hierarchical structure of the knowledge graph, to confirm whether the character information corresponding to the entity to be verified has errors.
  • the character information corresponding to the entity has an error, it is confirmed that the character information corresponding to the entity to be verified is the character information with the error.
  • the confirming whether there is an error in the character information corresponding to the entity to be verified according to the hierarchical structure of the knowledge graph includes: The hierarchical structure of the graph is classified; the level of the entity to be verified in the hierarchical structure of the knowledge graph is determined; the entity to be verified is calculated to have the same level and sum as the entity to be verified in the knowledge graph.
  • the entity to be verified is an entity to be verified with an error, and the character information corresponding to the entity to be verified has an error.
  • the minimum coding distance algorithm is used to acquire the similarity between the entity to be verified and all entities in the knowledge graph that have the same level and the same relationship as the entity to be verified. degree.
  • the correcting the character information with errors based on the knowledge graph includes: determining that the knowledge graph has the same level and the same as the entity to be verified The number of all entities in the relationship to obtain the number of entities.
  • the number of entities is equal to 1, directly replace the character information corresponding to the entity to be verified with errors with the character information corresponding to the entity having the same level and the same relationship as the entity to be verified, or, Calculate the probability that an entity with the same level and the same relationship as the entity to be verified is the entity to be verified with errors to obtain the entity probability, and in the case that the entity probability is greater than a predetermined entity probability threshold,
  • the character information corresponding to the entity to be verified with errors is replaced with the character information corresponding to the entity having the same level and the same relationship as the entity to be verified.
  • the following steps are performed: determine at least two candidate entities based on the similarity of the multiple entities; calculate each of the at least two candidate entities as the error to be corrected Verify the probability of the entity to obtain the candidate probability of each of the at least two candidate entities; and replace the character information corresponding to the entity to be verified with an error with the character corresponding to the candidate entity with the largest candidate probability information.
  • the determining at least two candidate entities based on the similarity of the multiple entities includes: comparing the pair of the multiple entity similarities with the to-be-understood in the knowledge graph based on the multiple entity similarity Verify that all entities with the same level and the same relationship are sorted in descending order, and a predetermined number of entities ranked at the top of the sequence are selected as the at least two candidate entities; or based on the similarity of the multiple entities to the knowledge All entities in the graph that have the same level and the same relationship as the entity to be verified are sorted in ascending order, and a predetermined number of entities at the end of the sequence are selected as the at least two candidate entities.
  • the calculating the probability that each of the at least two candidate entities is the entity to be verified with errors includes: determining whether the entity to be verified includes the following: A first-level entity to be verified to obtain a first determination result, wherein the next-level entity to be verified is all next-level entities belonging to the entity to be verified; it is determined whether the entity to be verified is Corresponding to the relevant entity to be verified to obtain the second determination result, wherein the relevant entity to be verified is at the same level as the entity to be verified in the hierarchical structure of the knowledge graph, and is the same as the entity to be verified.
  • Each of the entities is a method of the probability of the entity to be verified with errors.
  • the first determination result is the first determination sub-result in a case where the entity to be verified is determined to include the entity to be verified at the next level, or the The entity to be verified does not include the second determination sub-result in the case where the entity to be verified at the next level; and the second determination result is that the entity to be verified corresponds to the relevant entity to be verified.
  • the third decision sub-result in the case of the verification entity or the fourth decision sub-result in the case that it is determined that the entity to be verified does not correspond to the relevant entity to be verified.
  • the selection for calculating each of the at least two candidate entities based on the first determination result and the second determination result is the pending entity with errors.
  • the method for verifying the probability of an entity includes: when the first judgment result is a first judgment sub-result and the second judgment result is a third judgment sub-result, using the sum of the entities to be verified at the next level / Or the relevant entity to be verified calculates the candidate probability of each of the at least two candidate entities; where the first judgment result is a second judgment sub-result and the second judgment result is a third judgment sub-result
  • the candidate probability of each of the at least two candidate entities is calculated; when the first decision result is the first decision sub-result and the second decision result is the fourth In the case of judging the sub-result, use the next-level entity to be checked to calculate the candidate probability of each of the at least two candidate entities; and when the first judging result
  • the calculating the candidate probability of each of the at least two candidate entities by using the related entity to be verified includes: obtaining the related entity to be verified corresponding to the candidate entity. Verification entity; for each of the entities included in the related entity to be verified, the following method is performed: obtaining a related candidate entity, wherein the related candidate entity and each of the at least two candidate entities are in the
  • the knowledge graph is located at the same level in the hierarchical structure, and is associated with each of the at least two candidate entities, and the relationship with the upper level entity to which each of the at least two candidate entities belongs is equal to the correlation
  • the relationship between each of the entities included in the entity to be verified and the entity to be verified is calculated, and the similarity between each of the entities included in the related entity to be verified and the related candidate entity is calculated;
  • the similarity between each of the entities included in the related entity to be verified and the related candidate entity obtains the relative similarity of each of the at least two candidate entities; and based on each of the at least two candidate entities.
  • the acquisition of each of the at least two candidate entities is based on the similarity between each of the entities included in the related entity to be verified and the related candidate entity
  • the related similarity degree includes: determining the number of entities included in the related entity to be verified; when the number of entities included in the related entity to be verified is equal to 1, the related entity to be verified
  • the similarity between the entity included in the entity and the related candidate entity is used as the related similarity of each of the at least two candidate entities; when the number of entities included in the related entity to be verified is greater than one ,
  • the relative similarity of each of the at least two candidate entities and the entity similarity acquiring the candidate probability of each of the at least two candidate entities includes: combining the relative similarity of each of the at least two candidate entities and the entity The weighted sum of the similarities serves as the candidate probability of each of the
  • the calculating the candidate probability of each of the at least two candidate entities by using the next-level entity to be verified includes: acquiring the entity to be verified includes The next-level entity to be verified; based on the relationship between the entity to be verified and each entity in the next-level entity to be verified, the next-level entity to be verified is divided into at least one group of next-level entities Verification entity group; for each of the at least one next-level entity group to be verified, the following method is performed: obtaining a next-level candidate entity group, wherein the next-level candidate entity group is subordinate to the The relationship between each of the at least two candidate entities and each of the at least two candidate entities is equal to the relationship between each of the entities included in the next-level entity group to be verified and the entity to be verified Calculate the similarity between each of the at least one set of next-level candidate entity groups and the next-level candidate entity group to obtain the at least one set of next-level candidate entity groups At least one group similarity corresponding to the entity group
  • the acquiring at least one group similarity corresponding to the at least one group of next-level entities to be verified includes: acquiring the at least one group of next-level entities to be verified The maximum similarity of each of the multiple entities included in each of the entity groups to obtain the multiple maximum similarities of the multiple entities included in each of the at least one next-level entity group to be verified, where: The maximum similarity is a value of the similarity between each of the multiple entities included in each group of the at least one next-level entity group to be verified and all the entities included in the next-level candidate entity group A maximum similarity; and obtaining at least one group similarity corresponding to the at least one group of next-level entity groups to be verified based on the multiple maximum similarities.
  • acquiring at least one group similarity corresponding to the at least one group of next-level entity groups to be verified based on the multiple maximum similarities includes: The weighted sum of the similarities serves as the at least one group similarity corresponding to the at least one group of next-level entity groups to be verified.
  • the obtaining the next level similarity of each of the at least two candidate entities based on the at least one group similarity includes: determining the next level to be calibrated The number of entity groups to be verified at the next level included in the verification entity, and when the number of entity groups at the next level to be verified is equal to 1, the next level of entities to be verified includes The group similarity of the entity group to be verified at the next level is used as the next level similarity of each of the at least two candidate entities.
  • the calculation of the next level of similarity and entity similarity of each of the at least two candidate entities includes: calculating the candidate probability of each of the at least two candidate entities: The product of the similarity of the next level and the similarity of the entities is used as the candidate probability of each of the at least two candidate entities.
  • the calculating the candidate probability of each of the at least two candidate entities by using the next-level entity to be verified and the related entity to be verified includes: obtaining The relevant entity to be verified corresponding to the entity to be verified; for each of the entities included in the relevant entity to be verified, the following method is executed: obtaining a relevant candidate entity, wherein the relevant candidate entity and the at least Each of the two candidate entities is at the same level in the hierarchical structure of the knowledge graph, is associated with each of the at least two candidate entities, and is associated with the previous one to which each of the at least two candidate entities belongs.
  • the relationship between level entities is equal to the relationship between each of the entities included in the related entity to be verified and the entity to be verified, and the relationship between each of the entities included in the related entity to be verified and all the entities included in the related entity to be verified is calculated.
  • the similarity of the related candidate entities obtaining the related similarity of each of the at least two candidate entities based on the similarity between each of the entities included in the related entity to be verified and the related candidate entity;
  • the entity to be verified includes a next-level entity to be verified; based on the relationship between the entity to be verified and each entity in the entity to be verified, the next-level entity to be verified is divided into At least one set of next-level entity groups to be verified; for each of the at least one set of next-level entity groups to be verified, the following method is executed: obtaining a next-level candidate entity group, wherein the next-level candidate The entity group is subordinate to each of the at least two candidate entities and has a relationship with each of the at least two candidate entities equal to the sum of each of the
  • At least one embodiment of the present disclosure also provides a device for acquiring character information in a picture, which includes: a character information extraction device and a verification and correction device.
  • the character information extraction device is configured to obtain a picture and extract at least one item of character information included in the picture; and the verification and correction device is configured to verify the at least one item of character information based on the knowledge graph.
  • At least one embodiment of the present disclosure also provides an apparatus for acquiring character information in a picture, which includes a processor and a memory.
  • the memory stores computer program instructions suitable for execution by the processor, and when the computer program instructions are executed by the processor, the processor executes any acquisition method provided in at least one embodiment of the present disclosure.
  • At least one embodiment of the present disclosure also provides a non-transitory storage medium, which includes computer program instructions stored thereon.
  • the computer program instructions When the computer program instructions are executed by the processor, the computer executes any acquisition method provided in at least one embodiment of the present disclosure.
  • At least one embodiment of the present disclosure further provides a page processing method, which includes: obtaining a picture of a page; and obtaining a post-verification of the picture of the page by executing any of the obtaining methods provided in at least one embodiment of the present disclosure
  • the character information; and the checked character information of the picture of the page is structured to obtain a structured data combination.
  • the page processing method further includes: confirming that the picture of the page includes a medical image based on the knowledge graph; and adding the medical image to the structured data combination image.
  • the confirming that the picture of the page includes a medical image based on the knowledge graph includes: checking based on the knowledge graph extracted from the picture of the page The subsequent character information includes the text corresponding to the medical image, and it is determined that the picture of the page includes the medical image.
  • the page processing method further includes: based on the knowledge graph and multiple entities corresponding to the verified character information, in the structured data combination Add descriptions corresponding to the multiple entities.
  • At least one embodiment of the present disclosure also provides a method for constructing a knowledge graph, which includes: acquiring a data structure of a medical knowledge graph; at least deriving corresponding entities from a medical image archive and communication system and a laboratory information management system; The entity derived from the medical image archiving and communication system is aligned with the entity derived from the laboratory information management system to obtain a plurality of aligned entities; and the aligned multiple entities are based on the medical The data structure of the knowledge graph is filled into the medical knowledge graph.
  • the knowledge graph construction method further includes: obtaining a sub-item description of a clinical examination sub-item involved in the laboratory information management system; and processing the sub-item description Then fill it into the medical knowledge graph.
  • the knowledge graph construction method further includes: deriving a corresponding entity from a hospital information system; and aligning the entity derived from the hospital information system with the entity. Entity alignment is performed on the entities; multiple entities derived from the hospital information system and aligned with the entities are aligned with the entities and filled into the medical knowledge map based on the data structure of the medical knowledge map.
  • Fig. 1 is an exemplary flowchart of a method for acquiring character information in a picture provided by at least one embodiment of the present disclosure
  • 2A shows a schematic diagram of a process of extracting at least one item of character information included in a picture using optical character recognition technology
  • 2B shows another schematic diagram of a process of extracting at least one item of character information included in a picture using optical character recognition technology
  • Fig. 3 is a data structure of a medical knowledge graph provided by at least one embodiment of the present disclosure
  • Figure 4 is an example of a blood routine examination report
  • FIG. 5A is a schematic flowchart of verifying at least one item of character information based on a knowledge graph provided by at least one embodiment of the present disclosure
  • FIG. 5B is a schematic flowchart of parsing based on a knowledge graph provided by at least one embodiment of the present disclosure
  • 6A is a schematic flow chart of verifying character information in an image and adding explanations based on a knowledge graph provided by at least one embodiment of the present disclosure
  • FIG. 6B is a schematic flowchart of medical record analysis based on a knowledge graph provided by at least one embodiment of the present disclosure
  • Figure 7A is a picture corresponding to a partial blood routine checklist
  • Fig. 7B is a picture of part of the routine blood checklist shown in Fig. 7A after being contaminated;
  • FIG. 7C is a result obtained by using a software program based on the method for acquiring character information in a picture provided by at least one embodiment of the present disclosure to obtain character information in the picture in FIG. 7B;
  • FIG. 8 is a device for acquiring character information in a picture provided by at least one embodiment of the present disclosure.
  • Fig. 9 is another apparatus for acquiring character information in a picture provided by at least one embodiment of the present disclosure.
  • FIG. 10 is a non-transitory storage medium provided by at least one embodiment of the present disclosure.
  • FIG. 11 is an exemplary flowchart of a page processing method provided by at least one embodiment of the present disclosure.
  • FIG. 12 is a method for constructing a medical knowledge graph provided by at least one embodiment of the present disclosure.
  • FIG. 13 shows an exemplary scene diagram of a system for acquiring character information in a picture provided by at least one embodiment of the present disclosure.
  • Fig. 14 shows a computing device provided by at least one embodiment.
  • OCR optical character recognition
  • page parsing technology for example, page parsing technology
  • OCR technology uses parsing technology to integrate the extracted characters into a format that is consistent with the content of the picture and is convenient for users to view according to rules, so as to obtain information with specific meaning.
  • OCR technology and analysis technology only text content can be extracted from the picture, and the text content in the picture can be converted into a text format, so that the text content in the picture can be stored in the memory in the text format (also That is, ordinary OCR technology and analysis technology only solve the electronic problem and storage problem of medical examination reports).
  • the computer's ability to understand the text content in the above text format is poor, and cannot effectively retrieve and display (for example, using a graphical user interface to display) the above content, so the patient's medical examination report still cannot be effectively used.
  • the inventor of the present disclosure has also noticed in research that paper documents in the medical field (test slips, prescriptions, medical records, physical examination reports, etc.) involve rare characters, rare words, etc., and the characters of these rare words and rare words are complicated. Therefore, it may be difficult to recognize these rare words and words using an OCR module (that is, an ordinary OCR module) trained by a common method.
  • OCR module that is, an ordinary OCR module
  • the relevant medical OCR module has poor recognition performance for non-medical characters. Therefore, on the whole, the common OCR module and the medical OCR module have low accuracy in extracting the character information in the pictures corresponding to the paper documents in the medical field. Since the accuracy of the extracted characters is closely related to the health of the patient, there is an urgent need to improve the accuracy of the extracted characters.
  • Knowledge graph is a frontier research problem of intelligent big data. It conforms to the development of the information age with its unique technical advantages.
  • the knowledge graph has the following advantages: incremental data model design; good data integration capabilities; supported by existing Resource Description Framework (RDF) and Web Ontology Language (OWL, Web Ontology Language) standards ; Have the ability of semantic search and knowledge reasoning.
  • the knowledge graph has better new capabilities in querying complex related information, can understand user intentions from a semantic level, and can have a strong ability to describe data.
  • the knowledge graph can be used to describe various entities and concepts that exist in the real world. For example, a knowledge graph can be used to describe the association between two entities. For example, "Yao Ming was born in Shanghai” involves two entities “Yao Ming" and "Shanghai” and the relationship between the two entities "place of birth”.
  • Medical knowledge graph is a method to realize smart medical treatment, which is conducive to the development of more efficient and accurate medical services. Based on the reasoning ability of knowledge graph, medical knowledge graph has the advantages of interpretability and strong stability.
  • the application range of medical graphs is relatively small (for example, only for knowledge questioning and retrieval): There are few Chinese medical knowledge graphs, and medical terminology in different regions is not uniform. The construction of medical knowledge graphs requires medical experts’ expertise.
  • the medical knowledge map caused by deep participation has problems such as low efficiency, many restrictions, and poor scalability.
  • Edit Distance refers to the minimum number of editing operations required to transform one character string into another character string.
  • the permitted editing operations include replacing one character with another, inserting a character, and deleting a character.
  • the minimum edit distance algorithm may use edit[i][j] to represent the edit distance of the string consisting of i characters of the A string (string) and the string consisting of j characters of the B string.
  • edit[i][j] represents the edit distance from the string consisting of the 0th character to the i-th character of the A string and the string consisting of the 0th character to the jth character of the B string.
  • edit[5[5] represents the edit distance between the string "graph” and the string "grape”
  • edit[2[3] represents the character The edit distance between the string "gr" and the string "gra”.
  • i and j can be greater than or equal to 1.
  • the edit distance between A string and B string can also be expressed by dis[i][j].
  • dis[0][0] means that when both A string and B string are empty, their edit distance is 0 at this time.
  • dis[0][j] is the case where the A string is empty and the B string length is j.
  • their edit distance is j, that is, adding j characters from an empty string is converted to The minimum edit distance of string B is j;
  • dis[i][0] means that the length of string A is i and when string B is empty, string A needs to delete i characters before it can be converted to empty, so it is converted from string A to The minimum edit distance of string B is i.
  • the effective number of replacement flags can be expressed by the following equation.
  • the min() function (that is, the function used to calculate the minimum value) in the above dynamic programming equation contains three expressions, namely, edit[i-1][j]+1, edit[i][j -1]+1 and edit[i-1][j-1]+flag, the above three expressions respectively represent the operation of inserting characters, deleting characters, and replacing characters.
  • the specific instructions are as follows.
  • edit[i-1][j]+1 is equivalent to inserting the last character in the A string after the last character of the B string.
  • the above inserting character operation increases the compile distance by 1, and then calculates edit[i-1] [j] to calculate edit[i-1][j]+1 as a part of the compilation distance edit[i][j];
  • edit[i][j-1]+1 is equivalent to deleting the last character of the B string.
  • the above deleting character operation increases the compile distance by 1, and then calculates edit[i] by calculating edit[i][j-1] [j-1]+1, as part of calculating the coding distance edit[i][j];
  • edit[i-1][j-1]+flag is equivalent to replacing the last character of the B string with the last character of the A string.
  • flag is used to mark the effective number of character replacements.
  • the edit distance of two character strings calculated by the minimum edit distance algorithm can be used to obtain the similarity of the two character strings.
  • you can use the following formula and use the edit distance of two strings to calculate the similarity of two strings: S(A, B) 1-(DIS(A, B)/max(Len(A), Len (B)).
  • DIS(A,B) represents the edit distance between A string and B string
  • Len(A) and Len(B) represent the string length of A and string B respectively
  • max( Len(A), Len(B)) is equal to the maximum of the string length of A and the string length of B.
  • At least one embodiment of the present disclosure provides a method and device for acquiring character information in a picture, a non-transitory storage medium, a page processing method, and a method for constructing a medical knowledge graph.
  • the method for obtaining character information in the picture includes: obtaining the picture and extracting at least one item of character information included in the picture; and verifying the at least one item of character information based on the knowledge graph.
  • the method for obtaining character information in the picture checks at least one piece of character information extracted based on the knowledge graph, thereby improving the accuracy of the obtained character information.
  • Fig. 1 is an exemplary flowchart of a method for acquiring character information in a picture provided by at least one embodiment of the present disclosure. As shown in FIG. 1, the method for acquiring character information in the picture includes the following steps S10 and S20.
  • Step S10 Obtain a picture and extract at least one item of character information included in the picture.
  • Step S20 Verify at least one item of character information based on the knowledge graph.
  • step S10 the specific method for acquiring the picture can be set according to actual application requirements, which is not specifically limited in at least one embodiment of the present disclosure.
  • acquiring a picture may include acquiring (reading) a picture stored in the memory from the memory.
  • acquiring the picture may also include acquiring the picture remotely, for example, using an information sending and receiving device to receive the picture.
  • the information sending and receiving device can receive pictures from a client terminal (for example, a mobile phone terminal or a PC terminal).
  • the acquired picture may be a picture after picture preprocessing, or a picture before picture preprocessing.
  • picture preprocessing may also be performed on the picture.
  • the picture preprocessing includes, but is not limited to, at least one of the detection and extraction of the main region of the picture, the picture correction, the picture segmentation, the text direction correction of the picture, the picture binarization, and the picture denoising.
  • the detection and extraction of the main area of a picture includes detecting the main area of the picture (effective area, area containing useful information), and extracting the detected main area as a picture for subsequent processing.
  • the main area of the picture can be detected by locating (Text Detection) in the area where the text exists in the picture, that is, finding the bounding box (Word/Line-level) of the word or text line (Word/Line-level).
  • the user can confirm the detected main area before the extraction (that is, the user manually selects the effective area) to improve the accuracy of the detection and extraction of the main area of the picture .
  • picture correction e.g., keystone correction
  • gamma correction can be used to achieve picture correction (for example, keystone correction).
  • image segmentation is used to segment text (e.g., the text of a medical examination report) from a picture containing a background (e.g., a photo taken by a normal mobile phone) to help improve the accuracy of the text (e.g., character information) extracted from the picture .
  • a background e.g., a photo taken by a normal mobile phone
  • horizontal projection is used to count the elements of each row of a picture (that is, statistics along the horizontal direction), and then the statistical result graph can be drawn based on the results of the above statistics, so that the column direction of each row can be determined
  • the starting point and ending point of each line that is, the upper and lower limits of each line.
  • the projection direction of the vertical projection is downward, which is used to count the number of elements in each column and the projection position of the elements in each column, thereby determining the left and right limits of each column.
  • the horizontal projection of the picture can be achieved by the following methods: first, set a straight line (axis) extending in the vertical direction; secondly, project the pixels on the picture in the direction perpendicular to the straight line (axis) to the projection. On a straight line (axis), after projection, multiple pixels on the picture correspond to multiple black dots on the straight line (axis); third, count the number of black dots of pixels on the picture perpendicular to the vertical straight line, and add up And as the value of the position of the axis. After the above-mentioned horizontal projection is completed, cutting based on picture projection can be performed.
  • the cutting position (coordinates) of the picture is determined based on the above-mentioned characteristics, and the above-mentioned picture is cut with the above-mentioned coordinates to obtain the target picture (for example, a picture of a single text obtained by dividing the picture) ).
  • the target picture for example, a picture of a single text obtained by dividing the picture
  • vertical projection of a picture is similar to horizontal projection of the picture. The difference is that a straight line extending in the vertical direction is set, and the pixels on the picture are projected onto the above-mentioned straight line extending in the vertical direction.
  • the text orientation correction of a picture is used to correct the tilt of the text in the image caused by the process of taking the picture.
  • Fast Fourier Transformation Fast Fourier Transformation
  • technology can be used to correct the image, and then the corrected image can be inversely transformed to obtain the text with the orientation correction.
  • image binarization can be used to increase the contrast of text.
  • image binarization can be achieved by the following method. First, set a threshold; then, for multiple pixels in the picture, set the value of the pixel whose pixel value (for example, the gray value of the pixel) is greater than or equal to the threshold value to 1 (that is, make the pixel appear white) , Set the value of the pixel whose pixel value is less than the threshold value to 0 (that is, make the pixel appear black).
  • adaptive binarization can be used (each pixel uses a different threshold in the comparison process).
  • the two-dimensional matrix corresponding to the pixel values of the multiple pixels of the picture (that is, the matrix used to store the pixel values of the multiple pixels of the picture) can be converted into a vector, and then each pixel can be compared with a certain range based on The threshold calculated by the inner pixel (that is, the adaptive threshold) is compared.
  • picture denoising is used to reduce the noise of the picture to improve the accuracy of the text extracted from the picture.
  • picture denoising is used to reduce the noise of the picture to improve the accuracy of the text extracted from the picture.
  • the received picture is converted into a binary picture
  • the character information obtained from the picture may include a single item of character information or multiple items of character information.
  • each item of character information includes a single character or character string.
  • the character information may be text information or symbol information.
  • the symbol information may be a rising arrow symbol, a falling arrow symbol, and so on.
  • optical character recognition technology can be used to extract at least one item of character information included in the picture.
  • Fig. 2A shows a schematic diagram of a process of extracting at least one item of character information included in a picture using optical character recognition technology.
  • the process of optical character recognition technology to extract at least one item of character information included in a picture includes: character detection, character recognition, and result output.
  • text detection is used to detect individual characters in a picture.
  • text recognition is used to convert the detected text into a text format.
  • text recognition includes text feature extraction and comparison recognition, or it can also use, for example, a trained neural network (for example, a connected text proposal network or a convolutional recurrent neural network for detecting text in natural images) for text recognition.
  • a trained neural network for example, a connected text proposal network or a convolutional recurrent neural network for detecting text in natural images
  • contrast recognition is used to compare the features obtained through text feature extraction with a comparison database or a feature database (character set containing all the text to be recognized), so that the Chinese character of the picture can be found based on the characteristics of the text in the picture.
  • the word corresponding to the text in the text format For example, the specific method of using optical character recognition technology for text detection and text recognition can be referred to related technologies, which will not be repeated here.
  • text detection can be realized by detecting the coordinates of the upper and lower boundaries of the text line in the picture and the coordinates of the left and right boundaries of the text string.
  • CTPN Detecting Text in Natural Image with Connectionist Text Proposal Network
  • CRNN Convolutional Recurrent Neural Network
  • Open source libraries for example, Tesseract digital recognition library
  • FIG. 2B shows another schematic diagram of a process of extracting at least one item of character information included in a picture using optical character recognition technology.
  • the process of extracting at least one item of character information included in the picture by the optical character recognition technology may also include: picture input and picture processing.
  • picture processing refers to picture preprocessing
  • the specific method of picture preprocessing can refer to the related description of step S10, which will not be repeated here.
  • the picture obtained in step S10 may be the picture before the picture preprocessing, and there is no need to extract at least one item of character information included in the picture. Carry out additional image preprocessing.
  • step S20 by checking at least one item of character information based on the knowledge graph (for example, checking and correcting), it can be confirmed whether the character information extracted from the picture has errors and the character information with errors can be corrected.
  • This can improve the accuracy of character information obtained by using the method for obtaining character information in a picture provided by at least one embodiment of the present disclosure.
  • the knowledge graph is a medical knowledge graph.
  • the medical knowledge graph may be a medical examination knowledge graph.
  • the medical examination knowledge map can be used to provide relevant knowledge of the test indicators, test sheets, and so on.
  • the knowledge map of medical examination and inspection provides the possibility for accurate identification and analysis.
  • the architecture (framework or data structure) of the medical knowledge graph may use the data structure of the medical knowledge graph shown in FIG. 3.
  • the method for constructing a medical knowledge graph provided by at least one embodiment of the present disclosure can be used to construct a medical knowledge graph based on the data structure of the medical knowledge graph shown in FIG. 3, and the medical knowledge graph obtained by construction can be applied Go to step S20.
  • the method for constructing a medical knowledge graph provided by at least one embodiment of the present disclosure will be described in detail in the embodiment shown in FIG. 12, and will not be repeated here.
  • the medical knowledge map acquired based on the data structure of the medical knowledge map shown in FIG. 3 may include multiple entities and relationships derived from a medical image archiving and communication (PACS) system and a laboratory information management (LIS) system.
  • PACS medical image archiving and communication
  • LIS laboratory information management
  • the medical knowledge graph include entities derived from the PACS system, it is possible to confirm whether the picture contains medical images based on the medical knowledge graph, and save the medical images if the pictures contain medical images.
  • entities derived from the PACS system include clinical examination items, clinical examination sub-items, and entities related to medical imaging examinations in the clinical examination list.
  • entities derived from the PACS system include: computerized tomography images, nuclear magnetic resonance images (NMRI), ultrasound images, electrocardiograms, electroencephalograms, etc.
  • the entities derived from the LIS system include clinical examination items, clinical examination sub-items, and entities in the clinical examination list that are not related to medical imaging examinations.
  • the entities derived from the LIS system include various laboratory inspection items.
  • laboratory test items include blood test items, urine test items, stool test items, and so on.
  • blood test items include routine blood test items, blood lipid test items, blood sugar test items, and the like.
  • routine blood test items include a platelet count (PLT) sub-item, a platelet distribution width (PDW) sub-item, and a platelet packed volume (PCT) sub-item.
  • the medical knowledge graph obtained based on the data structure of the medical knowledge graph shown in FIG. 3 may also include entities derived from the hospital information system and/or entities related to the sub-item description of the clinical examination sub-item.
  • the entities derived from the hospital information system include entities related to patient information and entities related to medical orders.
  • the entity related to the description of the sub-item of the clinical examination sub-item provides the description of the sub-item of the clinical examination and the meaning of abnormality when the result of the clinical examination sub-item is abnormal, so that the patient can understand the examination result more clearly.
  • the data structure of the knowledge graph adopted in step S20 is not limited to the data structure of the knowledge graph shown in FIG. 3.
  • the data structure of the knowledge graph adopted in step S20 may not include the part corresponding to the hospital information system and/or the part corresponding to the medical significance of the clinical examination sub-item in the data structure of the knowledge graph shown in FIG. 3.
  • step S20 verifying at least one item of character information based on the knowledge graph includes the following step S210.
  • Step S210 Confirm that there is wrong character information in at least one item of character information based on the knowledge graph.
  • the aforementioned at least one item of character information includes multiple items of character information
  • the character information with errors includes character information that is incorrectly recognized during the recognition process (for example, "water” is incorrectly recognized as “small”) and/or during the recognition process. Unrecognized character information in.
  • step S210 confirming that there is an error in at least one item of character information based on the knowledge graph includes the following steps S211 and 212.
  • Step S211 Obtain multiple entities respectively according to multiple pieces of character information of the picture, and select the entity to be verified from the multiple entities to determine whether there is an error in the character information corresponding to the entity to be verified.
  • Figure 4 is an example of a blood routine examination report.
  • multiple entities obtained according to multiple character information of the picture may include: report form name, blood routine examination sub-item name (item name), sub-item abbreviation ( Code), sub-item inspection result (result), unit (unit) of the sub-item inspection result, whether there is an abnormality in the inspection result of the sub-item (indicated by the up or down arrow), and the reference value of the sub-item inspection result (item reference value) ).
  • the names of some sub-items of routine blood tests and the corresponding abbreviations include: platelet count (PLT), platelet distribution width (PDW), and platelet packed volume (PCT).
  • the multiple entities can be checked one by one until all entities contained in the multiple pieces of character information of the picture are checked.
  • the platelet count can be selected as the entity to be verified.
  • Step 212 According to the hierarchical structure of the knowledge graph, confirm whether the character information corresponding to the entity to be verified has errors, and, in the case of errors in the character information corresponding to the entity to be verified, confirm that the character information corresponding to the entity to be verified is There is wrong character information.
  • step 212 confirming whether there is an error in the character information corresponding to the entity to be verified includes the following steps 2121 to 2124.
  • Step 2121 Classify multiple entities according to the hierarchical structure of the knowledge graph.
  • multiple entities can be classified by means of analysis of the table structure in the picture and the typesetting, size, and boldness of the text in the picture.
  • optical recognition technology can be used to determine whether there is a table in the picture, and if there is a table, the structure of the table in the picture is further analyzed (or analyzed).
  • the specific method of analyzing the table structure can refer to the examples shown in FIG. 5A and FIG. 5B, which will not be repeated here.
  • the (relative) size of the text in the picture, whether it is bold or not, etc. can also be obtained through picture analysis, for example, part of it is realized by a classifier or neural network.
  • the following method can be used to rank.
  • the level in the report sheet can be associated with the level of the entity in the hierarchical structure of the knowledge graph.
  • "blood routine”, "item name (namely, sub-item name of blood routine examination)” and “platelet count” are located at the fifth, sixth and seventh levels respectively .
  • the "blood routine” in the picture of the blood routine examination report form shown in Figure 4 has the fifth level in the hierarchical structure of the knowledge graph; the "code” in the picture of the blood routine examination report form shown in Figure 4,
  • the level of the hierarchical structure of the knowledge graph of "Project name (that is, the name of the sub-item of routine blood test)", "Result", and "Item reference value” is the sixth level;
  • Figure 4 shows the picture of the routine blood test report
  • the "platelet count" in the hierarchical structure of the knowledge map is the seventh level.
  • the entity corresponding to the "bed number" in the blood routine examination report does not belong to the entity related to the blood routine examination result. Therefore, the corresponding entity such as the "bed number” will be performed on multiple entities. Ignored during the grading process.
  • Step 2122 Determine the level of the entity to be verified in the hierarchical structure of the knowledge graph.
  • the level of the entity to be verified in the hierarchical structure of the knowledge graph may be determined based on the classification result in step 2121. For example, in the case where "platelet count" is selected as the entity to be verified, based on the classification result in step 2121, it can be determined that the level of the hierarchical structure of the knowledge graph of the entity to be verified is the seventh level.
  • Step 2123 Calculate the similarity between the entity to be verified and all entities in the knowledge graph that have the same level and the same relationship with the entity to be verified, so as to obtain the similarity of multiple entities related to the entity to be verified.
  • the similarity between the entity to be verified and all entities that have the same level and the same relationship with the entity to be verified in the knowledge graph is between 0 and 1.
  • step 2123 "has the same relationship” means that the relationship between all entities that have the same level and the same relationship as the entity to be verified in the knowledge graph and the upper level entities of these entities is equal to the relationship between the entity to be verified and the entity to be verified. The relationship between the upper-level entities of the entity to be verified.
  • platelet count when “platelet count” is selected as the entity to be verified, all entities in the knowledge map that have the same level and the same relationship as the entity to be verified include “platelet count”, “platelet distribution width”, and “platelet count”. Packed volume” and “percentage of neutrophils” and so on.
  • the similarity between the entity to be verified ie, the “platelet count” and the “platelet count” in the knowledge map is 1; the entity to be verified (ie, the "platelet count”
  • the similarities between "count") and “platelet distribution width”, “platelet packing” and “neutrophil percentage” in the knowledge map are, for example, 0.5, 0.6, and 0, respectively.
  • platelet count is incorrectly identified as “platelet meter X”
  • the entity to be verified ie, "platelet meter X”
  • platelet count is related to the "platelet count", “platelet distribution width”, and “platelet count” in the knowledge map.
  • the degree of similarity between “platelet pack” and “percent neutrophils” that is, the entity similarity of "platelet count”, “platelet distribution width”, “platelet pack” and “percent neutrophils”, for example, respectively It is 0.8, 0.5, 0.6 and 0.
  • the "X” in the “platelet meter X” indicates that the character was recognized incorrectly, and the "X” in other characters or entities also have similar meanings, which will not be repeated here.
  • the minimum coding distance algorithm may be used to obtain the similarity between the entity to be verified and all entities that have the same level and the same relationship with the entity to be verified in the knowledge graph.
  • Step 2124 When the maximum value of the multiple entity similarities is less than the predetermined entity similarity threshold, it is determined that the entity to be verified is the entity to be verified with errors and the character information corresponding to the entity to be verified has errors.
  • the predetermined entity similarity threshold can be set according to actual application requirements.
  • the predetermined entity similarity threshold may be set to 1.
  • the predetermined entity similarity threshold may also be set to 0.95.
  • the entity is an entity to be verified with an error, and the character information corresponding to the entity to be verified has an error.
  • the similarity between the entity to be verified ie, "platelet meter X" and the "platelet count”, “platelet distribution width”, “platelet packed volume” and “neutrophil percentage” in the knowledge map, such as In the case of 0.8, 0.5, 0.6, and 0, since the maximum value of multiple entity similarities (that is, 0.8) is less than the predetermined entity similarity threshold (for example, 1), it can be determined that the entity to be verified corresponds to There is an error in the character information and the entity to be verified is an entity to be verified with an error.
  • the predetermined entity similarity threshold for example, 1
  • the similarity between the entity to be verified ie, "platelet count” and the “platelet count”, “platelet distribution width”, “platelet packed volume” and “neutrophil percentage” in the knowledge map are, for example, respectively
  • a predetermined entity similarity threshold for example, 1
  • step S20 verifying at least one item of character information based on the knowledge graph further includes the following step 220.
  • Step S220 Correct the character information with errors based on the knowledge graph.
  • correcting erroneous character information based on the knowledge graph includes: determining the number of all entities in the knowledge graph that have the same level and the same relationship with the entity to be verified to obtain the number of entities.
  • correcting the character information with errors based on the knowledge graph also includes: when the number of entities is equal to 1, directly replacing the character information corresponding to the entity to be verified with errors in the knowledge graph with the entity to be verified.
  • Character information corresponding to entities of the same level and same relationship or calculate the probability that an entity with the same level and the same relationship as the entity to be verified is the entity to be verified with errors to obtain the entity probability, and if the entity probability is greater than the predetermined In the case of the entity probability threshold of, replace the character information corresponding to the entity to be verified with an error with the character information corresponding to the entity with the same level and the same relationship as the entity to be verified.
  • the predetermined entity probability threshold can be set according to actual application requirements.
  • the predetermined entity probability threshold can be set to 0.5, 0.8, 0.9 or other applicable numerical values.
  • the character information with errors can be manually corrected.
  • correcting the character information with errors based on the knowledge graph also includes: when the number of entities is greater than one, performing the following steps S221 to S222.
  • Step S221 Determine at least two candidate entities based on the similarity of multiple entities.
  • determining at least two candidate entities based on multiple entity similarities includes: sorting all entities that have the same level and the same relationship with the entity to be verified in the knowledge graph in descending order based on multiple entity similarities; Select the predetermined number of entities at the top of the sequence as at least two candidate entities; or sort all the entities in the knowledge graph that have the same level and the same relationship with the entity to be verified in ascending order based on the similarity of multiple entities.
  • the predetermined number of entities at the end of the sequence serves as at least two candidate entities.
  • the predetermined number can be set according to actual application requirements.
  • the predetermined number can be set to 2, 3, 10, or other suitable numbers.
  • the "platelet count”, “platelet distribution width”, “platelet packing” and “medium platelet count” in the knowledge map can be classified based on the similarity of multiple entities (that is, 0.8, 0.5, 0.6, and 0).
  • the percentage of neutrophils is sorted in descending order into the following sequence, that is, “platelet count”, “platelet packing”, “platelet distribution width”, and "neutrophil percentage”.
  • the predetermined number is set to 2
  • the two entities arranged at the top of the sequence that is, “platelet count” and “platelet packed volume” may be used as at least two candidate entities.
  • the "platelet count”, “platelet distribution width”, “platelet packing” and “platelet packing” in the knowledge map may be calculated based on multiple entity similarities (that is, 0.8, 0.5, 0.6, and 0).
  • the percentage of neutrophils is sorted in ascending order into the following sequence, that is, “percent of neutrophils", “platelet distribution width”, “platelet packing”, and “platelet count”.
  • the predetermined number is set to 2
  • the two entities arranged at the end of the sequence that is, “platelet volume” and “platelet count” can be used as at least two candidate entities.
  • Step S222 Calculate the probability that each of the at least two candidate entities is the entity to be verified with errors, so as to obtain the candidate probability of each of the at least two candidate entities.
  • the probability that each of the at least two candidate entities is the entity to be verified with errors means that the character information corresponding to each of the at least two candidate entities is equivalent to the character information corresponding to the entity to be verified with errors.
  • the probability of character information being misrecognized means that the probability that "platelet compaction" is the entity to be verified with error "platelet meter X” means that the character information corresponding to "platelet compaction” is equivalent to the character information corresponding to "platelet meter X”. Probability of the character under.
  • step S222 calculating the probability that each of the at least two candidate entities is an entity to be verified with errors includes the following steps S2221-step S2223.
  • Step S2221 Determine whether the entity to be verified includes the next-level entity to be verified, so as to obtain the first determination result.
  • the next-level entities to be verified are all the next-level entities belonging to the entities to be verified.
  • the first determination result is the first determination sub-result in the case where it is determined that the entity to be verified includes the entity to be verified at the next level or when it is determined that the entity to be verified does not include the entity to be verified at the next level.
  • the second decision sub-result For example, in the case of selecting "item name X" as the entity to be verified, the first judgment result is the first judgment sub-result; in the case of selecting "platelet meter X" as the entity to be verified, the first judgment result is The second decision sub-result.
  • Step S2222 Determine whether the entity to be verified corresponds to the entity to be verified, so as to obtain the second determination result.
  • Related entities to be verified are all entities that are at the same level as the entity to be verified in the hierarchical structure of the knowledge graph, are associated with the entity to be verified, and have a different relationship with the upper level entity to which the entity to be verified belongs.
  • that the related entity to be verified is associated with the entity to be verified means that the related entity to be verified and the entity to be verified are used to describe different aspects of the same subject.
  • platelet count the full name of the platelet count check sub-item
  • PHT abbreviation of the platelet count check sub-item
  • 100-300 the reference value of the platelet count check sub-item
  • the entity to be verified corresponds to the entity to be verified.
  • the relevant entities to be verified corresponding to "platelet meter X” include "PLT (abbreviation of platelet count)", "100-300 (reference value of platelet count)” and so on.
  • PKT abbreviation of platelet count
  • 100-300 reference value of platelet count
  • the second determination result is the third determination sub-result in the case where it is determined that the entity to be verified corresponds to the relevant entity to be verified or the fourth determination sub-result in the case where it is determined that the entity to be verified does not correspond to the relevant entity to be verified. result.
  • the second judgment result is the fourth judgment sub-result; in the case of selecting "platelet meter X" as the entity to be verified, the second judgment result is The third decision sub-result.
  • Step S2223 Based on the first determination result and the second determination result, a method for calculating the probability that each of the at least two candidate entities is an entity to be verified with an error is selected.
  • step S2223 the method for calculating the probability that each of the at least two candidate entities is an entity to be verified with errors based on the first determination result and the second determination result includes the following steps S231 to S234 one of.
  • Step S232 In the case where the first determination result is the second determination sub-result and the second determination result is the third determination sub-result, use the relevant entities to be verified to calculate the candidate probability of each of the at least two candidate entities.
  • the candidate probability of each of the at least two candidate entities can be calculated by using the relevant entity to be verified.
  • the candidate entities of the entity to be verified include “platelet count”, “platelet packed volume”, and the like. It should be noted that, for the sake of clarity, it is assumed here that the candidate entities of the entity to be verified (that is, “platelet meter X”) only include “platelet count” and "platelet packed volume”.
  • calculating the candidate probability of each of the at least two candidate entities by using related entities to be verified includes the following steps S241 to S244.
  • Step S241 Obtain the relevant entity to be verified corresponding to the entity to be verified.
  • the relevant entity to be verified corresponding to the entity to be verified includes “PLT”, "100-300”, and so on.
  • Step S242 Perform the following steps S2421-step S2422 for each of the entities included in the relevant entity to be verified.
  • Step S2421 Obtain related candidate entities.
  • the related candidate entity and each of the at least two candidate entities are located at the same level in the hierarchical structure of the knowledge graph, are associated with each of the at least two candidate entities, and are related to the upper level to which each of the at least two candidate entities belongs
  • the relationship between the entities is equal to the relationship between each of the entities included in the related entity to be verified and the entity to be verified.
  • the related candidate entity corresponding to the candidate entity “platelet count” is “PLT”
  • the related candidate entity corresponding to the candidate entity “platelet count” is "PCT”.
  • the relevant candidate entity corresponding to the candidate entity “platelet count” is "100-300”
  • the relevant candidate entity corresponding to the candidate entity “platelet count” is "0.17-0.35" .
  • Step S2422 Calculate the similarity between each of the entities included in the relevant entity to be verified and the relevant candidate entity.
  • the similarity of the related candidate entity “PLT” corresponding to the related entity to be verified “PLT” and the candidate entity “Platelet Count” is 1, and the related entity “PLT” to be verified is related to the candidate entity “Platelet count”.
  • the similarity of the candidate entity "PCT” is 0.65.
  • the similarity of the related candidate entity "100-300" corresponding to the related entity to be verified “100-300” and the candidate entity “platelet count” is 1; the related entity to be verified “100-300” and the candidate entity “platelet count”
  • the similarity of related candidate entities "0.17-0.35" corresponding to "Packed product” is 0.
  • Step S243 Obtain the relative similarity of each of the at least two candidate entities based on the similarity between each of the entities included in the relevant entity to be verified and the relevant candidate entity.
  • step S243 obtaining the correlation similarity of each of the at least two candidate entities based on the similarity between each of the entities included in the relevant entity to be verified and the related candidate entity includes the following steps S2431-step S2433.
  • Step S2431 Determine the number of entities included in the relevant entity to be verified.
  • the relevant entity to be verified corresponding to the entity to be verified ie, "platelet meter X” only includes "PLT”
  • the number of entities included in the relevant entity to be verified is one.
  • the relevant entity to be verified corresponding to the entity to be verified ie, "platelet meter X”
  • the relevant entity to be verified can be determined The number of is 2.
  • Step S2432 When the number of entities included in the related entity to be verified is equal to 1, the similarity between the entity included in the related entity to be verified and the related candidate entity is taken as the correlation of each of the at least two candidate entities Similarity.
  • the relevant entity to be verified corresponding to the entity to be verified ie, "platelet count X" only includes "PLT”
  • the relevant entity to be verified "PLT” can be mapped to the candidate entity "platelet count”
  • the related candidate entity "PLT similarity that is, 1) is used as the related similarity of the candidate entity “platelet count”; and the related entity to be verified “PLT” can be correlated with the candidate entity “platelet count”.
  • the similarity (0.65) of the candidate entity "PCT” is used as the relative similarity of the candidate entity "Platelet Pack".
  • Step S2433 In the case that the number of entities included in the relevant entity to be verified is greater than 1, the weighted sum of the multiple similarities between the entities included in the relevant entity to be verified and the relevant candidate entity is taken as at least two The relative similarity of each of the candidate entities.
  • a weight may be set for each entity included in the relevant entity to be verified.
  • the relevant entity to be verified corresponding to the entity to be verified ie, "platelet meter X”
  • the sub-item can be abbreviated (for example, "PLT")
  • the weight of is set to 0.7
  • the project reference value for example, "100-300”
  • Step S244 Obtain the candidate probability of each of the at least two candidate entities based on the correlation similarity and the entity similarity of each of the at least two candidate entities.
  • step S244 obtaining the candidate probability of each of the at least two candidate entities based on the correlation similarity and the entity similarity of each of the at least two candidate entities includes: arranging the correlation of each of the at least two candidate entities to be similar The weighted sum of the degree and the entity similarity serves as the candidate probability of each of the at least two candidate entities.
  • the candidate probability of the candidate entity the weight of the related similarity of the candidate entity ⁇ the weight of the related similarity of the candidate entity+the entity similarity of the candidate entity ⁇ the weight of the entity similarity of the candidate entity.
  • the weights of correlation similarity and entity similarity can be set according to actual application requirements.
  • the weights of correlation similarity and entity similarity can be set at 0.6 and 0.4, respectively.
  • the entity similarity of the candidate entity “platelet count” and the candidate entity “platelet packing” are 0.8 and 0.6, respectively, and the correlation similarity of the candidate entity “platelet count” and the candidate entity “platelet packing” are 1 and respectively
  • the product of the correlation similarity and the entity similarity of each of the at least two candidate entities may also be used as the candidate probability of each of the at least two candidate entities.
  • Step S233 In a case where the first determination result is the first determination sub-result and the second determination result is the fourth determination sub-result, the next-level entity to be verified is used to calculate the candidate probability of each of the at least two candidate entities.
  • the first judgment result is the first judgment sub-result; the second judgment result is the fourth judgment sub-result (that is, the entity to be verified "item name X" has a next-level entity to be verified, but does not correspond to a related entity to be verified), the next-level entity to be verified can be used to calculate the candidate probability of each of the at least two candidate entities.
  • the candidate entities of "item name X" include "item name” and "item reference value".
  • calculating the candidate probability of each of the at least two candidate entities by using the next-level entity to be verified includes the following steps S251 to S255.
  • Step S251 Obtain the next-level entity to be verified included in the entity to be verified.
  • the entities to be verified at the next level of the entity to be verified "item name X" include “platelet count” and “platelet packed volume”. It should be noted that, for the sake of clarity, it is assumed here that the next-level entity to be verified of the entity to be verified "item name X" only includes “platelet count” and "platelet packed product”.
  • Step S252 Based on the relationship between the entity to be verified and each entity in the entity to be verified at the next level, the entity to be verified at the next level is divided into at least one group of entities to be verified at the next level.
  • the entities to be verified at the next level including “platelet count” and “platelet packed product” have the same relationship with the entity to be verified "item name X", here is only divided into a group of entities to be verified at the next level.
  • the test entity group that is, the entities included in the next-level entity group to be verified are "platelet count” and "platelet packed product”.
  • Step S253 Perform the following steps S2531-step S25312 for each group of at least one group of entities to be verified at the next level.
  • Step S2531 Obtain the next-level candidate entity group.
  • the next-level candidate entity group is subordinate to each of the at least two candidate entities, and the relationship with each of the at least two candidate entities is equal to each of the entities included in the next-level entity group to be verified and the entity to be verified. All entities that verify the relationship between entities.
  • next-level candidate entity group included in the candidate entity "project name” includes entities such as “platelet count”, “platelet packing”, and “platelet distribution width”; the next-level candidate entity included in the candidate entity “project reference value” The group includes “100-300”, “0.17-0.35", “9-17” and so on.
  • Step S2532 Calculate the similarity between each of the at least one group of next-level entity groups to be verified and the next-level candidate entity group to obtain at least one group similarity corresponding to the at least one group of next-level entity groups to be verified.
  • step S2532 obtaining at least one group similarity corresponding to at least one group of next-level entity groups to be verified includes:
  • Step 310 Obtain the maximum similarity of each of the multiple entities included in each group of the at least one next-level entity group to be verified, so as to obtain the maximum similarity of each group of the at least one next-level entity group to be verified.
  • Maximum similarity of multiple entities is the similarity between each of the multiple entities included in each group of at least one next-level entity group to be verified and all the entities included in the next-level candidate entity group with the largest value. degree.
  • the next-level entity group to be verified includes the entity "platelet count” and the candidate entity "item name”. All entities in the next-level candidate entity group (for example, “platelet count”, “platelet compaction”, “platelet count” The similarity of the distribution width") is 1, 0.6 and 0.5, respectively. Therefore, the entity "platelet count” included in the next-level entity group to be verified and the candidate entity "item name” include all entities in the next-level candidate entity group.
  • the similarity with the largest value in the similarity that is, for the candidate entity "item name", the maximum similarity of the entity "platelet count” included in the next-level entity group to be verified) is 1.
  • the next-level entity group to be verified includes all entities in the next-level candidate entity group of the entity "platelet packed product” and the candidate entity "item name" (for example, “platelet count”, “platelet packed product”, “ The similarity of platelet distribution width") is 0.6, 1 and 0.5, respectively. Therefore, all entities in the next-level candidate entity group including the entity “platelet compaction” and the candidate entity "item name” included in the next-level entity group to be verified Among the similarities of entities, the similarity with the largest value (that is, for the candidate entity "item name", the maximum similarity of the entity "platelet packed product” included in the next-level entity group to be verified) is 1.
  • the multiple entities included in the next-level entity group to be verified have multiple maximum similarities of 1 and 1, respectively.
  • the entity “platelet count” included in the next-level entity group to be verified has the highest similarity among all entities in the next-level candidate entity group of the candidate entity "item reference value" (that is, for Candidate entity “item reference value", the maximum similarity of the entity “platelet count” included in the next-level entity group to be verified) is 0; the entity “platelet packed product” included in the next-level entity group to be verified and the candidate entity
  • the similarity of all entities in the next-level candidate entity group of the "item reference value” has the largest value (that is, for the candidate entity "item reference value", the entities included in the next-level entity group to be verified "
  • the maximum similarity of platelet packed product) is 0; in this case, for the candidate entity “item reference value”, multiple entities included in the next-level entity group to be verified (ie, “platelet count” and “platelet count”)
  • the multiple maximum similarities of "packed product” are 0 and 0, respectively.
  • Step 320 Obtain at least one group similarity corresponding to at least one group of next-level entity groups to be verified based on the multiple maximum similarities.
  • acquiring at least one group similarity corresponding to at least one group of next-level entities to be verified based on multiple maximum similarities includes: taking a weighted sum of the multiple maximum similarities as at least one set of next-level entity groups to be verified Corresponding at least one group similarity.
  • the weights of multiple entities included in the next-level entity group to be verified can be set according to actual application requirements.
  • the weights of multiple entities included in the next-level entity group to be verified are all equal to 1/the number of multiple entities.
  • the weights of the entities "platelet count” and “platelet packing" of the entity group to be verified at the next level are both equal to 1/2.
  • Step S254 Obtain the next-level similarity of each of the at least two candidate entities based on the at least one group similarity.
  • step S254 obtaining the next-level similarity of each of the at least two candidate entities based on at least one group similarity includes: determining the number of the next-level entity group to be verified included in the next-level entity to be verified. Number; in the case that the number of the next-level entity groups to be verified is equal to 1, the group similarity of the next-level entity groups to be verified included in the next-level entity to be verified is taken as each of the at least two candidate entities When the number of entity groups to be verified at the next level is greater than 1, the similarity of multiple groups of entities to be verified included in the entity to be verified at the next level is weighted And as the next level of similarity of each of the at least two candidate entities.
  • next level of the entity to be verified in "Project name X” includes “platelet count” and “platelet compaction” and the entity to be verified "Project name X” all have the same relationship, that is, "Project name”
  • the number of entity groups to be verified at the next level of X is equal to 1, therefore, the next level similarity of the candidate entity "item name” and the next level similarity of the candidate entity “item reference value” are 1 and 0, respectively.
  • Step S255 Calculate the candidate probability of each of the at least two candidate entities based on the next-level similarity and entity similarity of each of the at least two candidate entities.
  • step S255 calculating the candidate probability of each of the at least two candidate entities based on the next-level similarity and entity similarity of each of the at least two candidate entities includes: calculating the candidate probability of each of the at least two candidate entities based on each of the at least two candidate entities.
  • the product of the next-level similarity of each and the entity similarity is used as the candidate probability of each of the at least two candidate entities.
  • Step S231 In the case that the first judgment result is the first judgment sub-result and the second judgment result is the third judgment sub-result, calculate at least two candidates using the next-level entity to be verified and/or related entities to be verified The candidate probability of each of the entities.
  • the next-level entity to be verified may be used to calculate the candidate probability of each of the at least two candidate entities, or Calculate the candidate probability of each of the at least two candidate entities using related entities to be verified.
  • the next-level entity to be verified and the related entity to be verified can be used to calculate at least two candidate entities The candidate probability of each.
  • calculating the candidate probability of each of the at least two candidate entities by using the next-level entity to be verified and related entities to be verified includes the above steps S241 to S244 and steps S251 to S255.
  • calculating the candidate probability of each of the at least two candidate entities by using the next-level entity to be verified and related entities to be verified further includes: based on the correlation similarity of each of the at least two candidate entities, and the next-level similarity. The degree of similarity with the entity obtains the candidate probability of each of the at least two candidate entities.
  • Step S234 When the first judgment result is the second judgment sub-result and the second judgment result is the fourth judgment sub-result, the entity similarity of each of the at least two candidate entities is taken as each of the at least two candidate entities. One is the probability of the entity to be verified with errors.
  • step S234 if the entity to be verified has neither the next-level entity to be verified nor the related entity to be verified, the entity similarity of each of the at least two candidate entities of the entity to be verified may be determined.
  • Each of the at least two candidate entities is the probability that there is an error entity to be verified.
  • step S220 correcting erroneous character information based on the knowledge graph further includes: in a case where the number of entities is greater than one, performing the following step S223.
  • Step S223 Replace the character information corresponding to the entity to be verified with the error with the character information corresponding to the candidate entity with the largest candidate probability.
  • the candidate entity “platelet count” of the entity to be verified “platelet meter X” has a candidate probability of 0.8
  • the candidate entity “platelet count” of the entity to be verified “platelet meter X” has a candidate probability of 0.273
  • the character information corresponding to the entity to be verified "platelet meter X" can be replaced with the character information corresponding to the candidate entity (ie, "platelet count") corresponding to the maximum candidate probability (ie, 0.8).
  • the candidate entity "project name” of the entity to be verified has a candidate probability of 0.75
  • the candidate entity "project name X" of the entity to be verified has a candidate entity "project reference value” that has a candidate probability of 0
  • the character information corresponding to the "item name X" of the entity to be verified may be replaced with the character information corresponding to the candidate entity (ie, "item name”) corresponding to the maximum candidate probability (ie, 0.75).
  • FIG. 5A is a schematic flow chart of verifying at least one item of character information based on a knowledge graph provided by at least one embodiment of the present disclosure.
  • FIG. 5B is a schematic flowchart of parsing based on a knowledge graph provided by at least one embodiment of the present disclosure.
  • the verification of at least one item of character information based on the knowledge graph includes table analysis (that is, table structure analysis) and Entity Linking. Associate or/and map an entity to an associated entity).
  • table analysis that is, table structure analysis
  • Entity Linking Associate or/and map an entity to an associated entity.
  • the result of the table analysis can be used to rank multiple entities included in at least one item of character information extracted from a picture.
  • entity linking multiple entities included in at least one item of character information extracted from a picture can be associated with entities in the knowledge graph, thereby determining the level of multiple entities in the knowledge graph.
  • At least one item of character information extracted from the picture can be verified based on the knowledge graph.
  • the result can be output (that is, the character after the verification is output).
  • table analysis may include table recognition and segmentation.
  • table recognition and segmentation include table horizontal line detection, table vertical line detection, and table vertex detection.
  • table horizontal line detection includes horizontal line detection, image binarization and secondary median filtering.
  • the pixels of each row in the picture can be detected, and the average pixel difference between adjacent pixels is greater than 120 as the standard, and the horizontal line that divides the two records can be identified, and the horizontal line list can be obtained.
  • the vertical line detection of a table includes detecting grayscale images and then detecting a straight line. The maximum spacing is 30 pixels, and the minimum line segment length is 500 pixels, and it is a vertical straight line.
  • table vertex detection is used to identify the vertex where each cell is located.
  • the table analysis may also include fine reconstruction of the table.
  • the table fine reconstruction includes at least one of row recognition (no horizontal lines), column recognition (no vertical lines), and cross-row recognition.
  • line recognition can be completed after statistics and analysis (line coordinates) of the location information returned after text recognition.
  • column recognition can be completed after comprehensive statistical analysis (column coordinates) based on the position information returned after character recognition.
  • the analysis based on the knowledge graph may also include at least one of page number recognition, page type recognition, line reconstruction and text block reconstruction, header and footer recognition, and image/text recognition classification.
  • page number recognition is used to use OCR to identify the page number of each report, and the order used to confirm and adjust the report is the positive sequence, and it is consecutive.
  • an error can be reported when the page is not continuous.
  • page type recognition is used to identify whether the page contains medical images.
  • HOCR a document format used by OCR
  • header and footer recognition is used to recognize the header and footer of a report sheet through similar recognition algorithms; for example, similar recognition means that if similar text appears multiple times in an inspection report sheet, it will be Classified as header and footer.
  • image/text recognition classification includes a general deep learning framework, using CTPN (Detecting Text in Natural Image with Connectionist Text Proposal Network) and/or CRNN (Convolution Recurrent neural network), using OCR recognition open source library (for example, Tesseract digital recognition library) for Chinese character recognition training.
  • CTPN Detecting Text in Natural Image with Connectionist Text Proposal Network
  • CRNN Convolution Recurrent neural network
  • OCR recognition open source library for example, Tesseract digital recognition library
  • FIG. 6A is a schematic flowchart of verifying character information in an image and adding explanations based on a knowledge graph provided by at least one embodiment of the present disclosure.
  • the verification of character information in an image based on the knowledge graph provided by at least one embodiment of the present disclosure and adding explanations include: image input; recognition result output; analysis result output; and graph verification.
  • OCR OCR
  • map verification includes using the knowledge map to verify the analysis result to confirm whether the character information extracted from the picture includes incorrect character information.
  • the knowledge graph can be used in the graph interpretation step to add a description (for example, a sub-item description) to the entity corresponding to the character information extracted from the picture.
  • a description for example, a sub-item description
  • the knowledge map can be used in the map correction step to correct the incorrect character information, and then the knowledge map can be used in the map interpretation step to correct the corrected character information.
  • Add a description to the entity corresponding to the character information For example, after completion of the diagram interpretation step, the character information after adding the description can be output.
  • use the knowledge graph to verify the analysis results to confirm whether the character information extracted from the picture includes incorrect character information use the knowledge graph to correct the incorrect character information, and use the knowledge graph to add descriptions to the entities corresponding to the character information
  • use the knowledge graph to add descriptions to the entities corresponding to the character information For the specific method, for example, refer to the method for obtaining character information in the picture and the related content of the page processing method provided in at least one embodiment of the present disclosure, which will not be repeated here.
  • FIG. 6B is a schematic flowchart of medical record analysis based on a knowledge graph provided by at least one embodiment of the present disclosure.
  • the medical record analysis based on the knowledge graph includes the following steps. First, obtain the document (ie, medical record) of the relevant user.
  • the document can be a paper or an electronic file (ie, a picture). If it is a paper, scan it to obtain a picture; for example, the medical record includes the patient’s medical record, Medical reports, etc.
  • the subject (subject area) of the picture is detected, and from this, the boundary of the subject (subject area) of the picture can be determined, and then the subject (subject area) can be extracted along the boundary of the subject (subject area).
  • Third, perform image enhancement on the subject (subject area) for example, de-noise, improve contrast and equivalence).
  • a recognition result that is, character information extracted from the picture
  • the recognition result that is, the character information extracted from the picture
  • the knowledge graph for example, the general knowledge graph or the medical examination knowledge graph
  • the parsed character information is output ; If it is confirmed that there is wrong character information in the character information obtained by the analysis, use the medical examination knowledge map for correction, and then output the corrected character information.
  • the use of the medical examination knowledge map for correction includes: the highest confirmation probability is based on the relationship between the item to be checked (for example, the character information to be corrected) and the checklist, the name of the item to be checked, the abbreviation of the item to be checked, etc.
  • the character information corresponding to the combination with the highest probability is used to overwrite (replace) the character information with errors.
  • the minimum coding distance algorithm can be used to obtain the coding distance (Levenstein Levenshtein distance) between at least one of the above multiple entities and the content to be verified (the entity to be verified), so as to compare the information extracted from the picture (for example, , Character information) for verification (inspection and correction).
  • calculating the minimum compilation distance probability of a block includes: using a block calculation method to calculate the minimum compilation of the content to be verified (including name, abbreviation, range, etc.) and the entity (or entity and relationship) in the knowledge map of medical examination and inspection. Distance, and calculate the probability based on the minimum code distance calculated above.
  • the calculation of the minimum edit distance probability of a block involves the following steps. First, obtain information (for example, inspection institution, clinical inspection item, clinical inspection sub-item, inspection list, inspection type, etc.) through layout analysis and form semantic identification and division; the entities corresponding to the above information are based on the entity name and Types are classified, and each entity module (for example, at least one of the entities corresponding to the above information and the corresponding entity in the knowledge graph) is calculated and the compiling distance is selected and the content corresponding to the combination with the highest probability (for example, character information) is overwritten. content.
  • information for example, inspection institution, clinical inspection item, clinical inspection sub-item, inspection list, inspection type, etc.
  • semantic identification and division the entities corresponding to the above information are based on the entity name and Types are classified, and each entity module (for example, at least one of the entities corresponding to the above information and the corresponding entity in the knowledge graph) is calculated and the compiling distance is selected and the content corresponding to the combination with the highest probability (for example
  • block calculation means the entity corresponding to the inspection organization only calculates the compilation distance with the entity corresponding to the inspection organization in the knowledge map; the entity corresponding to the clinical inspection item only calculates the compilation distance with the entity corresponding to the clinical inspection item in the knowledge map Calculate; the entity corresponding to the checklist name only calculates the compilation distance with the entity corresponding to the checklist name in the knowledge graph.
  • using the block minimum edit distance probability algorithm can make full use of the relationship between entities in the knowledge graph (including information and contained information, etc.), and can effectively reduce the scope of the minimum edit distance calculation for each entity (that is, reduce The amount of calculation) to improve the efficiency of error correction and the accuracy of the content after error correction.
  • P(B) is the probability of correct content (for example, correct character information)
  • a i is at least the corresponding entity of the checklist, check items, check details, word count, English abbreviation, etc. in the inspection knowledge graph One.
  • the entity corresponding to the surgical examination (2) only needs the entity of the examination item corresponding to the surgical examination (2) included in the physical examination item entity in the knowledge graph to calculate the edit distance probability.
  • Figure 7A is a picture corresponding to a part of the blood routine checklist.
  • Fig. 7B is a picture of part of the routine blood checklist shown in Fig. 7A after being contaminated.
  • the following character information extracted from the picture corresponding to the contaminated checklist has errors: "average blood XX volume”, “large platelet X cases”, and “lymphocyte XX ratio".
  • the character information in the first column-the seventh column of FIG. 7A and FIG. 7B respectively correspond to "sub-item serial number", "sub-item name”, “sub-item abbreviation”, “sub-item result”, “sub-item Prompt for abnormal project results", "sub-project result unit” and "sub-project reference value”.
  • Fig. 7C is a result obtained by using a software program based on the method for obtaining character information in a picture provided by at least one embodiment of the present disclosure to obtain the character information in the picture in Fig. 7B.
  • the characters in the solid-line box shown in FIG. 7C are character information obtained from the picture shown in FIG. 7B, and the "name" in the solid-line box represents the name of the inspection sub-item. As shown in FIG.
  • At least one embodiment of the present disclosure also provides an apparatus for acquiring character information in a picture.
  • Fig. 8 is an apparatus for acquiring character information in a picture provided by at least one embodiment of the present disclosure.
  • the device for acquiring character information in the picture includes a device for extracting character information and a device for checking and correcting.
  • the character information extraction device is configured to obtain a picture and extract at least one item of character information included in the picture.
  • the verification and correction device is configured to verify at least one item of character information based on the knowledge graph.
  • a specific method for acquiring a picture and extracting at least one item of character information included in the picture and a specific method for verifying at least one item of character information based on the knowledge graph please refer to at least one embodiment of the present disclosure.
  • a character in the picture is also provided. The method of obtaining the information will not be repeated here.
  • the character information extraction device and the verification and correction device can be implemented by software, firmware, hardware, and any combination thereof, respectively.
  • the hardware includes a Field Programmable Gate Array (FPGA) and the like.
  • a verification and correction device in the acquisition device for character information in a picture, at least one item of character information can be verified (for example, verification and correction) based on the knowledge graph, so that it can be confirmed whether the character information extracted from the picture is.
  • the presence of errors and the correction of the character information with errors can further improve the accuracy of the character information obtained by the apparatus for obtaining character information in a picture provided by at least one embodiment of the present disclosure.
  • At least one embodiment of the present disclosure also provides another device for acquiring character information in a picture.
  • Fig. 9 is another apparatus for acquiring character information in a picture provided by at least one embodiment of the present disclosure.
  • this other device for acquiring character information in a picture includes a processor and a memory.
  • the memory stores computer program instructions suitable for execution by the processor, and when the computer program instructions are executed by the processor, the processor executes the method for acquiring character information in any picture provided by at least one embodiment of the present disclosure.
  • the processor is, for example, a central processing unit (CPU), a graphics processing unit GPU, a tensor processor (TPU), or other forms of processing units with data processing capabilities and/or instruction execution capabilities.
  • the processor may It is implemented as a general-purpose processor, and can also be a single-chip microcomputer, a microprocessor, a digital signal processor, a dedicated image processing chip, or a field programmable logic array, etc.
  • the memory may include at least one of a volatile memory and a non-volatile memory.
  • the memory may include a read-only memory (ROM), a hard disk, a flash memory, and the like.
  • the memory may be implemented as one or more computer program products, and the computer program products may include various forms of computer-readable storage media, and one or more computer programs may be stored on the computer-readable storage medium.
  • the processor may run the program instructions to execute the method for acquiring character information in any picture provided by at least one embodiment of the present disclosure.
  • the memory may also store various other application programs and various data, for example, various data used and/or generated by the application programs.
  • At least one embodiment of the present disclosure also provides another device for acquiring character information in a picture, which can verify at least one item of character information (for example, verification and correction) based on the knowledge graph, so that it can be confirmed from the picture Whether the extracted character information has errors and correcting the character information with errors can further improve the accuracy of the character information obtained by using another apparatus for obtaining character information in a picture provided by at least one embodiment of the present disclosure.
  • another device for acquiring character information in a picture which can verify at least one item of character information (for example, verification and correction) based on the knowledge graph, so that it can be confirmed from the picture Whether the extracted character information has errors and correcting the character information with errors can further improve the accuracy of the character information obtained by using another apparatus for obtaining character information in a picture provided by at least one embodiment of the present disclosure.
  • FIG. 10 is a non-transitory storage medium provided by at least one embodiment of the present disclosure.
  • the non-transitory storage medium includes computer program instructions stored thereon.
  • the computer program instructions When the computer program instructions are executed by the processor, the computer executes the method for acquiring character information in any picture provided by at least one embodiment of the present disclosure.
  • non-transitory storage media may include magnetic storage media, optical storage media, semiconductor storage media, etc.; for example, non-transitory storage media may include read-only memory (ROM), hard disks, flash memory, and the like.
  • ROM read-only memory
  • the non-transitory storage medium shown in FIG. 10 can be used to improve the accuracy of character information obtained from pictures.
  • FIG. 11 is an exemplary flowchart of a page processing method provided by at least one embodiment of the present disclosure.
  • the page processing method includes the following steps S510 to S530.
  • Step S510 Obtain a picture of the page.
  • Step S520 Acquire the verified character information of the picture of the page by executing any of the acquisition methods provided in at least one embodiment of the present disclosure.
  • Step S530 Structure the verified character information of the picture of the page to obtain a structured data combination.
  • the page may be a medical report, a medical record, a prescription, and the like.
  • the medical report may include at least one of the following reports: inspection report, physical examination report, hospitalization report, discharge report, etc.
  • structured data is relational model data, which can be stored in a relational database.
  • unstructured data is data that has no fixed schema and can be stored in a non-relational database.
  • structured data can be understood by a computer, which is beneficial for retrieval and display on a user graphical interface. Therefore, by adopting the page processing method provided by at least one embodiment of the present disclosure to obtain a structured data combination from a picture of a page, not only Improving the accuracy of the character information obtained from the pictures on the page can also make the patient's medical examination test data more effective.
  • platelet count"-"value"-"215" is a kind of structured data; for example, “platelet count”-"value"-"0.2" is also a kind of structured data.
  • a plurality of structured data may be obtained based on the character information after verification of the picture of the page, and the plurality of structured data may be associated with each other to obtain a structured data combination.
  • the page processing method may further include the following steps S540 and S550.
  • Step S540 The picture on the confirmation page based on the knowledge graph includes medical images.
  • Step S550 adding medical images to the structured data combination.
  • At least one embodiment of the present disclosure also provides a page processing method that can be executed sequentially in the order of step S510, step S520, step S530, step S540, and step S550.
  • At least one embodiment of the present disclosure also provides a page processing method that can be executed in the order of step S510, step S520, step S540, and step S530+step S550 (that is, step S530 and step S550 are executed simultaneously).
  • the picture based on the knowledge graph confirming that the page includes medical images includes: according to character information extracted from the picture of the page and verified based on the knowledge graph (for example, the verified character obtained in step S520) Information)
  • the picture including the text determination page corresponding to the medical image includes the medical image.
  • the verified character information obtained in step S520 includes characters such as nuclear magnetic resonance images (NMRI) and does not include characters corresponding to the examination sub-items (for example, platelet count)
  • NMRI nuclear magnetic resonance images
  • the examination sub-items for example, platelet count
  • the page processing method provided by at least one embodiment of the present disclosure can effectively save more medical examination results.
  • the page processing method provided by at least one embodiment of the present disclosure can also have a page type recognition function.
  • step S550 the medical image detected and extracted from the picture of the page can be associated with the structured data in step S530, so that the medical image can be added to the updated structured data combination.
  • the page processing method may further include the following step S560.
  • Step S560 Add descriptions corresponding to the multiple entities to the structured data combination based on the multiple entities corresponding to the knowledge graph and the verified character information.
  • an entity corresponding to each of the multiple entities may be found in the knowledge graph (for example, check sub-items)
  • the descriptions corresponding to multiple entities are added to the structured data combination. For example, by adding descriptions corresponding to multiple entities in the structured data combination, the patient can better understand the meaning of the examination result, thereby allowing the patient to use the examination result more effectively.
  • Platelet count refers to the number of platelets contained in a unit volume of blood. Decreased platelet counts are common in reducing aplastic anemia. , Acute leukemia, radiotherapy and chemotherapy, thrombocytopenic purpura, hypersplenism, etc.; increased platelet count is common in pathological myeloproliferative diseases, essential thrombocythemia, acute hemolysis, acute purulent infection, etc.”.
  • the page processing method may also include page number recognition to combine and merge structured data located on different pages.
  • At least one embodiment of the present disclosure also provides a method for constructing a knowledge graph (for example, a method for constructing a medical knowledge graph).
  • Fig. 12 is a method for constructing a medical knowledge graph provided by at least one embodiment of the present disclosure.
  • the method for constructing the medical knowledge graph includes the following steps S610 to S640.
  • Step S610 Obtain the data structure of the medical knowledge graph.
  • the data structure of the medical knowledge graph may adopt the data structure of the medical knowledge graph shown in FIG. 3, but the embodiment of the present disclosure is not limited thereto.
  • the data structure of the medical knowledge map may not include the part corresponding to the hospital information system and/or the part corresponding to the medical significance of the clinical examination sub-item in the data structure of the medical knowledge map shown in FIG. 3.
  • constructing the data structure of the medical knowledge graph includes defining the types of entities and the types of relationships between entities.
  • Step S620 at least derive the corresponding entity from the medical image archive and communication system and the laboratory information management system.
  • the corresponding entity can be manually derived from the medical image archive and communication system and the laboratory information management system.
  • the export function of the medical image archive and communication system and the laboratory information management system can also be used to automatically derive the corresponding entities from the medical image archive and communication system and the laboratory information management system.
  • the medical knowledge graph include a medical image archiving and communication (PACS) system
  • PACS medical image archiving and communication
  • the knowledge map obtained by the method for constructing a medical knowledge map provided by an embodiment can be used to effectively save more medical examination results.
  • Step S630 Perform entity alignment on the entity derived from the medical image archive and communication system and the entity derived from the laboratory information management system to obtain multiple entities after entity alignment.
  • the entity derived from the medical image archive and communication system is aligned with the entity derived from the laboratory information management system.
  • the entity derived from the medical image archive and communication system is substantially the same as the entity derived from the laboratory information management system. (Consistent meaning) entities are associated.
  • the medical image archiving and communication system may use "abbreviation” to mean “the abbreviation of project name”
  • the laboratory information management system may use "code” to mean "the abbreviation of project name”.
  • step S630 Associate both "abbreviation” and “code” with “abbreviation of project name” (or directly link “abbreviation” and “code”), so that entities derived from the medical image archive and communication system and laboratory information The entities exported by the management system achieve entity alignment.
  • the knowledge graph obtained based on the method for constructing a medical knowledge graph provided by at least one embodiment of the present disclosure can be made Has wider applicability.
  • entity alignment between the entities derived from the medical image archive and communication system and the entities derived from the laboratory information management system it can also be combined with medical documents, guidelines, papers, etc., to align the entities derived from the medical image archive and communication system.
  • Other processing of entities and entities derived from the laboratory information management system includes data cleansing (for example, removal of unrelated entities) and data supplementation (replenishment of important entities that are not included in the two systems).
  • Step S640 Filling the aligned multiple entities into the medical knowledge graph based on the data structure of the medical knowledge graph.
  • the aligned entities that have completed other processing can be based on the data of the medical knowledge graph
  • the structure is filled into the medical knowledge graph.
  • the method for constructing the medical knowledge graph includes: deriving the corresponding entity from the hospital information system; in this case, the entity derived from the hospital information system can be aligned with multiple entities after the entity is aligned; from the hospital information
  • the multiple entities exported by the system and aligned with the entities are filled into the medical knowledge map based on the data structure of the medical knowledge map.
  • the corresponding entity may be derived from the hospital information system in step S620, and in step S630, the entity derived from the medical image archive and communication system can be obtained from Align the entities derived from the laboratory information management system and the corresponding entities derived from the hospital information system.
  • the method for constructing the medical knowledge graph further includes: obtaining the sub-item description of the clinical examination sub-item involved in the laboratory information management system; and processing the sub-item description and filling it into the medical knowledge graph.
  • a crawler ie, a network robot
  • the sub-item description can include "low hematocrit, which is common in patients with anemia and when the patient's blood is diluted.”
  • the processing described for the sub-items of the acquired clinical examination sub-items includes cleaning (that is, removal of irrelevant information).
  • the processing of the acquired sub-item descriptions of the clinical examination sub-items may also include merging (that is, merging descriptions obtained from multiple information sources) and cleaning (that is, removing irrelevant or redundant information).
  • the method and device for using the knowledge graph obtained by the method for constructing the medical knowledge graph provided by at least one embodiment of the present disclosure can present clinical information to the patient.
  • the description of the examination sub-items and the potential of the clinical examination sub-item results to have the meaning of abnormality under abnormal conditions, so that the patient can more clearly understand the meaning of the examination result, and the patient can use the examination result more effectively.
  • FIG. 13 shows an exemplary scene diagram of a system for acquiring character information in a picture provided by at least one embodiment of the present disclosure.
  • the system 300 for acquiring character information in the picture may include a user terminal 310, a network 320, a server 330, and a database 340.
  • the user terminal 310 may be the computer 310-1 or the portable terminal 310-2 shown in FIG. 13. It is understandable that the user terminal may also be any other type of electronic device capable of receiving, processing, and displaying data, which may include, but is not limited to, a desktop computer, a notebook computer, a tablet computer, a smart home device, a wearable device, In-vehicle electronic equipment, medical electronic equipment, etc.
  • the network 320 may be a single network, or a combination of at least two different networks.
  • the network 320 may include, but is not limited to, one or a combination of a local area network, a wide area network, a public network, a private network, the Internet, a mobile communication network, and the like.
  • the server 330 may be a single server or a server group, and each server in the server group is connected through a wired network or a wireless network.
  • the wired network may, for example, use twisted pair, coaxial cable, or optical fiber transmission for communication
  • the wireless network may use, for example, a 3G/4G/5G mobile communication network, Bluetooth, Zigbee, or WiFi.
  • the present disclosure does not limit the types and functions of the network here.
  • the server group may be centralized, such as a data center, or distributed.
  • the server can be local or remote.
  • the server 330 may be a general-purpose server or a dedicated server, and may be a virtual server or a cloud server.
  • the database 340 may be used to store various data used, generated, and output from the work of the user terminal 310 and the server 330.
  • the database 340 may be connected or communicated with the server 330 or a part of the server 330 via the network 320, or directly connected or communicated with the server 330, or may be connected or communicated with the server 330 through a combination of the above two methods.
  • the database 340 may be a stand-alone device.
  • the database 340 may also be integrated in at least one of the user terminal 310 and the server 340.
  • the database 340 may be set on the user terminal 310 or on the server 340.
  • the database 340 may also be distributed, a part of which is set on the user terminal 310, and the other part is set on the server 340.
  • the user terminal 310 can acquire pictures (for example, use the camera of the user terminal to take pictures taken from a paper medical examination report), and use the acquired pictures via the network 320 or other technologies (for example, Bluetooth communication, infrared communication, etc.).
  • pictures for example, use the camera of the user terminal to take pictures taken from a paper medical examination report
  • the network 320 or other technologies (for example, Bluetooth communication, infrared communication, etc.).
  • the server 330 may obtain the information of the knowledge graph from the database, and execute the method for obtaining character information in the picture and the method for page processing provided by at least one embodiment of the present disclosure to obtain the verified post-check extracted from the above picture And the structured data combination obtained by structuring the verified character information; then, the server 330 may store the structured data combination in the database 340, and send the structured data combination to the user terminal 310 ; Finally, the user terminal displays (for example, through a pre-designed graphical user interface display) the above-mentioned structured data combination.
  • the server 330 may use an application built in the server to execute the method for acquiring character information in the picture and the method for page processing provided in at least one embodiment of the present disclosure. In other implementation manners, the server 330 may execute the method for acquiring character information in the picture and the method for page processing provided in at least one embodiment of the present disclosure by calling an application program stored externally of the server.
  • the user terminal 310 may obtain a picture (for example, use the camera of the user terminal to take a picture obtained from a paper medical examination report), extract character information from the picture, and use the extracted character information via the network 320 or other Technology (for example, Bluetooth communication, infrared communication, etc.) is sent to the server 330; the server 330 can obtain the information of the knowledge graph from the database, and execute part and page processing of the method for obtaining character information in the picture provided by at least one embodiment of the present disclosure The method is to verify the received character information and structure the verified character information to obtain a structured data combination; then, the server 330 may store the structured data combination in the database 340 and store the structure The structured data combination is sent to the user terminal 310; finally, the user terminal displays (for example, through a pre-designed graphical user interface) the above structured data combination.
  • the user terminal can extract the character information in the picture by using an application built in the user terminal. In other implementations, the user terminal can extract the character information in the picture by
  • the method, device, or system according to the embodiments of the present application can also be implemented with the aid of the architecture of the computing device 400 shown in FIG. 14.
  • FIG. 14 shows the architecture of the computing device 400.
  • the computing device 400 may include a bus 410, one or at least two CPUs 420, a read only memory (ROM) 430, a random access memory (RAM) 440, a communication port 450 connected to a network, input/output Components 460, hard disk 470, etc.
  • the storage device (for example, ROM 430 or hard disk 470) in the computing device 400 can store at least one corresponding to the method for obtaining character information in the picture, the method for page processing, and the method for constructing a medical knowledge graph provided by at least one embodiment of the present disclosure. Instructions and various related data or documents.
  • the computing device 400 may also include a human-machine user interface 480.
  • the architecture shown in FIG. 14 is only exemplary. When implementing different devices, one or at least two components of the computing device shown in FIG. 14 may be omitted according to actual needs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Epidemiology (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Character Discrimination (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Databases & Information Systems (AREA)

Abstract

一种图片中字符信息的获取方法和装置、非暂时性存储介质、页面处理方法和知识图谱构建方法。该图片中字符信息的获取方法包括：获取图片并且提取图片包括的至少一项字符信息；基于知识图谱对至少一项字符信息进行校验。该图片中字符信息的获取方法通过基于知识图谱对提取得到的至少一项字符信息进行校验可以提升获取的字符信息的准确性。

Description

字符获取、页面处理与知识图谱构建方法及装置、介质 技术领域
本公开的实施例涉及一种图片中字符信息的获取方法和装置、存储介质、页面处理方法和知识图谱构建方法。
背景技术
患者(或体检者)会做很多医学检查检验，以便确认身体是否健康，医学检查检验结果作为医生进行诊断的辅助信息。然而，在获得纸质文件(化验单、处方、病历、体检报等)之后，患者可能扔掉纸质文件或者将纸质文件存储在家里的角落里。因此，患者的历史医学检查检验数据的保存率较低，并且难以再次利用。此外，纸质文件及其图像为非结构化数据，不便于查看和管理。
光学字符识别(Optical Character Recognition，OCR)技术可以将图片、照片上的文字内容直接转换为可编辑文本。例如，OCR技术可以提取JPG、PNG、GIF、BMP、DOC等格式的图片上的文字内容。
发明内容
本公开的至少一个实施例提供了一种图片中字符信息的获取方法，其包括：获取图片并且提取所述图片包括的至少一项字符信息；以及基于知识图谱对所述至少一项字符信息进行校验。
例如，在所述获取方法的至少一个示例中，所述基于所述知识图谱对至少一项字符信息进行校验包括：基于所述知识图谱确认所述至少一项字符信息中存在错误的字符信息；以及基于所述知识图谱对所述存在错误的字符信息进行校正。
例如，在所述获取方法的至少一个示例中，所述至少一项字符信息包括多项字符信息。所述基于所述知识图谱确认所述至少一项字符信息中存在错误的字符信息包括：根据所述图片的多项字符信息分别得到多个实体，且从所述多个实体中选择待校验实体以用于确定所述待校验实体对应的字符信息是否存在错误；以及根据所述知识图谱的分级结构，确认所述待校验实体对 应的字符信息是否存在错误，在所述待校验实体对应的字符信息存在错误的情形下，确认所述待校验实体对应的字符信息为所述存在错误的字符信息。
例如，在所述获取方法的至少一个示例中，所述根据所述知识图谱的分级结构，确认所述待校验实体对应的字符信息是否存在错误包括：对所述多个实体按照所述知识图谱的分级结构进行分级；确定所述待校验实体在所述知识图谱的分级结构的级别；计算所述待校验实体与在所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体的相似度，以获得与所述待校验实体相关的多个实体相似度；以及在所述多个实体相似度中的最大值小于预定的实体相似度阈值时，判定所述待校验实体为存在错误的待校验实体以及所述待校验实体对应的字符信息存在错误。
例如，在所述获取方法的至少一个示例中，使用最小编译距离算法获取所述待校验实体与在所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体的相似度。
例如，在所述获取方法的至少一个示例中，所述基于所述知识图谱对所述存在错误的字符信息进行校正包括：确定所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体的数目，以获得实体数目。在所述实体数目等于1的情况下，直接将所述存在错误的待校验实体对应的字符信息替换为与所述待校验实体具有相同级别和相同关系的实体对应的字符信息，或者，计算与所述待校验实体具有相同级别和相同关系的实体为所述存在错误的待校验实体的概率，以得到实体概率，并在所述实体概率大于预定的实体概率阈值的情况下，将所述存在错误的待校验实体对应的字符信息替换为与所述待校验实体具有相同级别和相同关系的实体对应的字符信息。在所述实体数目大于1的情况下，执行以下的步骤：基于所述多个实体相似度确定至少两个候选实体；计算所述至少两个候选实体的每个为所述存在错误的待校验实体的概率，以获取与所述至少两个候选实体的每个的候选概率；以及将所述存在错误的待校验实体对应的字符信息替换为对应于最大候选概率的候选实体对应的字符信息。
例如，在所述获取方法的至少一个示例中，所述基于所述多个实体相似度确定至少两个候选实体包括：基于所述多个实体相似度对在所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体进行降序排序，选择排在序列最前面的预定数目的实体作为所述至少两个候选实体；或者基于 所述多个实体相似度对在所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体进行升序排序，选择排在序列最后面的预定数目的实体作为所述至少两个候选实体。
例如，在所述获取方法的至少一个示例中，所述计算所述至少两个候选实体的每个为所述存在错误的待校验实体的概率包括：判定所述待校验实体是否包括下一级待校验实体，以获取第一判定结果，其中，所述下一级待校验实体为隶属于所述待校验实体的所有的下一级实体；判定所述待校验实体是否对应于相关待校验实体，以获取第二判定结果，其中，所述相关待校验实体为与所述待校验实体在所述知识图谱的分级结构中位于同一级、与所述待校验实体彼此关联、且与所述待校验实体所属的上一级实体具有不同关系的所有实体；基于所述第一判定结果和所述第二判定结果选择用于计算所述至少两个候选实体的每个为所述存在错误的待校验实体的概率的方法。
例如，在所述获取方法的至少一个示例中，所述第一判定结果为在判定所述待校验实体包括所述下一级待校验实体的情况下的第一判定子结果或者在判定所述待校验实体不包括所述下一级待校验实体的情况下的第二判定子结果；以及所述第二判定结果为在判定所述待校验实体对应于所述相关待校验实体情况下的第三判定子结果或者在判定所述待校验实体不对应于所述相关待校验实体情况下的第四判定子结果。
例如，在所述获取方法的至少一个示例中，所述基于所述第一判定结果和所述第二判定结果选择用于计算所述至少两个候选实体的每个为所述存在错误的待校验实体的概率的方法包括：在所述第一判定结果为第一判定子结果且所述第二判定结果为第三判定子结果的情况下，利用所述下一级待校验实体和/或所述相关待校验实体计算所述至少两个候选实体的每个的候选概率；在所述第一判定结果为第二判定子结果且所述第二判定结果为第三判定子结果的情况下，利用所述相关待校验实体计算所述至少两个候选实体的每个的候选概率；在所述第一判定结果为第一判定子结果且所述第二判定结果为第四判定子结果的情况下，利用所述下一级待校验实体计算所述至少两个候选实体的每个的候选概率；以及在所述第一判定结果为第二判定子结果且所述第二判定结果为第四判定子结果的情况下，将所述至少两个候选实体的每个的实体相似度作为所述至少两个候选实体的每个为所述存在错误的待校验实体的概率。
例如，在所述获取方法的至少一个示例中，所述利用所述相关待校验实体计算所述至少两个候选实体的每个的候选概率包括：获取所述待校验实体对应的相关待校验实体；针对所述相关待校验实体所包括的实体的每个执行以下的方法：获取相关候选实体，其中，所述相关候选实体与所述至少两个候选实体的每个在所述知识图谱的分级结构中位于同一级，与所述至少两个候选实体的每个彼此关联，且与所述至少两个候选实体的每个所属的上一级实体之间的关系等于所述相关待校验实体所包括的实体的每个与所述待校验实体之间的关系，计算所述相关待校验实体所包括的实体的每个与所述相关候选实体的相似度；基于所述相关待校验实体所包括的实体的每个与所述相关候选实体的相似度获取所述至少两个候选实体的每个的相关相似度；以及基于所述至少两个候选实体的每个的相关相似度以及实体相似度获取所述至少两个候选实体的每个的候选概率。
例如，在所述获取方法的至少一个示例中，所述基于所述相关待校验实体所包括的实体的每个与所述相关候选实体的相似度获取所述至少两个候选实体的每个的相关相似度包括：确定所述相关待校验实体所包括的实体的个数；在所述相关待校验实体所包括的实体的个数等于1的情况下，将所述相关待校验实体所包括的实体与所述相关候选实体的相似度作为所述至少两个候选实体的每个的相关相似度；在所述相关待校验实体所包括的实体的个数大于1的情况下，将所述相关待校验实体所包括的实体与所述相关候选实体的之间的多个相似度的加权和作为所述至少两个候选实体的每个的相关相似度；所述基于所述至少两个候选实体的每个的相关相似度以及实体相似度获取所述至少两个候选实体的每个的候选概率包括：将所述至少两个候选实体的每个的相关相似度和实体相似度的加权和作为所述至少两个候选实体的每个的候选概率。
例如，在所述获取方法的至少一个示例中，所述利用所述下一级待校验实体计算所述至少两个候选实体的每个的候选概率包括：获取所述待校验实体包括的下一级待校验实体；基于所述待校验实体与所述下一级待校验实体中每个实体的关系将所述下一级待校验实体划分为至少一组下一级待校验实体组；针对所述至少一组下一级待校验实体组的每组执行以下的方法：获取下一级候选实体组，其中，所述下一级候选实体组为隶属于所述至少两个候选实体的每个的、且与所述至少两个候选实体的每个的关系等于所述下一级 待校验实体组所包括的实体的每个与所述待校验实体之间关系的所有的实体；计算所述至少一组下一级待校验实体组的每组与所述下一级候选实体组的相似度，以获取所述至少一组下一级待校验实体组对应的至少一个组相似度；基于所述至少一个组相似度获取所述至少两个候选实体的每个的下一级相似度；以及基于所述至少两个候选实体的每个的下一级相似度和实体相似度的计算所述至少两个候选实体的每个的候选概率。
例如，在所述获取方法的至少一个示例中，所述获取所述至少一组下一级待校验实体组对应的至少一个组相似度包括：获取所述至少一组下一级待校验实体组的每组包括的多个实体的每个的最大相似度，以获取所述至少一组下一级待校验实体组的每组包括的多个实体的多个最大相似度，其中，所述最大相似度为所述至少一组下一级待校验实体组的每组包括的多个实体的每个与所述下一级候选实体组所包括的所有实体的相似度中取值最大的相似度；以及基于所述多个最大相似度获取所述至少一组下一级待校验实体组对应的至少一个组相似度。
例如，在所述获取方法的至少一个示例中，基于所述多个最大相似度获取所述至少一组下一级待校验实体组对应的至少一个组相似度包括：将所述多个最大相似度的加权和作为所述至少一组下一级待校验实体组对应的至少一个组相似度。
例如，在所述获取方法的至少一个示例中，所述基于所述至少一个组相似度获取所述至少两个候选实体的每个的下一级相似度包括：确定所述下一级待校验实体包括的下一级待校验实体组的个数，在所述下一级待校验实体组的个数等于1的情况下，将所述下一级待校验实体包括的下一级待校验实体组的组相似度作为所述至少两个候选实体的每个的下一级相似度，在所述下一级待校验实体组的个数大于1的情况下，将所述下一级待校验实体包括的下一级待校验实体组的多个组相似度的加权和作为所述至少两个候选实体的每个的下一级相似度；所述基于所述至少两个候选实体的每个的下一级相似度和实体相似度的计算所述至少两个候选实体的每个的候选概率包括：将所述基于所述至少两个候选实体的每个的下一级相似度和实体相似度的乘积作为所述至少两个候选实体的每个的候选概率。
例如，在所述获取方法的至少一个示例中，所述利用所述下一级待校验实体和所述相关待校验实体计算所述至少两个候选实体的每个的候选概率包 括：获取所述待校验实体对应的相关待校验实体；针对所述相关待校验实体所包括的实体的每个执行以下的方法：获取相关候选实体，其中，所述相关候选实体与所述至少两个候选实体的每个在所述知识图谱的分级结构中位于同一级，与所述至少两个候选实体的每个彼此关联，且与所述至少两个候选实体的每个所属的上一级实体之间的关系等于所述相关待校验实体所包括的实体的每个与所述待校验实体之间的关系，计算所述相关待校验实体所包括的实体的每个与所述相关候选实体的相似度；基于所述相关待校验实体所包括的实体的每个与所述相关候选实体的相似度获取所述至少两个候选实体的每个的相关相似度；获取所述待校验实体包括的下一级待校验实体；基于所述待校验实体与所述下一级待校验实体中每个实体的关系将所述下一级待校验实体划分为至少一组下一级待校验实体组；针对所述至少一组下一级待校验实体组的每组执行以下的方法：获取下一级候选实体组，其中，所述下一级候选实体组为隶属于所述至少两个候选实体的每个的、且与所述至少两个候选实体的每个的关系等于所述下一级待校验实体组所包括的实体的每个与所述待校验实体之间关系的所有的实体；计算所述至少一组下一级待校验实体组的每组与所述下一级候选实体组的相似度，以获取所述至少一组下一级待校验实体组对应的至少一个组相似度；基于所述至少一个组相似度获取所述至少两个候选实体的每个的下一级相似度；以及基于所述至少两个候选实体的每个的相关相似度、下一级相似度和实体相似度获取所述至少两个候选实体的每个的候选概率。
本公开的至少一个实施例还提供了一种图片中字符信息的获取装置，其包括：字符信息提取装置和检验校正装置。所述字符信息提取装置被配置为获取图片并提取所述图片包括的至少一项字符信息；以及所述检验校正装置被配置为基于知识图谱对所述至少一项字符信息进行校验。
本公开的至少一个实施例还提供了一种图片中字符信息的获取装置，其包括：处理器和存储器。所述存储器中存储有适于所述处理器执行的计算机程序指令，所述计算机程序指令被所述处理器运行时使得所述处理器执行本公开的至少一个实施例提供的任一获取方法。
本公开的至少一个实施例还提供了一种非暂时性存储介质，其包括存储其上的计算机程序指令。所述计算机程序指令被处理器运行时使得计算机执行本公开的至少一个实施例提供的任一获取方法。
本公开的至少一个实施例还提供了一种页面处理方法，其包括：获取页面的图片；以及通过执行本公开的至少一个实施例提供的任一获取方法获取所述页面的图片的校验后的字符信息；以及对所述页面的图片的校验后的字符信息进行结构化，以获得结构化数据组合。
例如，在所述页面处理方法的至少一个示例中，所述页面处理方法还包括：基于所述知识图谱确认所述页面的图片包括医学影像；以及在所述结构化数据组合中加入所述医学影像。
例如，在所述页面处理方法的至少一个示例中，所述基于所述知识图谱确认所述页面的图片包括医学影像包括：根据从所述页面的图片中提取的、基于所述知识图谱校验后的字符信息包括与所述医学影像对应的文字判定所述页面的图片包括所述医学影像。
例如，在所述页面处理方法的至少一个示例中，所述页面处理方法还包括：基于所述知识图谱以及所述校验后的字符信息对应的多个实体，在所述结构化数据组合中添加与所述多个实体对应的说明。
本公开的至少一个实施例还提供了一种知识图谱构建方法，其包括：获取医学知识图谱的数据结构；至少从医学影像存档与通讯系统和实验室信息管理系统导出对应的实体；将从所述医学影像存档与通讯系统导出的实体和从所述实验室信息管理系统导出的实体进行实体对齐，以得到实体对齐后的多个实体；以及将所述对齐后的多个实体基于所述医学知识图谱的数据结构填充到所述医学知识图谱中。
例如，在所述构建方法的至少一个示例中，所述知识图谱构建方法还包括：获取所述实验室信息管理系统涉及的临床检查子项目的子项目说明；以及将所述子项目说明进行处理后填充到所述医学知识图谱中。
例如，在所述构建方法的至少一个示例中，所述知识图谱构建方法还包括：从医院信息系统导出对应的实体；将从所述医院信息系统导出的实体与所述实体对齐后的多个实体进行实体对齐；将从所述医院信息系统导出、且与所述实体对齐后的多个实体进行对实体对齐的实体基于所述医学知识图谱的数据结构填充到所述医学知识图谱中。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例的附图作 简单地介绍，显而易见地，下面描述中的附图仅仅涉及本公开的一些实施例，而非对本公开的限制。
图1是本公开的至少一个实施例提供的图片中字符信息的获取方法的示例性流程图；
图2A示出了使用光学字符识别技术提取图片包括的至少一项字符信息的过程的一种示意图；
图2B示出了使用光学字符识别技术提取图片包括的至少一项字符信息的过程的另一种示意图；
图3是本公开的至少一个实施例提供的一种医学知识图谱的数据结构；
图4是血常规检查报告单的一个示例；
图5A是本公开的至少一个实施例提供的基于知识图谱对至少一项字符信息进行校验的一种示意性流程图；
图5B是本公开的至少一个实施例提供的基于知识图谱进行解析的一种示意性流程图；
图6A是本公开的至少一个实施例提供的基于知识图谱对图像中的字符信息进行校验以及添加解释的示意性流程图；
图6B是本公开的至少一个实施例提供的基于知识图谱的病案解析的示意性流程图；
图7A是一个部分血常规检查单对应的图片；
图7B是图7A所示的部分血常规检查单在受到污染之后的图片；
图7C是采用基于本公开的至少一个实施例提供的图片中字符信息的获取方法的软件程序获取图7B的图片中字符信息得到的结果；
图8是本公开的至少一个实施例提供的图片中字符信息的获取装置；
图9是本公开的至少一个实施例提供的另一种图片中字符信息的获取装置；
图10是本公开的至少一个实施例提供的非暂时性存储介质；
图11是本公开的至少一个实施例提供的页面处理方法的示例性流程图；
图12是本公开的至少一个实施例提供的医学知识图谱构建方法；
图13示出了本公开的至少一个实施例提供的图片中字符信息的获取系统的示例性的场景图；以及
图14示出了至少一个实施例提供的计算设备。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例的附图，对本发明实施例的技术方案进行清楚、完整地描述。显然，所描述的实施例是本发明的一部分实施例，而不是全部的实施例。基于所描述的本发明的实施例，本领域普通技术人员在无需创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。
除非另外定义，本公开使用的技术术语或者科学术语应当为本发明所属领域内具有一般技能的人士所理解的通常意义。本公开中使用的“第一”、“第二”以及类似的词语并不表示任何顺序、数量或者重要性，而只是用来区分不同的组成部分。同样，“一个”、“一”或者“该”等类似词语也不表示数量限制，而是表示存在至少一个。“包括”或者“包含”等类似的词语意指出现该词前面的元件或者物件涵盖出现在该词后面列举的元件或者物件及其等同，而不排除其他元件或者物件。“连接”或者“相连”等类似的词语并非限定于物理的或者机械的连接，而是可以包括电性的连接，不管是直接的还是间接的。“上”、“下”、“左”、“右”等仅用于表示相对位置关系，当被描述对象的绝对位置改变后，则该相对位置关系也可能相应地改变。
本公开的发明人在研究中注意到，尽管可以使用光学字符识别(Optical Character Recognition，OCR)技术从图片上提取文字内容，但是，OCR技术从图片上提取的文字内容仅仅是字符本身，还需要使用解析技术(例如，页面解析技术)将提取得到字符按照规则来整合成与图片内容一致的、且方便使用者查看的格式，从而得到有具体含义的信息。例如，通过普通的光学字符识别技术和和解析技术仅能从图片上提取文字内容，并将图片中文字内容转换为文本格式，由此使得图片中文字内容可以以文本格式存储在存储器中(也即，普通的OCR技术和解析技术仅解决了医学检查报告的电子化问题和存储问题)。然而，计算机对上述文本格式的文字内容的理解能力较差，不能有效检索和展示(例如，利用图形用户界面展示)上述内容，由此患者的医学检查报告依然不能得到有效的利用。
此外，本公开的发明人在研究中还注意到，医学领域的纸质文件(化验单、处方、病历、体检报等)涉及生僻字、生僻词语等，且这些生僻字和生僻词语的字符复杂，因此，使用通常方法训练出的OCR模块(也即，普通的 OCR模块)可能难以将这些生僻字和生僻词语识别出来。本公开的发明人在研究中还注意到，相关的医学OCR模块对非医学类文字的识别性能较差。因此，综合来看，普通的OCR模块和医学OCR模块在提取医学领域的纸质文件对应的图片中字符信息的准确率均较低。由于提取的字符的准确率与患者的身体健康息息相关，因此，亟需提升通过提取得到的字符的准确率的方法。
知识图谱是智能大数据的前沿研究问题，它以独有的技术优势顺应了信息化时代的发展。例如，知识图谱具有以下优点：渐增式的数据模式设计；良好的数据集成能力；得到现有的资源描述框架(Resource Description Framework，RDF)和网络本体语言(OWL，Web Ontology Language)等标准支持；具有语义搜索和知识推理能力等。例如，知识图谱在查询复杂的关联信息具有更好的新能，可以从语义层面理解用户意图，并且可以对数据具有较强的描述能力。知识图谱可以用于描述真实世界中存在的各种实体和概念。例如，可以用知识图谱描述两个实体之间的关联。例如，“姚明出生于上海”涉及两个实体“姚明”和“上海”以及两个实体之间的关系“出生地”。
在医学领域，随着区域卫生信息化及医疗信息系统的发展，积累了海量的医学数据。如何从这些数据中提炼信息，并加以管理、共享及应用，是推进医学智能化的关键问题，也是医学知识检索、临床诊断、医疗质量管理、电子病历及健康档案智能化处理的基础。医学知识图谱是实现智慧医疗的一种方法，有利于开展更高效和精准的医疗服务，基于知识图谱的推理能力，医学知识图谱具有可解释性和稳定性强两个优点。然而，由于以下问题，医学图谱的应用范围较小(例如，仅用于知识问答和检索)：现有中文医学知识图谱较少，不同地区的医学术语不统一，构建医学知识图谱需要医学专家的深度参与导致的医学知识图谱存在效率低、限制多、拓展性差等问题。
编辑距离(Edit Distance)是指由一个字符串转化成另一个字符串所需的最少编辑操作次数。允许的编辑操作包括将一个字符替换成另一个字符，插入一个字符，删除一个字符。一般来说，两个字符串之间编辑距离越小，表示两个字符串的相似度越大。
例如，最小编辑距离算法可以采用edit[i][j]表示A串(字符串)的i个字符组成的字符串以及B串的j个字符组成的字符串的编辑距离。edit[i][j]表示由A串的第0个字符开始到第i个字符组成的字符串以及由B串的第0个字符开始到第j个字符组成的字符串的编辑距离。例如，在A串为“graph”， B串为“grape”的情况下，edit[5[5]表示字符串“graph”和字符串“grape”的编辑距离；edit[2[3]表示字符串“gr”和字符串“gra”的编辑距离。例如，i和j可以大于等于1。
例如，A串和B串的的编辑距离还可以用dis[i][j]表示。例如，dis[0][0]表示A串和B串都为空的时候，此时它们的编辑距离为0。很明显可以得出的，dis[0][j]就是A串为空，B串长度为j的情况，此时他们的编辑距离为j，也就是，从空字符串添加j个字符转换成B串的最小编辑距离为j；同理dis[i][0]就是，A串长度为i，B串为空时，A串需要删除i个字符才能转换成空，所以从A串转换成B串的最小编辑距离为i。
通过以上的描述，可以得到以下的动态规划方程。
Figure PCTCN2019120634-appb-000001
其中,替换的有效次数flag可以使用以下方程表示。
Figure PCTCN2019120634-appb-000002
上述动态规划方程中的min()函数(也即，用于计算最小值的函数)中包含三个表达式，也即，edit[i-1][j]+1、edit[i][j-1]+1以及edit[i-1][j-1]+flag，上述三个表达式分别表示插入字符操作、删除字符操作以及替换字符操作，具体说明如下。
edit[i-1][j]+1相当于在B串的最后一个字符后插入了位于A串的最后的字符，上述插入字符操作使得编译距离增加1，之后通过计算edit[i-1][j]来计算edit[i-1][j]+1，以作为计算编译距离edit[i][j]的一部分；
edit[i][j-1]+1相当于将B串的最后一个字符删除，上述删除字符操作使得编译距离增加1，之后通过计算edit[i][j-1]来计算edit[i][j-1]+1，以作为计算编译距离edit[i][j]的一部分；
edit[i-1][j-1]+flag相当于通过将B串的最后一个字符替换为A串的最后一个字符。此处，使用flag标记字符替换的有效次数。
例如，可以利用最小编辑距离算法计算得到的两个字符串的编辑距离获取两个字符串的相似度。例如，可以使用以下公式并利用两个字符串的编辑 距离来计算两个字符串的相似度：S(A，B)＝1-(DIS(A，B)/max(Len(A)，Len(B))。此处，DIS(A，B)表示A串和B串的编辑距离，Len(A)和Len(B)分别表示A的字符串长度和B串的字符串长度，max(Len(A)，Len(B))等于A的字符串长度和B串的字符串长度中的最大值。
本公开的至少一个实施例提供了一种图片中字符信息的获取方法和装置、非暂时性存储介质、页面处理方法和医学知识图谱构建方法。该图片中字符信息的获取方法包括：获取图片并且提取图片包括的至少一项字符信息；基于知识图谱对至少一项字符信息进行校验。该图片中字符信息的获取方法通过基于知识图谱对提取得到的至少一项字符信息进行校验，由此可以提升获取的字符信息的准确性。
下面通过几个示例或实施例对根据本公开实施例提供的图片中字符信息的获取方法进行非限制性的说明，如下面所描述的，在不相互抵触的情况下这些具体示例或实施例中不同特征可以相互组合，从而得到新的示例或实施例，这些新的示例或实施例也都属于本公开保护的范围。
图1是本公开的至少一个实施例提供的图片中字符信息的获取方法的示例性流程图。如图1所示，该图片中字符信息的获取方法包括以下的步骤S10和步骤S20。
步骤S10：获取图片并且提取图片包括的至少一项字符信息。
步骤S20：基于知识图谱对至少一项字符信息进行校验。
在步骤S10中，获取图片的具体方法可以根据实际应用需求进行设定，本公开的至少一个实施例对此不作具体限定。在一个示例中，获取图片可以包括从存储器中获取(读取)存储在存储器中的图片。在另一个示例中，获取图片还可以包括远程获取图片，例如使用信息发送和接收装置接收图片。例如，信息发送和接收装置可以从客户端(例如，手机移动终端或PC端)接收图片。
例如，在步骤S10中，获取的图片可以为进行了图片预处理后的图片，也可以是进行图片预处理前的图片。例如，在获取的图片是进行图片预处理前的图片的情况下，在提取图片包括的至少一项字符信息之前，还可以对图片进行图片预处理。例如，图片预处理包括，但不限于，图片主体区域的检测和提取、图片校正、图片分割、图片的文字方向矫正、图片二值化以及图片去噪的至少一个。
例如，图片主体区域的检测和提取包括检测上述图片的主体区域(有效区域，包含有用信息的区域)，并将检测到的主体区域提取出来，以作为后续处理的图片。例如，可以通过对图片中存在文字的区域进行定位(Text Detection)，即找到单词或文本行(Word/Line-level)的边界框(Bounding Box)，来检测图片的主体区域。例如，在图片主体区域的检测和提取中，用户可以在提取之前，对检测到的主体区域进行确认(也即，通过用户人工选取有效区域)，以提升图片主体区域的检测和提取的准确性。
例如，图片校正(例如，梯形校正)用于矫正图片(例如，手机拍摄的图片)存在的变形。例如，可以采用伽玛校正来实现图片校正(例如，梯形校正)。
例如，图片分割用于从包含背景的图片(例如，普通手机拍摄的照片)中分割出文字(例如，体检报告的文字)，以利于提升从图片提取的文字(例如，字符信息)的准确性。例如，可以通过以下的方法从包含背景的图片中分割出文字。
首先，对图片进行水平投影，找到每一行的上界限和下界限；其次，沿着每一行的上界限和下界限进行行切割，以获得多行；第三，对于切割得到的每一行，进行垂直投影，以找到每一个字符的左边界和右边界；第四，沿着每一个字符的左边界和右边界进行单个字符的切割。例如，水平投影用于对一张图片的每一行元素进行统计(也即，沿水平方向统计)，然后，可以根据上述统计的结果画出统计结果图，由此可以确定每一行的沿列方向的起始点和结束点，也即，每一行的上界限和下界限。例如，垂直投影的投影方向是向下的，其用于统计每一列的元素个数以及每一列的元素的投影位置，由此确定每一列的左界限和右界限。
例如，可以通过以下的方法实现对图片进行水平投影：首先，设定沿垂直方向延伸的一条直线(轴)；其次，将图片上的像素沿垂直于直线(轴)的方向投影该投影到该直线(轴)上，在投影之后，图片上的多个像素对应于直线(轴)上的多个黑点；第三，统计垂直于该垂直直线的图片上的像素的黑点数量，累加求和作为该轴该位置的值。在完成上述水平投影之后，可以执行基于图片投影的切割。具体而言，在将完成图片映射并得到相关特征后，基于上述特征判定图片的切割位置(坐标)，用上述坐标来切割上述图片，得到目标图片(例如，通过分割图片获取的单个文字的图片)。例如，对图片进 行垂直投影类似于对图片进行水平投影，区别在于，设置沿垂直方向延伸的直线，并将图片上的像素投影到上述沿垂直方向延伸的直线上。
下面结合一个示例对水平投影做示例性说明。例如，可以通过以下的方法实现水平投影。首先，对图片进行二值化，以使得文字为黑色(例如，该文字对应的像素值为Lb，例如，Lb＝0)，背景为白色(例如，该文字对应的像素值为Lw，例如，Lw＝255)。其次，针对图片的每一行执行以下步骤：依次判断每一列的像素值是否等于Lb，如果像素值等于Lb，则判定位于该行该列的像素很像文字；统计该行所有很像文字的个数，假设该行共有n个像素的像素值等于Lb，则把该行从第一列到第n列均设置为黑。
例如，图片的文字方向矫正用于矫正图片拍摄过程导致的图像中文字的倾斜。例如，可以使用快速傅里叶变换(Fast Fourier Transformation)技术对图片进行校正，然后，通过对矫正后的图片进行逆变换得到方向矫正后的文字。
例如，图片二值化可用于提升文字的对比度。例如，可以通过以下方法实现图片二值化。首先，设置一个阈值；然后，对于图片的多个像素，将像素值(例如，像素的灰度值)大于或等于阈值的像素的取值设置为1(也即，使得该像素呈现为白色)，将像素值小于阈值的像素的取值设置为0(也即，使得该像素呈现为黑色)。例如，可以采用自适应二值化(每个像素在比对过程中采用的阈值不同)。例如，可以将图片的多个像素的像素值对应的二维矩阵(也即，用于存储图片的多个像素的像素值的矩阵)转换为向量，然后，将每个像素与基于前面一定范围内像素计算出来的阈值(也即，自适应阈值)进行比对。
例如，图片去噪用于降低图片的噪声，以提升从图片提取的文字的准确性。例如，在将接收到的图片转换成二值图片之后，二值图片中可能存在很多噪声；对此，可以使用中值滤波或者均值滤波对二值图片进行去噪处理。
例如，从图片中获取的字符信息可以包括单项字符信息或者多项字符信息。例如，每项字符信息包括单个字符或字符串。例如，字符信息可以是文字信息，也可以为符号信息。例如，符号信息可以为上升箭头符号、下降箭头符号等。
例如，可以使用光学字符识别技术提取图片包括的至少一项字符信息。图2A示出了使用光学字符识别技术提取图片包括的至少一项字符信息的过 程的一种示意图。
例如，如图2A所示，光学字符识别技术提取图片包括的至少一项字符信息的过程包括：文字检测、文字识别以及结果输出。例如，文字检测用于检测图片中单个的字符。例如，文字识别用于将检测到的文字转换为文本格式。例如，文字识别包括文字特征抽取以及对比识别，或者还可以使用例如训练得到的神经网络(例如，用于检测自然图像中的文本的连接文本提议网络或者卷积循环神经网络)进行文字识别。例如，对比识别用于将通过文字特征抽取获取的特征与比对数据库或特征数据库(包含所有待识别文字的字集)来进行比对，由此可以基于图片中文字的特征找到与该图片中文字对应的文本格式的文字。例如，使用光学字符识别技术进行文字检测和文本识别的具体方法可以参见相关技术，在此不再赘述。例如，可以通过检测图片中文字行的上下边界的坐标以及文字列的左右边界的坐标实现文字检测。例如，可以采用CTPN(用于检测自然图像中的文本的连接文本提议网络，Detecting Text in Natural Image with Connectionist Text Proposal Network)和/或CRNN(卷积循环神经网络)进行文字识别，可以用OCR识别开源库(例如，Tesseract数字识别库)进行汉字识别训练。
图2B示出了使用光学字符识别技术提取图片包括的至少一项字符信息的过程的另一种示意图。例如，如图2B所示，光学字符识别技术提取图片包括的至少一项字符信息的过程还可以包括：图片输入和图片处理。例如，图片处理是指图片预处理，图片预处理的具体方法可以参见步骤S10的相关描述，在此不再赘述。例如，在光学字符识别技术还包括图像处理(例如，图片预处理)的情况下，步骤S10获取的图片可以是进行图片预处理前的图片，且在提取图片包括的至少一项字符信息之前无需开展额外的图片预处理。
例如，在步骤S20中，通过基于知识图谱对至少一项字符信息进行校验(例如，检验和校正)可以确认从图片中提取的字符信息是否存在错误以及对存在错误的字符信息进行校正，由此可以提升采用了本公开的至少一个实施例提供的图片中字符信息的获取方法获取的字符信息的准确率。
例如，在步骤S20中，知识图谱为医学知识图谱。例如，该医学知识图谱可以为医学检查检验知识图谱。例如，医学检查检验知识图谱可用于提供检测指标、检测单的相关知识等。例如，医学检查检验知识图谱为精准识别和解析提供可能。
例如，医学知识图谱的架构(框架或数据结构)可以使用图3所示的医学知识图谱的数据结构。例如，可以利用本公开的至少一个实施例提供的种医学知识图谱的构建方法并基于图3所示的医学知识图谱的数据结构构建医学知识图谱，并可以将上述通过构建得到的医学知识图谱应用到步骤S20中。本公开的至少一个实施例提供的种医学知识图谱的构建方法将在图12所示的实施例中进行详细描述，在此不再赘述。
例如，基于图3所示的医学知识图谱的数据结构获取的医学知识图谱可以包含从医学影像存档与通讯(PACS)系统和实验室信息管理(LIS)系统导出的多个实体和关系。例如，通过使得医学知识图谱包括从PACS系统导出的实体，可以基于该医学知识图谱确认图片是否包含医学影像，并且在图片包含医学影像的情况下保存医学影像。
例如，从PACS系统导出的实体包括临床检查项目、临床检查子项目、临床检查单中与医学影像检查相关的实体。例如，从PACS系统导出的实体包括：电子计算机X射线断层扫描图像、核磁共振图像(NMRI)、超声图像、心电图、脑电图等。例如，从LIS系统导出的实体包括临床检查项目、临床检查子项目、临床检查单中与医学影像检查不相关的实体。例如，从LIS系统导出的实体包括各种实验室检查项目。例如，实验室检查项目包括血液检查项目、尿液检查项目、大便检查项目等。例如，血液检查项目包括血常规检查项目、血脂检查项目、血糖检查项目等。例如，血常规检查项目包括血小板计数(PLT)子项目、血小板分布宽度(PDW)子项目和血小板压积(PCT)子项目。
例如，基于图3所示的医学知识图谱的数据结构获取的医学知识图谱还可以包含从医院信息系统导出的实体和/或与临床检查子项目的子项目说明相关的实体。例如，从医院信息系统导出的实体包括与患者信息相关的实体以及与医嘱相关的实体。例如，与临床检查子项目的子项目说明相关的实体给出了临床检查子项目的说明以及临床检查子项目结果存在异常情况下异常代表的含义，由此患者可以更为清楚的了解检查结果。
需要说明的是，步骤S20采用的知识图谱的数据结构不限于图3所示的知识图谱的数据结构。例如，步骤S20采用的知识图谱的数据结构还可以不包括图3所示的知识图谱的数据结构中医院信息系统对应的部分和/或临床检查子项目的医学意义对应的部分。
例如，在步骤S20中，基于知识图谱对至少一项字符信息进行校验包括以下的步骤S210。
步骤S210：基于知识图谱确认至少一项字符信息中存在错误的字符信息。
例如，上述至少一项字符信息包括多项字符信息，存在错误的字符信息包括在识别过程中被识别错误的字符信息(例如，“水”被错误识别为“小”)和/或在识别过程中未被识别的字符信息。
例如，在步骤S210中，基于知识图谱确认至少一项字符信息中存在错误的字符信息包括以下的步骤S211和步骤212。
步骤S211：根据图片的多项字符信息分别得到多个实体，且从多个实体中选择待校验实体以用于确定待校验实体对应的字符信息是否存在错误。
图4是血常规检查报告单的一个示例。例如，对于图4所示血常规检查报告单的图片，根据图片的多项字符信息分别得到多个实体可以包括：报告单名称、血常规检查的子项目名称(项目名称)、子项目缩写(代码)、子项目检验结果(结果)、子项目检验结果的单位(单位)、子项目的检验结果是否存在异常(使用向上或向下箭头表示)以及子项目检验结果的参考值(项目参考值)。例如，血常规检查的部分子项目名称以及对应的缩写包括：血小板计数(PLT)、血小板分布宽度(PDW)和血小板压积(PCT)。
例如，在根据图片的多项字符信息分别得到多个实体，可以对多个实体进行逐个校验，直至检验完图片的多项字符信息包含的所有实体。例如，可以选择血小板计数作为待校验实体。
步骤212：根据知识图谱的分级结构，确认待校验实体对应的字符信息是否存在错误，并且，在待校验实体对应的字符信息存在错误的情形下，确认待校验实体对应的字符信息为存在错误的字符信息。
例如，在步骤212中，根据知识图谱的分级结构，确认待校验实体对应的字符信息是否存在错误包括以下的步骤2121-步骤2124。
步骤2121：对多个实体按照知识图谱的分级结构进行分级。
例如，在步骤2121中，可以借助对图片中表格结构的分析以及图片中文字的排版、尺寸、是否加粗等来对多个实体进行分级。例如，可以使用光学识别技术确定图片中是否存在表格，如果存在表格，则进一步对图片中表格结构进行分析(或解析)。例如，表格结构分析的具体方法可以参见图5A和图5B所示的示例，在此不再赘述。例如，图片中文字的(相对)尺寸、是否 加粗等也可以通过图片分析得到的，例如，部分采用分类器或神经网络实现。
例如，对于图4所示血常规检查报告单的图片的多个实体，可以采用以下的方法进行分级。
首先，可以将位于报告单第一行的、居中且字体较大的字符串“血常规”对应的实体(为简单起见，本公开中后面的描述直接使用“血常规”进行表示，对其它实体也进行了类似的处理)作为报告单中的第一级实体；将位于表格第一行的“代码”、“项目名称(也即，血常规检查的子项目名称)”、“结果”、“项目参考值”等对应的实体作为报告单中的第二级实体；将位于表格中且位于表格的第一行之外的所有实体作为报告单中的第三级实体。
然后，可以将报告单中的等级与知识图谱的分级结构中实体的级别进行关联。例如，假如在知识图谱的分级结构中，“血常规”、“项目名称(也即，血常规检查的子项目名称)”和“血小板计数”分别位于第五级、第六级和第七级，则图4所示的血常规检查报告单的图片中的“血常规”在知识图谱的分级结构的级别为第五级；图4所示血常规检查报告单的图片中的“代码”、“项目名称(也即，血常规检查的子项目名称)”、“结果”、“项目参考值”在知识图谱的分级结构的级别为第六级；图4所示血常规检查报告单的图片中的“血小板计数”等在知识图谱的分级结构的级别为第七级。
例如，基于知识图谱可知，血常规检查报告单中的“床号”等对应的实体不属于与血常规检查结果相关的实体，因此，“床号”等对应的实体将在对多个实体进行分级的过程中被忽略。
步骤2122：确定待校验实体在知识图谱的分级结构的级别。
例如，在步骤2122中，可以基于步骤2121中的分级结果确定待校验实体在知识图谱的分级结构的级别。例如，在选择了“血小板计数”作为待校验实体的情况下，基于步骤2121中的分级结果，可以确定待校验实体在知识图谱的分级结构的级别为第七级。
步骤2123：计算待校验实体与在知识图谱中与待校验实体具有相同级别和相同关系的所有实体的相似度，以获得与待校验实体相关的多个实体相似度。
例如，待校验实体与在知识图谱中与待校验实体具有相同级别和相同关系的所有实体的相似度位于0和1之间。
例如，在步骤2123中，“具有相同关系”是指在知识图谱中与待校验实 体具有相同级别和相同关系的所有实体与这些实体的上一级实体之间的关系等于待校验实体与待校验实体的上一级实体之间的关系。
例如，在选择了“血小板计数”作为待校验实体的情况下，在知识图谱中与待校验实体具有相同级别和相同关系的所有实体包括“血小板计数”、“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”等。
例如，如果“血小板计数”不存在错误，则待校验实体(也即，“血小板计数”)与知识图谱中的“血小板计数”的相似度为1；待校验实体(也即，“血小板计数”)与知识图谱中的“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”的相似度例如分别为0.5、0.6和0。
例如，如果“血小板计数”被错误的识别为“血小板计X”，则待校验实体(也即，“血小板计X”)与知识图谱中的“血小板计数”、“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”的相似度(也即，“血小板计数”、“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”的实体相似度)例如分别为0.8、0.5、0.6和0。需要说明的是，“血小板计X”中的“X”表示该字符被识别错误，其它字符或实体中的“X”也具有类似含义，不再赘述。
例如，在步骤2123中，可以使用最小编译距离算法获取待校验实体与在知识图谱中与待校验实体具有相同级别和相同关系的所有实体的相似度。
步骤2124：在多个实体相似度中的最大值小于预定的实体相似度阈值时，判定待校验实体为存在错误的待校验实体以及待校验实体对应的字符信息存在错误。
例如，预定的实体相似度阈值可以根据实际应用需求进行设定。例如，预定的实体相似度阈值可以设为1。又例如，预定的实体相似度阈值还可以设为0.95。例如，在预定的实体相似度阈值设为1的情况下，只要在知识图谱中与待校验实体具有相同级别和相同关系的所有实体不包括待校验实体的情况下，则判定待校验实体为存在错误的待校验实体以及待校验实体对应的字符信息存在错误。
例如，在待校验实体(也即，“血小板计X”)与知识图谱中的“血小板计数”、“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”的相似度例如分别为0.8、0.5、0.6和0的情况下，由于多个实体相似度中的最大值(也即，0.8)小于预定的实体相似度阈值(例如，1)，则可以判定待校验实体对应的字符信息存在错误以及待校验实体为存在错误的待校验实体。
例如，在多个实体相似度中的最大值大于等于预定的实体相似度阈值时，判定待校验实体对应的字符信息不存在错误。
例如，在待校验实体(也即，“血小板计数”)与知识图谱中的“血小板计数”、“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”的相似度例如分别为1、0.5、0.6和0的情况下，由于多个实体相似度中的最大值(也即，1)大于等于预定的实体相似度阈值(例如，1)，则可以判定待校验实体对应的字符信息不存在错误。
例如，在步骤S20中，基于知识图谱对至少一项字符信息进行校验还包括以下的步骤220。
步骤S220：基于知识图谱对存在错误的字符信息进行校正。
例如，基于知识图谱对存在错误的字符信息进行校正包括：确定所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体的数目，以获得实体数目。
例如，基于知识图谱对存在错误的字符信息进行校正还包括：在实体数目等于1的情况下，直接将存在错误的待校验实体对应的字符信息替换为在知识图谱中与待校验实体具有相同级别和相同关系的实体对应的字符信息，或者，计算与待校验实体具有相同级别和相同关系的实体为存在错误的待校验实体的概率，以得到实体概率，并在实体概率大于预定的实体概率阈值的情况下，将存在错误的待校验实体对应的字符信息替换为与待校验实体具有相同级别和相同关系的实体对应的字符信息。
例如，预定的实体概率阈值可以根据实际应用需求进行设定。例如，预定的实体概率阈值可以设置为0.5、0.8、0.9或其它适用的数值。例如，在该候选实体的实体概率小于等于预定的实体概率阈值的情况下，可以对存在错误的字符信息进行人工校正。
例如，基于知识图谱对存在错误的字符信息进行校正还包括：在实体数目大于1的情况下，执行以下的步骤S221-步骤S222。
步骤S221：基于多个实体相似度确定至少两个候选实体。
例如，在步骤S221中，基于多个实体相似度确定至少两个候选实体包括：基于多个实体相似度对在知识图谱中与待校验实体具有相同级别和相同关系的所有实体进行降序排序，选择排在序列最前面的预定数目的实体作为至少两个候选实体；或者基于多个实体相似度对在知识图谱中与待校验实体具有 相同级别和相同关系的所有实体进行升序排序，选择排在序列最后面的预定数目的实体作为至少两个候选实体。
例如，可以根据实际应用需求设定预定数目。例如，可以将预定数目设定为2、3、10或其它适用的数目。
例如，在步骤S221中，可以基于多个实体相似度(也即，0.8、0.5、0.6和0)将知识图谱中的“血小板计数”、“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”按照降序排序排列为以下序列，也即，“血小板计数”、“血小板压积”、“血小板分布宽度”和“中性粒细胞百分比”。例如，在将预定数目设定为2的情况下，可以将排列在序列最前面的两个实体，也即，“血小板计数”和“血小板压积”作为至少两个候选实体。
又例如，在步骤S221中，可以基于多个实体相似度(也即，0.8、0.5、0.6和0)将知识图谱中的“血小板计数”、“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”按照升序排序排列为以下序列，也即，“中性粒细胞百分比”、“血小板分布宽度”、“血小板压积”和“血小板计数”。例如，在将预定数目设定为2的情况下，可以将排列在序列最后面的两个实体，也即，“血小板压积”和“血小板计数”作为至少两个候选实体。
步骤S222：计算至少两个候选实体的每个为存在错误的待校验实体的概率，以获取与至少两个候选实体的每个的候选概率。
例如，至少两个候选实体的每个为存在错误的待校验实体的概率是指至少两个候选实体的每个对应的字符信息等价于存在错误的待校验实体对应的字符信息在未被错误识别下的字符信息的概率。例如，“血小板压积”为存在错误的待校验实体“血小板计X”的概率是指“血小板压积”对应的字符信息等价于“血小板计X”对应的字符信息在未被错误识别下的字符的概率。
例如，在步骤S222中，计算至少两个候选实体的每个为存在错误的待校验实体的概率包括以下的步骤S2221-步骤S2223。
步骤S2221：判定待校验实体是否包括下一级待校验实体，以获取第一判定结果。下一级待校验实体为隶属于待校验实体的所有的下一级实体。
例如，对于图4所示血常规检查报告单的图片的多个实体，在选择“血小板计X”作为待校验实体的情况下，可以判定待校验实体不包括下一级待校验实体；在选择“项目名X”作为待校验实体的情况下，可以判定待校验实体包括下一级待校验实体，例如，“项目名X”包括下一级待校验实体包括“血 小板计数”、“血小板分布宽度”、“血小板压积”和“中性粒细胞百分比”等。
例如，第一判定结果为在判定待校验实体包括下一级待校验实体的情况下的第一判定子结果或者在判定待校验实体不包括下一级待校验实体的情况下的第二判定子结果。例如，在选择“项目名X”作为待校验实体的情况下，第一判定结果为第一判定子结果；在选择“血小板计X”作为待校验实体的情况下，第一判定结果为第二判定子结果。
步骤S2222：判定待校验实体是否对应于相关待校验实体，以获取第二判定结果。相关待校验实体为与待校验实体在知识图谱的分级结构中位于同一级、与待校验实体彼此关联、且与待校验实体所属的上一级实体具有不同关系的所有实体。例如，相关待校验实体为与待校验实体彼此关联是指相关待校验实体和待校验实体用于描述同一主体的不同方面。例如，“血小板计数(血小板计数检查子项目的全称)”以及“PLT(血小板计数检查子项目的缩写)”、“100-300(血小板计数检查子项目的参考值)用于描述血小板计数检查子项目的不同方面，因此，待校验实体“血小板计数”与“PLT”和“100-300”彼此相关。
例如，对于图4所示血常规检查报告单的图片的多个实体，在选择“血小板计X”作为待校验实体的情况下，可以判定待校验实体对应于相关待校验实体。例如，“血小板计X”对应的相关待校验实体包括“PLT(血小板计数的缩写)”、“100-300(血小板计数的参考值)”等。在选择“项目名X”作为待校验实体的情况下，可以判定待校验实体不对应于相关待校验实体。
例如，第二判定结果为在判定待校验实体对应于相关待校验实体情况下的第三判定子结果或者在判定待校验实体不对应于相关待校验实体情况下的第四判定子结果。例如，在选择“项目名X”作为待校验实体的情况下，第二判定结果为第四判定子结果；在选择“血小板计X”作为待校验实体的情况下，第二判定结果为第三判定子结果。
步骤S2223：基于第一判定结果和第二判定结果选择用于计算至少两个候选实体的每个为存在错误的待校验实体的概率的方法。
例如，在步骤S2223中，基于第一判定结果和第二判定结果选择用于计算至少两个候选实体的每个为存在错误的待校验实体的概率的方法包括以下的步骤S231-步骤S234中的一个。
步骤S232：在第一判定结果为第二判定子结果且第二判定结果为第三判 定子结果的情况下，利用相关待校验实体计算至少两个候选实体的每个的候选概率。
例如，在选择“血小板计X”作为待校验实体的情况下，第一判定结果为第二判定子结果，第二判定结果为第三判定子结果(也即，“血小板计X”不具有下一级待校验实体，但是对应于相关待校验实体)，可以利用相关待校验实体计算至少两个候选实体的每个的候选概率。
例如，待校验实体(也即，“血小板计X”)的候选实体包括“血小板计数”、“血小板压积”等。需要说明的是，为清楚起见，此处假设待校验实体(也即，“血小板计X”)的候选实体仅包括“血小板计数”和“血小板压积”。
例如，利用相关待校验实体计算至少两个候选实体的每个的候选概率包括以下的步骤S241-步骤S244。
步骤S241：获取待校验实体对应的相关待校验实体。
例如，待校验实体(也即，“血小板计X”)对应的相关待校验实体包括“PLT”、“100-300”等。
步骤S242：针对相关待校验实体所包括的实体的每个执行以下的步骤S2421-步骤S2422。
步骤S2421：获取相关候选实体。相关候选实体与至少两个候选实体的每个在知识图谱的分级结构中位于同一级，与至少两个候选实体的每个彼此关联，且与至少两个候选实体的每个所属的上一级实体之间的关系等于相关待校验实体所包括的实体的每个与待校验实体之间的关系。
例如，针对相关待校验实体“PLT”，候选实体“血小板计数”对应的相关候选实体为“PLT”，候选实体“血小板压积”对应的相关候选实体为“PCT”。例如，针对相关待校验实体“100-300”，候选实体“血小板计数”对应的相关候选实体为“100-300”，候选实体“血小板压积”对应的相关候选实体为“0.17-0.35”。
步骤S2422：计算相关待校验实体所包括的实体的每个与相关候选实体的相似度。
例如，相关待校验实体“PLT”与候选实体“血小板计数”对应的相关候选实体“PLT的相似度为1，而相关待校验实体“PLT”与候选实体“血小板压积”对应的相关候选实体“PCT”的相似度为0.65。
例如，相关待校验实体“100-300”与候选实体“血小板计数”对应的相关候选实体“100-300”的相似度为1；相关待校验实体“100-300”与候选实体“血小板 压积”对应的相关候选实体“0.17-0.35”的相似度为0。
步骤S243：基于相关待校验实体所包括的实体的每个与相关候选实体的相似度获取至少两个候选实体的每个的相关相似度。
例如，在步骤S243中，基于相关待校验实体所包括的实体的每个与相关候选实体的相似度获取至少两个候选实体的每个的相关相似度包括以下的步骤S2431-步骤S2433。
步骤S2431：确定相关待校验实体所包括的实体的个数。
例如，在待校验实体(也即，“血小板计X”)对应的相关待校验实体仅包括“PLT”情况下，可以确定相关待校验实体所包括的实体的个数为1。又例如，在待校验实体(也即，“血小板计X”)对应的相关待校验实体包括“PLT”和“100-300”的情况下，可以确定相关待校验实体所包括的实体的个数为2。
步骤S2432：在相关待校验实体所包括的实体的个数等于1的情况下，将相关待校验实体所包括的实体与相关候选实体的相似度作为至少两个候选实体的每个的相关相似度。
例如，在待校验实体(也即，“血小板计X”)对应的相关待校验实体仅包括“PLT”情况下，可以将相关待校验实体“PLT”与候选实体“血小板计数”对应的相关候选实体“PLT的相似度(也即，1)作为候选实体“血小板计数”的相关相似度；并且，可以将相关待校验实体“PLT”与候选实体“血小板压积”对应的相关候选实体“PCT”的相似度(0.65)作为候选实体“血小板压积”的相关相似度。
步骤S2433：在相关待校验实体所包括的实体的个数大于1的情况下，将相关待校验实体所包括的实体与相关候选实体的之间的多个相似度的加权和作为至少两个候选实体的每个的相关相似度。
例如，在相关待校验实体所包括的实体的个数大于1的情况下，可以针对相关待校验实体所包括的每个实体设置权重。例如，在待校验实体(也即，“血小板计X”)对应的相关待校验实体包括“PLT”和“100-300”的情况下，可以将子项目缩写(例如，“PLT”)的权重设置为0.7，并将项目参考值(例如，“100-300”)设置为0.3。
例如，候选实体“血小板计数”的相关相似度＝0.7×1+0.3×1＝1，也即，候选实体“血小板计数”的相关相似＝0.7(子项目缩写的权重)×1(相关待校验实体“PLT”与候选实体“血小板计数”对应的相关候选实体“PLT”的相似度)+0.3(项 目参考值的权重)×1(相关待校验实体“100-300”与候选实体“血小板计数”对应的相关候选实体“100-300”的相似度)。
例如，候选实体“血小板压积”的相关相似度＝0.7×0.65+0.3×0＝0.455，也即，候选实体“血小板压积”的相关相似度＝0.7(子项目缩写的权重)×0.65(相关待校验实体“PLT”与候选实体“血小板压积”对应的相关候选实体“PCT”的相似度)+0.3(项目参考值的权重)×0(相关待校验实体“100-300”与候选实体“血小板压积”对应的相关候选实体“0.17-0.35”的相似度)。
步骤S244：基于至少两个候选实体的每个的相关相似度以及实体相似度获取至少两个候选实体的每个的候选概率。
例如，在步骤S244中，基于至少两个候选实体的每个的相关相似度以及实体相似度获取至少两个候选实体的每个的候选概率包括：将至少两个候选实体的每个的相关相似度和实体相似度的加权和作为至少两个候选实体的每个的候选概率。
例如，候选实体的候选概率＝候选实体的相关相似度×候选实体的相关相似度的权重+候选实体的实体相似度×候选实体的实体相似度的权重。例如，相关相似度和实体相似度的权重可以根据实际应用需求进行设定。例如，相关相似度和实体相似度的权重可以分别设置在0.6和0.4。
例如，在候选实体“血小板计数”和候选实体“血小板压积”的实体相似度分别为0.8和0.6，且候选实体“血小板计数”和候选实体“血小板压积”的相关相似度分别为1和0.455的情况下，候选实体“血小板计数”的候选概率＝1×0.6+0.8×0.4＝0.92；候选实体“血小板压积”的候选概率＝0.455×0.6+0.6×0.4＝0.513。
在另一些示例中，还可以将至少两个候选实体的每个的相关相似度和实体相似度的乘积作为至少两个候选实体的每个的候选概率。此种情况下，候选实体“血小板计数”的候选概率＝1×0.8＝0.8；候选实体“血小板压积”的候选概率＝0.455×0.6＝0.273。
步骤S233：在第一判定结果为第一判定子结果且第二判定结果为第四判定子结果的情况下，利用下一级待校验实体计算至少两个候选实体的每个的候选概率。
例如，在选择“项目名X”作为待校验实体的情况下，第一判定结果为第一判定子结果；第二判定结果为第四判定子结果(也即，待校验实体“项 目名X”具有下一级待校验实体，但是不对应于相关待校验实体)，可以利用下一级待校验实体计算至少两个候选实体的每个的候选概率。例如，“项目名X”的候选实体包括“项目名称”和“项目参考值”。
例如，利用下一级待校验实体计算至少两个候选实体的每个的候选概率包括以下的步骤S251-步骤S255。
步骤S251：获取待校验实体包括的下一级待校验实体。
例如，待校验实体“项目名X”的下一级待校验实体包括“血小板计数”和“血小板压积”等。需要说明的是，为清楚起见，此处假设待校验实体“项目名X”的下一级待校验实体仅包括“血小板计数”和“血小板压积”。
步骤S252：基于待校验实体与下一级待校验实体中每个实体的关系将下一级待校验实体划分为至少一组下一级待校验实体组。
例如，由于下一级待校验实体包括“血小板计数”和“血小板压积”与待校验实体“项目名X”具有相同的关系，因此，此处仅划分为一组下一级待校验实体组，也即，下一级待校验实体组包括的实体为“血小板计数”和“血小板压积”。
步骤S253：针对至少一组下一级待校验实体组的每组执行以下的步骤S2531-步骤S25312。
步骤S2531：获取下一级候选实体组。下一级候选实体组为隶属于至少两个候选实体的每个的、且与至少两个候选实体的每个的关系等于下一级待校验实体组所包括的实体的每个与待校验实体之间关系的所有的实体。
例如，候选实体“项目名称”包括的下一级候选实体组包括“血小板计数”、“血小板压积”、“血小板分布宽度”等实体；候选实体“项目参考值”包括的下一级候选实体组包括“100-300”、“0.17-0.35”、“9-17”等。
步骤S2532：计算至少一组下一级待校验实体组的每组与下一级候选实体组的相似度，以获取至少一组下一级待校验实体组对应的至少一个组相似度。
例如，在步骤S2532中，获取至少一组下一级待校验实体组对应的至少一个组相似度包括：
步骤310：获取至少一组下一级待校验实体组的每组包括的多个实体的每个的最大相似度，以获取至少一组下一级待校验实体组的每组包括的多个实体的多个最大相似度。此处，最大相似度为至少一组下一级待校验实体组的每组包括的多个实体的每个与下一级候选实体组所包括的所有实体的相似度 中取值最大的相似度。
例如，下一级待校验实体组包括的实体“血小板计数”与候选实体“项目名称”的下一级候选实体组的所有实体(例如，“血小板计数”、“血小板压积”、“血小板分布宽度”)的相似度分别为1、0.6和0.5、因此，下一级待校验实体组包括的实体“血小板计数”与候选实体“项目名称”的下一级候选实体组的所有实体的相似度中取值最大的相似度(也即，针对候选实体“项目名称”，下一级待校验实体组包括的实体“血小板计数”的最大相似度)为1。
例如，下一级待校验实体组包括的实体“血小板压积”与候选实体“项目名称”的下一级候选实体组的所有实体(例如，“血小板计数”、“血小板压积”、“血小板分布宽度”)的相似度分别为0.6、1和0.5、因此，下一级待校验实体组包括的实体“血小板压积”与候选实体“项目名称”的下一级候选实体组的所有实体的相似度中取值最大的相似度(也即，针对候选实体“项目名称”，下一级待校验实体组包括的实体“血小板压积”的最大相似度)为1。
因此，针对候选实体“项目名称”，下一级待校验实体组包括的多个实体(也即，“血小板计数”和“血小板压积”)的多个最大相似度分别为1和1。
例如，下一级待校验实体组包括的实体“血小板计数”与候选实体“项目参考值”的下一级候选实体组的所有实体的相似度中取值最大的相似度(也即，针对候选实体“项目参考值”，下一级待校验实体组包括的实体“血小板计数”的最大相似度)为0；下一级待校验实体组包括的实体“血小板压积”与候选实体“项目参考值”的下一级候选实体组的所有实体的相似度中取值最大的相似度(也即，针对候选实体“项目参考值”，下一级待校验实体组包括的实体“血小板压积”的最大相似度)为0；此种情况下，针对候选实体“项目参考值”，下一级待校验实体组包括的多个实体(也即，“血小板计数”和“血小板压积”)的多个最大相似度分别为0和0。
步骤320：基于多个最大相似度获取至少一组下一级待校验实体组对应的至少一个组相似度。
例如，基于多个最大相似度获取至少一组下一级待校验实体组对应的至少一个组相似度包括：将多个最大相似度的加权和作为至少一组下一级待校验实体组对应的至少一个组相似度。
例如，下一级待校验实体组包括的多个实体的权重可以根据实际应用需求进行设定。例如，下一级待校验实体组包括的多个实体的权重均等于1/多个实体的数目。例如，下一级待校验实体组的实体“血小板计数”和“血小板压积”的权重均等于1/2。
例如，针对候选实体“项目名称”，下一级待校验实体组(包括实体“血小板计数”和“血小板压积”)的组相似度＝0.5×1+0.5×1＝1。例如，针对候选实体“项目参考值”，下一级待校验实体组(包括实体“血小板计数”和“血小板压积”)的组相似度＝0.5×0+0.5×0＝0。
步骤S254：基于至少一个组相似度获取至少两个候选实体的每个的下一级相似度。
例如，在步骤S254中，基于至少一个组相似度获取至少两个候选实体的每个的下一级相似度包括：确定下一级待校验实体包括的下一级待校验实体组的个数；在下一级待校验实体组的个数等于1的情况下，将下一级待校验实体包括的下一级待校验实体组的组相似度作为至少两个候选实体的每个的下一级相似度；在下一级待校验实体组的个数大于1的情况下，将下一级待校验实体包括的下一级待校验实体组的多个组相似度的加权和作为至少两个候选实体的每个的下一级相似度。
例如，由于“项目名X”的下一级待校验实体包括的“血小板计数”和“血小板压积”与待校验实体“项目名X”均具有相同的关系，也即，“项目名X”的下一级待校验实体组的个数等于1，因此，候选实体“项目名称”下一级相似度和候选实体“项目参考值”的下一级相似度分别为1和0。
步骤S255：基于至少两个候选实体的每个的下一级相似度和实体相似度的计算至少两个候选实体的每个的候选概率。
例如，在步骤S255中，基于至少两个候选实体的每个的下一级相似度和实体相似度的计算至少两个候选实体的每个的候选概率包括：将基于至少两个候选实体的每个的下一级相似度和实体相似度的乘积作为至少两个候选实体的每个的候选概率。
例如，在“项目名称”和“项目参考值”的实体相似度分别为0.75和0.3，且“项目名称”和“项目参考值”的下一级相似度分别为1和0的情况下，候选实体“项目名称”的候选概率＝0.75×1＝0.75，“项目参考值”的候选概率＝0.3×0＝0。
步骤S231：在第一判定结果为第一判定子结果且第二判定结果为第三判定子结果的情况下，利用下一级待校验实体和/或相关待校验实体计算至少两个候选实体的每个的候选概率。
例如，在第一判定结果为第一判定子结果且第二判定结果为第三判定子结果的情况下，可以利用下一级待校验实体计算至少两个候选实体的每个的候选概率或者利用相关待校验实体计算至少两个候选实体的每个的候选概率。又例如，在第一判定结果为第一判定子结果且第二判定结果为第三判定子结果的情况下，可以利用下一级待校验实体和相关待校验实体计算至少两个候选实体的每个的候选概率。
例如，利用下一级待校验实体和相关待校验实体计算至少两个候选实体的每个的候选概率包括以上的步骤S241-步骤S244以及步骤S251-步骤S255。
例如，利用下一级待校验实体和相关待校验实体计算至少两个候选实体的每个的候选概率还包括：基于至少两个候选实体的每个的相关相似度、下一级相似度和实体相似度获取至少两个候选实体的每个的候选概率。
例如，基于至少两个候选实体的每个的相关相似度、下一级相似度和实体相似度获取至少两个候选实体的每个的候选概率包括：将至少两个候选实体的每个的相关相似度与实体相似度的加权和与下一级相似度的乘积作为至少两个候选实体的每个的候选概率。例如，在候选实体的相关相似度、下一级相似度和实体相似度分别为0.9、1和0.8，且相关相似度和实体相似度的权重分别为0.6和0.4的情况下，候选实体的相关相似度＝(0.6×0.9+0.4×0.8)×1＝0.86。
又例如，基于至少两个候选实体的每个的相关相似度、下一级相似度和实体相似度获取至少两个候选实体的每个的候选概率包括：将至少两个候选实体的每个的相关相似度、下一级相似度和实体相似度的乘积作为至少两个候选实体的每个的候选概率。例如，在候选实体的相关相似度、下一级相似度和实体相似度分别为0.9、1和0.8的情况下，候选实体的相关相似度＝0.9×1×0.8＝0.72。
步骤S234：在第一判定结果为第二判定子结果且第二判定结果为第四判定子结果的情况下，将至少两个候选实体的每个的实体相似度作为至少两个候选实体的每个为存在错误的待校验实体的概率。
例如，在步骤S234中，如果待校验实体既没有下一级待校验实体，又没 有相关待校验实体，则可以将待校验实体的至少两个候选实体的每个的实体相似度作为至少两个候选实体的每个为存在错误的待校验实体的概率。
例如，在步骤S220中，基于知识图谱对存在错误的字符信息进行校正还包括：在实体数目大于1的情况下，执行以下的步骤S223。
步骤S223：将存在错误的待校验实体对应的字符信息替换为对应于最大候选概率的候选实体对应的字符信息。
例如，在确认待校验实体“血小板计X”的候选实体“血小板计数”的候选概率为0.8，且待校验实体“血小板计X”的候选实体“血小板压积”的候选概率为0.273的情况下，可以将待校验实体“血小板计X”对应的字符信息替换为对应于最大候选概率(也即，0.8)的候选实体(也即，“血小板计数”)对应的字符信息。
例如，在确认待校验实体“项目名X”的候选实体“项目名称”的候选概率为0.75，且待校验实体“项目名X”的候选实体“项目参考值”的候选概率为0的情况下，可以将待校验实体“项目名X”对应的字符信息替换为对应于最大候选概率(也即，0.75)的候选实体(也即，“项目名称”)对应的字符信息。
图5A是本公开的至少一个实施例提供的基于知识图谱对至少一项字符信息进行校验的一种示意性流程图。图5B是本公开的至少一个实施例提供的基于知识图谱进行解析的一种示意性流程图。
如图5A和图5B所示，基于知识图谱对至少一项字符信息进行校验包括表格分析(也即，表格结构分析)和实体链指(Entity Linking，例如，实体链指可以将对应的实体相关联或/和将一个实体映射到相关联的实体)。例如，表格分析的结果可用于对从图片提取的至少一项字符信息包括的多个实体进行分级。例如，通过实体链指，可以将从图片提取的至少一项字符信息包括的多个实体与知识图谱中的实体相关联，由此可以确定多个实体在知识图谱中的等级。
例如，如图5A和图5B所示，在完成实体链指之后，可以基于知识图谱对从图片提取的至少一项字符信息进行校验。例如，基于知识图谱对从图片提取的至少一项字符信息进行校验的具体方法可以参见本公开的至少一个实施例提供的图片中字符信息的获取方法，在此不再赘述。例如，在完成校验之后，可以将结果输出(也即，将校验后的字符输出)。
例如，如图5B所示，表格分析可以包括表格识别和分割。例如，可以使用OpenCV提供的程序识别表格，从而能够将对表格进行识别和分割。例如，通过知识图谱确认页面类型为表格。
例如，表格识别和分割包括表格横线检测、表格竖线检测和表格顶点检测。例如，表格横线检测包括横向直线检测、图像二值化和二次中值滤波。例如，可以对图片中的每一行的像素进行检测，并以相邻的平均像素差大于120为标准，识别出分割两条记录的横线，获得横向直线列表。例如，表格竖线检测包括对灰度图片进行检测，然后进行直线检测，最大间距为30个像素，线段长度最小为500个像素，并且为竖直直线。例如，表格顶点检测用于识别每个单元格所在的顶点。
例如，如图5B所示，表格分析还可以包括表格精细重构。例如，表格精细重构包括行识别(无横线)、列识别(无竖线)和跨行识别的至少一个。例如，可以通过文字识别后返回的位置信息进行统计和分析后(行坐标)完成行识别。例如，可以通过文字识别后返回的位置信息进行综合统计分析后(列坐标)完成列识别。例如，可以通过行识别(向上对齐)和列识别(向左对齐)后的位置信息，以行数最多的列中的行数作为表格的最大行数，通过相邻行的同列位置的文字信息，如果没有文字，则定义为文字跨行，将内容整合为一行(上下都没有信息时，以中间为基准调整)。
例如，如图5B所示，基于知识图谱进行解析的还可以包括页码识别、页面类型识别、行重构和文字块重构、页眉页脚识别、图片/文字识别分类的至少一项。例如，页码识别用于利用OCR识别出每张报告的页码，以用于确认和调整报告的顺序为正序，且前后连续。例如，在页面不连续的情况下可以报错。例如，页面类型识别用于识别页面是否包含医学影像。例如，根据版面分析和OCR的结果，可以重构出包含文字信息和版面信息的电子文档，进行行重构和文字块重构，并且将其标准化输出为HOCR(OCR采用的一种文档的格式)格式的文档。例如，页眉页脚识别用于通过相似识别算法实现报告单的页眉页脚识别；例如，相似识别指的是在一份多张的检查报告单中，如果相似文字多次出现，则被分类为页眉页脚。例如，图片/文字识别分类包括基于通用的深度学习框架，采用CTPN(用于检测自然图像中的文本的连接文本提议网络，Detecting Text in Natural Image with Connectionist Text Proposal Network)和/或CRNN(卷积循环神经网络)，用OCR识别开源库(例如， Tesseract数字识别库)进行汉字识别训练。
图6A是本公开的至少一个实施例提供的基于知识图谱对图像中的字符信息进行校验以及添加解释的示意性流程图。
例如，如图6A所示，本公开的至少一个实施例提供的基于知识图谱对图像中的字符信息进行校验以及添加解释包括：图片输入；识别结果输出；解析结果输出；以及图谱验证。
例如，可以利用OCR从图片中提取字符信息，并将字符信息作为识别结果输出；例如，可以对OCR从图片中提取字符信息进行解析，以将图片中的字符信息转换为文本格式的字符信息，并将文本格式的字符信息作为解析结果输出。例如，还可以对图片中表格进行分析，并将表格分析结果用于将解析后字符信息对应的实体与知识图谱中的实体对应。例如，图谱验证包括使用知识图谱对解析结果进行验证，以确认从图片中提取字符信息是否包括存在错误的字符信息。例如，在从图片中提取字符信息不包括存在错误的字符信息的情况下，可以在图谱解释步骤中使用知识图谱对从图片中提取字符信息对应的实体添加说明(例如，子项目说明)。例如，在从图片中提取字符信息包括存在错误的字符信息的情况下，可以在图谱校正步骤中使用知识图谱对存在错误的字符信息进行校正，然后在图谱解释步骤中使用知识图谱对校正后的字符信息对应的实体添加说明。例如，在完成图谱解释步骤之后，可以将添加说明后的字符信息输出。
例如，使用知识图谱对解析结果进行验证，以确认从图片中提取字符信息是否包括存在错误的字符信息，使用知识图谱对存在错误的字符信息进行校正以及使用知识图谱对字符信息对应的实体添加说明的具体方法，例如，可以参见本公开的至少一个实施例提供的图片中字符信息的获取方法以及页面处理方法相关内容，在此不做赘述。
图6B是本公开的至少一个实施例提供的基于知识图谱的病案解析的示意性流程图。
例如，如图6B所示，该基于知识图谱的病案解析包括以下的步骤。首先，获取相关用户的文档(也即，病案)，该文档可以为纸件或电子文件(即图片)，如果为纸件，则将其进行扫描以获得图片；例如，病案包括患者的病历、医学报告等。其次，对图片进行主体(主体区域)检测，并由此可以确定图片的主体(主体区域)的边界，接着可以沿着主体(主体区域)的边界提取主 体(主体区域)。第三，对主体(主体区域)执行图像增强(例如，去噪声，提高对比对等)。第四，对图像增强后的主体区域进行图片识别(也即，提取图片中的字符信息)，以获得识别结果(也即，从图片中提取的字符信息)。第五，对识别结果(也即，从图片中提取的字符信息)进行解析，以将提取的字符信息转换为文本格式。第六，使用知识图谱(例如，普通知识图谱或医学检查检验知识图谱)对解析得到的字符信息进行检查；如果确认解析得到的字符信息不存在错误的字符信息，则将解析得到的字符信息输出；如果确认解析得到的字符信息存在错误的字符信息，则使用医学检查检验知识图谱进行校正，然后将校正后的字符信息输出。
如图6B所示，使用医学检查检验知识图谱进行校正包括：基于待检查项(例如，待校正的字符信息)和检查单的关系、待检查项的名称、待检查项的缩写等确认概率最高的组合，并且利用概率最高的组合对应的字符信息覆盖(替代)存在错误的字符信息。
例如，可以基于图3所示知识图谱中的多个实体(也即，检查单、检查项目、检查子项目、英文缩写、医嘱编码、医嘱名称、内容、诊断、别名等)以及上述多个实体之间的关系对从图片中提取的信息进行校验。例如，可以使用最小编译距离算法获取上述多个实体的至少一个与待校验的内容(待校验的实体)的编译距离(莱文斯坦Levenshtein距离)，以对从图片中提取的信息(例如，字符信息)进行校验(检查和校正)。
例如，可以通过计算分块最小编译距离概率的方法确定存在错误的内容(例如，存在错误的字符信息；例如，“血小板计X”)对应的候选正确内容(例如，候选的字符信息；例如，“血小板压积”)为存在错误的内容在未被错误识别情况下对应的内容(例如，“血小板计数”)的概率，并将对应于最大概率的候选正确内容覆盖存在错误的内容。例如，计算分块最小编译距离概率包括：采用分块计算的方法计算待校验的内容(包括名称，缩写，范围等)与医学检查检验知识图谱中的实体(或者实体和关系)的最小编译距离，并基于上述计算得到的最小编译距离计算概率。
例如，分块最小编辑距离概率计算涉及以下步骤。首先，通过版面分析以及表格语义识别和划分获取信息(例如，检查机构、临床检查项目、临床检查子项目、检查单、检查类型等)；将上述信息对应的实体按照知识图谱中的实体名称和类型进行分类，将每个实体模块(例如，上述信息对应的实体 的至少一个与知识图谱中对应的实体)计算编译距离后选择对应于概率最高组合的内容(例如，字符信息)覆盖存在错误的内容。例如，分块计算是指：检查机构对应的实体只和知识图谱中的检查机构对应的实体做编译距离计算；临床检查项目对应的实体仅与知识图谱中的临床检查项目对应的实体做编译距离计算；检查单名称对应的实体仅与知识图谱中的检查单名称对应的实体做编译距离计算。例如，采用分块最小编辑距离概率算法可以充分利用知识图谱中实体之间的关系(包含信息以及被包含信息等)，可以有效地降低每个实体的最小编辑距离计算的范围(也即，减少运算量)，提高纠错的效率以及纠错后的内容的准确率。
例如，可以采用以下的公式计算概率。
Figure PCTCN2019120634-appb-000003
此处，P(B)为正确内容(例如，正确的字符信息)的概率，A i为检查检验知识图谱中的检查单、检查项目、检查细项、字数、英文缩写等对应的实体的至少一个。
例如，在分块最小编辑距离概率计算中，外科检查(二)对应的实体仅需要与知识图谱中体检项目实体包括的外科检查(二)对应的检查项目的实体进行编辑距离概率计算，不需要与知识图谱中体检机构包括的相关实体进行编译距离概率计算。例如，在对外科检查(二)对应的实体以及知识图谱中体检项目实体包括的外科检查(二)对应的检查项目的实体进行编辑距离概率种，外科检查(二)包括的外科检查子项目“皮肤颜色检查”仅需要与知识图谱中外科检查(二)包括的至少一个外科检查子项目对应的实体(例如，全称、英文缩写、字数等)进行编译距离计算，由此可以确定正确内容。
图7A是一个部分血常规检查单对应的图片。图7B是图7A所示的部分血常规检查单在受到污染之后的图片。
如图7B所示，从受到污染之后的检查单对应的图片中提取的以下字符信息均存在错误：“平均血XX体积”、“大血小板X例”以及“淋巴细胞XX比”。需要说明的是，图7A和图7B的第一列-第七列的字符信息分别对应于“子项目序号”、“子项目名称”、“子项目缩写”、“子项目结果”、“子项目结果异常提示”、“子项目结果单位”和“子项目参考值”。
图7C是采用基于本公开的至少一个实施例提供的图片中字符信息的获 取方法的软件程序获取图7B的图片中字符信息得到的结果。图7C所示的实线框中的字符为从图7B所示的图片获取的字符信息，其中，实线框中的“name”表示检查子项目名称。如图7C所示，采用基于本公开的至少一个实施例提供的图片中字符信息的获取方法的软件程序将均存在错误的字符信息“平均血XX体积”、“大血小板X例”以及“淋巴细胞XX比”分别校正为“平均血小板体积”、“大血小板比例”以及“淋巴细胞百分比”后输出。
需要说明的是，由于图7C中实线框外的字符与本公开的实施例的技术方案无关，因此，此处对实线框外的字符的含义不再赘述。
本公开的至少一个实施例还提供了一种图片中字符信息的获取装置。图8是本公开的至少一个实施例提供的图片中字符信息的获取装置。
例如，如图8所示，该图片中字符信息的获取装置包括字符信息提取装置和检验校正装置。
例如，字符信息提取装置被配置为获取图片并提取图片包括的至少一项字符信息。例如，检验校正装置被配置为基于知识图谱对至少一项字符信息进行校验。
例如，获取图片并提取图片包括的至少一项字符信息的具体方法以及基于知识图谱对至少一项字符信息进行校验的具体方法可以参见本公开的至少一个实施例还提供了一种图片中字符信息的获取方法，在此不做赘述。
例如，字符信息提取装置和检验校正装置可以分别通过软件、固件、硬件及其任意组合实现，例如，该硬件包括现场可编程门阵列(Field Programmable Gate Array，FPGA)等。
例如，通过在图片中字符信息的获取装置中设置检验校正装置，可以基于知识图谱对至少一项字符信息进行校验(例如，检验和校正)，由此可以确认从图片中提取的字符信息是否存在错误以及对存在错误的字符信息进行校正，进而可以提升采用了本公开的至少一个实施例提供的图片中字符信息的获取装置获取的字符信息的准确率。
本公开的至少一个实施例还提供了另一种图片中字符信息的获取装置。图9是本公开的至少一个实施例提供的另一种图片中字符信息的获取装置。例如，如图9所示，该另一种图片中字符信息的获取装置包括：处理器和存储器。存储器中存储有适于处理器执行的计算机程序指令，计算机程序指令被处理器运行时使得处理器执行本公开的至少一个实施例提供的任一图片中 字符信息的获取方法。
例如，该处理器例如是中央处理单元(CPU)、图形处理器GPU、张量处理器(TPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元，例如，该处理器可以实现为通用处理器，并且也可以为单片机、微处理器、数字信号处理器、专用的图像处理芯片、或现场可编程逻辑阵列等。例如，存储器可以包括易失性存储器和非易失性存储器的至少一种，例如存储器可以包括只读存储器(ROM)、硬盘、闪存等。相应地，该存储器可以实现为一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，在所述计算机可读存储介质上可以存储一个或多个计算机程序指令。处理器可以运行所述程序指令，以执行本公开的至少一个实施例提供的任一图片中字符信息的获取方法。该存储器还可以存储其他各种应用程序和各种数据，例如，所述应用程序使用和/或产生的各种数据等。
例如，本公开的至少一个实施例还提供了另一种图片中字符信息的获取装置可以基于知识图谱对至少一项字符信息进行校验(例如，检验和校正)，由此可以确认从图片中提取的字符信息是否存在错误以及对存在错误的字符信息进行校正，进而可以提升采用了本公开的至少一个实施例提供的另一种图片中字符信息的获取装置获取的字符信息的准确率。
本公开的至少一个实施例还提供了一种非暂时性存储介质。图10是本公开的至少一个实施例提供的非暂时性存储介质。例如，如图10所示，该非暂时性存储介质包括存储其上的计算机程序指令。计算机程序指令被处理器运行时使得计算机执行本公开的至少一个实施例提供的任一图片中字符信息的获取方法。
例如，非暂时性存储介质可以包括磁性存储介质、光存储介质、半导体存储介质等；例如，非暂时性存储介质可以包括只读存储器(ROM)、硬盘、闪存等。例如，图10所示的非暂时性存储介质可用于提升从图片中获取的字符信息的准确率。
本公开的至少一个实施例还提供了一种页面处理方法。图11是本公开的至少一个实施例提供的页面处理方法的示例性流程图。例如，如图11所示，该页面处理方法包括以下的步骤S510-步骤S530。
步骤S510：获取页面的图片。
步骤S520：通过执行本公开的至少一个实施例提供的任一获取方法获取 页面的图片的校验后的字符信息。
步骤S530：对页面的图片的校验后的字符信息进行结构化，以获得结构化数据组合。
例如，在步骤S510中，页面可以为医学报告、病历、处方等。例如，医学报告可以包括以下报告的至少一种：检查检验报告、体检报告、住院报告、出院报告等。
例如，结构化数据为关系模型数据，可以存储在关系型数据库中。例如，非结构化数据是没有固定模式的数据，可以存储在非关系型数据库中。例如，结构化数据可以被计算机理解，因此有利于检索以及在用户图形界面展示，由此，通过采用本公开的至少一个实施例提供的页面处理方法从页面的图片获得结构化数据组合，不仅可以提升从页面的图片获取的字符信息的准确度，还能够使得患者的医学检查检验数据可以得到更为有效的利用。例如，“血小板计数”-“值”-“215”是一种结构化数据；例如，“血小板压积”-“值”-“0.2”也是一种结构化数据。例如，可以基于对页面的图片的校验后的字符信息得到多个结构化数据，并将上述多个结构化数据彼此关联，以得到结构化数据组合。
例如，页面处理方法还可以包括以下的步骤S540和步骤S550。
步骤S540：基于知识图谱确认页面的图片包括医学影像。
步骤S550：在结构化数据组合中加入医学影像。
例如，本公开的至少一个实施例还提供了一种页面处理方法可以按照步骤S510、步骤S520、步骤S530、步骤S540和步骤S550的顺序顺次执行。
又例如，本公开的至少一个实施例还提供了一种页面处理方法可以按照步骤S510、步骤S520、步骤S540以及步骤S530+步骤S550(也即，步骤S530和步骤S550同时执行)的顺序执行。
例如，在步骤S540中，基于知识图谱确认页面的图片包括医学影像包括：根据从页面的图片中提取的、基于知识图谱校验后的字符信息(例如，通过步骤S520得到的校验后的字符信息)包括与医学影像对应的文字判定页面的图片包括医学影像。例如，在通过步骤S520得到的校验后的字符信息包括核磁共振图像(NMRI)等文字，且不包括检查子项目(例如，血小板计数)对应的文字，可以判定页面的图片包括医学影像，并可以将医学影像从页面的图片中提取出来，由此本公开的至少一个实施例提供的页面处理方法可以有 效保存更多的医学检查结果。例如，通过使得页面处理方法还可以包括基于知识图谱确认页面的图片包括医学影像，可以使得本公开的至少一个实施例提供的页面处理方法还具有页面类型识别功能。
例如，在步骤S550中，可以将从页面的图片检测到且提取出的医学影像与步骤S530中的到结构化数据相关联，由此可以在更新后的结构化数据组合中加入医学影像。
例如，页面处理方法还可以包括以下的步骤S560。
步骤S560：基于知识图谱以及校验后的字符信息对应的多个实体，在结构化数据组合中添加与多个实体对应的说明。
例如，可以针对上述多个实体的每个，在基于多个实体的每个在知识图谱中的等级，在知识图谱中找到与上述多个实体的每个对应的实体(例如，检查子项目)相关联(检查子项目的医学意义)的检查项目解释和异常解释，并基于该检查项目解释和异常解释在结构化数据组合中添加与多个实体对应的说明。例如，通过在结构化数据组合中添加与多个实体对应的说明，可以使得患者更为了解检查结果的含义，由此使得患者可以更为有效的利用检查结果。
例如，对于实体“血小板计数”可以添加以下的说明：“血小板是人类血液中的一种细胞，血小板计数是指单位体积血液中所含的血小板数目。血小板计数减小常见于减少再生障碍性贫血、急性白血病、放射治疗和化疗、血小板减少性紫癜、脾功能亢进等；血小板计数增加常见于病理性的骨髓增生性疾病、原发性血小板增多症、急性溶血、急性化脓性感染等”。
例如，该页面处理方法还可以包括页码识别，以用于将位于不同页面的结构化数据组合、合并。
本公开的至少一个实施例还提供了一种知识图谱构建方法(例如，医学知识图谱的构建方法)。图12是本公开的至少一个实施例提供的医学知识图谱的构建方法。
例如，如图12所示，该医学知识图谱的构建方法包括以下的步骤S610-步骤S640。
步骤S610：获取医学知识图谱的数据结构。
例如，在步骤S610中，医学知识图谱的数据结构可以采用图3所示的医学知识图谱的数据结构，但本公开的实施例不限于此。例如，医学知识图谱 的数据结构还可以不包括图3所示的医学知识图谱的数据结构中医院信息系统对应的部分和/或临床检查子项目的医学意义对应的部分。例如，构建医学知识图谱的数据结构(医学知识图谱框架)包括定义实体的类型以及实体之间的关系的类型。
步骤S620：至少从医学影像存档与通讯系统和实验室信息管理系统导出对应的实体。
例如，可以人工从医学影像存档与通讯系统和实验室信息管理系统导出对应的实体。又例如，还可以利用医学影像存档与通讯系统和实验室信息管理系统的导出功能自动从医学影像存档与通讯系统和实验室信息管理系统导出对应的实体。
例如，通过使得医学知识图谱包括医学影像存档与通讯(PACS)系统，可以确认图片是否包含医学影像，并且在图片包含医学影像的情况下从页面的图片中提取出来，由此基于本公开的至少一个实施例提供的医学知识图谱的构建方法获取的知识图谱可用于有效保存更多的医学检查结果中。
步骤S630：将从医学影像存档与通讯系统导出的实体和从实验室信息管理系统导出的实体进行实体对齐，以得到实体对齐后的多个实体。
例如，从医学影像存档与通讯系统导出的实体和从实验室信息管理系统导出的实体进行实体对齐用于将医学影像存档与通讯系统导出的实体和从实验室信息管理系统导出的实体中实质相同(含义一致)的实体进行关联。例如，医学影像存档与通讯系统可能使用“缩写”表示“项目名称的缩写”，而实验室信息管理系统可能使用“代码”表示“项目名称的缩写”，此种情况下，可以在步骤S630中将“缩写”和“代码”均与“项目名称的缩写”相关联(或者，将“缩写”和“代码”直接关联)，以使得从医学影像存档与通讯系统导出的实体和从实验室信息管理系统导出的实体实现了实体对齐。
例如，通过将从医学影像存档与通讯系统导出的实体和从实验室信息管理系统导出的实体进行实体对齐，可以使得基于本公开的至少一个实施例提供的医学知识图谱的构建方法获取的知识图谱具有更广泛的适用性。
例如，除了对从医学影像存档与通讯系统导出的实体和从实验室信息管理系统导出的实体进行实体对齐之外，还可以结合医学文献、指南、论文等对从医学影像存档与通讯系统导出的实体和从实验室信息管理系统导出的实体其它处理。例如，其它处理包括数据清洗(例如，去除无关实体)和数据 补充(补充未包含在两个系统中，但重要的实体)。
步骤S640：将对齐后的多个实体基于医学知识图谱的数据结构填充到医学知识图谱中。
例如，在对从医学影像存档与通讯系统导出的实体和从实验室信息管理系统导出的实体进行实体对齐和其它处理的情况下，可以将对齐后且完成其它处理的实体基于医学知识图谱的数据结构填充到医学知识图谱中。
例如，该医学知识图谱的构建方法包括：从医院信息系统导出对应的实体；此种情况下，可以将从医院信息系统导出的实体与实体对齐后的多个实体进行实体对齐；将从医院信息系统导出、且与实体对齐后的多个实体进行对实体对齐的实体基于医学知识图谱的数据结构填充到医学知识图谱中。
例如，在需要从医院信息系统导出对应的实体的情况下，可以在步骤S620中执行从医院信息系统导出对应的实体，并且在步骤S630，同时将从医学影像存档与通讯系统导出的实体，从实验室信息管理系统导出的实体以及从医院信息系统导出对应的实体进行实体对齐。
例如，该医学知识图谱的构建方法还包括：获取实验室信息管理系统涉及的临床检查子项目的子项目说明；以及将子项目说明进行处理后填充到医学知识图谱中。
例如，可以利用爬虫(也即，网络机器人)获取的互联网记载的实验室信息管理系统涉及的临床检查子项目的子项目说明。例如，子项目说明可以包括“红细胞压积偏低，常见于贫血的病人和当患者血液稀释的时候”。
例如，对获取的临床检查子项目的子项目说明的处理包括清洗(也即，去除无关信息)。又例如，对获取的临床检查子项目的子项目说明的处理还可以包括合并(也即，将从多个信息源获取的说明合并)和清洗(也即，去除无关或冗余的信息)。
例如，通过将子项目说明进行处理后填充到医学知识图谱中，可以使得利用了基于本公开的至少一个实施例提供的医学知识图谱的构建方法获取的知识图谱的方法和装置具有向患者呈现临床检查子项目的说明以及临床检查子项目结果存在异常情况下异常代表的含义的潜力，由此使得患者可以更为清楚的了解检查结果含义，并使得患者可以更为有效的利用检查结果。
图13示出了本公开的至少一个实施例提供的图片中字符信息的获取系统的示例性的场景图。如图13所示，该图片中字符信息的获取系统300可以包 括用户终端310、网络320、服务器330以及数据库340。
例如，用户终端310可以是图13中示出的电脑310-1、便携式终端310-2。可以理解的是，用户终端还可以是能够执行数据的接收、处理和显示的任何其他类型的电子设备，其可以包括但不限于台式电脑、笔记本电脑、平板电脑、智能家居设备、可穿戴设备、车载电子设备、医疗电子设备等。
例如，所述网络320可以是单个网络，或至少两个不同网络的组合。例如，网络320可以包括但不限于局域网、广域网、公用网络、专用网络、因特网、移动通信网络等中的一种或几种的组合。
例如，该服务器可330以为一个单独的服务器，或者为一个服务器群组，服务器群组内的各个服务器通过有线网络或无线网络进行连接。有线网络例如可以采用双绞线、同轴电缆或光纤传输等方式进行通信，无线网络例如可以采用3G/4G/5G移动通信网络、蓝牙、Zigbee或者WiFi等通信方式。本公开对网络的类型和功能在此不作限制。该一个服务器群组可以是集中式的，例如数据中心，也可以是分布式的。服务器可以是本地的或远程的。例如，该服务器330可以为通用型服务器或专用型服务器，可以为虚拟服务器或云服务器等。
例如，数据库340可用于存储从用户终端310和服务器330工作中所利用、产生和输出的各种数据。数据库340可以经由网络320与服务器330或服务器330的一部分相互连接或通信，或直接与服务器330相互连接或通信，或者经由上述两种方式的结合实现与服务器330相互连接或通信。在一些实施例中，数据库340可以是独立的设备。在另一些实施例中，数据库340也可以集成在用户终端310和服务器340中的至少一个中。例如，数据库340可以设置在用户终端310上，也可以设置在服务器340上。又例如，数据库340也可以是分布式的，其一部分设置在用户终端310上，另一部分设置在服务器340上。
在一个示例中，用户终端310可以获取图片(例如，利用用户终端的摄像头拍摄纸医学检查报告单取的图片)，并将获取的图片经由网络320或其它技术(例如，蓝牙通信、红外通信等)发送至服务器330；服务器330可以从数据库获取知识图谱的信息，并执行本公开的至少一个实施例提供的图片中字符信息的获取方法和页面处理方法，以得到从上述图片提取的校验后的字符信息以及对校验后的字符信息进行结构化得到的结构化数据组合；然后， 服务器330可以将上述结构化数据组合存储在数据库340中，并将上述结构化数据组合发送给用户终端310；最后，用户终端显示(例如，通过预先设计的图形用户界面显示)上述结构化数据组合。
在一些实现方式中，服务器330可以利用服务器内置的应用程序执行本公开的至少一个实施例提供的图片中字符信息的获取方法和页面处理方法。在另一些实现方式中，服务器330可以通过调用服务器外部存储的应用程序执行本公开的至少一个实施例提供的图片中字符信息的获取方法和页面处理方法。
在另一个示例中，用户终端310可以获取图片(例如，利用用户终端的摄像头拍摄纸医学检查报告单获取的图片)，从图片中提取字符信息，并将提取得到的字符信息经由网络320或其它技术(例如，蓝牙通信、红外通信等)发送至服务器330；服务器330可以从数据库获取知识图谱的信息，并执行本公开的至少一个实施例提供的图片中字符信息的获取方法的部分和页面处理方法，以对接收到的字符信息进行校验以及对校验后的字符信息进行结构化得到结构化数据组合；然后，服务器330可以将上述结构化数据组合存储在数据库340中，并将上述结构化数据组合发送给用户终端310；最后，用户终端显示(例如，通过预先设计的图形用户界面显示)上述结构化数据组合。在一些实现方式中，用户终端可以利用用户终端内置的应用程序提取图片中的字符信息。在另一些实现方式中，用户终端可以通过调用用户终端外部存储的应用程序提取图片中的字符信息。
根据本申请实施例的方法、装置或系统也可以借助于图14所示的计算设备400的架构来实现。
图14示出了该计算设备400的架构。如图14所示，计算设备400可以包括总线410、一个或至少两个CPU 420、只读存储器(ROM)430、随机存取存储器(RAM)440、连接到网络的通信端口450、输入/输出组件460、硬盘470等。计算设备400中的存储设备(例如，ROM 430或硬盘470)可以存储本公开的至少一个实施例提供的图片中字符信息的获取方法、页面处理方法、医学知识图谱的构建方法中至少一个对应的指令以及各种相关的数据或文件。计算设备400还可以包括人机用户界面480。当然，图14所示的架构只是示例性的，在实现不同的设备时，根据实际需要，可以省略图14示出的计算设备中的一个或至少两个组件。
虽然上文中已经用一般性说明及具体实施方式，对本公开作了详尽的描述，但在本公开实施例基础上，可以对之作一些修改或改进，这对本领域技术人员而言是显而易见的。因此，在不偏离本公开精神的基础上所做的这些修改或改进，均属于本公开要求保护的范围。
以上所述仅是本公开的示范性实施方式，而非用于限制本公开的保护范围，本公开的保护范围由所附的权利要求确定。

Claims (20)

  1. 一种图片中字符信息的获取方法，包括：
    获取图片并且提取所述图片包括的至少一项字符信息；以及
    基于知识图谱对所述至少一项字符信息进行校验。
  2. 根据权利要求1所述的获取方法，其中，所述基于所述知识图谱对至少一项字符信息进行校验包括：
    基于所述知识图谱确认所述至少一项字符信息中存在错误的字符信息；以及
    基于所述知识图谱对所述存在错误的字符信息进行校正。
  3. 根据权利要求2所述的获取方法，其中，所述至少一项字符信息包括多项字符信息；以及
    所述基于所述知识图谱确认所述至少一项字符信息中存在错误的字符信息包括：
    根据所述图片的多项字符信息分别得到多个实体，且从所述多个实体中选择待校验实体以用于确定所述待校验实体对应的字符信息是否存在错误；以及
    根据所述知识图谱的分级结构，确认所述待校验实体对应的字符信息是否存在错误，在所述待校验实体对应的字符信息存在错误的情形下，确认所述待校验实体对应的字符信息为所述存在错误的字符信息。
  4. 根据权利要求3所述的获取方法，其中，所述根据所述知识图谱的分级结构，确认所述待校验实体对应的字符信息是否存在错误包括：
    对所述多个实体按照所述知识图谱的分级结构进行分级；
    确定所述待校验实体在所述知识图谱的分级结构的级别；
    计算所述待校验实体与在所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体的相似度，以获得与所述待校验实体相关的多个实体相似度；以及
    在所述多个实体相似度中的最大值小于预定的实体相似度阈值时，判定所述待校验实体为存在错误的待校验实体以及所述待校验实体对应的字符信息存在错误。
  5. 根据权利要求4所述的获取方法，其中，所述基于所述知识图谱对所述存在错误的字符信息进行校正包括：
    确定所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体的数目，以获得实体数目；
    在所述实体数目等于1的情况下，
    直接将所述存在错误的待校验实体对应的字符信息替换为与所述待校验实体具有相同级别和相同关系的实体对应的字符信息，或者，
    计算与所述待校验实体具有相同级别和相同关系的实体为所述存在错误的待校验实体的概率，以得到实体概率，并在所述实体概率大于预定的实体概率阈值的情况下，将所述存在错误的待校验实体对应的字符信息替换为与所述待校验实体具有相同级别和相同关系的实体对应的字符信息；以及
    在所述实体数目大于1的情况下，执行以下的步骤：
    基于所述多个实体相似度确定至少两个候选实体；
    计算所述至少两个候选实体的每个为所述存在错误的待校验实体的概率，以获取与所述至少两个候选实体的每个的候选概率；以及
    将所述存在错误的待校验实体对应的字符信息替换为对应于最大候选概率的候选实体对应的字符信息。
  6. 根据权利要求5所述的获取方法，其中，所述基于所述多个实体相似度确定至少两个候选实体包括：
    基于所述多个实体相似度对在所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体进行降序排序，选择排在序列最前面的预定数目的实体作为所述至少两个候选实体；或者
    基于所述多个实体相似度对在所述知识图谱中与所述待校验实体具有相同级别和相同关系的所有实体进行升序排序，选择排在序列最后面的预定数目的实体作为所述至少两个候选实体。
  7. 根据权利要求5或6所述的获取方法，
    其中，所述计算所述至少两个候选实体的每个为所述存在错误的待校验实体的概率包括：
    判定所述待校验实体是否包括下一级待校验实体，以获取第一判定结果，其中，所述下一级待校验实体为隶属于所述待校验实体的所 有的下一级实体；
    判定所述待校验实体是否对应于相关待校验实体，以获取第二判定结果，其中，所述相关待校验实体为与所述待校验实体在所述知识图谱的分级结构中位于同一级、与所述待校验实体彼此关联、且与所述待校验实体所属的上一级实体具有不同关系的所有实体；以及
    基于所述第一判定结果和所述第二判定结果选择用于计算所述至少两个候选实体的每个为所述存在错误的待校验实体的概率的方法；
    所述第一判定结果为在判定所述待校验实体包括所述下一级待校验实体的情况下的第一判定子结果或者在判定所述待校验实体不包括所述下一级待校验实体的情况下的第二判定子结果；以及
    所述第二判定结果为在判定所述待校验实体对应于所述相关待校验实体情况下的第三判定子结果或者在判定所述待校验实体不对应于所述相关待校验实体情况下的第四判定子结果。
  8. 根据权利要求7所述的获取方法，其中，所述基于所述第一判定结果和所述第二判定结果选择用于计算所述至少两个候选实体的每个为所述存在错误的待校验实体的概率的方法包括：
    在所述第一判定结果为第一判定子结果且所述第二判定结果为第三判定子结果的情况下，利用所述下一级待校验实体和/或所述相关待校验实体计算所述至少两个候选实体的每个的候选概率；
    在所述第一判定结果为第二判定子结果且所述第二判定结果为第三判定子结果的情况下，利用所述相关待校验实体计算所述至少两个候选实体的每个的候选概率；
    在所述第一判定结果为第一判定子结果且所述第二判定结果为第四判定子结果的情况下，利用所述下一级待校验实体计算所述至少两个候选实体的每个的候选概率；以及
    在所述第一判定结果为第二判定子结果且所述第二判定结果为第四判定子结果的情况下，将所述至少两个候选实体的每个的实体相似度作为所述至少两个候选实体的每个为所述存在错误的待校验实体的概率。
  9. 根据权利要求8所述的获取方法，其中，所述利用所述相关待校验实体计算所述至少两个候选实体的每个的候选概率包括：
    获取所述待校验实体对应的相关待校验实体；
    针对所述相关待校验实体所包括的实体的每个执行以下的方法：
    获取相关候选实体，其中，所述相关候选实体与所述至少两个候选实体的每个在所述知识图谱的分级结构中位于同一级，与所述至少两个候选实体的每个彼此关联，且与所述至少两个候选实体的每个所属的上一级实体之间的关系等于所述相关待校验实体所包括的实体的每个与所述待校验实体之间的关系，
    计算所述相关待校验实体所包括的实体的每个与所述相关候选实体的相似度；
    基于所述相关待校验实体所包括的实体的每个与所述相关候选实体的相似度获取所述至少两个候选实体的每个的相关相似度；以及
    基于所述至少两个候选实体的每个的相关相似度以及实体相似度获取所述至少两个候选实体的每个的候选概率。
  10. 根据权利要求9所述的获取方法，
    其中，所述基于所述相关待校验实体所包括的实体的每个与所述相关候选实体的相似度获取所述至少两个候选实体的每个的相关相似度包括：
    确定所述相关待校验实体所包括的实体的个数；
    在所述相关待校验实体所包括的实体的个数等于1的情况下，将所述相关待校验实体所包括的实体与所述相关候选实体的相似度作为所述至少两个候选实体的每个的相关相似度；
    在所述相关待校验实体所包括的实体的个数大于1的情况下，将所述相关待校验实体所包括的实体与所述相关候选实体的之间的多个相似度的加权和作为所述至少两个候选实体的每个的相关相似度；
    所述基于所述至少两个候选实体的每个的相关相似度以及实体相似度获取所述至少两个候选实体的每个的候选概率包括：将所述至少两个候选实体的每个的相关相似度和实体相似度的加权和作为所述至少两个候选实体的每个的候选概率。
  11. 根据权利要求8所述的获取方法，其中，所述利用所述下一级待校验实体计算所述至少两个候选实体的每个的候选概率包括：
    获取所述待校验实体包括的下一级待校验实体；
    基于所述待校验实体与所述下一级待校验实体中每个实体的关系将所述下一级待校验实体划分为至少一组下一级待校验实体组；
    针对所述至少一组下一级待校验实体组的每组执行以下的方法：
    获取下一级候选实体组，其中，所述下一级候选实体组为隶属于所述至少两个候选实体的每个的、且与所述至少两个候选实体的每个的关系等于所述下一级待校验实体组所包括的实体的每个与所述待校验实体之间关系的所有的实体，
    计算所述至少一组下一级待校验实体组的每组与所述下一级候选实体组的相似度，以获取所述至少一组下一级待校验实体组对应的至少一个组相似度；
    基于所述至少一个组相似度获取所述至少两个候选实体的每个的下一级相似度；以及
    基于所述至少两个候选实体的每个的下一级相似度和实体相似度的计算所述至少两个候选实体的每个的候选概率。
  12. 根据权利要求11所述的获取方法，
    其中，所述获取所述至少一组下一级待校验实体组对应的至少一个组相似度包括：
    获取所述至少一组下一级待校验实体组的每组包括的多个实体的每个的最大相似度，以获取所述至少一组下一级待校验实体组的每组包括的多个实体的多个最大相似度，其中，所述最大相似度为所述至少一组下一级待校验实体组的每组包括的多个实体的每个与所述下一级候选实体组所包括的所有实体的相似度中取值最大的相似度；以及
    基于所述多个最大相似度获取所述至少一组下一级待校验实体组对应的至少一个组相似度；
    基于所述多个最大相似度获取所述至少一组下一级待校验实体组对应的至少一个组相似度包括：将所述多个最大相似度的加权和作为所述至少一组下一级待校验实体组对应的至少一个组相似度。
  13. 根据权利要求11或12所述的获取方法，
    其中，所述基于所述至少一个组相似度获取所述至少两个候选实体的每个的下一级相似度包括：
    确定所述下一级待校验实体包括的下一级待校验实体组的个数，
    在所述下一级待校验实体组的个数等于1的情况下，将所述下一级待校验实体包括的下一级待校验实体组的组相似度作为所述至少两个候 选实体的每个的下一级相似度，
    在所述下一级待校验实体组的个数大于1的情况下，将所述下一级待校验实体包括的下一级待校验实体组的多个组相似度的加权和作为所述至少两个候选实体的每个的下一级相似度；
    所述基于所述至少两个候选实体的每个的下一级相似度和实体相似度的计算所述至少两个候选实体的每个的候选概率包括：将所述基于所述至少两个候选实体的每个的下一级相似度和实体相似度的乘积作为所述至少两个候选实体的每个的候选概率。
  14. 一种图片中字符信息的获取装置，包括：字符信息提取装置和检验校正装置，
    其中，所述字符信息提取装置被配置为获取图片并提取所述图片包括的至少一项字符信息；以及
    所述检验校正装置被配置为基于知识图谱对所述至少一项字符信息进行校验。
  15. 一种非暂时性存储介质，包括存储其上的计算机程序指令，
    其中，所述计算机程序指令被处理器运行时使得计算机执行如权利要求1-13任一项所述的获取方法。
  16. 一种页面处理方法，包括：
    获取页面的图片；以及
    通过执行如权利要求1-13任一所述的获取方法获取所述页面的图片的校验后的字符信息；以及
    对所述页面的图片的校验后的字符信息进行结构化，以获得结构化数据组合。
  17. 根据权利要求16所述的页面处理方法，还包括：
    基于所述知识图谱确认所述页面的图片包括医学影像；以及
    在所述结构化数据组合中加入所述医学影像。
  18. 根据权利要求16或17所述的页面处理方法，还包括：
    基于所述知识图谱以及所述校验后的字符信息对应的多个实体，在所述结构化数据组合中添加与所述多个实体对应的说明。
  19. 一种知识图谱构建方法，包括：
    获取医学知识图谱的数据结构；
    至少从医学影像存档与通讯系统和实验室信息管理系统导出对应的实体；
    将从所述医学影像存档与通讯系统导出的实体和从所述实验室信息管理系统导出的实体进行实体对齐，以得到实体对齐后的多个实体；以及
    将所述对齐后的多个实体基于所述医学知识图谱的数据结构填充到所述医学知识图谱中。
  20. 根据权利要求19所述的知识图谱构建方法，还包括：
    获取所述实验室信息管理系统涉及的临床检查子项目的子项目说明，以及将所述子项目说明进行处理后填充到所述医学知识图谱中；或/和
    从医院信息系统导出对应的实体，将从所述医院信息系统导出的实体与所述实体对齐后的多个实体进行实体对齐，以及将从所述医院信息系统导出、且与所述实体对齐后的多个实体进行实体对齐的实体基于所述医学知识图谱的数据结构填充到所述医学知识图谱中。
PCT/CN2019/120634 2019-11-25 2019-11-25 字符获取、页面处理与知识图谱构建方法及装置、介质 Ceased WO2021102632A1 (zh)

Priority Applications (6)

Application Number Priority Date Filing Date Title
PCT/CN2019/120634 WO2021102632A1 (zh) 2019-11-25 2019-11-25 字符获取、页面处理与知识图谱构建方法及装置、介质
JP2022504256A JP2023510667A (ja) 2019-11-25 2019-11-25 キャラクタ取得、ページ処理と知識グラフ構築方法及び装置、媒体
EP19945445.5A EP4068121B1 (en) 2019-11-25 2019-11-25 Method and apparatus for acquiring character, page processing method, method for constructing knowledge graph, and medium
CN201980002570.5A CN113260988B (zh) 2019-11-25 2019-11-25 字符获取、页面处理与知识图谱构建方法及装置、介质
US16/982,185 US12131809B2 (en) 2019-11-25 2019-11-25 Character acquisition, page processing and knowledge graph construction method and device, medium
US18/885,106 US20250006320A1 (en) 2019-11-25 2024-09-13 Character acquisition, page processing and knowledge graph construction method and device, medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/120634 WO2021102632A1 (zh) 2019-11-25 2019-11-25 字符获取、页面处理与知识图谱构建方法及装置、介质

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US16/982,185 A-371-Of-International US12131809B2 (en) 2019-11-25 2019-11-25 Character acquisition, page processing and knowledge graph construction method and device, medium
US18/885,106 Continuation US20250006320A1 (en) 2019-11-25 2024-09-13 Character acquisition, page processing and knowledge graph construction method and device, medium

Publications (1)

Publication Number Publication Date
WO2021102632A1 true WO2021102632A1 (zh) 2021-06-03

Family

ID=76128989

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/120634 Ceased WO2021102632A1 (zh) 2019-11-25 2019-11-25 字符获取、页面处理与知识图谱构建方法及装置、介质

Country Status (5)

Country Link
US (2) US12131809B2 (zh)
EP (1) EP4068121B1 (zh)
JP (1) JP2023510667A (zh)
CN (1) CN113260988B (zh)
WO (1) WO2021102632A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114117082A (zh) * 2022-01-28 2022-03-01 北京欧应信息技术有限公司 用于对待校正数据校正的方法、设备和介质
CN115374287A (zh) * 2022-04-19 2022-11-22 北京中科凡语科技有限公司 一种实体的链接方法、装置和存储介质
US20230055391A1 (en) * 2021-07-26 2023-02-23 Tata Consultancy Services Limited Method and system for table structure recognition via deep spatial association of words
CN119939120A (zh) * 2025-01-03 2025-05-06 武汉爱迪科技股份有限公司 一种基于多模态学习的档案信息提取管理方法及系统
CN120015356A (zh) * 2025-04-21 2025-05-16 杭州瑞检软件科技有限公司 一种用于整合多源医学数据集的集成方法及系统
CN120144780A (zh) * 2025-02-26 2025-06-13 中国地质科学院矿产资源研究所 矿产知识图谱的社区发现方法、装置及存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117556034B (zh) * 2023-11-14 2024-07-30 生命奇点(北京)科技有限公司 一种对电子病历问答模型输出结果标准化的数据处理系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060210171A1 (en) * 2005-03-16 2006-09-21 Kabushiki Kaisha Toshiba Image processing apparatus
CN108073569A (zh) * 2017-06-21 2018-05-25 北京华宇元典信息服务有限公司 一种基于多层级多维度语义理解的法律认知方法、装置和介质
CN109284396A (zh) * 2018-09-27 2019-01-29 北京大学深圳研究生院 医学知识图谱构建方法、装置、服务器及存储介质
CN109918640A (zh) * 2018-12-22 2019-06-21 浙江工商大学 一种基于知识图谱的中文文本校对方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0528323A (ja) * 1991-07-18 1993-02-05 Oki Electric Ind Co Ltd 文字認識装置
JP4660504B2 (ja) * 2007-05-22 2011-03-30 株式会社東芝 テキスト処理装置及びプログラム
US8560300B2 (en) * 2009-09-09 2013-10-15 International Business Machines Corporation Error correction using fact repositories
US20120065997A1 (en) * 2010-09-09 2012-03-15 Siemens Medical Solutions Usa, Inc. Automatic Processing of Handwritten Physician Orders
KR101718159B1 (ko) 2014-05-12 2017-03-22 연세대학교 산학협력단 의료 영상으로부터 관심영역(roi) 값을 추출하는 방법 및 이 방법을 수행하는 프로그램이 기록된 컴퓨터 판독 가능한 기록 매체
CN104268814A (zh) 2014-10-11 2015-01-07 北京网秦天下科技有限公司 生成电子病历的方法和装置
CN106021281A (zh) * 2016-04-29 2016-10-12 京东方科技集团股份有限公司 医学知识图谱的构建方法、其装置及其查询方法
AU2016424629A1 (en) * 2016-09-30 2019-04-11 Rovi Guides, Inc. Systems and methods for correcting errors in caption text
CN109614501B (zh) * 2018-12-13 2020-09-25 浙江工商大学 一种基于知识图谱的工业隐患规范化上报方法及系统
CN110059195A (zh) 2019-04-10 2019-07-26 华侨大学 一种基于lis的医学检验知识图谱构建方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060210171A1 (en) * 2005-03-16 2006-09-21 Kabushiki Kaisha Toshiba Image processing apparatus
CN108073569A (zh) * 2017-06-21 2018-05-25 北京华宇元典信息服务有限公司 一种基于多层级多维度语义理解的法律认知方法、装置和介质
CN109284396A (zh) * 2018-09-27 2019-01-29 北京大学深圳研究生院 医学知识图谱构建方法、装置、服务器及存储介质
CN109918640A (zh) * 2018-12-22 2019-06-21 浙江工商大学 一种基于知识图谱的中文文本校对方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4068121A4 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230055391A1 (en) * 2021-07-26 2023-02-23 Tata Consultancy Services Limited Method and system for table structure recognition via deep spatial association of words
US11868387B2 (en) * 2021-07-26 2024-01-09 Tata Consultancy Services Limited Method and system for table structure recognition via deep spatial association of words
CN114117082A (zh) * 2022-01-28 2022-03-01 北京欧应信息技术有限公司 用于对待校正数据校正的方法、设备和介质
CN115374287A (zh) * 2022-04-19 2022-11-22 北京中科凡语科技有限公司 一种实体的链接方法、装置和存储介质
CN119939120A (zh) * 2025-01-03 2025-05-06 武汉爱迪科技股份有限公司 一种基于多模态学习的档案信息提取管理方法及系统
CN120144780A (zh) * 2025-02-26 2025-06-13 中国地质科学院矿产资源研究所 矿产知识图谱的社区发现方法、装置及存储介质
CN120015356A (zh) * 2025-04-21 2025-05-16 杭州瑞检软件科技有限公司 一种用于整合多源医学数据集的集成方法及系统

Also Published As

Publication number Publication date
US12131809B2 (en) 2024-10-29
US20250006320A1 (en) 2025-01-02
JP2023510667A (ja) 2023-03-15
EP4068121B1 (en) 2025-07-16
US20220415456A1 (en) 2022-12-29
CN113260988B (zh) 2025-01-24
EP4068121A1 (en) 2022-10-05
EP4068121A4 (en) 2023-04-26
CN113260988A (zh) 2021-08-13

Similar Documents

Publication Publication Date Title
CN113260988B (zh) 字符获取、页面处理与知识图谱构建方法及装置、介质
CN113241135B (zh) 一种基于多模态融合的疾病风险预测方法和系统
US11669965B2 (en) AI-based label generating system and methods for use therewith
JP7392120B2 (ja) 自然言語処理を使用する病理報告内の自動化された情報の抽出及び改良
US20220044812A1 (en) Automated generation of structured patient data record
CN114023414A (zh) 一种体检报告多层次结构录入方法、系统以及存储介质
CN105868526B (zh) 基于鲁棒张量保持的儿童社区获得性肺炎数据处理系统及方法
WO2022222943A1 (zh) 科室推荐方法、装置、电子设备及存储介质
CN113408323A (zh) 表格信息的提取方法、装置、设备及存储介质
US20150149215A1 (en) System and method to detect and visualize finding-specific suggestions and pertinent patient information in radiology workflow
US20240177818A1 (en) Methods and systems for summarizing densely annotated medical reports
CN118116611B (zh) 基于多源医养大数据融合集成的数据库构建方法
CN117954090A (zh) 一种基于多模态缺失数据患者的死亡率预测方法及系统
CN113284572A (zh) 多模态异构的医学数据处理方法及相关装置
US20200293528A1 (en) Systems and methods for automatically generating structured output documents based on structural rules
CN119381009A (zh) 基于结构熵层次知识树的检索增强方法及其系统
Adamo et al. An automatic document processing system for medical data extraction
CN119128067A (zh) 基于知识图谱的肺炎分析与智能信息推荐方法及系统
CN118762370B (zh) 肺结节检验报告单信息提取方法、装置、设备及存储介质
CN121215261A (zh) 一种基于多模态数据融合的妊娠期糖尿病风险预测系统及方法
CN114385845A (zh) 基于图聚类的影像分类管理方法及系统
Alhassan Thresholding Chaotic Butterfly Optimization Algorithm with Gaussian Kernel (TCBOGK) based segmentation and DeTrac deep convolutional neural network for COVID-19 X-ray images
CN114694780A (zh) 用于数据处理的方法、设备和介质
US20240177814A1 (en) Test result processing and standardization across medical testing laboratories
CN118571462A (zh) 一种医学信息提取及骨质疏松智能分析方法和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19945445

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022504256

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019945445

Country of ref document: EP

Effective date: 20220627

WWG Wipo information: grant in national office

Ref document number: 201980002570.5

Country of ref document: CN

WWG Wipo information: grant in national office

Ref document number: 2019945445

Country of ref document: EP