CN109727188A - Image processing method and device, safe driving method and device - Google Patents

Abstract

The present invention provides an image processing method and device, a safe driving method and device thereof, wherein the image processing method includes: acquiring a first image and a second image based on a dual camera device on a vehicle; Perform image fusion to form a target image; identify each object from the target image; wherein, the object is a pedestrian object or a vehicle object. The image processing method forms a target image by image fusion of the images captured by the dual cameras, which can ensure the quality of the target image, thereby recognizing the target image, improving the accuracy of object recognition, and thus ensuring the safety of vehicle driving.

Description

Image processing method and device, safe driving method and device

technical field

The present invention relates to the technical field of vehicle control, and in particular, to an image processing method and device thereof, a safe driving method and device thereof.

Background technique

As the number of vehicles continues to increase, the incidence of traffic accidents also increases. In order to effectively ensure the safety of life and property of the driver and the passengers in the vehicle, vehicle manufacturers are committed to developing more reliable safety assistance systems.

In the prior art, an Advanced Driver Assistant System (ADAS) of a vehicle uses a visual method to collect environmental data outside the vehicle, and then performs object recognition on the collected data. Specifically, a visible light camera device is used to acquire an image outside the vehicle, and then object recognition is performed on the acquired image.

In this way, since the visible light camera can only capture high-quality images in scenes with sufficient light, and in scenes with weak light, the captured images are blurred and noisy. In the process of object recognition from images, the false recognition rate and missed recognition rate of objects are relatively large, which directly affects the driving safety of vehicles.

SUMMARY OF THE INVENTION

The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

Therefore, the first object of the present invention is to propose an image processing method, which can ensure the quality of the target image by performing image fusion on the images captured by the dual cameras to form a target image, so that the target image can be recognized, and the target image can be improved. The accuracy of recognition, thereby ensuring the safety of vehicle driving, is used to solve the problem that the existing ADAS system uses a visible light camera device to obtain images outside the vehicle. Therefore, in the subsequent process of object recognition on the image, the false recognition rate and the missed recognition rate of the object are relatively large, which directly affects the technical problem of the driving safety of the vehicle.

The second object of the present invention is to provide a safe driving method.

The third object of the present invention is to provide an image processing apparatus.

The fourth object of the present invention is to provide a safe driving device.

A fifth object of the present invention is to provide a computer device.

A sixth object of the present invention is to propose a non-transitory computer-readable storage medium.

A seventh object of the present invention is to propose a computer program product.

In order to achieve the above purpose, the embodiment of the first aspect of the present invention proposes an image processing method, including:

acquiring the first image and the second image based on the dual camera device on the vehicle;

performing image fusion on the first image and the second image to form a target image;

Each object is identified from the target image; wherein the object is a pedestrian object or a vehicle object.

The image method of the embodiment of the present invention obtains a first image and a second image based on a dual camera device on a vehicle; performs image fusion on the first image and the second image to form a target image; and identifies each object from the target image. In this embodiment, the target image is formed by image fusion of the images captured by the dual cameras, which can ensure the quality of the target image, thereby identifying the target image, improving the accuracy of object recognition, and thus ensuring the safety of vehicle driving.

In order to achieve the above purpose, a second aspect of the present invention provides a safe driving method, including:

acquiring a target image; wherein, the target image is obtained according to the image processing method described in the embodiment of the first aspect of the present invention;

identify each object from the target image;

According to the identified object and the current running information of the vehicle, a safe driving strategy is generated and executed.

In the safe driving method of the embodiment of the present invention, a target image is acquired, each object is identified from the target image, and then a safe driving strategy is generated and executed according to the identified object and the current operation information of the vehicle, which can effectively ensure the driving safety of the vehicle. safety.

In order to achieve the above-mentioned purpose, an image processing apparatus is provided in an embodiment of the third aspect of the present invention, including:

an image acquisition module for acquiring the first image and the second image based on the dual camera device on the vehicle;

a forming module, configured to perform image fusion on the first image and the second image to form a target image;

A first recognition module for recognizing each object from the target image.

The image processing device according to the embodiment of the present invention obtains a first image and a second image based on a dual camera device on a vehicle; performs image fusion on the first image and the second image to form a target image; and identifies each object from the target image . In this embodiment, the target image is formed by image fusion of the images captured by the dual cameras, which can ensure the quality of the target image, thereby identifying the target image, improving the accuracy of object recognition, and thus ensuring the safety of vehicle driving.

In order to achieve the above purpose, a fourth aspect of the present invention provides a safe driving device, including:

an acquisition module for acquiring a target image; wherein, the target image is obtained according to the image processing method according to any one of claims 1-9;

a second identification module for identifying each object from the target image;

The generating module is configured to generate and execute a safe driving strategy according to the identified object and the current running information of the vehicle.

The safe driving device according to the embodiment of the present invention can obtain a target image, identify each object from the target image, and then generate and execute a safe driving strategy according to the identified object and the current running information of the vehicle, which can effectively ensure the driving safety of the vehicle. safety.

In order to achieve the above object, a fifth aspect of the present invention provides a computer device, including: a processor and a memory;

Wherein, the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the image processing method according to the first method embodiment of the present invention Or the safe driving method described in the embodiment of the second aspect.

In order to achieve the above object, the embodiment of the sixth aspect of the present invention provides a non-transitory computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the first embodiment of the present invention is implemented The image processing method described in the method embodiment or the safe driving method described in the second aspect embodiment.

In order to achieve the above object, an embodiment of the seventh aspect of the present invention provides a computer program product. When an instruction in the computer program product is executed by a processor, the image processing method according to the first method embodiment of the present invention is executed. Or the safe driving method described in the embodiment of the second aspect.

Additional aspects and advantages of the present invention will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a schematic flowchart of a first image processing method provided by an embodiment of the present invention;

2 is a schematic flowchart of a second image processing method provided by an embodiment of the present invention;

3 is a schematic diagram of a calibration template of a dual camera device in an embodiment of the present invention;

4 is a schematic flowchart of a first safe driving method provided by an embodiment of the present invention;

5 is a schematic flowchart of a second safe driving method provided by an embodiment of the present invention;

6 is a schematic flowchart of a third safe driving method provided by an embodiment of the present invention;

7 is a schematic diagram of the position of each object in the target image according to the embodiment of the present invention;

8 is a schematic structural diagram of a safe driving system in an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention;

10 is a schematic structural diagram of another image processing apparatus provided by an embodiment of the present invention;

11 is a schematic structural diagram of a safe driving device according to an embodiment of the present invention;

12 is a schematic structural diagram of another safe driving device provided by an embodiment of the present invention;

FIG. 13 is a schematic structural diagram of a safety assistance system provided by an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to explain the present invention and should not be construed as limiting the present invention.

The image processing method and device thereof, the safe driving method and the device thereof according to the embodiments of the present invention are described below with reference to the accompanying drawings. Before the specific description of the embodiments of the present invention, in order to facilitate understanding, common technical terms are first introduced:

YUV is a color coding method, in which "Y" represents the brightness (Luminance or Luma), that is, the grayscale value; while "U" and "V" represent the chromaticity (Chrominance or Chroma), which are used to describe the color of the image and saturation, which specify the color of the pixel. The characteristic of YUV color space is that its luminance signal Y and chrominance signals U and V are separated. If there is only Y signal component but no U and V components, it indicates that the image is a black and white grayscale image.

Multi-Scale decomposition (MSD) refers to scaling the input image at multiple scales to generate reduced images with multiple resolutions, and then analyze and process the scaled images at each scale. MSD can separate the high and low frequency detail information contained in the image into zoomed images of various scales, and then analyze and process the information in different frequency bands of the image.

FIG. 1 is a schematic flowchart of a first image processing method provided by an embodiment of the present invention.

As shown in Figure 1, the image processing method includes the following steps:

Step 101, acquiring a first image and a second image based on the dual camera device on the vehicle.

In this embodiment of the present invention, two camera devices with different resolutions may be installed side by side on the vehicle, for example, they may be recorded as camera device A and camera device B, wherein, the fields of view captured by camera devices A and B are the same, so that they can be captured by cameras A and B. image for subsequent processing. For example, the camera A may be a visible light camera, and the camera B may be an infrared camera. The resolution of the infrared camera can be lower than that of the visible light camera, and it is relatively insufficient to describe the detailed information in the scene. Therefore, the visible light camera device can choose a high-definition camera device, which can ensure that in the case of sufficient light, visible light camera. The images captured by the device describe the detailed information of the scene relatively clearly, and in the case of weak light, the images captured by the infrared camera device describe the detailed information of the scene relatively clearly.

Thus, the first image may be acquired based on one of the dual cameras on the vehicle and the second image may be acquired based on the other camera. For example, the first image may be acquired based on camera A and the second image may be acquired based on camera B, or the first image may be acquired based on camera B and the second image may be acquired based on camera A.

In this embodiment of the present invention, an example is that the camera device A captures the first image, and the camera device B captures the second image.

Step 102: Perform image fusion on the first image and the second image to form a target image.

In this embodiment of the present invention, since the resolutions of the dual camera devices may be different, before image fusion is performed on the first image and the second image, the resolutions of the first image and the second image need to be adjusted so that the first image and the second image The images are of the same resolution.

For example, based on the resolution of one of the two images, the resolution of the other image can be adjusted so that the resolution of the two images is the same. Alternatively, according to the resolution of the first image and the resolution of the second image, a compromised resolution may be obtained as the target resolution, and then the resolutions of the first image and the second image may be simultaneously adjusted as the target resolution. For example, when the resolution of the first image is 1600*1200 and the resolution of the second image is 1024*768, the target resolution can be 1280*960, and the resolution of the second image of the first image is adjusted to 1280* 960.

It should be noted that although the dual camera devices are installed side by side and have the same field of view, the two images cannot be completely overlapped after the resolution adjustment due to the different positions of the dual camera devices. Therefore, in this embodiment, two images with the same resolution can be registered, and then the registered first image and the second image can be fused to obtain the target image. Specifically, one image can be selected as the reference image, and then another image is subjected to geometric transformation processing according to the reference image, and the processed image is fused with the reference image, so that the two images are completely coincident.

Step 103: Identify each object from the target image; wherein, the object is a pedestrian object or a vehicle object.

Generally speaking, the input image of the camera is a color image, and the color space is YUV. In order to reduce the amount of calculation in the image fusion process, only the Y component in the color space is calculated in the calculation process of the image fusion, and UV components do not participate in the calculation.

In this embodiment, when recognizing the object in the target image, the Y component can be extracted from the fused target image. Regardless of whether the target image is a color image or a black and white image, the process of extracting the Y component is the grayscale of the image. This reduces the amount of computation and improves the real-time performance of the system.

After the Y component is extracted from the target image, that is, after grayscale processing is performed on the target image, the grayscale image of the target image can be obtained. In order to improve the contrast of the grayscale image and the change of the grayscale tone, and make the grayscale image clearer, in this embodiment, a histogram equalization process may be performed on the grayscale image to obtain an equalized grayscale image.

In the embodiment of the present invention, since the identification rules of objects such as pedestrians and vehicles are different, after the equalized grayscale images are obtained, the equalized grayscale images can be divided into at least two channels to form at least two equalized grayscale images. Then, pedestrian recognition can be performed on one of the equalized grayscale images to obtain identification information of pedestrian objects and pedestrian objects, and vehicle identification can be performed on the other equalized grayscale images to obtain vehicle objects and vehicle object identification information. It should be noted that, the above two identification processing processes are performed simultaneously, so as to improve the real-time performance of the system.

The identification information may include coordinate information, width information, height information, distance information, and the like.

As a possible implementation, for the recognition of pedestrian objects, the Laplacian pyramid decomposition algorithm can be used to perform multi-level scaling processing on the equalized grayscale image, and then perform directional gradients on the scaled images at each level. Histogram of Oriented Gradient (referred to as hog in this embodiment) feature extraction, and further classification and identification based on the hog feature can be performed to identify pedestrian objects from objects. For vehicle object recognition, the Laplacian pyramid decomposition algorithm can be used to perform multi-level scaling processing on the equalized grayscale image, and then Haar feature extraction is performed on the scaled image on each level, which can be based on The haar feature is used for classification and recognition, and the vehicle object is recognized from the object.

Further, in order to improve the recognition accuracy of pedestrian objects or vehicle objects, after identifying pedestrian objects or vehicle objects in the target image, a tracking algorithm for pedestrian objects and vehicle objects, such as Kalman filtering algorithm, can be used to detect pedestrian objects and vehicle objects. Vehicle objects are tracked, and misidentified pedestrian objects and vehicle objects are eliminated.

In the image processing method of this embodiment, a first image and a second image are acquired based on a dual camera device on a vehicle; image fusion is performed on the first image and the second image to form a target image; and each object is identified from the target image. In this embodiment, the target image is formed by image fusion of the images captured by the dual cameras, which can ensure the quality of the target image, thereby identifying the target image, improving the accuracy of object recognition, and thus ensuring the safety of vehicle driving.

As a possible implementation manner of the embodiment of the present invention, referring to FIG. 2 , on the basis of the embodiment shown in FIG. 1 , step 102 specifically includes the following sub-steps:

Step 201: Adjust the resolution of the first image and/or the second image so that the resolutions of the two images are the same.

As a possible implementation manner of the embodiment of the present invention, the resolution of one image in the two images may be adjusted based on the resolution of the other image, so that the resolutions of the two images are the same. Specifically, one of the first image and the second image may be selected as a reference image, and then the resolution of the other image may be adjusted according to the resolution of the reference image. For example, when the reference image is the first image, the resolution of the second image can be adjusted so that the resolution of the second image is the same as the resolution of the first image, or when the reference image is the second image, the first image can be adjusted resolution, so that the resolution of the first image is the same as the resolution of the second image.

Preferably, the one with the smaller resolution can be selected from the first image and the second image as the reference image. For example, when the resolution of the first image is lower than that of the second image, the resolution of the first image can be used as the reference image. Referring to the reference image, the second image can then be scaled to reduce the resolution of the second image, so that the resolutions of the two images are the same. Thereby, the calculation amount of the system can be reduced, and the real-time performance of the system can be improved.

As another possible implementation manner of the embodiment of the present invention, a target resolution may be obtained according to the resolution of the first image and the resolution of the second image; and the resolution of the first image and the second image is adjusted simultaneously as the target resolution. For example, when the resolution of the first image is 1600*1200 and the resolution of the second image is 1024*768, the target resolution can be 1280*960, and the resolution of the second image of the first image is adjusted to 1280* 960.

Step 202, register the first image and the second image with the same resolution.

In the embodiment of the present invention, one of the two images with the same resolution can be selected as the reference image, and then another image is subjected to geometric transformation processing according to the reference image, so that the processed image can be well coincident with the reference image. Specifically, a transformation coefficient for performing affine transformation on another image can be obtained according to the reference image, and then affine transformation is performed on the other image according to the transformation coefficient to obtain the registered first image and second image, wherein the transformation The coefficients are obtained by pre-calibrating the dual-camera device.

In this embodiment of the present invention, the first image is used as an example of a reference image, and the camera that captures the first image is camera A. Therefore, geometric transformation processing can be performed on the second image according to the first image captured by the camera device A, so that the processed second image can well coincide with the first image. Specifically, a transformation coefficient for performing radiological transformation on the second image can be obtained according to the first image, and then affine transformation is performed on the second image according to the transformation coefficient to obtain the registered first image and the second image.

In this embodiment, the calibration process of the transform coefficients may be as follows:

The calibration template can be made as shown in FIG. 3 (the calibration template in FIG. 3 is just an example, and can be made according to the actual situation during the specific implementation), and then printed out with paper. Then, place the calibration template directly in front of the dual-camera device, and adjust the distance between the calibration template and the dual-camera device, so that the black rectangles at the four corners above the calibration template all fall into the four corners of the image captured by the dual-camera device. in a corner area. Then, the images captured by the dual-camera device can be collected, and the coordinates of all the vertices of the black rectangular frame at the four corners can be obtained by using the "corner detection" method.

Specifically, the vertex coordinates of all black rectangular frames on the image captured by camera A and the vertex coordinates of the corresponding black rectangular frames on the image captured by camera B can be substituted into the affine as shown in formula (1). In the transformation matrix equation, the formula (2) is then derived.

Wherein, x and y represent the vertex coordinates of the black rectangular frame on the image captured by the camera A, x' and y' represent the vertex coordinates of the black rectangular frame corresponding to the image captured by the camera B and the image captured by the camera A, m ₁ , m ₂ , m ₃ , m ₄ , m ₅ , and m ₆ are transform coefficients of the affine transform.

Among them, k represents the number of vertex coordinates of the black rectangular frame (the number of k in Fig. 3 is 28), x _k and y _k represent the vertex coordinates of the kth black rectangular frame on the image captured by the camera A, and x _k ' and y _k ′ represent the vertex coordinates corresponding to the kth black rectangular frame on the image captured by the camera B and the image captured by the camera A.

Finally, using the least squares method, the transformation coefficients m ₁ , m ₂ , m ₃ , m ₄ , m ₅ and m ₆ of the affine transformation can be obtained by solving.

After the transformation coefficients of the affine transformation are obtained, affine transformation may be performed on the second image captured by the camera B according to the transformation coefficients to obtain the registered first image and the second image.

Step 203 , fuse the registered first image and the second image to obtain a target image.

When the registered first image and the second image are fused, first, a fusion coefficient of the two images needs to be calculated. For example, the MSD method can be used to calculate the fusion coefficients of the registered first image and the second image, and then the target image can be obtained according to the fusion coefficients.

Specifically, multi-scale decomposition can be performed on the registered first image and second image to obtain two sets of multi-scale decomposition coefficients:

Among them, i=1,2,...,n, n represents the number of layers of multi-scale decomposition, represents the multiscale decomposition coefficients of the first image, Represents the multiscale decomposition coefficients of the second image.

After obtaining the two sets of multi-scale decomposition coefficients, the two sets of multi-scale decomposition coefficients can be fused according to the preset fusion rules to obtain the fusion coefficients:

in, represents the fusion coefficient, and θ represents the preset fusion rule.

After getting the fusion coefficient After that, according to the fusion coefficient Perform multi-scale inverse transformation to reconstruct the target image in reverse, as shown in the following formula:

Among them, image _r represents the fused target image.

In the image processing method of this embodiment, the resolution of the first image and/or the second image is adjusted so that the resolutions of the two images are the same, the first image and the second image with the same resolution are registered, and the matching The calibrated first image and the second image are fused to obtain the target image. In this way, a good coincidence of the two images can be achieved, thereby improving the accuracy of image recognition.

FIG. 4 is a schematic flowchart of a first safe driving method provided by an embodiment of the present invention.

As shown in Figure 4, the safe driving method may include the following steps:

Step 301, acquiring a target image.

In this embodiment of the present invention, after image fusion is performed on the first image and the second image in step 102 to form a target image, the target image can be acquired.

Step 302, identify each object from the target image.

Generally speaking, the input image of the camera is a color image, and the color space is YUV. In order to reduce the amount of calculation in the image fusion process, only the Y component in the color space is calculated in the calculation process of the image fusion, and UV components do not participate in the calculation.

In this embodiment, when recognizing the object in the target image, the Y component can be extracted from the fused target image. Regardless of whether the target image is a color image or a black and white image, the process of extracting the Y component is the grayscale of the image. This reduces the amount of computation and improves the real-time performance of the system.

After the Y component is extracted from the target image, that is, after grayscale processing is performed on the target image, the grayscale image of the target image can be obtained. In order to improve the contrast of the grayscale image and the change of the grayscale tone, and make the grayscale image clearer, in this embodiment, a histogram equalization process may be performed on the grayscale image to obtain an equalized grayscale image.

In the embodiment of the present invention, since the identification rules of objects such as pedestrians and vehicles are different, after the equalized grayscale images are obtained, the equalized grayscale images can be divided into at least two channels to form at least two equalized grayscale images. Then, pedestrian recognition can be performed on one of the equalized grayscale images to obtain identification information of pedestrian objects and pedestrian objects, and vehicle identification can be performed on the other equalized grayscale images to obtain vehicle objects and vehicle object identification information. It should be noted that, the above two identification processing processes are performed simultaneously, so as to improve the real-time performance of the system.

The identification information may include coordinate information, width information, height information, distance information, and the like.

As a possible implementation, for the recognition of pedestrian objects, the Laplacian pyramid decomposition algorithm can be used to perform multi-level scaling processing on the equalized grayscale image, and then perform directional gradients on the scaled images at each level. Histogram of Oriented Gradient (referred to as hog in this embodiment) feature extraction, and further classification and identification based on the hog feature can be performed to identify pedestrian objects from objects. For vehicle object recognition, the Laplacian pyramid decomposition algorithm can be used to perform multi-level scaling processing on the equalized grayscale image, and then Haar feature extraction is performed on the scaled image on each level, which can be based on The haar feature is used for classification and recognition, and the vehicle object is recognized from the object.

Further, in order to improve the recognition accuracy of pedestrian objects or vehicle objects, after identifying pedestrian objects or vehicle objects in the target image, a tracking algorithm for pedestrian objects and vehicle objects, such as Kalman filtering algorithm, can be used to detect pedestrian objects and vehicle objects. Vehicle objects are tracked, and misidentified pedestrian objects and vehicle objects are eliminated.

Step 303 , according to the identified objects and the current running information of the vehicle, generate a safe driving strategy and execute it.

In this embodiment of the present invention, the current running information of the vehicle may include: the current running speed (vehicle speed) of the vehicle, the current position of the vehicle, the state of the accelerator pedal and/or the state of the brake pedal, and the like. Specifically, the current running information of the vehicle can be collected through the CAN bus of the vehicle.

It can be understood that when the distance between the object and the vehicle is different, and the current running information of the vehicle is different, the risk level affecting the safe driving of the vehicle is different. For example, when the object is close to the vehicle and the current speed of the vehicle is high, its risk level for safe driving of the vehicle is high, or when the object is far away from the vehicle and the current speed of the vehicle is low, its The risk level for safe driving of the vehicle is low. Therefore, in this embodiment, a safe driving strategy of the vehicle can be generated and executed according to the object and the current running information of the vehicle.

For example, when the object is far away from the vehicle and the current driving speed of the vehicle is low, due to its low risk level to the safe driving of the vehicle, at this time, the driver can be reminded through voice reminders, warning lights, etc. to slow down the driver. However, when the object is close to the vehicle and the current driving speed of the vehicle is high, due to its high risk level for the safe driving of the vehicle, at this time, the steering wheel vibration can be used to prompt the driver, automatic braking, etc., to ensure the safety of the vehicle and the vehicle. Passenger safety.

In the safe driving method of the embodiment of the present invention, a target image is acquired, each object is identified from the target image, and then a safe driving strategy is generated and executed according to the identified object and the current operation information of the vehicle, which can effectively ensure the driving safety of the vehicle. safety.

As a possible implementation manner of the embodiment of the present invention, referring to FIG. 5 , on the basis of the embodiment shown in FIG. 4 , step 303 specifically includes the following sub-steps:

Step 401, from all the objects, determine the object with the closest distance as the first object that is at risk with the vehicle.

It can be understood that when the vehicle is driving, it is the objects located in the lane where the vehicle is located that threatens the driving safety, while the objects in other lanes do not pose a threat to the driving safety of the vehicle. Therefore, in this embodiment, the identification information of the pedestrian object and the identification information of the vehicle object can be screened, and the object located in the lane where the vehicle is located is obtained as the candidate object, and then according to the distance between each candidate object and the vehicle, A first object is identified from all candidate objects.

Step 402 , according to the first object and the operation information, generate and execute a safe driving strategy of the vehicle.

It can be understood that when the distance between the first object and the vehicle is different, and the running information of the vehicle is different, the risk levels affecting the safe driving of the vehicle are different. For example, when the first object is closer to the vehicle and the vehicle speed is higher, the risk level of the first object to the safe driving of the vehicle is higher, or, when the first object is far away from the vehicle and the vehicle speed is lower, It has a low level of risk to the safe driving of the vehicle.

Therefore, in this embodiment, the first distance between the first object and the vehicle can be obtained, and then the running time required for the vehicle to reach the first object can be obtained according to the vehicle speed and the first distance in the running information, and then the running time can be obtained according to the running time. The risk level between the first object and the vehicle is determined, and a safe driving strategy is generated and executed according to the risk level.

For example, when the first object is far away from the vehicle and the vehicle speed is low, the running time required for the vehicle to reach the first object is longer, and at this time, the risk level of the first object to the safe driving of the vehicle is low. Therefore, The driver can be prompted through voice reminders, warning lights, etc., so that the driver can slow down. However, when the first object is closer to the vehicle and the vehicle speed is higher, the running time required for the vehicle to reach the first object is shorter, and at this time, the risk level of the first object to the safe driving of the vehicle is higher. Steering wheel vibration alerts the driver, automatic braking, etc., to ensure the safety of the vehicle and its passengers.

In the safe driving method of this embodiment, the object closest to the vehicle is determined from all the identified objects as the first object that has a risk with the vehicle; according to the first object and the operation information, a safe driving strategy for the vehicle is generated and executed . In this way, the first object that is at risk with the vehicle can be determined, and the safety of the vehicle can be ensured while the processing amount of the system can be reduced, thereby improving the real-time performance of the system.

As a possible implementation manner of the embodiment of the present invention, referring to FIG. 6 , on the basis of the embodiment shown in FIG. 5 , step 401 specifically includes the following sub-steps:

Step 501: Screen according to the identification information of the pedestrian object and the identification information of the vehicle object, and obtain the pedestrian object and the vehicle object that meet the preset conditions as candidate objects.

Since it is the object located in the lane where the vehicle is located that threatens driving safety, in this embodiment of the present invention, the preset condition may be, for example, whether the object is located in the lane where the current vehicle is located.

As a possible implementation manner of the embodiment of the present invention, the edge pixels of each pedestrian object and each vehicle object may be acquired according to the coordinate information in the identification information. Specifically, the coordinate information is the coordinates of the center point of the object, and the connected domain can be identified based on the image information of the center point, so that the boundary of each connected domain can be identified. In this embodiment, edge pixels can be determined from the boundary, and then it is determined whether at least one pixel of the edge pixels is located in the first area, where the first area is the area formed by the lane where the vehicle is located in the target image . When at least one edge pixel is located in the first area, it indicates that the object is located in the first area, that is, in the lane where the vehicle is located. Therefore, all edge pixels can be classified as pedestrian objects or pedestrian objects in the first area. Vehicle objects are used as candidates.

As another possible implementation manner of the embodiment of the present invention, an identification frame may be marked for each pedestrian object and each vehicle object according to the coordinate information in the identification information. Specifically, the coordinate information is the coordinates of the center point of the object, and a recognition frame centered on the coordinate point can be formed according to a preset size, and the recognition frame can be a rectangle. For example, referring to FIG. 7 , the recognition frames 1, 2, 3, and 4 and 5 are different objects, and each recognition box has four boundary points. Then, it can be determined whether each recognition frame has at least one designated boundary point in the first area, wherein the designated boundary points can be the lower left boundary point and the lower right boundary point of the recognition frame. When there is at least one specified boundary point in the first area, it indicates that the object is located in the first area. At this time, the recognition frame with at least one specified boundary point in the first area can be used as the target recognition frame, and then the target The object corresponding to the recognition frame is used as a candidate object.

As an example, referring to FIG. 7 , the lower right boundary point of the recognition frame 2 is located in the first area, the lower left boundary point and the lower right boundary point of the recognition frame 3 are both located in the first area, and the lower left boundary point of the recognition frame 4 is located in the first area. point in the first area, therefore, the recognition boxes 2, 3 and 4 can be used as target recognition boxes.

Step 502: Identify the first object from all the candidate objects according to the distance between each candidate object and the vehicle.

Optionally, the distance between each candidate object and the vehicle may be obtained according to the distance information in the identification information, and then the object with the closest distance may be used as the first object that is at risk with the vehicle.

In the safe driving method of this embodiment, the pedestrian objects and vehicle objects that satisfy the preset conditions are obtained as candidate objects by screening according to the identification information of pedestrian objects and the identification information of vehicle objects. distance to identify the first object from all candidate objects. In this way, the first object that is at risk with the vehicle can be determined, and the safety of the vehicle can be ensured while the processing amount of the system can be reduced, thereby improving the real-time performance of the system.

As an example, referring to FIG. 8 , FIG. 8 is a schematic structural diagram of a safe driving system in an embodiment of the present invention. 8 includes: a camera A2011, a camera B2012, an image processing chip 202, and an execution mechanism 203, wherein the image processing chip 202 includes: an image fusion unit 2021, an image recognition unit 2022, and a system decision unit 2023.

In the prior art, multiple chips are used to complete different processing processes of image data, each individual chip needs to use an exclusive DDR memory, and memory sharing cannot be performed between different chips, resulting in a relatively low transmission of data such as image data and intermediate results. difficulty.

However, in the embodiment of the present invention, the camera A2011 and the camera B2012 are both connected to the image processing chip 202, and the image processing chip 202 may use a SOC chip, and the SOC chip may integrate multiple central processing units (Central Processing Unit, CPU), which can The main frequency of the above-mentioned integrated CPU is divided into low, medium and high grades according to the frequency. Among them, the main frequency of the low grade can be about 200MHz, the main frequency of the middle grade can be 500~700MHz, and the main frequency of the high grade can be 500~700MHz. It can be 1GHz or more. Each CPU can be responsible for different image processing tasks. In addition, different CPUs can share data such as image data and intermediate results through external DDR memory.

After the image recognition unit 2022 recognizes the object, the system decision unit 2023 can generate a safe driving strategy according to the recognition result, and then control the execution mechanism 203 according to the safe driving strategy, wherein the execution mechanism 203 can issue an alarm in the form of sound, light, etc. Reminders, and actions such as controlling steering wheel shake or automatic braking.

In order to realize the above embodiments, the present invention also provides an image processing apparatus.

FIG. 9 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention.

As shown in FIG. 9 , the image processing apparatus 900 includes: an image acquisition module 910 , a formation module 920 , and an identification module 930 . in,

The image acquisition module 910 is configured to acquire the first image and the second image based on the dual camera device on the vehicle.

The forming module 920 is configured to perform image fusion on the first image and the second image to form a target image.

The first recognition module 930 is used to recognize each object from the target image.

Further, in a possible implementation manner of the embodiment of the present invention, referring to FIG. 10 , on the basis of the embodiment shown in FIG. 9 , the image processing apparatus 900 may further include:

In this embodiment of the present invention, the forming module 920 includes:

The adjustment sub-module 921 is configured to adjust the resolution of the first image and/or the second image so that the resolutions of the two images are the same.

In this embodiment of the present invention, the adjustment submodule 921 is specifically configured to select one of the first image and the second image as a reference image; adjust the resolution of the other image according to the resolution of the reference image; or, according to the resolution of the first image The resolution and the resolution of the second image are obtained to obtain a target resolution; at the same time, the resolutions of the first image and the second image are adjusted to be the target resolution.

The registration sub-module 922 is used for registering the first image and the second image with the same resolution.

In this embodiment of the present invention, the registration sub-module 922 is specifically configured to select one of the first image and the second image with the same resolution as a reference image; and obtain a transformation coefficient for performing affine transformation on the other image according to the reference image. wherein, the transformation coefficient is obtained by pre-calibrating the dual camera device; and the other image is affine transformed according to the transformation coefficient to obtain the registered first image and the second image.

As a possible implementation manner, the registration sub-module 922 is specifically configured to perform multi-scale decomposition on the registered first image and the second image respectively, and obtain two sets of multi-scale decomposition coefficients; Two sets of multi-scale decomposition coefficients are fused to obtain fusion coefficients; according to the fusion coefficients, multi-scale inverse transformation is performed to reversely reconstruct the target image.

The fusion sub-module 923 fuses the registered first image and the second image to obtain the target image.

In this embodiment of the present invention, the first identification module 930 includes:

The first processing sub-module 931 is configured to perform grayscale processing on the target image to obtain a grayscale image of the target image.

The second processing sub-module 932 is configured to perform histogram equalization processing on the grayscale image to obtain an equalized grayscale image.

The branching sub-module 933 is used for branching the equalized grayscale images to form at least two equalized grayscale images.

The pedestrian recognition sub-module 934 is used to perform pedestrian recognition on the grayscale image after equalization, and obtain the pedestrian object and the identification information of the pedestrian object.

In the embodiment of the present invention, the pedestrian identification sub-module 934 is specifically configured to use the Laplace pyramid decomposition algorithm to perform multi-level scaling processing on the equalized grayscale image; perform hog feature extraction on the scaled image on each level; Pedestrian objects are identified from objects for classification and recognition based on hog features.

The vehicle identification sub-module 935 is used to perform vehicle identification on another grayscale image after equalization, and obtain identification information of vehicle objects and vehicle objects.

In the embodiment of the present invention, the vehicle identification sub-module 935 is specifically configured to use the Laplace pyramid decomposition algorithm to perform multi-level scaling processing on the equalized grayscale image; perform haar feature extraction on the scaled image on each level; Classification and recognition based on haar features, vehicle objects identified from objects.

The tracking and eliminating sub-module 936 is used for tracking the identified pedestrian objects and vehicle objects, and eliminating the wrongly identified pedestrian objects and vehicle objects.

It should be noted that, the foregoing explanations on the image processing method embodiment are also applicable to the image processing apparatus 900 of this embodiment, and are not repeated here.

The image processing device of this embodiment obtains the first image and the second image based on the dual camera device on the vehicle; performs image fusion on the first image and the second image to form a target image; and identifies each object from the target image. In this embodiment, image fusion is performed on the images captured by the dual cameras to form a target image, which can ensure the quality of the target image, thereby recognizing the target image, improving the accuracy of object recognition, and thus ensuring the safety of vehicle driving.

In order to realize the above embodiments, the present invention also provides a safe driving device.

FIG. 11 is a schematic structural diagram of a safe driving device according to an embodiment of the present invention.

As shown in FIG. 11 , the safe driving device 1100 includes: an acquisition module 1110 , a second identification module 1120 , and a generation module 1130 . in,

The acquiring module 1110 is configured to acquire a target image; wherein, the target image is obtained according to the image processing method of any one of claims 1-9.

The second recognition module 1120 is used to recognize each object from the target image.

In this embodiment of the present invention, the second identification module 1120 is specifically configured to perform grayscale processing on the target image to obtain a grayscale image of the target image; perform histogram equalization processing on the grayscale image to obtain an equalized grayscale image; After equalization, the grayscale images are split to form at least two equalized grayscale images; pedestrian identification is performed on one equalized grayscale image to obtain identification information of pedestrian objects and pedestrian objects; vehicle identification is performed on the other equalized grayscale images. , to obtain vehicle object and vehicle object identification information.

The generating module 1130 is configured to generate and execute a safe driving strategy according to the identified objects and the current running information of the vehicle.

Further, in a possible implementation manner of the embodiment of the present invention, referring to FIG. 12 , on the basis of the embodiment shown in FIG. 11 , the safe driving device 1100 may further include:

In this embodiment of the present invention, the generating module 1130 includes:

A determination sub-module 1131 is configured to determine, from all the identified objects, the object closest to the vehicle as the first object that is at risk with the vehicle.

In this embodiment of the present invention, the determination sub-module 1131 includes:

The obtaining unit 11311 is configured to screen according to the identification information of the pedestrian object and the identification information of the vehicle object, and obtain the pedestrian object and the vehicle object that meet the preset conditions as candidate objects.

As a possible implementation, the acquiring unit 11311 is specifically configured to acquire the edge pixels of each pedestrian object and each vehicle object according to the coordinate information in the identification information; determine whether at least one pixel of the edge pixels is in the first In an area; the first area is the area formed by the lane where the vehicle is located in the target image; the pedestrian objects or vehicle objects whose edge pixels have at least one pixel point in the first area are used as candidate objects.

As another possible implementation, the obtaining unit 11311 is specifically configured to mark a recognition frame for each pedestrian object and each vehicle object according to the coordinate information in the recognition information; determine whether each recognition frame has at least one designated boundary point is in the first area; the first area is the area formed by the lane where the vehicle is located in the target image; the recognition frame with at least one specified boundary point in the first area is used as the target recognition frame; the object corresponding to the target recognition frame is used as the target recognition frame. candidates.

The identifying unit 11312 is configured to identify the first object from all the candidate objects according to the distance between each candidate object and the vehicle.

The generating sub-module 1132 is configured to generate and execute a safe driving strategy of the vehicle according to the first object and the operation information.

In the embodiment of the present invention, the generating sub-module 1132 is specifically used to obtain the first distance between the first object and the vehicle; according to the vehicle speed and the first distance in the operation information, obtain the running time required for the vehicle to reach the first object; Determine the risk level between the first object and the vehicle according to the running time; generate and execute a safe driving strategy according to the risk level.

It should be noted that, the foregoing explanations of the embodiment of the safe driving method are also applicable to the safe driving device 1100 of this embodiment, which will not be repeated here.

The safe driving device according to the embodiment of the present invention can obtain a target image, identify each object from the target image, and then generate and execute a safe driving strategy according to the identified object and the current running information of the vehicle, which can effectively ensure the driving safety of the vehicle. safety.

In order to realize the above embodiments, the present invention also proposes a safety assistance system.

FIG. 13 is a schematic structural diagram of a safety assistance system provided by an embodiment of the present invention.

As shown in FIG. 13 , the safety assistance system 1300 includes: the image processing device described in the foregoing embodiments of FIGS. 9-10 and the safe driving device described in the embodiments of FIGS. 11-12 .

The safety assistance system of this embodiment forms a target image by image fusion of the images captured by the dual cameras, which can ensure the quality of the target image, thereby recognizing the target image and improving the accuracy of object recognition. In addition, according to the first object and operation information that is at risk with the vehicle, the safe driving strategy of the vehicle is generated and executed, which can effectively ensure the safety of the vehicle driving, which is used to solve the problem that the existing ADAS system adopts the visible light camera device to obtain the For the image outside the vehicle, in the scene with weak light, the captured image is relatively blurry and noisy. Therefore, in the subsequent process of object recognition on the image, the false recognition rate and the missed recognition rate of the object are relatively large. Technical issues that directly affect the driving safety of vehicles.

In order to realize the above-mentioned embodiments, the present invention also provides a computer device, comprising: a processor and a memory; wherein, the processor runs and executes the executable program code by reading the executable program code stored in the memory. A corresponding program is used to implement the image processing method or the safe driving method as proposed in the foregoing embodiments of the present invention.

In order to realize the above-mentioned embodiments, the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the image as proposed in the foregoing embodiments of the present invention is realized. How to deal with it or how to drive safely.

In order to realize the above embodiments, the present invention also provides a computer program product, when the instructions in the computer program product are executed by the processor, the image processing method or the safe driving method proposed in the foregoing embodiments of the present invention is executed.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the invention includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present invention belong.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program is stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present invention have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (21)

1. An image processing method, comprising:

acquiring a first image and a second image based on a double-camera device on a vehicle;

carrying out image fusion on the first image and the second image to form a target image;

identifying each object from the target image; wherein the object is a pedestrian object or a vehicular object.

2. The image processing method according to claim 1, wherein the image fusing the first image and the second image to form a target image comprises:

adjusting the resolution of the first image and/or the second image to make the resolution of the two images the same;

registering the first and second images of the same resolution;

and fusing the first image and the second image after registration to obtain the target image.

3. The method according to claim 2, wherein the adjusting the resolution of the first image and/or the second image to make the resolution of the two images the same comprises:

selecting one of the first image and the second image as a reference image;

adjusting the resolution of the other image according to the resolution of the reference image; or,

acquiring a target resolution according to the resolution of the first image and the resolution of the second image;

and simultaneously adjusting the resolution of the first image and the second image to be the target resolution.

4. The image processing method of claim 2, wherein said registering the first and second images of the same resolution comprises:

selecting one of the first image and the second image with the same resolution as a reference image;

acquiring a transformation coefficient for performing affine transformation on another image according to the reference image; the transformation coefficient is obtained by calibrating the double-camera device in advance;

and carrying out affine transformation on the other image according to the transformation coefficient to obtain the first image and the second image after registration.

5. The image processing method according to claim 4, wherein the fusing the registered first image and the second image to obtain the target image comprises:

respectively carrying out multi-scale decomposition on the registered first image and the registered second image to obtain two groups of multi-scale decomposition coefficients;

fusing the two groups of multi-scale decomposition coefficients according to a preset fusion rule to obtain fusion coefficients;

and carrying out multi-scale inverse transformation and reverse reconstruction according to the fusion coefficient to obtain the target image.

6. The image processing method according to any one of claims 1 to 5, wherein the identifying each object from the target image comprises:

carrying out gray level processing on the target image to obtain a gray level image of the target image;

performing histogram equalization processing on the gray level image to obtain an equalized gray level image;

dividing the balanced gray level images to form at least two paths of balanced gray level images;

carrying out pedestrian recognition on one path of the equalized gray level image to obtain a pedestrian object and recognition information of the pedestrian object;

and carrying out vehicle identification on the other path of the equalized gray scale image to acquire a vehicle object and identification information of the vehicle object.

7. The image processing method according to claim 6, wherein the performing pedestrian recognition on one path of the equalized gray-scale map to obtain a pedestrian object comprises:

performing multilevel scaling processing on the equalized gray level image by using a Laplacian pyramid decomposition algorithm;

carrying out hog feature extraction on the zoomed image on each level;

and performing classification identification based on the hog features, wherein the pedestrian object is identified from the objects.

8. The image processing method according to claim 6, wherein the performing vehicle identification on the other path of the equalized gray scale map to obtain a vehicle object comprises:

performing multilevel scaling processing on the equalized gray level image by using a Laplacian pyramid decomposition algorithm;

extracting haar characteristics of the zoomed image on each level;

and performing classification identification based on the haar features, and identifying vehicle objects from the objects.

9. The image processing method according to claim 7 or 8, characterized by, after identifying a pedestrian object or a vehicular object from the objects, further comprising:

and tracking the identified pedestrian objects and the identified vehicle objects, and eliminating the pedestrian objects and the vehicle objects which are identified by mistake.

10. A safe driving method, characterized in that the method comprises:

acquiring a target image; wherein the target image is obtained by the image processing method according to any one of claims 1 to 9;

identifying each object from the target image;

and generating and executing a safe driving strategy according to the identified object and the current running information of the vehicle.

11. The safe driving method according to claim 10, wherein the identifying each object from the target image includes:

carrying out gray level processing on the target image to obtain a gray level image of the target image;

performing histogram equalization processing on the gray level image to obtain an equalized gray level image;

dividing the balanced gray level images to form at least two paths of balanced gray level images;

carrying out pedestrian recognition on one path of the equalized gray level image to obtain a pedestrian object and recognition information of the pedestrian object;

and carrying out vehicle identification on the other path of the equalized gray scale image to acquire a vehicle object and identification information of the vehicle object.

12. The safe driving method according to claim 11, wherein the generating and executing of the safe driving strategy according to the identified object and the current operation information of the vehicle comprises:

determining an object closest to the vehicle from all the identified objects as a first object at risk with the vehicle;

and generating and executing a safe driving strategy of the vehicle according to the first object and the operation information.

13. The safe driving method according to claim 12, wherein the determining, as the first object at risk with the vehicle, an object closest to the vehicle from among all the objects identified, comprises:

screening according to the identification information of the pedestrian object and the identification information of the vehicle object, and acquiring the pedestrian object and the vehicle object which meet preset conditions as candidate objects;

the first object is identified from all candidate objects according to the distance between each candidate object and the vehicle.

14. The safe driving method according to claim 13, wherein the performing the filtering according to the identification information of the pedestrian object and the identification information of the vehicle object, from which the pedestrian object and the vehicle object that satisfy a preset condition are acquired as candidate objects, includes:

acquiring edge pixel points of each pedestrian object and each vehicle object according to the coordinate information in the identification information;

judging whether at least one pixel point of the edge pixel points is in a first area or not; the first area is an area formed by a lane where the vehicle is located in the target image;

and taking the pedestrian object or the vehicle object of which at least one pixel point is located in the first region of the edge pixel points as the candidate object.

15. The safe driving method according to claim 13, wherein the performing the filtering according to the identification information of the pedestrian object and the identification information of the vehicle object, from which the pedestrian object and the vehicle object that satisfy a preset condition are acquired as candidate objects, includes:

marking an identification frame for each pedestrian object and each vehicle object according to the coordinate information in the identification information;

judging whether each identification frame has at least one appointed boundary point in the first area; the first area is an area formed by a lane where the vehicle is located in the target image;

taking the recognition frame with at least one designated boundary point in the first area as a target recognition frame;

and taking the object corresponding to the target recognition frame as the candidate object.

16. The safe driving method according to any one of claims 12 to 15, wherein the generating and executing of the safe driving strategy of the vehicle based on the first object and the operation information includes:

obtaining a first distance between the first object and the vehicle;

acquiring the running time required by the vehicle to reach the first object according to the vehicle speed and the first distance in the running information;

determining a risk level between the first object and the vehicle as a function of the runtime;

and generating and executing the safe driving strategy according to the risk level.

17. An image processing apparatus characterized by comprising:

the system comprises an image acquisition module, a first image acquisition module and a second image acquisition module, wherein the image acquisition module is used for acquiring a first image and a second image based on a double-camera device on a vehicle;

the forming module is used for carrying out image fusion on the first image and the second image to form a target image;

a first recognition module for recognizing each object from the target image.

18. A safe driving apparatus, comprising:

the acquisition module is used for acquiring a target image; wherein the target image is obtained by the image processing method according to any one of claims 1 to 9;

a second recognition module for recognizing each object from the target image;

and the generating module is used for generating and executing a safe driving strategy according to the identified object and the current running information of the vehicle.

19. A computer device comprising a processor and a memory;

wherein the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory for implementing the image processing method according to any one of claims 1 to 9 or the safe driving method according to any one of claims 10 to 16.

20. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the image processing method according to any one of claims 1 to 9 or the safe driving method according to any one of claims 10 to 16.

21. A computer program product, wherein instructions, when executed by a processor, perform the image processing method of any of claims 1-9 or the safe driving method of any of claims 10-16.