阅读说明：本技术 图像的处理方法、装置、设备以及存储介质 (Image processing method, device, equipment and storage medium ) 是由 衣英林 于 2020-03-18 设计创作，主要内容包括：本申请提供一种图像的处理方法、装置、设备以及存储介质。该方法通过获取实时拍摄的原始鱼眼图像,并对原始鱼眼图像进行渲染处理,得到整体鱼眼图像,再根据整体鱼眼图像中预设的局部选择框对原始鱼眼图像中对应的待展开区域进行鱼眼展开,得到局部展开图像,将整体鱼眼图像和局部展开图像进行拼接显示,得到多个不同角度的、或者原始鱼眼图像中不同细节处的局部展开图像,并将整体鱼眼图像和局部展开图像一同显示,使用户从视频图像中能够获得更全面的信息,提高了用户体验。(The application provides an image processing method, an image processing device, image processing equipment and a storage medium. According to the method, an original fisheye image shot in real time is obtained, the original fisheye image is rendered to obtain an integral fisheye image, then fisheye expansion is carried out on a corresponding region to be expanded in the original fisheye image according to a local selection frame preset in the integral fisheye image to obtain a local expanded image, the integral fisheye image and the local expanded image are subjected to splicing display to obtain a plurality of local expanded images with different angles or different details in the original fisheye image, and the integral fisheye image and the local expanded image are displayed together, so that a user can obtain more comprehensive information from a video image, and user experience is improved.)

1. A method of processing an image, comprising:

acquiring an original fisheye image shot in real time;

rendering the original fisheye image to obtain an integral fisheye image;

according to a preset local selection frame in the whole fisheye image, carrying out fisheye unfolding on a corresponding area to be unfolded in the original fisheye image to obtain a local unfolded image; the local selection frame is used for indicating a region to be developed in the original fisheye image;

and splicing and displaying the whole fisheye image and the local unfolded image.

2. The method according to claim 1, wherein the rendering the original fisheye image to obtain an overall fisheye image comprises:

and cutting the original fisheye image according to a preset display proportion to obtain an integral fisheye image, so that the integral fisheye image is matched with a display area.

3. The method according to claim 1, wherein the fish-eye unfolding the to-be-unfolded region of the original fish-eye image to obtain a locally unfolded image comprises:

cutting the area to be expanded of the original fisheye image to obtain a local fisheye image corresponding to the area to be expanded;

and carrying out two-dimensional transformation on the local fisheye image to obtain the local unfolded image.

4. The method according to claim 3, wherein the stitching display of the whole fisheye image and the locally unfolded image comprises:

reducing the whole fisheye image, placing the reduced whole fisheye image in a first target area of a first image layer, and performing transparency processing on the image layer area outside the first target area;

reducing the local expansion image, then placing the reduced local expansion image in a second target area of a second image layer, and carrying out transparent processing on the image layer area outside the second target area;

overlapping and displaying the first image layer and the second image layer in a display area of a display;

wherein the first target region and the second target region do not overlap.

5. The method according to any one of claims 1 to 4, wherein after the rendering process is performed on the original fisheye image to obtain an overall fisheye image, the method further comprises:

receiving the selection operation of a user on any local area in the whole fisheye image;

generating a local selection frame corresponding to the local area in response to the selection operation;

displaying the local selection frame on the local area of the overall fisheye image.

6. The method according to any one of claims 1 to 4, wherein after said stitching and displaying said whole fisheye image and said locally unfolded image, said method further comprises:

receiving an adjusting operation of a user on the local selection frame; the adjusting operation comprises moving a position of the local selection box or changing a size of the local selection box;

and responding to the adjustment operation, and adjusting the position or the size of the area to be expanded corresponding to the local selection frame.

7. The method of claim 4, wherein prior to said stitching displaying said whole fisheye image and said locally unfolded image, said method further comprises:

and determining the position coordinate and the area size of the first target area and the position coordinate and the area size of the second target area according to the number of the areas to be expanded.

8. An apparatus for processing an image, the apparatus comprising:

the acquisition module is used for acquiring an original fisheye image shot in real time;

the processing module is used for rendering the original fisheye image to obtain an integral fisheye image;

the processing module is further used for carrying out fisheye unfolding on the corresponding to-be-unfolded area in the original fisheye image according to a preset local selection frame in the whole fisheye image to obtain a local unfolded image; the local selection frame is used for indicating a region to be developed in the original fisheye image;

and the display module is used for splicing and displaying the whole fisheye image and the local unfolded image.

9. An electronic device, comprising: the fish-eye camera, the display, the memory and the processor;

the camera is used for shooting an original fisheye image in real time;

the display is used for displaying corresponding content according to the control of the processor;

the memory stores computer-executable instructions;

the processor executes the computer-executable instructions stored by the memory to cause the edge energizer to perform the method of processing an image according to any one of claims 1 to 7.

10. A storage medium, comprising: a readable storage medium and a computer program for implementing the method of processing an image according to any one of claims 1 to 7.

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for processing an image.

Background

With the continuous development of image processing technology, users have higher and higher requirements on the comprehensiveness and definition of video pictures which can be provided in monitoring videos.

In an actual application scene, a fisheye video image is collected through a fisheye camera in video monitoring to obtain a wide-angle image beyond the range which can be seen by human eyes.

However, the angle of the fisheye image provided in the prior art is single, and the content of the video image that can be provided is greatly limited, resulting in a low user experience.

Disclosure of Invention

The application provides an image processing method, device, equipment and storage medium, which can obtain fisheye expanded images at different angles and different details in an original picture aiming at the same original fisheye image.

In a first aspect, an embodiment of the present application provides an image processing method, including:

acquiring an original fisheye image shot in real time;

rendering the original fisheye image to obtain an integral fisheye image;

according to a preset local selection frame in the whole fisheye image, carrying out fisheye unfolding on a corresponding area to be unfolded in the original fisheye image to obtain a local unfolded image; the local selection frame is used for indicating a region to be developed in the original fisheye image;

and splicing and displaying the whole fisheye image and the local unfolded image.

In a specific implementation manner, the rendering the original fisheye image to obtain an overall fisheye image includes:

and cutting the original fisheye image according to a preset display proportion to obtain an integral fisheye image, so that the integral fisheye image is matched with a display area.

In a specific implementation manner, the fish-eye unfolding the to-be-unfolded region of the original fish-eye image to obtain a locally unfolded image includes:

cutting the area to be expanded of the original fisheye image to obtain a local fisheye image corresponding to the area to be expanded;

and carrying out two-dimensional transformation on the local fisheye image to obtain the local unfolded image.

In a specific implementation manner, the displaying the whole fisheye image and the locally expanded image in a mosaic manner includes:

reducing the whole fisheye image, placing the reduced whole fisheye image in a first target area of a first image layer, and performing transparency processing on the image layer area outside the first target area;

reducing the local expansion image, then placing the reduced local expansion image in a second target area of a second image layer, and carrying out transparent processing on the image layer area outside the second target area;

overlapping and displaying the first image layer and the second image layer in a display area of a display;

wherein the first target region and the second target region do not overlap.

Further, after the rendering processing is performed on the original fisheye image to obtain an overall fisheye image, the method further includes:

receiving the selection operation of a user on any local area in the whole fisheye image;

generating a local selection frame corresponding to the local area in response to the selection operation;

displaying the local selection frame on the local area of the overall fisheye image.

Optionally, after the splicing and displaying the whole fisheye image and the locally unfolded image, the method further includes:

receiving an adjusting operation of a user on the local selection frame; the adjusting operation comprises moving a position of the local selection box or changing a size of the local selection box;

and responding to the adjustment operation, and adjusting the position or the size of the area to be expanded corresponding to the local selection frame.

Further, before the splicing and displaying the whole fisheye image and the locally unfolded image, the method further comprises:

and determining the position coordinate and the area size of the first target area and the position coordinate and the area size of the second target area according to the number of the areas to be expanded.

In a second aspect, an embodiment of the present application provides an apparatus for processing an image, the apparatus including:

the acquisition module is used for acquiring an original fisheye image shot in real time;

the processing module is used for rendering the original fisheye image to obtain an integral fisheye image;

the processing module is further used for carrying out fisheye unfolding on the corresponding to-be-unfolded area in the original fisheye image according to a preset local selection frame in the whole fisheye image to obtain a local unfolded image; the local selection frame is used for indicating a region to be developed in the original fisheye image;

and the display module is used for splicing and displaying the whole fisheye image and the local unfolded image.

In a third aspect, an embodiment of the present application provides an electronic device, including: the fish-eye camera, the display, the memory and the processor;

the camera is used for shooting an original fisheye image in real time;

the display is used for displaying corresponding content according to the control of the processor;

the memory stores computer-executable instructions;

the processor executes computer-executable instructions stored by the memory to cause the sharps to perform the method of processing an image as described in the first aspect.

In a fourth aspect, an embodiment of the present application provides a storage medium, including: a readable storage medium and a computer program for implementing the method of processing an image according to the first aspect.

The image processing method provided by the embodiment of the application comprises the steps of obtaining an original fisheye image shot in real time, rendering the original fisheye image to obtain a whole fisheye image, carrying out fisheye expansion on a corresponding region to be expanded in the original fisheye image according to a local selection frame preset in the whole fisheye image to obtain a local expanded image, splicing and displaying the whole fisheye image and the local expanded image to obtain local expanded images at different angles or different details in the original fisheye image, displaying the whole fisheye image and the local expanded image together, enabling a user to obtain more comprehensive information from a video image, and improving user experience.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1 is a schematic flowchart of a first embodiment of an image processing method according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a second embodiment of an image processing method according to an embodiment of the present application;

fig. 3 is a schematic diagram of a display effect of layer stacking according to an embodiment of the present application;

fig. 4 is a schematic diagram of a display effect of layer stacking according to an embodiment of the present application;

fig. 5 is a schematic flowchart of a third embodiment of an image processing method according to the present application;

fig. 6 is a schematic flowchart of a fourth embodiment of an image processing method according to an embodiment of the present application;

fig. 7 is a data flow diagram illustrating an image processing method according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of a first embodiment of an image processing apparatus according to an embodiment of the present application;

fig. 9 is a schematic hardware structure diagram of an electronic device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

As used herein, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Reference throughout the specification to "one embodiment" or "another embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in this embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

In current supervisory equipment, adopt the fisheye camera to carry out image acquisition to target monitoring area usually, the fisheye camera has that the angle of vision is big, can hold the scene many, advantages such as adaptable narrow and small space shooting, and common fisheye camera can reach the angle of vision of 120 degrees to 180 degrees, and some fisheye cameras can reach 360 degrees angles of vision even, can gather the surveillance image at no dead angle on a large scale. Based on the abundant image data of the collected fisheye image, the monitoring equipment needs to perform fisheye unfolding processing of different degrees on the fisheye image of each frame so as to present an image which is combined with human vision of a user, and further, fish eye unfolding processing is performed on different areas at the same time, so that different image details can be presented to the user, and the image information which needs to be acquired by the user is met. The embodiment of the application can be applied to the scenes, can present fish eye expansion images of the required viewing areas for the user according to the requirements of the user, and presents the whole fish eye images and one to more local expansion images at the same time, so that the user can obtain image information from different areas or different angles, the user experience is improved, and the interactivity of the monitoring equipment is enriched.

The execution main body of the application is an electronic device with image processing capability, and the electronic device can be applied to any terminal device with a fisheye camera, such as a mobile phone, a notebook, a tablet, a personal computer, an intelligent wearable product, an intelligent household product, a security monitoring device and the like, or can be applied to a medical device, or can be applied to an industrial control device, or can be applied to a server and the like. The present application provides an image processing method, which can obtain a plurality of expanded images at different angles and at different details in an original fisheye image for the same original fisheye image, and is described below with reference to several specific embodiments.

Fig. 1 is a schematic flowchart of a first embodiment of an image processing method according to an embodiment of the present application. In order to present image details required by the user, in this embodiment, when the entire fisheye image is presented to the user, the fisheye unfolding is performed on the corresponding to-be-unfolded region in the original fisheye image according to the preset local selection frame, and the unfolded local unfolded image and the entire fisheye image are displayed in the display region of the display together. As shown in fig. 1, the image processing method includes:

s101: and acquiring an original fisheye image shot in real time.

In this step, an original fisheye image is acquired to provide an object of image processing. The method comprises the steps that an original fisheye image can be acquired through an image acquisition device, such as a fisheye camera, arranged in the electronic equipment or connected with the electronic equipment, and an image of a target object is acquired in real time to obtain the original fisheye image; alternatively, the original fisheye image transmitted by the external device may be received. It should be understood that the original fisheye image is a frame of image in a video, which may be a surveillance video, a game video, a home video, etc. The video content is not limited by the scheme, and any processing on the original fisheye image in the scheme can be applied to each frame of image in the video.

S102: and rendering the original fisheye image to obtain the whole fisheye image.

In this step, the original fisheye image acquired needs to be adaptively clipped according to the display area to be displayed, so that the original fisheye image can be matched with the display area. The display proportion, namely the length-width ratio, of the display area is predetermined, it is understood that in different application scenes, the integral fisheye image and different numbers of locally unfolded images can be set to be spliced and displayed together, so that the length-width ratio of the display area corresponding to each image can be determined by determining how to divide the whole display area according to the number of the locally unfolded images, and the original fisheye image is cut according to the length-width ratio of the display area corresponding to the integral fisheye image, so that the integral fisheye image can achieve a better display effect.

Optionally, the rendering processing on the original fisheye image may further include color rendering, texture rendering, shadow rendering, and the like, so as to achieve a better presentation effect of the whole fisheye image.

S103: and carrying out fisheye unfolding on the corresponding to-be-unfolded area in the original fisheye image according to a preset local selection frame in the whole fisheye image to obtain a local unfolded image.

The local selection frame is displayed in the whole fisheye image and used for indicating the region to be expanded in the original fisheye image, and it should be understood that the whole fisheye image is obtained after the original fisheye image is rendered, and each region in the whole fisheye image has a region corresponding to the region in the original fisheye image, so that the local selection frame in the whole fisheye image corresponds to the region to be expanded in the original fisheye image, and the local selection frame is a marking mode of the region to be expanded.

And performing fisheye unfolding on the corresponding to-be-unfolded region in the original fisheye image to obtain a local unfolded image relative to the whole fisheye image. The number of the preset local selection frames is one to more, correspondingly, the number of the local expansion images is also one to more, and the scheme is not limited by the number of the local expansion images.

Optionally, in an actual application scenario, when the number of the preset to-be-expanded regions is greater than or equal to two, the preset to-be-expanded regions may be laid out on the same horizontal axis or vertical axis of the original fisheye image, or laid out at different corners of the original fisheye image, and each to-be-expanded region may overlap.

S104: and splicing and displaying the whole fisheye image and the local unfolded image.

In the step, the obtained integral fisheye image and the at least one local expanded image can be spliced to obtain a final display picture, and the final display picture is displayed through a display.

For example, an original fisheye image may be rendered through an Open-Graphics Library (OpenGL), and a canvas is provided for OpenGL rendering by an Embedded-System Graphics Library (EGL), that is, an off-screen buffer is provided as an interface for OpenGL off-screen rendering. In this embodiment, an original fisheye image is input to OpenGL, the OpenGL outputs the processed whole fisheye image and the processed local expanded image, the whole fisheye image is stored in the off-screen buffer area, the whole fisheye image and the local expanded image stored in the off-screen buffer area are sent to the memory corresponding to the display area, and the electronic device waits for displaying by reading data in the memory corresponding to the display area.

In order to implement the splicing display of the whole fisheye image and the partially expanded image, for example, layer processing needs to be performed on both the whole fisheye image and the partially expanded image, the whole fisheye image is reduced and then placed in a first target region of a first layer, and a layer region outside the first target region is subjected to transparency processing, and correspondingly, the partially expanded image is reduced and then placed in a second target region of a second layer, and a layer region outside the second target region is subjected to transparency processing. Optionally, the layer processing process is completed through OpenGL, the processed whole fisheye image and the processed local expanded image are stored in a memory corresponding to the display area, the electronic device reads the whole fisheye image and the local expanded image stored in the memory at the same time, so that the whole fisheye image and the local expanded image are displayed in an overlapping manner, and the display effect of the splicing display is presented in the display because the first target area and the second target area are not overlapped.

The image processing method provided by the embodiment obtains an original fisheye image shot in real time, performs rendering processing on the original fisheye image to obtain a whole fisheye image, performs fisheye unfolding on a corresponding region to be unfolded in the original fisheye image according to a preset local selection frame in the whole fisheye image to obtain a local unfolded image, and finally performs splicing display on the whole fisheye image and the local unfolded image, so that the same original fisheye image is rendered to obtain unfolded images at a plurality of different angles or different details in the original fisheye image, so that a user can obtain more comprehensive information from a video image, and user experience is improved.

On the basis of the above embodiment, the present embodiment provides a specific implementation manner for how to determine the positions of the target areas of the whole fisheye image and the partially expanded image in the respective image layers: and determining the position coordinate and the area size of the first target area and the position coordinate and the area size of the second target area according to the number of the areas to be expanded. For example, the number of the regions to be expanded is 3, that is, the number of the partially expanded images is 3, the image layer 4 is equally divided into a first target region and 3 second target regions, which are respectively located at 4 different corners of the respective image layers, that is, the upper left corner, the lower left corner, the upper right corner, and the lower right corner.

As an example, fig. 3 is a schematic diagram of a display effect of layer superposition provided in an embodiment of the present application, and fig. 4 is a schematic diagram of a display effect of layer superposition provided in an embodiment of the present application. As shown in fig. 3, taking the number of the local selection frames as 3 for example, fish-eye expansion is performed on the to-be-expanded region corresponding to each local selection frame to obtain 3 local expanded images, so that the final display screen should include 4 display regions, such as display regions 1 to 4 as shown in the figure, and the whole fish-eye image and each local expanded image are respectively displayed in one display region. With reference to fig. 4, the entire fisheye image 01 is displayed in the display area 4 shown in fig. 3, and then the first target area where the entire fisheye image is located should be the lower right corner of the first layer, and each local developed image 02 may be respectively displayed in the display areas 1 to 3 according to a preset sequence, that is, each local developed image 02 determines the position of the layer where the image is located after being reduced according to the position of the display area where the local developed image 02 is located.

Fig. 2 is a schematic flowchart of a second embodiment of an image processing method according to the present application. As shown in fig. 2, in the process of rendering an original fisheye image to obtain an overall fisheye image, the embodiment includes the following steps:

s1031: and cutting the region to be expanded of the original fisheye image to obtain a local fisheye image corresponding to the region to be expanded.

S1032: and carrying out two-dimensional transformation on the local fisheye image to obtain a local expanded image.

In step S1031 and step S1032, the region to be expanded of the original fisheye image is located, the region to be expanded is cut to obtain a local fisheye image, and the local fisheye image is subjected to two-dimensional transformation to obtain a local expanded image.

Optionally, in this embodiment, the original fisheye image may also be first fisheye-expanded to obtain an overall expanded image, and then the local area corresponding to the area to be expanded in the overall expanded image is located, and the local expanded image is obtained by clipping.

In order to enable a user to select or adjust a local area to be expanded according to requirements, the embodiment of the application provides a personalized scheme, so that the user can set the area to be expanded by setting or adjusting a local selection frame, and the interchangeability of the electronic device is improved. On the basis of the embodiment shown in fig. 2, as an example, the original fisheye image is collected in real time, the whole fisheye image obtained according to the original fisheye image is also a section of continuous video frame (hereinafter referred to as a fisheye video), when the whole fisheye image is displayed, that is, in the process of displaying the fisheye video, a user may operate the local selection frame in the whole fisheye image of any frame of the fisheye video according to a requirement, so as to implement setting or adjustment of the region to be expanded, and finally change at least one displayed local expanded image, optionally, the electronic device may receive an operation input by the user through a human-computer interaction interface, and may also receive an operation input by the user through a mouse or a remote control device through an input interface. The method comprises the following specific implementation modes:

the method comprises the following steps of firstly, setting a local selection frame in a whole fisheye image: receiving a selection operation of a user on any local area in the whole fisheye image, wherein the selection operation is used for indicating to create a local selection frame, responding to the selection operation, generating the local selection frame corresponding to the local area, and displaying the local selection frame on the local area of the whole fisheye image. For example, the user clicks the whole fisheye image through the human-computer interface, and a local selection frame with a preset size range is generated by taking the clicked position as the center, or the local selection frame generated by the user through the human-computer interface is defined.

And secondly, adjusting a local selection frame displayed in the whole fisheye image: receiving an adjusting operation of a user on any local selection frame, wherein the adjusting operation comprises moving the position of the local selection frame or changing the size of the local selection frame, for example, the user drags the local selection frame by touching a human-computer interaction interface or operating a mouse or a remote control device to change the position or the size of the local selection frame; and responding to the adjustment operation, adjusting the position or size of the to-be-expanded region corresponding to the local selection frame so as to adjust the rendered local expanded image based on the to-be-expanded region.

And a third mode, deleting local selection frames in the whole fisheye image: receiving a deletion operation of a user in the whole fisheye image, wherein the deletion operation is used for indicating to delete the local selection frame, for example, the user triggers a closing control of the local selection frame by touching a human-computer interaction interface or operating a mouse or a remote control device so as to delete the local selection frame; and in response to the deletion operation, deleting the to-be-expanded region corresponding to the local selection frame, without rendering the to-be-expanded region of the original fisheye image, which is equivalent to deleting the corresponding local expanded image.

In a specific implementation manner, the splicing and displaying the whole fisheye image and the at least one locally unfolded image in the above embodiment may specifically include: and respectively storing the whole fisheye image and the at least one local expansion image in different storage spaces corresponding to the display area. It should be understood that in the present solution, corresponding storage spaces, which may be memory spaces or display card memory spaces, are allocated to the display areas 1 to 4 shown in fig. 2, and are used for storing the images to be displayed in each display area. Exemplarily, an original fisheye image is rendered to obtain an overall fisheye image and at least one partially unfolded image, each image is stored in a Memory space corresponding to a display region for displaying the image, and further, data in the Memory space is copied to a corresponding display region (specifically, in a display card Memory corresponding to the display region) through a hardware copy module, for example, a Direct Memory Access (DMA) module, so that the current display channel is enabled to be in the same display channel, and each fisheye image is displayed through the corresponding display region, thereby realizing a display effect of a tiled display.

Fig. 5 is a schematic flow chart of a third embodiment of an image processing method provided in an embodiment of the present application, before the foregoing embodiments, the method further includes an initialization step shown in fig. 5, specifically including:

the method comprises the steps of creating an image processing thread, initializing display of the electronic device, applying for a memory space for a display area, initializing EGL and initializing OpenGL.

Fig. 6 is a schematic flowchart of a fourth embodiment of a method for processing an image according to an embodiment of the present application, as shown in fig. 6, the method includes:

the method comprises the steps of enabling an image processing function of the electronic equipment, obtaining areas to be unfolded selected by a user, calculating parameters of display areas according to the number of the areas to be unfolded selected by the user, wherein the parameters comprise the number of the display areas, the size of each display area and the position coordinate of each display area, creating the display areas according to the area parameters of the display areas, obtaining decoding data of an original fisheye image after the creation of the display areas is completed, rendering the whole original fisheye image and the corresponding areas according to the selected areas to be unfolded to obtain a whole fisheye image and at least one local unfolded image, copying the whole fisheye image and the at least one local unfolded image to the corresponding display areas, and splicing and displaying the whole fisheye image and the at least one local unfolded image through the corresponding display areas.

Fig. 7 is a data flow diagram of an image processing method according to an embodiment of the present application, and as shown in fig. 7, the rendering processing of the original fisheye image in the present solution includes performing layer processing on the entire fisheye image and the local expanded image, storing the processed entire fisheye image and at least one local expanded image in a memory space of a corresponding display area, copying an image in the memory space to the corresponding display area, and displaying the images in an overlapping manner to present a stitching effect of each image displayed in the respective display area.

Fig. 8 is a schematic structural diagram of a first embodiment of an image processing apparatus according to an embodiment of the present application, and as shown in fig. 8, the image processing apparatus 10 includes:

the acquisition module 11 is used for acquiring an original fisheye image shot in real time;

the processing module 12 is configured to perform rendering processing on the original fisheye image to obtain an overall fisheye image;

the processing module 12 is further configured to perform fisheye unfolding on a corresponding to-be-unfolded region in the original fisheye image according to a preset local selection frame in the whole fisheye image to obtain a local unfolded image; the local selection frame is used for indicating a region to be developed in the original fisheye image;

and the display module 13 is configured to perform mosaic display on the whole fisheye image and the locally expanded image.

The embodiment provides a processing apparatus 10 for image, including obtaining module 11, processing module 12 and display module 13, through obtaining the original fisheye image of shooting in real time, and render processing to original fisheye image, obtain whole fisheye image, according to the local select frame that predetermines in the whole fisheye image, the area that waits to expand that corresponds in the original fisheye image carries out fisheye expansion, obtain local expansion image, at last splice whole fisheye image and local expansion image and show, it renders processing to same original fisheye image to have realized, obtain a plurality of different angles, or the expansion image of different details department in the original fisheye image, make the user can obtain more comprehensive information in following the video image, user experience has been improved.

In one possible design, the processing module 12 is specifically configured to:

and cutting the original fisheye image according to a preset display proportion to obtain an integral fisheye image, so that the integral fisheye image is matched with a display area.

In one possible design, the processing module 12 is specifically configured to:

cutting the area to be expanded of the original fisheye image to obtain a local fisheye image corresponding to the area to be expanded;

and carrying out two-dimensional transformation on the local fisheye image to obtain the local unfolded image.

In one possible design, the display module 13 is specifically configured to:

reducing the whole fisheye image, placing the reduced whole fisheye image in a first target area of a first image layer, and performing transparency processing on the image layer area outside the first target area;

reducing the local expansion image, then placing the reduced local expansion image in a second target area of a second image layer, and carrying out transparent processing on the image layer area outside the second target area;

overlapping and displaying the first image layer and the second image layer in a display area of a display;

wherein the first target region and the second target region do not overlap.

In a possible design, the obtaining module 11 is further configured to receive a selection operation of a user on any local area in the whole fisheye image;

the processing module 12 is further configured to generate a local selection box corresponding to the local region in response to the selection operation;

the display module 13 is further configured to display the local selection frame on the local area of the overall fisheye image.

In a possible design, the obtaining module 11 is further configured to receive an adjustment operation of the local selection box by the user; the adjusting operation comprises moving a position of the local selection box or changing a size of the local selection box;

the processing module 12 is further configured to adjust a position or a size of the to-be-deployed area corresponding to the local selection box in response to the adjustment operation.

In a possible design, the processing module 12 is further configured to determine the position coordinate and the area size of the first target area and the position coordinate and the area size of the second target area according to the number of the areas to be expanded.

The image processing apparatus provided in the foregoing embodiment may execute the technical solutions of the foregoing method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.

Referring to fig. 9, in the embodiment of the present application, an electronic device is further provided, and the embodiment of the present application is only described with reference to fig. 9 as an example, which does not mean that the present application is limited thereto.

Fig. 9 is a schematic hardware structure diagram of an electronic device according to an embodiment of the present application. As shown in fig. 9, the electronic device 200 provided in this embodiment may include: a fisheye camera 204, a display 205, a memory 201, and a processor 202; optionally, a bus 203 may also be included. The bus 203 is used to realize connection between the elements.

The fisheye camera 204 is used for shooting an original fisheye image in real time;

the display 205 is configured to display corresponding content according to the control of the processor;

the memory 201 stores computer-executable instructions;

the processor 202 executes the computer-executable instructions stored in the memory 201, so that the processor executes the image processing method provided by any one of the foregoing embodiments.

Wherein, the memory 201 and the processor 202 are electrically connected directly or indirectly to realize the data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines, such as via bus 203. The memory 201 stores computer-executable instructions for implementing the data access control method, including at least one software functional module that can be stored in the memory 201 in the form of software or firmware, and the processor 202 executes various functional applications and data processing by running software programs and modules stored in the memory 201.

The Memory 201 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 201 is used for storing programs, and the processor 202 executes the programs after receiving the execution instructions. Further, the software programs and modules in the memory 201 may also include an operating system, which may include various software components and/or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.

The processor 202 may be an integrated circuit chip having signal processing capabilities. The Processor 202 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and so on. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. It will be appreciated that the configuration of fig. 9 is merely illustrative and may include more or fewer components than shown in fig. 9 or have a different configuration than shown in fig. 9. The components shown in fig. 9 may be implemented in hardware and/or software.

The embodiment of the present application further provides a computer-readable storage medium, on which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the method for processing an image provided in any one of the method embodiments described above may be implemented.

The computer-readable storage medium in this embodiment may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, etc. that is integrated with one or more available media, and the available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., SSDs), etc.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.