Abstract
Human face plays an indispensable role in emotional expression and information exchange, which attracts large number of researchers to study face recognition. Nowadays, with the rapid development of computer graphics, artificial intelligence, and other technologies, the ability of the human vision system to recognize facial expressions and facial organs is enhanced. More and more experts think about how to make the computer vision system have this capability. By combining artificial intelligence and computer graphics, this paper studies how to optimize the 3D face network model and extract geometric features for 3D face recognition. We propose a 3D face network model based on Poisson equation to realize face hole recognition and boundary preprocessing. Besides, we also establish the 3D face surface equation and equal face value extraction and enhance the face feature based on facial semantic information. We study the hole repair of 3D face model based on Poisson equation integrating semantic information and achieve the purpose of optimizing the 3D face model. After the optimization, the method is compared with mesh repair and Poisson repair, which demonstrates that our Poisson-based 3D face hole repairing model obtains the best results among compared methods.
1. Introduction
In recent years, the research focus is to reconstruct realistic 3D face models. With the rapid development of artificial intelligence and computer graphics, 3D face models have been widely used in various fields, such as human-computer interaction, remote conference, multimedia communication, and virtual movie roles. In the field of computer graphics, data processing and geometric optimization of 3D face model have also become hot research issues. In the field of artificial intelligence, face recognition technology is also gradually developing to the field of 3D face application. The real 3D face image data are collected and reconstructed through the high-precision 3D face camera [1], and a 3D face model similar to the real one is obtained. However, the collected three-dimensional information often lacks measurement data, resulting in the lack of holes in the reconstructed three-dimensional model, which cannot be applied in practice. Therefore, artificial intelligence and computer graphics should be used to optimize the 3D face geometric model.
Through combining artificial intelligence and computer graphics, this paper studies how to optimize the 3D face network model and extract geometric features for 3D face recognition. In essence, we propose a 3D face network model based on Poisson equation to realize face hole recognition and boundary preprocessing. Besides, we also establish the 3D face surface equation and equal face value extraction and enhance the face feature based on facial semantic information. We study the hole repair of 3D face model based on Poisson equation integrating semantic information and achieve the purpose of optimizing the 3D face model. The major innovations of the research conducted in this paper are as follows:(1)We use the artificial intelligence technology and 3D scanner to establish 3D face projection map; the face feature point detection and face detection are conducted on 2D image based on the convolution neural network. We extract face features to formulate the conformal mapping two-dimensional feature map based on the face model co-mapping algorithm; the method introduces multiple geometry feature fusion approach to analyze various 3D face feature information and conducts feature fusion by feature concatenation and weighted overlay.(2)We fuse the face semantic information with Poisson equation to optimize the 3D face network. For the hole problems in the model, the hole detection is implemented, and the edge hole boundary network is marked. The Poisson surface is established based on the principle of Poisson equation, and the feature of Poisson algorithm is enhanced by using face semantic information to achieve the purpose of repairing the 3D face model. The holes in the 3D face network model are repaired based on Poisson equation.
The rest of the paper is organized as follows. In Section 2, we offer an overview of the related work. Section 3 is about 3D face recognition based on geometric features. In Section 3 face mesh model optimization based on Poisson equation is discussed. Analysis of experimental results of 3D face geometry optimization methods is explained in Section 5. Moreover, experimental details are also presented. Finally, Section 6 concludes this paper along with directions for future research.
2. Related Work
3D face recognition was proposed from the late 1980s to the early 1990s [2]. In this period, the three-dimensional data acquisition equipment was backward and the computing power of the processor was poor, so this research could not be fully carried out, and the number of selected datasets was small, which could not be applied in practice. Nowadays, with the rapid development of computer technology, various advanced storage devices and 3D data acquisition devices have been developed to promote the rapid development of 3D image face recognition technology. In the twenty-first century, experts at home and abroad have focused on 3D face recognition technology. Zulkifli et al. [3] studied the thinking mode of face feature and proposed an extraction method based on characteristic head and face features. Its essence is to represent frontal face with a 3D space function and then locate it on a 2D plane to estimate the function parameters by projecting the shape function into a 2D gray image, making full use of the prior knowledge of face structure to effectively deal with many problems in the process of shape restoration by projection. [4] proposed a novel face imaging model which is based on the gray plane method. In detail, this model can establish a deformable 3D mesh surface, which transforms the face matching problem into an elastic matching problem between changeable surfaces. At the same time, through the surface deformation which can be completed by the finite element analysis method, we can judge if the identities of different people in the image are the same based on the actual deformation situation.
[5] proposed a multi-pose face recognition method under different lighting conditions based on a three-dimensional deformation model. The remarkable feature of this method is to use the three-dimensional information of face to recognize the face and better deal with the impact of changing lighting and face pose on the recognition accuracy in face recognition. This method is a synthetic analysis technology. Based on three-dimensional face texture statistics and face shape deformation model [6], the camera model and illumination model during image acquisition are established by using graphics simulation, so that the characteristics of face shape and texture are completely separated from various external parameters, which refer to illumination, camera configuration, etc. This can improve the accuracy of face recognition. Eun et al. [7] used cylinders to represent the three-dimensional head and used projection photography to evaluate the head posture. Hua et al. [8] combined expectation maximization algorithm, AAM algorithm, and 3dmm algorithm to predict the pose of a single face image.
3. 3D Face Recognition Based on Geometric Features
3.1. Principles and Characteristics of Face Modeling
The three-dimensional scanner [9] is a high-precision instrument in the optical field. Its principle is to obtain the three-dimensional coordinates of the human face by using the triangulation method, emit light through the laser, pass through the parallel equidistant straight line to form an amplitude grating, and form a linear interference fringe to map to the human face. The fringe is deformed due to the change of the depth and curvature of the object surface. The deformed fringe image can be captured by the CCD camera [10], and the laser beam CCD and the laser beam emission angle can be combined to form an internal imaging device, and the position coordinates or distance data of the detected point can be obtained based on the triangular geometric relationship. The center point of CCD camera lens is the origin of measurement coordinate system. The imaging point of the face to be measured is represented by (x, y, z), f represents the focal length of the camera, B represents the distance between the camera and the laser projection center, and ? represents the angle between the center of the light source and the monitored point forming a straight line and the x-axis. Based on the 3D face scanner, the face projection in Figure 1 is constructed.
The system parameters in the figure above are b, f, and based on the camera calibration technology; u and V represent the pixel coordinates on the CCD camera, and the following relationship exists on the XOZ plane:
The YOZ plane has the following relationships:
The three-dimensional coordinates of the following measured points (x, y, z) can be obtained:
3.2. Face Model Co-Mapping Algorithm
In the mapping process, the conformal mapping algorithm converts the Gaussian curvature angles of different points of the 3D face point cloud into the adjacent triangular mesh measurement [11]. Firstly, define the internal angle at the fixed point Vi as fijk. Define the side length of the triangular mesh of the center of two circles in the circle packing as the inversion distance Iij; Ki represents the Gaussian curvature of the fixed point position.
By carefully observing the facial features, the highest point in the face model is the nose position, and a three-dimensional spatial coordinate system is constructed. The nose coordinate is represented by the maximum value on the Z axis. If P is the tip of the nose, the starting point selected in the solution process based on Ricci flow algorithm is p, and the 3D face cloud model is mapped into a 2D face image. Figure 2 shows the face model co-mapping algorithm process [12].(1)The initial Riemannian metric radius of all orders on the 3D model is calculated based on the initial triangular mesh edge length. The following is the calculation formula: After calculation, the radius of the fixed point Vi position is .(2)Using the triangular mesh edge length and the initial circular pattern radius, the inversion distance of all edges is obtained by cosine theorem. The conformal mapping algorithm only changes the circular pattern radius, but the inversion distance does not change.(2)Calculate all internal angles in the grid, and the formula is as follows:(3)Calculate the discrete Gaussian curvature Ki of all vertices, and its formula is as follows:(4)The initial energy is defined according to the initial circle radius defined by each vertex, which is calculated by the following formula:(5)Define the Gaussian curvature of all points on the surface as K′, define the target Gaussian curvature as O, and adjust the energy value on each point according to the difference between the target curvature and the initial Gaussian curvature to adjust the circle radius. The calculation formula is as follows:(6)Based on the radius between the new circle mode and the adjacent circle mode and the reverse distance of the edge where the two points are located, the edge length of the triangular mesh is calculated, and the following is the calculation formula:
This should be noted that after traversing all the points in the 3D model, the conformal plane metric of the triangular network in the original 3D face model can be obtained. Based on the corresponding concerns in the triangular mesh of the original 3D model, the plane metric can be rearranged in the 2D plane disk to obtain the conformal mapping 2D feature map.
3.3. Multiple Geometric Feature Fusion Algorithms
The recognition ability of different face features is also different, and the role of face recognition is also different. There is less information in a single feature and strong information limitations. By fusing a variety of feature information and comprehensively analyzing [13] various feature data, the original data can be retained to the greatest extent. Based on the in-depth analysis of various features, this paper proposes a face recognition algorithm integrating a variety of different features. Because different channels express different information in different ways and there are differences in the amount of information, each channel should be normalized and the weight should be evenly distributed to each channel.
In terms of fusion, the most widely used methods are weighted superposition and feature concatenation [14]. After feature concatenation, it has a high feature vector dimension. Here, the following weighted superposition methods are used to realize multivariate feature fusion:where ωi is the role of different types of recognition features in recognition devices.
4. 3D Face Mesh Model Optimization Based on Poisson Equation
4.1. Poisson Equation
Poisson equation [15] belongs to partial differential equation. Poisson surface is reconstructed by combining implicit fitting, local features, and global features, Poisson equation is calculated, a three-dimensional model is established to represent the hidden layer surface, and the surface model with geometric entity information is obtained by equivalent extraction. Combined with Poisson equation reconstruction method, it can better deal with the surface smoothness and surface detail features on the closed feature model [16].
When dealing with the scaling function V : R3⟶R3, this method is realized by using the specified vector domain V : R3⟶R3. On a certain basis of the solid model , the solid surface is represented by , and the gradient of the scaling function makes it infinitely close to the composition of the vector domain. The minimum value of the scaling function is calculated as follows:
Since the vector field cannot be integrated directly, it is necessary to convert the problem into the problem of calculating Poisson equation, and the following divergence operator formula is introduced:
4.2. 3D Face Mesh Hole Repair Based on Poisson Equation
Figure 3 shows the flowchart of 3D face mesh hole repair algorithm based on Poisson equation.
4.2.1. Hole Identification and Boundary Pretreatment
The closed ring formed by the head and tail connecting boundary is a hole. As evident from the state of the art, different types of holes have boundary structures with high similarity. The shape of the hole can be obtained by identifying the boundary edge. In a complete triangular mesh structure, each edge has two corresponding triangular patches. The hole boundary is a triangular network, and any edge in the triangular network belongs to only one triangular network. Based on this property, different hole regions can be identified. As shown in Figure 4, the hole area is a blank position. The information of the original point set of the hole boundary is relatively sparse. During the reconstruction, the hole boundary will produce long and narrow triangular patches that are inconsistent with the properties of Delaunay triangle [17,18]. To further refine the triangular patches of the hole boundary area, we can make full use of the hole boundary area data and the stitching process to provide a better interface and enhance the transition effect [19].
4.2.2. Establishing Surface Equation
The algorithm runs based on the network model vertex information and the original point cloud information and obtains the network model vertex information according to the topology. A three-dimensional surface represents the changes in the surface at different locations, i e., constants. The normal vector in the point on the surface of the model is the same as the gradient of the function [20]. If M represents the input network model, all directed point sets C ⸦ m {C1, C2,…, CN} of M in R3 space are obtained, and Nci is the internal normal vector of directed point Ci. The vector field of this point set is represented by V, and the x implicit three-dimensional surface equation is obtained by fitting the point vector set. By converting this problem into , the surface equation can be closer to the vector domain. This problem is converted into a Poisson equation problem, as follows:
The octree segmentation algorithm is introduced into the segmentation space in this algorithm to solve the discretization problem. Then, the 3D surface vector field uses the small surface patch information and area obtained by 3D surface segmentation, which is calculated by the following formula:
In the above formula, the offset between q and p is represented by Fp (q) = F(q-p), C is the set of input points, so as to establish octree O, and the sample points are saved in the leaf node. Define Fo at node o:
The node closest to the eight positions of node C is represented by NgbrD (c), ωo, c is the weight of all interpolation points, and then the three-dimensional surface vector domain is approximately represented by the following:
4.2.3. Equal Face Value Withdrawal
After obtaining the most ideal fitting equation of X three-dimensional surface, we should extract the equation isosurface and further obtain the three-dimensional surface. If the point on the isosurface meets the requirements of function F(ci) = 0, F(ci) < 0, it indicates that the point is located in the surface, and if f (ci) > 0, it indicates that the point is located outside the surface. In this paper, MC (marching cubes) algorithm is used to extract the equivalent value, and the R threshold is set to ensure that the extracted equivalent surface is consistent with the original model surface, and the average value of each point is calculated, which can reduce. Scaling affects the effect of extracting isosurfaces [21].
4.2.4. Hole Patch Generation and Stitching
Based on the above process, the prediction surface is obtained. In this algorithm, Delaunay triangulation is used to segment the surface with protection constraints to obtain the meshed prediction surface. Subdivide the hole area, and the hole indefinite M ′can be obtained without refining the triangular mesh area. Strengthening the protection constraint is to reduce the quadratic angle triangulation triangular network model, which can save a lot of time and retain the original state of the network. Finally, the following hole patches are obtained by matching the predicted surface with the neighborhood triangle of the hole area. The protected areas are defined as
The distance between points m and P of the triangular network is represented by dis (m, P) and the threshold radius is represented by R. Assuming that the hole patch is the average triangular mesh side length, the known information can be stitched better by subdividing the hole patch, protecting the constrained triangular mesh, and reserving the interface at the boundary position. Figure 5 shows different situations of stitching.
4.2.5. Feature Enhancement
Based on the semantic information of face features [22], the feature vector p of 3D face region is obtained by multiplying the patch matrix vector of 3D feature region by the weight of hidden layer of face feature classification. According to the Poisson surface equation and isosurface extraction constructed above, the hole surface is obtained by Delaunay triangular section algorithm, and then the feature points near the hole are extracted to construct the feature point set. Combined with the above missing data and mixed feature model, the hole eigenvalue is improved [23]. If the hole is in the face feature area, assume that the complete part is S = (S1, S2,…,Sn)T, and transform the feature point feature matrix to obtain S’:
The triangular mesh patch is deformed and combined with Poisson equation with Dirichlet boundary conditions, as follows:where F represents unknown scalar function, divh represents h divergence, and f∗ represents scalar function constrained by boundary conditions. Finally, seamlessly stitch the original model and feature enhancement hole patch to end the hole patch. Function expansion module is shown in Figure 6.
5. Analysis of Experimental Results of 3D Face Geometry Optimization
5.1. 3D Face Network Model Quality Evaluation System
Here, we need to detect holes first, so we design and develop a three-dimensional face network model quality detection system [24], which can obtain holes through boundary properties. There are many formats of 3D models. When designing algorithms, we should convert different formats and realize reading operation. By analyzing the data structure of the 3D face model, we can better process the model. Starting from the model properties existing in the reconstruction model, this paper designs the following three-dimensional face network model quality evaluation system.
The system reflects the quality model of 3D face model reconstruction. In the system, the detection module should analyze the number of holes, the smoothness of the model, and whether there are burrs on the model. Eventually the data are taken as the evaluation standard. When detecting the number of holes, it is necessary to read and analyze the data on the 3D face model to obtain the 3D point cloud information. It is convenient to distinguish triangular patches by using the inner and outer boundary extraction algorithms to obtain the corresponding number of holes.
The HTML page reflects the model quality evaluation module, displays the read three-dimensional information on the page, and evaluates the model score. The model quality evaluated in this way lacks integrity. It is necessary to expand the number of holes, form a specific evaluation criterion, and then formulate an accurate evaluation quality to optimize the frame page.
Function Expansion Module. This module is used for quality detection, which is conducive to promoting three-dimensional reconstruction and greatly improving data integrity. The extended function is related to the classification management of various quality models, such as repairing holes and removing debris information. In this paper, the quality evaluation framework of 3D face reconstruction is used to provide management basis for the construction of 3D face database.
5.2. Hole Repair Analysis of 3D Face Mesh Model Based on Poisson Equation
The original 3D face network model is compared, and the results of mesh hole repair by various methods are shown in Figure 7. As shown in the figure, the mesh repair method cannot completely retain the surface features on the mesh. Poisson equation adopted in this paper has the best effect and can significantly improve the surface smoothness of the hole model.
(a)
(b)
(c)
(d)
According to the above table, the number of original vertices of the network model is 104029 and the number of triangular meshes is 206132. After repair, the number of fixed points and meshes has changed significantly, and there is a certain difference in time consumption. Combined with the repair results in Figure 7 above and compared with the original Poisson repair method, this algorithm integrates the face semantic information and establishes a new feature enhancement algorithm based on Poisson equation to repair holes. When the number of networks and vertices meets the requirements, the repair effect is the best.
6. Conclusions and Future Work
In this paper, the 3D face model with high authenticity in the measured data is obtained by 3D reconstruction. However, the reconstructed model data are incomplete and have some defects. Therefore, the optimization and repair of the face model is very important in 3D face model and can also effectively improve the robustness and availability of the 3D face model. The traditional way uses geometric heuristic method to repair. Starting from the face features, this paper uses semantic information to greatly improve the repair quality of the 3D face model, and the repair model accuracy is also improved significantly. By combining artificial intelligence and computer graphics, this paper proposed a 3D face recognition system based on geometric features and used this algorithm to extract and segment the feature semantics of the 3D face model as an important basis for repairing the 3D face mesh. The feature enhancement of the 3D face network model based on Poisson equation is used to repair the holes, detect the 3D face holes, mark the boundary network, and clarify the location of the specific holes.
Using the principle of Poisson equation and isosurface extraction technology, combined with the face detection model, the hidden layer weight in the convolution neural network is extracted to obtain the feature vectors of different regions. The feature vector is introduced into the Poisson equation to enhance the surface fitting features, so as to realize the hole repair of the 3D face network model based on Poisson equation, which integrates the face semantics, and the 3D face network model is also significantly optimized. In the future, we will consider integrating machine learning-based methods to improve the system performance.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The author declares that there are no conflicts of interest.
Acknowledgments
This study was supported by the Research on the innovation and reform of talent training mode under the background of “mass entrepreneurship and innovation”—Take the “art and design” major as an example (no. 2017107).