Abstract
In this paper, a new geometric structure of projective invariants is proposed. Compared with the traditional invariant calculation method based on 3D reconstruction, this method is comparable in the reliability of invariant calculation. According to this method, the only thing needed to find out is the geometric relationship between 3D points and 2D points, and the invariant can be obtained by using a single frame image. In the method based on 3D reconstruction, the basic matrix of two images is estimated first, and then, the 3D projective invariants are calculated according to the basic matrix. Therefore, in terms of algorithm complexity, the method proposed in this paper is superior to the traditional method. In this paper, we also study the projection transformation from a 3D point to a 2D point in space. According to this relationship, the geometric invariant relationships of other point structures can be easily derived, which have important applications in model-based object recognition. At the same time, the experimental results show that the eight-point structure invariants proposed in this paper can effectively describe the essential characteristics of the 3D structure of the target, without the influence of view, scaling, lighting, and other link factors, and have good stability and reliability.
1. Introduction
3D object recognition is one of the important contents of computer vision research, but the object imaging will be affected by the shooting angle, illumination, camera parameters, and other factors, which will make the same object in different shooting conditions get different images, leading to recognition difficulties [1]. Therefore, it is very important to find an invariant feature from the object which is not affected by these external factors. With the development of recognition technology, people not only find this invariant feature, geometric invariant [2–5], but also give some application examples. For example, Wei and Qiu studied the recognition method of a man-made object with a geometric structure line segment feature [6], Cao and Yan et al. studied deep learning with geometric invariance, [7], and Ding and Feng et al. studied the geometric correction of 3D human face with geometric invariance [8]. The geometric invariants were obtained by Cao et al. from another point of view by using the geometric algebra method [9–12]. At present, although many geometric invariants have been proposed by researchers and applied to 3D object recognition, face recognition, medical image recognition, and other fields, these invariants are still difficult to effectively eliminate the adverse effects caused by the difference of object parameters [13–16].
The recognition of 3D objects by a computer is to describe the features of the object by extracting the points of interest in the image and to recognize the object according to these features. Because the point features generally exist in any image, it is easy to extract, and the derivation and calculation of invariants based on point features is relatively simple, so it has been favored by more and more researchers. At the same time, the use of a single frame image to calculate its invariant has also been a widespread concern and promotion [17–19], but the previous calculation of the point, line, curve feature invariant requires that the extracted elements are coplanar, but the reality is that most of the elements obtained are not in a plane; thus, the previous methods have great limitations.
In view of this kind of present situation, based on the theory of Conformal Geometric Algebra (CGA), this paper proposes an eight-point geometric structure invariant in three-dimensional space, which has the following advantages.
First of all, because the invariant calculation mainly relies on the extracted point features and line features, which are commonly found in any image, the extraction method is relatively mature. Compared with the five, six, and seven-point structure invariants, the calculation of invariants based on point features and line features is relatively accurate and easy; second, the calculation of the invariant only needs a single frame image, which avoids the calculation of the fundamental matrix between multiple images and camera calibration, compared with the invariant obtained by using multiple frames, and is simple and convenient; finally, the invariants of other geometric structures can be easily deduced by using the algorithm proposed in this paper, and it is also beneficial to find more invariants of unknown structures, which shows that the invariant algorithm proposed in this paper has good generality and simplicity. At the same time, the simulation and real experiments also prove that the invariants extracted in this paper are not affected by the camera viewpoint and other factors and basically remain unchanged in different perspective images, which has good reliability and stability.
2. The Projective Transformation Relationship between 3D Space Points and Two-Dimensional Plane Points
Geometric invariant: let be a transformation group, is an element of and a geometric entity of in a space, stand for the coordinate parameter vector and the vector, respectively, and is the number of algebras with respect to and only related to the parameter vector . When acts on , there are
are the transformed coordinate vector and parameter vector, respectively, and is the number of generations with respect to and only with respect to the parameter vector ; if present,
Then, we show that is an invariant of the geometric body under the transformation group, where is a quantity which depends on only and is independent of the coordinates and parameters, and is an absolute invariant if equals (3).
Euclidean transformation, projective transformation, similar transformation, and affine transformation are all typical transformations in mathematics, which can be reduced to the transformation group in mathematics.
Let a point in 3D space project to a corresponding point through a projective centre O, and let and be the coordinates of and , respectively, in which case the relationship between the 3D coordinate and the 2D coordinate can be described by a projective matrix .
, which iswhere is a scale factor. is a projective matrix, related to parameters such as cameras.
Projective transformation is the key content of graphic transformation, and it is a bridge connecting 3D objects with 2D graphics. The projective transformation generally involves the projective matrix, but the projective matrix has high computational complexity and poor real-time performance and the results are easily affected by image noise. This problem has not been solved well yet. In order to avoid the specific calculation process of the projective matrix and to more conveniently represent the relationship between space points and their projection points, the following transformation is performed.
First, change Z in the spatial coordinate point to 0. We set to the rotation and translational operator in the Euclidean transformation, by projecting the i-th point on the space plane onto the image plane to derive from (3).
We set
Then, (3) changes intowhere is a matrix and are any constants, which are converted from . Since the third row Z of is 0, the third column of that can be ignored makes equation (6) simpler:where the matrix is obtained by removing the third column of and is obtained by removing the third row of .
For any point in space, the projected image point can be described by (7). On this basis, we can get the invariants of many spatial point structures.
2.1. Invariants of Eight-Point Geometry on Two Adjacent Planes in Space
In this paper, the relationship between the 3D point and its corresponding 2D plane projection point is obtained. Using this relationship, this paper proposes a new invariant of the geometric structure composed of points, that is, the geometric structure of eight points on two adjacent planes in space, as shown in Figure 1. Here, points A, B, C, D, and E are located on a plane , any three points of the five points are not collinear, points D, E, F, G, and H are located on the plane , likewise, any three of the five points are not collinear, and their corresponding 2D plane projection points are , respectively; then, by using equation (7), we get the following series of relationships.
For A, B, C, D, and E on the plane ,
We write points ; ; ; and , respectively, in a matrix form to get
The determinant is obtained from both sides of the equation:
Similarly, for on the plane , we have
We write points ; ; ; and , respectively, in a matrix form to get
The determinants on both sides are
If we setwherethen (15)∼(18) can be written in simpler forms as follows:
Then, the variant of the eight-point structure should be
The calculation of the invariant requires eight points. For the structure of the eight points, the flexibility is relatively large, and it is not easy to obtain the same projection structure, which leads to errors. So, it is highly applicable for the actual matching.
2.2. Comparison with Other Methods
Using the abovementioned method, it is convenient to obtain the invariant of the general position of the space at 5 o’clock and the invariant of the 7-point structure of the space proposed by Zhu et al. [18].
We use the algorithm of this paper to calculate the projection invariant of 5 points coplanar.
For a general position on the space plane , , , and any 3 points are not collinear. According to the formula, the relationship of the projection points can be obtained for each point as follows:
We write points 1, 2, 4; 1, 2, 5; 1, 3, 4; 1, 3, 5 in a matrix form, respectively.
Taking the determinant on both sides of the abovementioned four equations, we get
Similarly, we setwhere , and then, (25)∼(28), can be written in simpler forms as follows:
At this point, we can get the invariant of the 5 points as
The traditional method is to use the cross ratio to calculate its invariant, also known as the cross ratio invariant. It can be defined by a dot column or by a wire harness. If any four points are on line three, the cross ratio can be defined as
The ratio of the two secondary cross ratios is the geometric projection invariant of the four points of the collinear line:
However, when the calculation ratio reaches 5, the calculation complexity is high. At the same time, when using the cross ratio to calculate the invariant, there will be cases where the mismatched point ratios are equal, resulting in inaccurate matching, and the invariants can be easily obtained by the method in this section, and the matching efficiency is relatively high.
We use the algorithm of this paper to calculate the invariants of the spatial six-point structure proposed by Zhu et al.
Zhu et al. designed a geometric structure that intersects two planes in 3D space, taking two points C, D, E, and F at the intersection line AB and beyond the intersection line on the two planes. The geometric structure of 6 points is shown in Figure 2.
For two groups of coplanar points and , any three points in each group are not collinear, and their corresponding two-dimensional plane projection points are , respectively. According to the theory of the previous section, the relationship between the points in the 3D space and their projection points can be written into the following form.
For points on the plane and their projection points are
Writing the three points and the three points in a matrix, we get
Taking the determinant on both sides of the abovementioned two equations, we get
Similarly, for points on a plane and their projection points ,
Writing the three points and the three points in a matrix, we get
Taking the determinant on both sides of the abovementioned two equations, we get
Similarly, we set , , where , , and then, (39)∼(40) can be written in simpler forms as follows:
So, we can get its invariants as
The invariant is the same as the invariant proposed by Zhu et al., but it can be seen that Zhu et al. obtained by introducing an intersection of seven lines to obtain its invariant. The calculation method is more complicated, and the calculation process is relatively simple by using the algorithm in this section, and it is easy to extend to the invariants of other point structures in space.
2.3. Algorithm Design
Based on the abovementioned derivation, for the proposed eight-point invariant geometric structure, this section first gives the specific framework of the algorithm design. As shown in Figure 3, by using the 3D images obtained by using the 3D object and the camera, we can calculate the 3D invariant and 2D invariants and, then, compare and analyse the structures of the two invariants.
Since the invariant algorithm proposed in this paper is about the calculation of the point, the experiment must extract the feature of the point in the picture. The corner point is used as the feature point on the image and contains important information, which plays an important role in the understanding and analysis of the image. Here, we use the Harris and Piersol [19] corner detection algorithm to extract the points that need to be used in the experiment.
In addition, since the coplanar five points in the eight-point structure have any three points that are not collinear, the noncollinearity test needs to be performed first on the selected points. The so-called collinearity test means that, in a conformal geometric algebra, the straight line determined by any two points can be expressed as and, then, for any point , if is on a straight line, then the point satisfies the equation , but in the actual image application, the processing is easy to produce errors, the three points that seem to be noncollinear may be collinear, so we choose to use the area formed by the three points to measure whether they are collinear. To set a very small threshold , if the area meets the equation , then the three points are not collinear [20].
3. Experimental Results and Analysis
3.1. Experimental Platform
Hardware: personal PC, Intel (R) T2400 processor, 1.83 GHz, 1.00 GB memory, and one camera, 8 MP.
Software platform: Windows XP operating system, 3D graphics software AutoCAD, and image processing platform software MATLAB 7.5.0 (R2007b).
3.2. Simulation Experiment
The simulated image is drawn in AutoCAD 3D space. In order to obtain a geometric structure of two adjacent planes and eight points, we construct a model as shown in Figure 4. In order to see the point structure model at a glance, we extracted the model. The wireframe structure is shown in Figure 5, and select 8 points A–H as our experimental points.
First, we calculate the 3D invariant value of the point model, that is, the true value. If you use the point shown in the Figure 5 to establish the coordinate system as the origin, you can use AutoCAD to get the coordinates of the eight points, as shown in Table 1 and, then, use the formula to calculate its 3D invariant. The 3D projective invariant of the model is calculated to be 0.4135.
Second, in order to calculate the 2D projective invariants, we use the same camera to select the 12 frames of images acquired by the model under different viewpoints, rotation angles, and scaling, as shown in Figure 6, and, then, extract the 20 corner points in the subimage, as shown in Figure 7 by using the Harris corner detection algorithm. Also, their 2D invariants are, respectively, calculated.
After detecting the corner points, we extract the coordinates of the 8 points we need and, then, use the collinearity test described above to perform the noncollinearity test on the five points on the coplanar surface. Here, we select 12 frames in which the constraint is satisfied and calculate a 2D invariant of the 3D model. As shown in Table 2, represents the projective invariant values calculated from (3)–(20). The last column represents the absolute value of the relative error of each invariant to the true value of the 3D invariant calculated above 0.4135. The penultimate row represents all 2D mean values of the invariant and the error of the obtained average relative to the true 3D value, and the last line is the standard deviation of the obtained invariant value. We also used the methods of Zhang and Mui [3] and Deng et al. [21] to calculate the invariants of the abovementioned 12 graphs. The absolute values of the relative errors are listed in the third and fourth columns of the table below, as can be seen from the table. The method in this article has certain advantages.
The abovementioned table is more clearly expressed as shown in Figure 8.
It can be seen from Tables 1 and 2 and Figure 8 that as it adopts the simulation image, the 3D invariant calculated by the algorithm is 0.4135, and the average of the 2D invariant is 0.4176, that is, despite the different viewpoint positions of the camera and the changes of 3D model, the calculated invariant results are ideal, and the projective invariants remain basically unchanged in 3D objects and 2D images. At the same time, the same object remains basically unchanged under different viewpoints, and the error is small. This invariable property can facilitate target recognition and object matching in computer vision.
In order to verify the stability and accuracy of the eight-point geometric structure, the invariants of the eight-point geometric structure obtained in this paper are compared with those of six-point and seven-point geometric structures proposed by Zhu et al. The error percentage of the invariant of the six-point-seven-point geometric structure relative to the true value and the error percentage of the invariant of the eight-point geometric structure relative to the true value of the three-dimensional invariant in this experiment are calculated, respectively, to get Figure 9, in which the blue line represents the structure invariant error rate of the space six-point structure and the red line represents the structure invariant error rate of the space seven-point structure. The black line represents the spatial eight-point structure, the structure invariant, and the error rate. As can be seen from the figure, the accuracy of the eight-point invariant geometric structure proposed in this paper is slightly higher than that of the six-point and seven-point invariant structures, and the stability is also improved.
3.3. Real Image Experiment
In order to further verify the reliability of the geometric invariant structure proposed in this paper in real images, we use books to form two adjacent planes, which constitute a 3D scene and, then, select eight points on the plane to form the proposed the eight-point invariant structure, as shown in Figure 10. The figure is marked with 8 points, among which are located in one plane and on another adjacent plane. Also, are located on the intersection of the two planes. In order to verify that the invariant structure is protectively invariant, 20 views are acquired from different positions and angles using the same camera, as shown in Figure 11. Then, the corner points of each image are extracted by using the Harris corner extraction, as shown in Figure 12, and finally, one 2D geometric invariant of each image is calculated using the extracted corner points.
Figure 12 shows a Harris corner extraction map of the 20 images.
It can be seen from the corner extraction diagram that the eight points required for the experiment are basically detected. Now, we use the extracted corner points to calculate the 2D invariant and compare them with the methods of Zhang and Mui [3] and Deng et al. [21]. The calculation results are shown in Table 3. The first column is the image number, the second column is the invariant value of the corresponding image, and the third, fourth, and fifth columns use, as in the work of Zhang and Mui [3] and Deng et al. [21], the error percentage of the invariant value of each image calculated by the method in this paper relative to the average value. The last line is the average value of the invariant results of the 20 images and the average value of the error.
Using the method in this paper to compare with the work of Zhang and Mui [3] and Deng et al. [21], it was seen that, except the error on the image 11, it was 18 points lower than the method proposed by Deng et al. [21], the error of other image invariant points is better than that in the work of Deng et al. [21], the error of the image was 11, and 18 points of this method is completely better than that of that in the work of Zhang and Mui [3], which proves that the method has a certain cutting edge.
As can be seen from Table 3, the invariant value is basically stable around 6.4, and the calculated results are generally ideal, which shows that the geometric structure of the eight-point projective invariant proposed in this paper remains basically unchanged under different viewpoints, and the error is small. However, there is still a problem in the experiment; when the view is scaled to a very small size, the corner cannot be detected accurately, resulting in a large error in the calculation of invariants, so it is very important to find a better structure of invariants or other corner extraction methods. In order to more clearly see the distribution of invariants, MATLAB is used to draw its invariant distribution, as shown in Figure 13; from the figure, it can be seen that invariant value distribution is flat, basically unchanged.
4. Conclusions
In this paper, we propose a new geometric structure of projective invariants. Compared with the traditional invariant calculation method based on 3D reconstruction, the reliability of the invariant calculation is comparable. Moreover, the method proposed in this paper only needs to find out the geometric relationship between 3D empty points and two-dimensional space points, and its invariants can be obtained by using a single frame image. In the method based on 3D reconstruction, the basic matrix of two images is estimated first, and then, the 3D projective invariants are calculated according to the basic matrix. Therefore, in terms of algorithm complexity, the method proposed in this paper is superior to the traditional method.
In addition, this paper studies the projection transformation relationship from a 3D point to a 2D point in space. According to this relationship, the geometric invariant relationships of other point structures can be easily derived, which have important applications in model-based object recognition. At the same time, the experimental results show that the eight-point structure invariant proposed in this paper can effectively describe the essential characteristics of the 3D structure of the target, is not affected by view, scaling, lighting, and other link factors, and has good stability and reliability.
Data Availability
The (PNG images) data used to support the findings of this study have not been made available because of privacy issues.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant nos. 61871186 and 61771322 and the Zhejiang Natural Science Foundation under Grant no. LY14F030007.