Incremental Graph Regulated Nonnegative Matrix Factorization for Face Recognition

Abstract

In a real world application, we seldom get all images at one time. Considering this case, if a company hired an employee, all his images information needs to be recorded into the system; if we rerun the face recognition algorithm, it will be time consuming. To address this problem, In this paper, firstly, we proposed a novel subspace incremental method called incremental graph regularized nonnegative matrix factorization (IGNMF) algorithm which imposes manifold into incremental nonnegative matrix factorization algorithm (INMF); thus, our new algorithm is able to preserve the geometric structure in the data under incremental study framework; secondly, considering we always get many face images belonging to one person or many different people as a batch, we improved our IGNMF algorithms to Batch-IGNMF algorithms (B-IGNMF), which implements incremental study in batches. Experiments show that (1) the recognition rate of our IGNMF and B-IGNMF algorithms is close to GNMF algorithm while it runs faster than GNMF. (2) The running times of our IGNMF and B-IGNMF algorithms are close to INMF while the recognition rate outperforms INMF. (3) Comparing with other popular NMF-based face recognition incremental algorithms, our IGNMF and B-IGNMF also outperform then both the recognition rate and the running time.

1. Introduction

Nonnegative matrix factorization (NMF) is a widely used method for low-rank approximation of a nonnegative matrix (matrix with only nonnegative entries), where nonnegative constraints are imposed on factor matrices in the decomposition. There are large bodies of past work on NMF [1]. Lee and Seung [2, 3] proposed NMF for learning parts of faces, and in their work the reconstruction error function 𝐹(𝑈,𝑉) is introduced: 𝐹(𝑈,𝑉)=‖𝑋−𝑈𝑉‖²_𝐹, where 𝑋 denotes the data matrix, 𝑈 can be considered as the basic matrix, and 𝑉 can be considered as the coefficient matrix; all elements of 𝑋, 𝑈, and 𝑉 are nonnegative. Sparse coding is a famous parts-based representation method, by minimizing a 𝐿₁ regularization-related objective function of NMF-based algorithms, sparse constraints can be achieved. Hoyer [4] proposed a method by keeping 𝐿₂ norm unchanged in each iteration, but 𝐿₁ norm set to achieve desired sparseness. Li et al. [5] proposed another sparse representation method which focused on sparse NMF algorithm with Kullback-Leibler based cost function. Discrimination method was also introduced into NMF algorithm. Wang et al. [6] proposed a Fisher nonnegative matrix factorization which introduces Fisher constraint (discrimination) method into NMF algorithm. later, Nikitidis et al. [7] introduced subclass discriminant into NMF algorithm, by separating each class into several subclasses; this method was able to ensure that the underlying data distribution in each subclass is unimodal. Because the convergence of canonical NMF algorithms is slow, gradient descent based methods are introduced to NMF to improve its speed of convergence. Guan et. al [8] applied Nesterov’s optimal gradient method to alternatively optimize one factor with another. By introducing fast gradient descent method into search of the optimal step size for gradient descent based NMF algorithm, Guan et al. [9] introduced nonnegative patch alignment framework (NPAF) and nonnegative discriminative locality alignment (NDLA). Canonical NMF algorithm aims to minimize the Euclidean distance or the Kullback-Leibler distance between the data matrix 𝑋 and its reconstruction matrix 𝑈×𝑉. By introducing Manhattan distance into NMF algorithm, Guan et al. [10] introduced Manhattan nonnegative matrix factorization (MahNMF), then rank-one residual iteration method and Nesterov’s smoothing method are both introduced to optimizing MahNMF. The geometrical information of the original data space is an important information for face recognition. Zhang et al. [11] proposed a novel topology preserving nonnegative matrix factorization (TPNMF) which considers the gradient distance instead of the Euclidean distance or the Kullback-Leibler distance. By constructing the intrinsic and penalty graphs, Liu et al. [12] proposed the projective nonnegative graph embedding (PNGE). Cai et al. [13] proposed graph regularized nonnegative matrix factorization (GNMF) algorithm for face recognition. GNMF imposes manifold into NMF, which makes it able to preserve geometric structure of data after it maps the original image into low dimension space.

NMF-based algorithms are notoriously slow to converge. In practice, if new image comes, algorithm needs to rerun which is time consuming. To address this problem, incremental process used for online study has attracted a lot of attention. During an incremental study process of NMF, once a factorization, such as 𝑈 and 𝑉 which mentioned above, is obtained for a group of face images, the representation of 𝑈 should be updated with a small computational cost, but the canonical NMF and many other algorithms are too expensive to compute 𝑈 and 𝑉 every time when new face image comes. Bucak and Gunsel [14] introduced an Incremental nonnegative matrix factorization algorithm (INMF), which imposes NMF into incremental study. This makes it possible to represent data content online and reduce dimension significantly. Chen et al. [15] proposed another incremental nonnegative matrix factorization (INMF) for face representation and recognition. In order to distinguish Chen’s algorithms [15] from Bucak and Gunsel’s [14], we called Chen’s algorithms CINMF. There are two main differences between CINMF and INMF: first, INMF is an unsupervised method while CINMF is a supervised method; second, INMF can only deal with one new image while CINMF can only deal with a batch of images which belong to a new class. Wang and Lu [16] introduced an incremental orthogonal projective nonnegative matrix factorization algorithm (IOPNMF) which introduced orthogonal projective constraint into incremental NMF. Lefèvre et al. [17] introduced incremental study into Itakura-Saito NMF algorithm; the Istakura-Saito divergence was defined as 𝑑_IS(𝑦,𝑥)=∑_𝑖(𝑦_𝑖/𝑥_𝑖−log𝑦_𝑖/𝑥_𝑖−1). The proposed algorithm is used in the field of audio source separation. Guan et al. [18] proposed an efficient online RSA-NMF algorithm. RSA-NMF is a NMF algorithm based which introduces Robust Stochastic approximation (RSA) to optimize NMF algorithm. Benefiting from the smartly chosen learnig rate and averaging technique, online RSA-NMF algorithm converges at the rate of 𝑂(1/√𝑘) in each update of the bases. later Guan et al. also gave some extensions on the proposed online RSA-NMF algorithm. Cao et al. [19] introduced an online NMF algorithm. Once we get the factorization of 𝑋 as 𝑈×𝑉, by separating the new data matrix 𝑋, and the new coefficient matrix 𝑉 into two corresponding parts, the inequation can be rewritten as 𝑋𝐱≈𝑈^′×𝑉^′𝐯; every iteration, 𝑉^′𝐯, can be recomputed with a small computation cost while 𝑈^′ can be recomputed as 𝑈^′=𝑈×(𝑉^′)⁻¹; obviously there is an assumption that 𝑋≈𝑈×𝑉 are full-rank decompositions and 𝑉^′ is invertible.

Although the incremental algorithms we mentioned above dramatically reduce the cost of incremental study, they did not consider the geometrical information of the original data space on the original manifold. The geometrical information of the original data space is an important information for face recognition [20, 21]. Data set often lay in a high dimension manifold, so the intrinsic geodesic distance in the manifold between two data points is more suitable than Euclidean distance. Thus, in this paper, By introducing manifold into Incremental nonnegative matrix factorization algorithm (INMF), first, we propose a new incremental study algorithm called incremental graph regulated nonnegative matrix factorization (IGNMF). Second, as human face images always come in batches, we improve our IGNMF algorithms to Batch-IGNMF algorithms (B-IGNMF). The contributions of the research presented in this paper are as follows.(1)We propose a novel subspace incremental method called incremental graph regularized nonnegative matrix factorization algorithm (IGNMF), which is able to preserve discrimination information under incremental study framework.(2)We further improve our IGNMF algorithms, thus developing Batch-IGNMF (B-IGNMF), which is able to perform incremental study with image batches no mater whether the batch of images belongs to the same class or different classes.

The remainder of this paper is organized as follows. Section 2 introduces the GNMF algorithm and INMF algorithm. Section 3 presents our IGNMF and B-IGNMF algorithms. Experiments on face databases are reported in Section 4. Section 5 concludes the paper.

2. A Brief Review of GNMF and INMF

In this section, we will give a brief review on graph regularized nonnegative matrix factorization (GNMF) which was proposed by Cai et al. [13] and incremental nonnegative matrix factorization algorithm (INMF) which was proposed by Bucak and Gunsel [14]; both algorithms are closely related to our new proposed algorithms.

2.1. GNMF

The geometrical information of the original data space is an important information for face recognition [20, 21]; data set often lay in a high dimension manifold, so the intrinsic geodesic distance in the manifold between two data points is more suitable than Euclidean distance. Thus in this section, we will give a brief review on graph regularized NMF (GNMF) [13].

Given a data matrix 𝑋=[𝑥₁,𝑥₂,…,𝑥_𝑘]∈ℝ^𝑛×𝑘, there are total 𝑘 face image vectors and 𝑥_𝑖(1⩽𝑖⩽𝑘) denotes a vector of 𝑛-dimensional facial image, 𝑈∈ℝ^𝑛×𝑟 and 𝑉∈ℝ^𝑟×𝑘, so the objective function 𝐹(𝑈,𝑉) for GNMF is defined as (1); by minimizing 𝐹(𝑈,𝑉) a factorization, 𝑈 and 𝑉 in GNMF, is obtained: 𝐹=‖𝑋−𝑈𝑉‖²_𝐹+𝜆Tr𝑉𝐿𝑉^𝑇. (1)

In the above equation 𝐹=‖⋅‖²_𝐹 denotes the matrix Frobenius norm [2]; Tr(⋅) denotes the trace of a matrix; 𝜆>0 is the regularization parameter; 𝐿 is called graph Laplacian [22]; 𝐿=𝐷−𝑊, where 𝑊 is the weight matrix, 𝑊_𝑖𝑗=1 if 𝑥_𝑖∈𝑁_𝑝(𝑥_𝑗), and 𝐷 is a diagonal matrix and 𝐷_𝑖𝑖=∑_𝑗𝑊_𝑖𝑗. There are two methods to define 𝑊 [13, 22].(1)Supervised method is achieved by defining 𝑥_𝑖∈𝑁_𝑝(𝑥_𝑗) if 𝑥_𝑖 and 𝑥_𝑗 belong to the same class.(2)Unsupervised method is achieved by defining 𝑁_𝑝(𝑥_𝑖) as the set of 𝑝 nearest neighbors of 𝑥_𝑖 under the Euclidean distance.

The iterative update algorithm proposed by Cai et al. [13] is as follows (2): 𝑈^𝑡+1_𝑖𝑗=𝑈^𝑡_𝑖𝑗⋅𝑋𝑉^𝑇_𝑖𝑗𝑈𝑉𝑉^𝑇_𝑖𝑗𝑉^𝑡+1_𝑖𝑗=𝑉^𝑡_𝑖𝑗⋅𝑈^𝑇𝑋+𝜆𝑉𝑊_𝑖𝑗𝑉^𝑇𝑈^𝑇𝑈+𝜆𝑉𝐷_𝑖𝑗, (2) where 𝑈_𝑖𝑗 and 𝑉_𝑖𝑗 are the elements of 𝑈 and 𝑉, respectively.

2.2. INMF

Incremental nonnegative factorization (INMF) [14] is a popular incremental study algorithm which is NMF-based. Considering the canonical reconstruction inequation of NMF algorithm as 𝑋≈𝑈×𝑉, and each column of both sides is 𝑋_𝑖≈𝑈×𝑉_𝑖; from this inequation we can find that if we consider each column of the matrix 𝑈 as the building block of the database and 𝑉_𝑖 can be considered as the reconstruction coefficient matrix, by summing each building block of 𝑈 under the coefficient matrix 𝑉_𝑖, original image 𝑋_𝑖 will be approximately reconstructed [2, 3].

Every time when new image 𝐱 comes, the inequation of the reconstruction function will become 𝑋𝐱≈𝑈×𝑉𝐯; by assuming that the previous coefficient matrix 𝑉 would not change during the incremental process, the computation cost will be reduced significantly.

If we defined the 𝐹_𝑘 as the cost function corresponding to the NMF as (3) where 𝑘 denotes the total number of samples before incremental study, then 𝐹_𝑘=‖𝑋−𝑈𝑉‖²=^𝑛𝑘_𝑖=1𝑗=1𝑋_𝑘𝑖𝑗−𝑈_𝑘𝑉_𝑘𝑖𝑗². (3)

Everytime, when the (𝑘+1)th sample 𝐱 arrives, the corresponding cost function 𝐹_𝑘+1 can be defined as (4) 𝐹_𝑘+1=^𝑛_𝑖=1^𝑘+1_𝑗=1𝑋_{𝑘+1𝑖𝑗}−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}². (4)

Note 𝐹_𝑘+1 can be separated into two parts as follows: 𝐹_𝑘+1=^𝑛_𝑖=1^𝑘+1_𝑗=1𝑋_{𝑘+1𝑖𝑗}−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}²=^𝑛𝑘_𝑖=1𝑗=1𝑋_{𝑘+1𝑖𝑗}−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}²+^𝑛_𝑖=1𝐱−𝑈_𝑘+1𝐯_𝑖,𝑘+1², (5) where 𝐱 and 𝐯 can be considered as the new coming image and its corresponding coefficient vector. By assuming that the first 𝑘 columns of 𝑉_𝑘+1 would not change after the incremental study, the first part of (5) can be rewritten as ^𝑛𝑘_𝑖=1𝑗=1𝑋_{𝑘+1𝑖𝑗}−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}²≈𝐹_𝑘=^𝑛𝑘_𝑖=1𝑗=1𝑋_𝑘𝑖𝑗−𝑈_𝑘𝑉_𝑘𝑖𝑗². (6) So 𝐹_𝑘+1 can be rewritten as 𝐹_𝑘+1=𝐹_𝑘+^𝑛_𝑖=1𝐱−𝑈_𝑘+1𝐯_𝑖,𝑘+1²=𝐹_𝑘+𝑓. (7)

Considering that 𝐹_𝑘 is the function which is independent of 𝑣_𝑖, the partial derivative of 𝜕𝐹_𝑘/𝐯_𝑖=0. Thus the partial derivative of 𝜕𝐹_𝑘+1/𝐯_𝑖(=𝜕𝑓/𝐯_𝑖) and 𝜕𝐹_𝑘+1/𝑈_𝑖𝑗 can be deduced. Then the update rule of 𝐯_𝑖 and 𝑈_𝑖𝑗 can be formulated within the framework of gradient descent algorithm.

In order to save space, here we just list the iterative update algorithms of INMF in (8): 𝐯^′_𝑖⟵𝐯_𝑖⋅𝑈^𝑇_𝑘+1𝐱_𝑖𝑈^𝑇_𝑘+1𝑈_𝑘+1𝐯_𝑖,𝑈^′_{𝑘+1𝑖𝑎}⟵𝑈_{𝑘+1𝑖𝑎}⋅𝑋_𝑘𝑉^𝑇_𝑘+𝑋_𝑘+1𝐯^𝑇_𝑖𝑎𝑈_𝑘+1𝑈_𝑘𝑉^𝑇_𝑘+𝑈_𝑘+1𝐯𝐯^𝑇_𝑖𝑎. (8)

3. IGNMF and B-IGNMF

Bucak and Gunsel [14] introduced an incremental nonnegative matrix factorization algorithm (INMF) which imposed the NMF algorithm into incremental study, so INMF inherits the disadvantage of the NMF algorithm; that is, it does not consider the geometric structure in the data. In this section, we introduced an incremental graph regularized nonnegative matrix factorization algorithm (IGNMF), in which manifold is introduced to overcome this limitation.

3.1. IGNMF

Let 𝑋_𝑘, 𝑈_𝑘, 𝑉_𝑘, 𝐿_𝑘, and 𝐹_𝑘 denote the corresponding 𝑋, 𝑈, 𝑉, 𝐿, and 𝐹(𝑈,𝑉) in (1) under the initial 𝑘 samples, so the objective function 𝐹_𝑘 can be rewriten as 𝐹_𝑘=^𝑛𝑘_𝑖=1𝑗=1𝑋_𝑘𝑖𝑗−𝑈_𝑘𝑉_𝑘𝑖𝑗²+𝜆Tr𝑉_𝑘𝐿_𝑘𝑉^𝑇_𝑘. (9)

Let 𝑋_𝑘+1, 𝑈_𝑘+1, 𝑉_𝑘+1, 𝐿_𝑘+1, and 𝐹_𝑘+1 denote the corresponding 𝑋, 𝑈, 𝑉, 𝐿, and 𝐹(𝑈,𝑉) in (1) when the (𝑘+1)th sample arrives. So objective function 𝐹_𝑘+1 can be rewriten as 𝐹_𝑘+1=^𝑛_𝑖=1^𝑘+1_𝑗=1𝑋_{𝑘+1𝑖𝑗}−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}²+𝜆Tr𝑉_𝑘+1𝐿_𝑘+1𝑉^𝑇_𝑘+1. (10)

𝑈 can be considered as the basic matrix and 𝑉 can be considered as the coefficient matrix, so the reconstruction process can be thought as adding columns of matrix 𝑈 under the coefficient matrix 𝑉_𝑖, just as 𝑋_𝑖≈𝑈×𝑉_𝑖 [3]. Thus we make the assumption that during the incremental process, when the (𝑘+1)th sample arrives, the first 𝑘 columns of 𝑉_𝑘+1 does not change, which means the fist 𝑘 columns of 𝑉_𝑘+1 is approximately equal to 𝑉_𝑘. This assumption would reduce the computation cost significantly. Experiments show that IGNMF would iterate less than 5 times and then the objective function converges to its minimal value, because we just need to update the last column of 𝑉_𝑘+1 meanwhile 𝑈_𝑘+1 needs to be updated completely which will dramatically reduce the cost of incremental study. For more details about this assumption, please refer to [14].

Assuming 𝐹^𝑘_𝑘+1 refers to the objective function corresponding to GNMF representation of the first 𝑘 sample when the (𝑘+1)th sample arrives, 𝐿^𝑘_𝑘+1 refers to the 𝑘×𝑘 dimensional matrix which equals the first 𝑘 rows and first 𝑘 columns of 𝐿_𝑘+1, and 𝑟 is a predefined parameter which indicates that the 𝑛-dimensional facial image vector maps in to an 𝑟-dimensional vector. So 𝐹^𝑘_𝑘+1 can be rewritten as 𝐹^𝑘_𝑘+1=^𝑛𝑘_𝑖=1𝑗=1𝑋_𝑖𝑗−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}²+𝜆^𝑟𝑘_𝑙=1^𝑘_𝑖=1𝑗=1𝑉_{𝑘+1𝑙𝑖}𝐿^𝑘_{𝑘+1𝑖𝑗}𝑉^𝑇_{𝑘+1𝑗𝑙}≈^𝑛𝑘_𝑖=1𝑗=1𝑋_𝑖𝑗−𝑈_𝑘+1𝑉_𝑘𝑖𝑗²+𝜆^𝑟𝑘_𝑙=1^𝑘_𝑖=1𝑗=1𝑉_𝑘𝑙𝑖𝐿^𝑘_{𝑘+1𝑖𝑗}𝑉^𝑇_𝑘𝑗𝑙. (11)

Consequently, objective function 𝐹_𝑘+1 can be rewriten as follows: 𝐹_𝑘+1=^𝑛_𝑖=1^𝑘+1_𝑗=1𝑋_𝑖𝑗−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}²+𝜆^𝑟_𝑙=1^𝑘+1_𝑖=1^𝑘+1_𝑗=1𝑉_{𝑘+1𝑙𝑖}𝐿_{𝑘+1𝑖𝑗}𝑉^𝑇_{𝑘+1𝑗𝑙}≈^𝑛𝑘_𝑖=1𝑗=1𝑋_𝑖𝑗−𝑈_𝑘+1𝑉_𝑘𝑖𝑗²+𝜆^𝑟𝑘_𝑙=1^𝑘_𝑖=1𝑗=1𝑉_𝑘𝑙𝑖𝐿^𝑘_{𝑘+1𝑖𝑗}𝑉^𝑇_𝑘𝑗𝑙+^𝑛_𝑖=1𝐱−𝑈_𝑘+1𝐯_𝑖²+𝜆^𝑟_𝑙=1𝐯²𝐿_{𝑘+1𝑘+1,𝑘+1}+^𝑘_𝑗=12𝐯𝑉_𝑘𝑙𝑗𝐿_{𝑘+1𝑗,𝑘+1}=𝐹^𝑘_𝑘+1+𝑓_𝑘+1, (12) where 𝐱 means the new coming facial vector and 𝐯 means the coefficient vector corresponding to 𝐱.

After constructing the objective function given by (12), gradient descent optimization that yields IGNMF can be performed. The update rule of 𝐯 can be formulated within the framework of gradient descent algorithm as follows: 𝐯^′_𝑎⟵𝐯_𝑎−𝜇_𝑎⋅𝜕𝐹_𝑘+1𝜕𝐯_𝑎. (13)

In (13), 𝜕𝐹_𝑘+1/𝜕𝐯_𝑎 is the partial derivative of 𝐹_𝑘+1 with respect to 𝐯_𝑎 and 𝜕𝐹_𝑘+1/𝜕𝐯_𝑎=𝜕𝐹^𝑘_𝑘+1/𝜕𝐯_𝑎+𝜕𝑓_𝑘+1/𝜕𝐯_𝑎, because 𝐹^𝑘_𝑘+1 is the function which is not relevant to 𝐯_𝑎, so 𝜕𝐹^𝑘_𝑘+1/𝜕𝐯_𝑎=0, and we just need to compute the partial derivative of 𝑓_𝑘+1:𝜕𝐹_𝑘+1𝜕𝐯_𝑎=𝜕𝐹^𝑘_𝑘+1𝜕𝐯_𝑎+𝜕𝑓_𝑘+1𝜕𝐯_𝑎=𝜕𝜕𝐯_𝑎^𝑛_𝑖=1𝐱−𝑈_𝑘+1𝐯_𝑖²+𝜆^𝑟_𝑙=1𝐯²𝐿_{𝑘+1𝑘+1,𝑘+1}+^𝑘_𝑗=12𝐯𝑉_𝑘𝑙𝑗𝐿_{𝑘+1𝑗,𝑘+1}+𝑉_𝑘²_𝑗,𝑘+1𝑊_{𝑘+1𝑙,𝑘+1}=2−𝑈^𝑇_𝑘+1𝐱+𝑈^𝑇_𝑘+1𝑈_𝑘+1𝐯+𝜆𝐯𝐿_{𝑘+1𝑘+1,∶𝑎}. (14)

𝜇_𝑎 is the step size and is calculated by 𝜇_𝑎=𝐯_𝑎2𝑈^𝑇_𝑘+1𝑈_𝑘+1𝐯+𝜆⋅𝐯𝐷_{𝑘+1𝑘+1,∶𝑎}, (15) where (𝐿_𝑘+1)_𝑘+1,∶, (𝐷_𝑘+1)_𝑘+1,∶ and (𝑊_𝑘+1)_𝑘+1,∶ are vectors which are equal to the (𝑘+1)th row of 𝐿_𝑘+1, 𝐷_𝑘+1 and 𝐷_𝑘+1, respectively.

After substituting (14) and (15) into (13), The update rule equation for 𝐯_𝑎 yields as 𝐯^′_𝑎⟵𝐯_𝑎⋅𝑈^𝑇_𝑘+1𝐱+𝜆⋅𝐯𝑊_{𝑘+1𝑘+1,∶𝑎}𝑈^𝑇_𝑘+1𝑈_𝑘+1𝐯+𝜆⋅𝐯𝐷_{𝑘+1𝑘+1,∶𝑎}. (16)

The update rule of (𝑈_𝑘+1)_𝑖𝑎 can also be formulated within the framework of gradient descent algorithm as 𝑈^′_{𝑘+1𝑖𝑎}⟵𝑈_{𝑘+1𝑖𝑎}−𝜂_𝑖𝑎⋅𝜕𝐹_𝑘+1𝜕𝑈_{𝑘+1𝑖𝑎}. (17) In (17), 𝜕𝐹_𝑘+1/𝜕(𝑈_𝑘+1)_𝑖𝑎 is the partial derivative of 𝐹_𝑘+1 with respect to (𝑈_𝑘+1)_𝑖𝑎 and is given as 𝜕𝐹_𝑘+1𝜕𝑈_{𝑘+1𝑖𝑎}=𝜕𝜕𝑈_{𝑘+1𝑖𝑎}^𝑛_𝑖=1^𝑘+1_𝑗=1𝑋_𝑖𝑗−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}²+𝜆^𝑟_𝑙=1^𝑘+1_𝑖=1^𝑘+1_𝑗=1𝑉_{𝑘+1𝑙𝑖}𝐿_{𝑘+1𝑖𝑗}𝑉^𝑇_{𝑘+1𝑗𝑙}=𝜕𝜕𝑈_𝑘+1^𝑛_{𝑖𝑎𝑖=1}^𝑘+1_𝑗=1𝑋_𝑖𝑗−𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}²=2^𝑘+1_𝑗=1−𝑋_𝑖𝑗𝑈_{𝑘+1𝑎𝑗}+𝑉_{𝑘+1𝑎𝑗}𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}. (18)𝜂 is the step size and is calculated by 𝜂_𝑖𝑎=𝑈_{𝑘+1𝑖𝑎}2∑^𝑘+1_𝑗=1𝑈_𝑘+1𝑉_{𝑘+1𝑖𝑗}𝑉^𝑇_𝑗𝑎. (19) After substituting (18) and (19) into (17), The update rule equation for (𝑈_𝑘+1)_𝑖𝑎 yields as 𝑈^′_{𝑘+1𝑖𝑎}⟵𝑈_{𝑘+1𝑖𝑎}⋅𝑋_𝑘𝑉^𝑇_𝑘+𝑋_𝑘+1𝐯^𝑇_𝑖𝑎𝑈_𝑘+1𝑈_𝑘𝑉^𝑇_𝑘+𝑈_𝑘+1𝐯𝐯^𝑇_𝑖𝑎. (20)

We omit the proof of its convergence as it is similar to that of GNMF while IGNMF just assumes the first 𝑘 columns of 𝑉 would not be updated during iterations.

3.2. Batch-IGNMF

In practice, images often come in batches. For example, a face recognition system needs to record a batch of images belonging to a new class or belonging to many different classes.

Once several images come, our IGNMF needs to run one time for each image, which is time consuming if there are too many images. In this section, we propose an improved version of IGNMF algorithm which can deal with a batch of images. This improved version is named Batch-IGNMF (B-IGNMF), which is able to perform incremental study in batch of images no matter whether the batch of images belongs to the same class or different classes.

Let 𝑋_𝑙 denote the new coming 𝑙 images. So 𝑋_𝑘+𝑙 is the total sample with its first 𝑘 columns equal to 𝑋_𝑘 and the rest 𝑙 columns equal to 𝑋_𝑙. 𝑈_𝑘+𝑙 and 𝑉_𝑘+𝑙 denote the optimized factor matrices of 𝑋_𝑘+𝑙, assuming 𝑉_𝑘 as the first 𝑘 columns of 𝑉_𝑘+𝑙 which would not change during incremental study, 𝑉_𝑙 is the last 𝑙 columns of 𝑉_𝑘+𝑙 denoting the corresponding coefficient matrix of 𝑋_𝑙, and (𝑊_𝑘+𝑙)_{𝑘+1∼𝑘+𝑙,∶} denotes the rows of 𝑊_𝑘+𝑙 from 𝑘+1 to 𝑘+𝑙.

So the objective function 𝐹_𝑘+𝑙 can be rewriten as 𝐹_𝑘+𝑙=^𝑛_𝑖=1^𝑘+𝑙_𝑗=1𝑋_𝑖𝑗−𝑈_𝑘+𝑙𝑉_{𝑘+𝑙𝑖𝑗}²+𝜆Tr𝑉_𝑘+𝑙𝐿_𝑘+𝑙𝑉^𝑇_𝑘+𝑙. (21) We omit the detailed derivation process due to its similarity to the derivation process of IGNMF. We just list the multiplicative update rules as 𝑉^′_𝑙𝑖𝑎⟵𝑉_𝑙𝑖𝑎⋅𝑈^𝑇_𝑘+1𝑋_𝑙+𝜆⋅𝑉_𝑘+𝑙𝑊_{𝑘+𝑙𝑘+𝑙∼𝑘+𝑙,∶𝑖𝑎}𝑈^𝑇_𝑘+𝑙𝑈_𝑘+𝑙𝑉_𝑙+𝜆⋅𝑉_𝑘+𝑙𝐷_{𝑘+𝑙𝑘+𝑙∼𝑘+𝑙,∶𝑖𝑎},𝑈^′_{𝑘+𝑙𝑖𝑎}⟵𝑈_{𝑘+𝑙𝑖𝑎}⋅𝑋_𝑘𝑉^𝑇_𝑘+𝑋_𝑙𝑉^𝑇_𝑙𝑖𝑎𝑈_𝑘+𝑙𝑈_𝑘𝑉^𝑇_𝑘+𝑈_𝑘+𝑙𝑉_𝑙𝑉^𝑇_𝑙𝑖𝑎. (22)

4. Experiments

In this section, the FERET database [23, 24] and CMU-PIE database [25] are selected to evaluate the performance of our IGNMF and B-IGNMF algorithms, along with two canonical face recognition algorithms: supervised GNMF (GNMF-S) and unsupervised GNMF (GNMF-U) and three incremental algorithms: INMF, CINMF, and IOPNMF. NMF is selected as the baseline. The stopping condition of iterative update is defined as (23) in all experiments: 𝐹^(𝑛−1)−𝐹^(𝑛)𝐹^(𝑛)<𝛿, (23) where 𝐹^(𝑛) is the 𝑛th iteration of the update criterion function defined in (9) under the 𝑛th iteration.

In the FERET database [23, 24], we select 200 individual faces (7 images for each). All images in the database are aligned by the centers of eyes and mouth and then resized into 25×25. The CMU-PIE database includes total of 68 individuals [25] and we select all these 68 individuals, each of which has 56 images, including 7 poses with neural expression in different views and 38 different lighting conditions and 11 talking images in frontal view. All images in the database are aligned by the centers of eyes and mouth and then resized into 28×23. Examples of these two datasets are shown in Figures 1 and 2. Each experiment is run for 10 trials, and the parameters for our IGNMF and B-IGNMF are set as follows: 𝜆=0.01 and 𝛿=0.01 for both databases and 𝑟=200 for the FERET database and 𝑟=68 for the CMU-PIE database. Then the average accuracies and the mean running time are recorded.

Figure 1 

All images of one person from FERET database.

Figure 2 

All images of one person from CMU-PIE database.

4.1. Incremental Study for Single Image

In these experiments, we choose 5 images for each individual as the training set from the FERET database and 42 images for each individual from the CMU-PIE database. Our experiments are performed as follows: first, GNMF and NMF are chosen for initialization; second, IGNMF and INMF are performed to incremental study; GNMF and NMF are also performed by rerunning GNMF and NMF every time when new image comes. We choose one image of each individual from the rest of the database to incremental study, so all algorithms were run at a total of 200 times for the FERET database and 68 times for the CMU-PIE database. Notice INMF is an unsupervised learning algorithm, so unsupervised IGNMF is performed.

Figures 3 and 4 illustrate the curves of recognition rates for INMF, IGNMF, NMF, and GNMF during incremental study. We can see that when a small number of new images come, incremental methods (INMF and IGNMF) perform better than canonical methods (NMF and GNMF). We also observe that incremental methods can keep the recognition rate increasing smoothly with the number of images increasing; while canonical methods would make the recognition rate fluctuating. This is because every time when we rerun the canonical methods, the initial values for 𝑈 and 𝑉 are randomly assigned; different initial values would lead to different local minimum value, which in turn affect the recognition rate significantly. It is also noted that the recognition rates of IGNMF and GNMF are better than those of NMF and INMF; this is because the use of the manifold method, which preserves the geometrical information of the original data, contributes to the improvement of the recognition rate.

Figure 3 

The average recognition rate during incremental study for the FERET database.

Figure 4 

The average recognition rate during incremental study for the CMU-PIE database.

Figures 5 and 6 illustrate the running time of the selected algorithms in both databases, from the figure we can see that the mean running time of IGNMF for single image is close then INMF, while faster than GNMF and NMF, which means IGNMF achieved better recognition rate than INMF while faster than it.

Figure 5 

The mean running time during incremental study with single image for the FERET database.

Figure 6 

The mean running time during incremental study with single image for the CMU-PIE database.

4.2. Incremental Study for One Batch of Images Belonging to One Class

In this section, our B-IGNMF, and the other two typical incremental algorithms (CINMF and IOPNMF) are performed along with supervised GNMF, unsupervised GNMF, and NMF. Because CINMF is a supervised learning algorithm while IOPNMF is an unsupervised one, supervised B-IGNMF (BIGNMF-S) and unsupervised B-IGNMF (BIGNMF-U) are both performed.

We choose 5 images for each individual as the training set from the FERET database and 42 images for each individual from the CMU-PIE database. First, we choose 195 individuals from the FERET database and 63 individuals from the CMU-PIE database for study. Second, one of the rest of five individuals is chosen to add for incremental study, total 5 times. The parameters of CINMF are set to 𝛼=10, 𝛽=10⁻⁷, and 𝑟₀=6 for the FERET database and 𝛼=0.1, 𝛽=10⁻⁴, and 𝑟₀=6 for the CMU-PIE database; the parameters of IOPNMF are set to 𝑟=200 for the FERET database and 𝑟=136 for the CMU-PIE database. Tables 1 and 2 showed the recognition rates for two databases; “Add 1” means the recognition rate of adding one batch of images into “Start,” and “Add 2” means the recognition rate of adding another one batch of images into “Add 1,” and so on.

Table 1 

The recognition rates (%) for the FERET database during adding 5 batches of images belonging to one class.

Table 2 

The recognition rates (%) for the CMU-PIE database during adding 5 batches of images belonging to one class.

Tables 1 and 2 showed the recognition rates for two databases. We can see that the recognition rate of our BIGNMF is close to GNMF, both for the supervised one and the unsupervised one. In some cases BIGNMF is slightly better than GNMF. The recognition rates of GNMF-U, GNMF-S, and NMF are fluctuating during incremental study. In the experiments we found that CINMF was the fastest one to start (study without incremental); it needs less than 40 s for the FERET database and less than 15 s for CMU-PIE database, but during the incremental process, the recognition rate of our BIGNMF outperforms CINMF and is much more faster than CINMF. It is noted that the recognition rate of IOPNMF performs close to NMF for the FERET database while the worst is for the CMU-PIE database; the reason does not concern us.

Figures 7 and 8 illustrate the running time of the selected algorithms in both databases; from the figure we can see that the mean running time of BIGNMFs for one batch of images belonging to one class is faster than other NMF-based incremental algorithms, both for the supervised one and the unsupervised one, which means our proposed algorithms achieved better recognition rate than other NMF-based incremental algorithms, while faster than then.

Figure 7 

The mean running time during incremental study with images belonging to one class for the FERET database.

Figure 8 

The mean running time during incremental study with images belonging to one class for the CMU-PIE database.

4.3. Incremental Study for One Batch of Images Belonging to Different Classes

In this section, incremental study for one batch of images belonging to different classes is performed. Our B-IGNMF, both supervised method and unsupervised method, is selected, along with supervised GNMF, unsupervised GNMF, IOPNMF, and NMF.

Experiments are designed as follows: we choose 4 images for each individual (totally 7 images) as the starting training set from the FERET database; the rest of the images are considered as the testing set. For each incremental study, one image for each individual from the testing set is selected as one batch of images to incremental study, totally 200 images for each batch. The incremental study for FERET is performed twice. For the CMU-PIE database, we choose 42 images for each individual (totally 56 images) as the starting training set, then the rest of the images are considered as the testing set. For each incremental study, 2 images for each individual from the testing set are selected as one batch of images to incremental study, total 68×2 images for each batch. The incremental study for FERET is performed 5 times. Tables 3 and 4 showed the recognition rates for two databases; “Add 1” means the recognition rate of adding one batch of images into “Start,” and “Add 2” means the recognition rate of adding another one batch of images into “Add 1,” and so on.

Table 3 

The recognition rates (%) for the FERET database during adding 2 batches of images belonging to different classes.

Table 4 

The recognition rates (%) for the CMU-PIE database during adding 5 batches of images belonging to different classes.

Tables 3 and 4 showed the recognition rates for two databases; we can see that the recognition rate of our BIGNMF, both the supervised one and the unsupervised one, is close to that of GNMF and better than that of NMF. Also, we can see that the recognition rate for both BIGNMFs is increasing while the new batch of images comes, which illustrate the effectiveness of the new incremental study algorithms. Note in the experment of the FERET database, in the starting, there are 4 images for each individual, totally 800 images; after adding the second batch of images, we added totally 400 images (2 images for each individual). Which means during the 50% amount of new images to incremental study, our BIGNMF still works. It is noted that the recognition rate of IOPNMF performs close to NMF for the FERET database while the worst for the CMU-PIE database; the reason is unknown.

Figures 9 and 10 illustrate the mean running times for all the selected algorithms in both database. From the figures we can see that the mean running time of our BIGNMFs for one batch of images belonging to different classes is faster than other NMF-based incremental algorithms, both for the supervised one and the unsupervised one, which means the recognition rate of our proposed algorithms is close to rerunning GNMF algorithms (sometimes slightly better), but faster.

Figure 9 

The mean running time during incremental study with images belonging to different classes for the FERET database.

Figure 10 

The mean running time during incremental study with images belonging to different classes for the CMU-PIE database.

5. Conclusions

In this paper, a novel incremental study method named incremental graph regularized nonnegative matrix factorization (IGNMF) has been proposed for face recognition. IGNMF introduces the graph regularized nonnegative matrix factorization algorithm into incremental study. The proposed IGNMF is able to preserve discrimination information under incremental study framework. In addition, we adapted our IGNMF algorithm to deal with learning from image batches, resulting in another new learning method called Batch IGNMF (B-IGNMF). Experiments show that the recognition rates of our IGNMF and B-IGNMF algorithms are close to GNMF algorithms while they run faster than GNMF algorithms. The running times of our IGNMF and B-IGNMF algorithms are close to INMF, faster than other popular NMF-based face recognition incremental algorithms, while the recognition rate of our IGNMF and B-IGNMF algorithms outperforms them.

Finally we point out the fact that if IGNMF and BIGNMF run too many times, the recognition rate would worsen than that of rerun GNMF. The reason for this is the assumption that 𝑉_𝑘 remains unchanged during the iterations and cannot hold if algorithms run too many times. So our future work will be focusing on how to resolve this issue.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported by the Ph.D. Programs Foundation of Ministry of Education of China under Grant no. 20120061110045, the Natural Science Foundation of Jilin province of China under Grant no. 201115022, and Graduate Innovation Fund of Jilin University Project no. 20121103. The corresponding author is Zhe-Zhou Yu.