A Note on a Lower Bound on the Minimum Rank of a Positive Semidefinite Hankel Matrix Rank Minimization Problem

Xu, Yi; Ren, Xiaorong; Yan, Xihong

doi:https://doi.org/10.1155/2021/2524016

Mathematical Problems in Engineering

On this page

Abstract Introduction Preliminaries Conclusion Data Availability Conflicts of Interest Authors’ Contributions Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2021 | Article ID 2524016 | https://doi.org/10.1155/2021/2524016

A Note on a Lower Bound on the Minimum Rank of a Positive Semidefinite Hankel Matrix Rank Minimization Problem

Yi Xu,^1,2Xiaorong Ren,³and Xihong Yan³

Academic Editor: He Chen

Received05 Aug 2020

Revised12 Dec 2020

Accepted22 Jan 2021

Published11 Feb 2021

Abstract

This paper investigates the problem of approximating the global minimum of a positive semidefinite Hankel matrix minimization problem with linear constraints. We provide a lower bound on the objective of minimizing the rank of the Hankel matrix in the problem based on conclusions from nonnegative polynomials, semi-infinite programming, and the dual theorem. We prove that the lower bound is almost half of the number of linear constraints of the optimization problem.

1. Introduction

The matrix rank minimization problem, or minimizing the rank of a matrix subject to linear constraints, recently has gained tremendous popularity due to wide applications in various fields including machine learning [1], compressed sensing [2, 3], system identification and control [4, 5], and computer vision [6]. This problem can be formulated as the following program:where is the optimization variable, is the rank of , is a linear map, and . A special case of this problem is the matrix completion problem [2], recovering an unknown low-rank matrix from a sampling of its entries. For this special case, the constraints are simply a subset of the entries of the matrix. The majority of existing work on algorithms for problem (1) has concentrated on this special case. Many papers addressed methods to solve the matrix completion problem, for instance, the singular value thresholding algorithm [7], the accelerated proximal gradient algorithm [8], the matrix factorization-based approach [9], and the facial reduction [10–12]. Various other methods are well documented in [13–15] and references therein.

In many engineering and statistical applications, for example, dynamical systems problems [16] and time-domain signal problems [17], the recovered matrix of problem (1), in addition to being low-rank, is required to have a Hankel structure. Driven by the wide applicability of the Hankel matrix minimization problem, there have been several algorithms available for solving it. Based on the nuclear norm model which minimizes the nuclear norm of the matrix instead of its rank, Fazel et al. [4] discussed several first-order algorithms for solving the structured rank minimization problem, including gradient projection methods, proximal point algorithms, and alternating direction methods of multipliers. Several evolutionary algorithms were proposed by Cai, including an iterative hard thresholding algorithm based on the nuclear norm model [18], an alternating direction method of multipliers based on Vandermonde factorization of a Hankel matrix model [19], and a projected Wirtinger gradient algorithm [20]. The major drawbacks of these methods are that the solutions obtained are not guaranteed to be a good approximate global solution for the problem since there have been no approximation bounds developed for these methods. Moreover, they are highly sensitive to some parameters which depend on a guess of the rank of the concerned matrix. Hence, in view of improving performance of algorithms for solving the problem, it is important to find an approximation for the rank of the optimal Hankel matrix. Along this direction, in this paper, we provide a lower bound on the minimum rank of a positive semidefinite Hankel matrix minimization problem with linear constraints, which is modeled as the following program:

Note that the positive semidefinite Hankel matrix completion problem is a special case of problem (2).

Most literature about estimation of the rank of a matrix concentrates on the rank of the optimal matrix of a positive semidefinite program [21–24] or a band matrix [25–28]. Though the minimum rank of problem (2) is guaranteed theoretically through these methods from the previous work, they are not efficient since the structure emerging from a positive semidefinite Hankel matrix is not completely utilized. To our knowledge, this work is one of the first efforts to address the lower bound of the objective value of the positive semidefinite Hankel matrix rank minimization problem. We first construct a semidefinite program to describe the feasible region of problem (2); then derive an equivalent semi-infinite program for the dual of the semidefinite program using a conclusion from nonnegative polynomials; and finally, by the optimality conditions for the semi-infinite problem, we prove that a lower bound for the minimum objective of problem (2) is almost half of the number of its constraints.

The remainder of the paper is organized as follows. Section 2 summarizes some notations and preliminaries that serve as fundamentals of analysis. In Section 3, a detailed stepwise analysis is shown to convert our problem into a semi-infinite program. Moreover, we establish the main results under some mild assumptions. Finally, we conclude this paper in Section 4.

2. Preliminaries

In this section, we introduce some notations and basic definitions which will be used in the remaining part of the paper and some lemmas that serves as fundamentals to recast a positive semidefinite system as an equivalent semi-infinite program.

Notations. Let be the -dimensional Euclidean space and be the set of real matrices. and denote the rank and trace of a matrix , respectively. A matrix is symmetric positive definite (resp., positive semidefinite) and is denoted by (resp., ). represents the -th component of vector , and represents the entry of matrix . is the number of elements of set . The symbol represents the transpose. represents the identity matrix. is the set consisting of the polynomials with coefficients in and degree at most . An -degree polynomial : is written as . Denote coefficients of polynomial by . means is a nonnegative polynomial. Considering a collection of matrices , where , we writefor the operations between a collection of matrices and a vector (resp., a matrix), where and . Let be a base of the Hankel matrix space, which is defined as follows:

Definition 1. An real Hankel matrix is a matrix of the form:where . It is obvious that an Hankel matrix could be completely determined by its last row and first column, a total of entries. Let . The Hankel matrix can be constructed by vector and can be expressed uniquely as based on (3) and (4). Thus, program (2) can be rewritten as follows:where .
Problem (6) is a special case of the matrix rank minimization problem (1) due to the special structure of the positive semidefinite Hankel matrix. However, the additional positive semidefinite Hankel structure requirement comes at a price, with theoretical and practical difficulties involved in the process of solving such a problem. Hence, we present some lemmas to provide tools to simplify the high technical requirement.

Lemma 1. (see [29]). Suppose that and . Then, is a sum of squares, that is,where .
We are now ready to state a lemma, which accords us with a mechanism to transform a semidefinite positive system to a nonnegative polynomial, which is the most useful part of our main result.

Lemma 2. Suppose and is the associated coefficient vector. Then, the following statements are equivalent:(a)(b)There exists a matrix such thatwhere is defined in (4)

Proof. Because and , according to Lemma 1, we havewhere .
Let and , then and , which means thatDefine ; then,Based on the equivalence of the coefficients of on the two sides of the above equation, we obtain and complete the proof in one direction.
For the other directions, since , ( is the rank of ), which together with , it implies that

The following lemma, which can be obtained from ([21], Theorem 2), provides us a useful property of the optimal matrix of program (6).

Lemma 3. Let be the cone of positive semidefinite matrices and be an affine subspace of the space of symmetric matrices such that the intersection is nonempty and bounded. If codim , where , then there is a matrix such that .

3. A Lower Bound on the Optimal Objective of a Positive Seimdefinite Hankel Rank Minimization Problem

In this section, we investigate a lower bound on the minimum rank of problem (5). Beginning with a lemma, we first explore an important property of the optimal value of problem (6).

Lemma 4. The objective value of problem (6) is small than .

Proof. An equivalent formulation for program (2) is stated as follows:where represents the floor function. It is clear that the affine subspace of the above program, denoted by , can be expressed by the following system:It is evident that codim , which meanswhere is as defined in Lemma 3. A direct application of the statement of Lemma 3 to estimating the objective value of program (2) results in . Recalling , it follows that , which implies . Hence, we prove the desired conclusion of the lemma due to the equivalence of programs (2) and (6).
Lemma 4 shows that optimal solutions of program (6) lie exactly on the boundary of the feasible region since the rank of that optimal solutions is smaller than . This says that there exists a nonzero vector such that program (6) and the following semidefinite program have the same solution:Hence, it suffices to estimate the objective value of program (16). It is easy to see that system (16) is a semidefinite program, which can be solved by many methods, for example, the interior point method [30] and the feasible direction method [31]. However, these methods just give numerical solutions for given and and cannot provide analytical solutions for any and . Hence, we explore a novelty method to solve this issue.
Recall that the dual of program (16) is given bywhere and are associated multipliers.

Hereafter, we state a lemma showing that the strict complementarity holds almost everywhere, whose most essential nature is given by ([32], Lemma 2).

Lemma 5. Suppose that and are primal and dual optimal solutions, respectively. Then, we have thatholds almost everywhere.

Proof. For program (16), the KKT conditions areLet and . Using the complementarity conditions , we have and eigendecompositions as follows:with and . Plunging eigendecompositions (20) into the KKT system, (19) yields the following system of equations:This is a system of equations in variables . If , the system is unsolvable almost. Hence, holds almost everywhere. We derive the conclusion of the lemma.
Let . According to Lemma 2, program (17) can be represented as the following semi-infinite program,The lemma below states that the feasible set of program (22) is bounded.

Next, we will establish the main result on a lower bound on the optimal value of the positive semidefinite Hankel matrix rank minimization problem.

Theorem 1. If is full rank and the Slater condition holds for program (16), then the objective value of the positive semidefinite Hankel matrix rank minimization problem (program (6)), denoted as , is almost not smaller than , that is, .

Proof. Based on Lemma 4 and the Slater condition, the solution set of program (22) is nonempty and finite. Let be an optimal solution of program (22). Then, is a local minimizer of the following program:where is an finite index set of the active constraints of problem (22) at since if is infinite, we can select its finite subset such that, for all in the subset holds. Thus, satisfies the following KKT system:where is the corresponding Lagrange multiplier vector.
Equation (24) is a system of equations in variables . If , the system is almost unsolvable. Hence, holds almost everywhere. Moreover, for , we have , which means since , where . Thus, since . This says that . Consequently, according to the conclusion of Lemma 5, we almost have

4. Conclusion

This paper focuses on approximating the global optimal value of a Hankel matrix rank minimization problem subjective to linear constraints. A lower bound for this optimization problem is provided based on some conclusions from nonnegative polynomials, dual theorem, and semi-infinite problems. We conclude that the lower bound is almost half of the number of linear constraints of the problem. This result is useful for simplifying the requirement of guessing the rank of the objective matrix in existing methods for solving such problem.

Data Availability

No data were used to support the study.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

All authors contributed equally and significantly in this paper. All authors read and approved the final manuscript.

Acknowledgments

Y. Xu was supported in part by the National Natural Science Foundation of China (nos. 11501100, 11571178, 11671082, and 11871149). X. Yan was supported in part by the National Natural Science Foundation of China (no. 11901424) and NSF of Shanxi province No. 201801D121022.

References

A. Argyriou, T. Evgeniou, and M. Pontil, “Multi-task feature learning,” Advances in Neural Information Processing Systems, vol. 19, pp. 41–48, 2007.
View at: Google Scholar
E. J. Candès and B. Recht, “Exact matrix completion via convex optimization,” Foundations of Computational Mathematics, vol. 9, no. 6, pp. 717–772, 2009.
View at: Publisher Site | Google Scholar
Y. Chen and Y. Chi, “Spectral compressed sensing via structured matrix completion,” IEEE Transactions on Information Theory, vol. 60, pp. 6576–6601, 2013.
View at: Google Scholar
M. Fazel, T. K. Pong, D. Sun, and P. Tseng, “Hankel matrix rank minimization with applications to system identification and realization,” SIAM Journal on Matrix Analysis and Applications, vol. 34, no. 3, pp. 946–977, 2013.
View at: Publisher Site | Google Scholar
Z. Liu and L. Vandenberghe, “Interior-point method for nuclear norm approximation with application to system identification,” SIAM Journal on Matrix Analysis and Applications, vol. 31, no. 3, pp. 1235–1256, 2010.
View at: Publisher Site | Google Scholar
C. Tomasi and T. Kanade, “Shape and motion from image streams under orthography: a factorization method,” International Journal of Computer Vision, vol. 9, no. 2, pp. 137–154, 1992.
View at: Publisher Site | Google Scholar
J.-F. Cai, E. J. Candès, and Z. Shen, “A singular value thresholding algorithm for matrix completion,” SIAM Journal on Optimization, vol. 20, no. 4, pp. 1956–1982, 2010.
View at: Publisher Site | Google Scholar
K. C. Toh and S. Yun, “An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems,” Pacific Journal of Optimization, vol. 6, pp. 615–640, 2010.
View at: Google Scholar
R. Sun and Z.-Q. Luo, “Guaranteed matrix completion via non-convex factorization,” IEEE Transactions on Information Theory, vol. 62, no. 11, pp. 6535–6579, 2016.
View at: Publisher Site | Google Scholar
D. Drusvyatskiy and H. Wolkowicz, “The many faces of degeneracy in conic optimization,” Foundations and Trends in Optimization, vol. 3, no. 2, pp. 77–170, 2017.
View at: Publisher Site | Google Scholar
S. Huang and H. Wolkowicz, “Low-rank matrix completion using nuclear norm minimization and facial reduction,” Journal of Global Optimization, vol. 72, no. 1, pp. 5–26, 2018.
View at: Publisher Site | Google Scholar
S. Ma, F. Wang, L. Wei, and H. Wolkowicz, “Robust principal component analysis using nuclear norm minimization and facial reduction,” Optimization and Engineering, vol. 21, pp. 1195–1219, 2020.
View at: Google Scholar
M. Fazel, “Matrix rank minimization with applications,” Stanford University, Stanford, CA, USA, 2002, Ph.D. thesis.
View at: Google Scholar
Y. Hu, D. Zhang, J. Liu, J. Ye, and X. He, “Accelerated singular value thresholding for matrix completion,” in Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 298–306, London, UK, August 2012.
View at: Google Scholar
S. Ma, D. Goldfarb, and L. Chen, “Fixed point and Bregman iterative methods for matrix rank minimization,” Mathematical Programming, vol. 128, no. 1-2, pp. 321–353, 2011.
View at: Publisher Site | Google Scholar
B. D. Schutter, “Minimal state-space realization in linear system theory: an overview,” Journal of Computational and Applied Mathematics, vol. 121, no. 1-2, pp. 331–354, 2000.
View at: Publisher Site | Google Scholar
J.-F. Cai, X. Qu, W. Xu, and G.-B. Ye, “Robust recovery of complex exponential signals from random Gaussian projections via low rank Hankel matrix reconstruction,” Applied and Computational Harmonic Analysis, vol. 41, no. 2, pp. 470–490, 2016.
View at: Publisher Site | Google Scholar
J.-F. Cai, T. Wang, and K. Wei, “Fast and provable algorithms for spectrally sparse signal reconstruction via low-rank Hankel matrix completion,” Applied and Computational Harmonic Analysis, vol. 46, no. 1, pp. 94–121, 2019.
View at: Publisher Site | Google Scholar
J. Ying, J.-F. Cai, D. Guo, G. Tang, Z. Chen, and X. Qu, “Vandermonde factorization of Hankel matrix for complex exponential signal recovery-application in fast NMR spectroscopy,” IEEE Transactions on Signal Processing, vol. 66, no. 21, pp. 5520–5533, 2018.
View at: Publisher Site | Google Scholar
M. Cho, J. Cai, S. Liu, Y. C. Eldar, and W. Xu, “Fast alternating projected gradient descent algorithms for recovering spectrally sparse signals,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, October 2016.
View at: Google Scholar
A. Barvinok, “A remark on the rank of positive semidefinite matrices subject to affine constraints,” Discrete & Computational Geometry, vol. 25, no. 1, pp. 23–31, 2001.
View at: Publisher Site | Google Scholar
H. Fawzi, J. Gouveia, P. A. Parrilo, R. Z. Robinson, and R. R. Thomas, “Positive semidefinite rank,” Mathematical Programming, vol. 153, no. 1, pp. 133–177, 2015.
View at: Publisher Site | Google Scholar
J. Gouveia, R. Z. Robinson, and R. R. Thomas, “Worst-case results for positive semidefinite rank,” Mathematical Programming, vol. 153, no. 1, pp. 201–212, 2015.
View at: Publisher Site | Google Scholar
G. Pataki, “On the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues,” Mathematics of Operations Research, vol. 23, no. 2, pp. 339–358, 1998.
View at: Publisher Site | Google Scholar
N. Cohen and J. Dancis, “Inertias of block band matrix completions,” SIAM Journal on Matrix Analysis and Applications, vol. 19, no. 3, pp. 583–612, 1998.
View at: Publisher Site | Google Scholar
R. L. Ellis and D. C. Lay, “Rank-preserving extensions of band matrices,” Linear and Multilinear Algebra, vol. 26, no. 3, pp. 147–179, 1990.
View at: Publisher Site | Google Scholar
H. J. Woerdeman, “Minimal rank completions of partial banded matrices,” Linear and Multilinear Algebra, vol. 36, no. 1, pp. 59–68, 1993.
View at: Publisher Site | Google Scholar
H. J. Woerdeman, “Hermitian and normal completions,” Linear and Multilinear Algebra, vol. 42, no. 3, pp. 239–280, 1997.
View at: Publisher Site | Google Scholar
J. B. Lasserre, “A sum of squares approximation of nonnegative polynomials,” SIAM Review, vol. 49, no. 4, pp. 651–669, 2007.
View at: Publisher Site | Google Scholar
C. Helmberg, F. Rendl, R. J. Vanderbei, and H. Wolkowicz, “An interior-point method for semidefinite programming,” SIAM Journal on Optimization, vol. 6, no. 2, pp. 342–361, 1996.
View at: Publisher Site | Google Scholar
Y. Xu and X. Yan, “On a box-constrained linear symmetric cone optimization problem,” Journal of Optimization Theory and Applications, vol. 181, no. 3, pp. 946–971, 2019.
View at: Publisher Site | Google Scholar
F. Alizadeh, J.-P. A. Haeberly, and M. L. Overton, “Complementarity and nondegeneracy in semidefinite programming,” Mathematical Programming, vol. 77, no. 1, pp. 111–128, 1997.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2021 Yi Xu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies