Constraint Consensus Methods for Finding Interior Feasible Points in Second-Order Cones

Abstract

Optimization problems with second-order cone constraints (SOCs) can be solved efficiently by interior point methods. In order for some of these methods to get started or to converge faster, it is important to have an initial feasible point or near-feasible point. In this paper, we study and apply Chinneck's Original constraint consensus method and DBmax constraint consensus method to find near-feasible points for systems of SOCs. We also develop and implement a new backtracking-like line search technique on these methods that attempts to increase the length of the consensus vector, at each iteration, with the goal of finding interior feasible points. Our numerical results indicate that the new methods are effective in finding interior feasible points for SOCs.

1. Introduction

We consider a system of second-order cone inequalities as follows:𝑐^𝑇_𝑖𝑥+𝑑_𝑖−𝐴_𝑖𝑥+𝑏_𝑖≥0𝑖=1,2,…,𝑞, (1.1) where 𝑥∈ℝ^𝑛,  𝐴_𝑖 is an 𝑚_𝑖×𝑛 matrix, 𝑏_𝑖 is a vector in ℝ^𝑚𝑖, 𝑐_𝑖 is a vector in ℝ^𝑛, and 𝑑_𝑖 is a scalar. The norm is the standard Euclidean norm. We assume the interior of the feasible region 𝑆={𝑥∈ℝ^𝑛∣𝑐^𝑇_𝑖𝑥+𝑑_𝑖−‖𝐴_𝑖𝑥+𝑏_𝑖‖≥0,𝑖=1,…,𝑞} is nonempty.

Second-order cones (SOCs) are important to study because there exist many optimization problems where the constraints can be written in this form. For instance, SOCs can be used to easily represent problems dealing with norms, hyperbolic constraints, and robust linear programming. There are also a huge number of real world applications in areas such as antenna array weight design, grasping force optimization, and portfolio optimization (see [1] for more information on these and other applications). Furthermore, there exist efficient interior point methods for solving optimization problems with SOC constraints, such as the primal-dual interior-point method. Because of these applications and the success of interior point methods when applied to SOCs, there exists a need to be able to efficiently find a near-feasible or feasible point for a system of SOCs [1–4]. One approach to feasibility is given in [5].

In this paper, we will describe two of Chinneck's constraint consensus algorithms and apply them to SOCs to find near-feasible points. These are the Original constraint consensus method and DBmax constraint consensus method. One could of course add a small value to the final consensus vector of these methods to make it enter the interior of the feasible region [6]. However, this would not work if the final consensus vector is far away from the boundary. We propose a different approach, one which increases the step size of the consensus vector at each iteration using a backtracking technique. The goal is to find interior feasible points and to reduce the number of iterations and amount of time. The technique works by extending the consensus vector by a given value and then backtracking until it is as close as possible to the feasible region at a point where the number of satisfied constraints does not decrease. Finally, we investigate the effectiveness of the modification on applied to Original and DBmax methods by testing them upon randomly generated problems. The results show that the backtracking technique is effective at finding interior feasible points. It also greatly reduces the number of necessary iterations and time of the methods.

We also study how our work for SOCs apply to the special case of convex quadratic constraints (CQCs). More information on the importance of CQCs in the field of optimization can be found in [7, 8]. Our results for CQCs show that backtracking also significantly reduces the number of necessary iterations and time of the Original method. It also was able to find interior feasible points on many of the test problems. However, backtracking does not work well with the DBmax method in the case of CQCs. We note that in both constraint types (SOCS or CQCs), DBmax method outperforms the Original method, as expected.

2. The Constraint Consensus Methods Applied to Second-Order Cones

In this section, we study and apply Chinneck's Original constraint consensus and DBmax constraint consensus methods to SOCs in order to find near-feasible points of the system (1.1).

The first constraint consensus method, hereby, called Original was developed by Chinneck in [3] (see Algorithm 1). The method starts from an infeasible point 𝑥₀. The method associates each constraint 𝑓_𝑖(𝑥)≥0 with a feasibility vector 𝑓𝑣_𝑖  (𝑖=1,2,…,𝑞). In the case of our SOCs, 𝑓_𝑖=𝑐^𝑇_𝑖𝑥+𝑑_𝑖−‖𝐴_𝑖𝑥+𝑏_𝑖‖. The feasibility vector is defined by 𝑓𝑣_𝑖=𝑣∇𝑓_𝑖(𝑥₀)/‖∇𝑓_𝑖(𝑥₀)‖², where 𝑣=max{0,−𝑓_𝑖(𝑥₀)} is the constraint violation at 𝑥₀ and ∇𝑓_𝑖(𝑥₀) is the gradient of 𝑓_𝑖(𝑥)≥0 at point 𝑥₀. We assume that ∇𝑓_𝑖(𝑥) exists. Note that if 𝑓_𝑖(𝑥) is linear, 𝑥₀+𝑓𝑣_𝑖 extends from 𝑥₀ to its orthogonal projection on the constraint 𝑓_𝑖(𝑥)≥0. The length of the feasibility vector ‖𝑓𝑣_𝑖‖=𝑣/‖∇𝑓_𝑖(𝑥₀)‖ is called the feasibility distance. We define 𝑥₀ to be near feasible with respect to 𝑓_𝑖(𝑥)≥0 if ‖𝑓𝑣_𝑖‖<𝛼, where 𝛼 is a preassigned feasibility tolerance. We say that 𝑥₀ is feasible with respect to 𝑓_𝑖(𝑥)≥0 if 𝑓_𝑖(𝑥₀)≥0 and is an interior feasible point if 𝑓_𝑖(𝑥₀)>0.

INPUTS: set of q constraints (1.1), an initial point 𝑥, a feasibility distance tolerance 𝛼, a movement tolerance 𝛽, maximum number of iterations 𝜇. while steps <𝜇  do  𝑁𝐼𝑁𝐹=0, for all 𝑗∶𝑛𝑗=0,  𝑠𝑗=0  for every constraint 𝑖  do   if constraint 𝑖 is violated then    Calculate feasibility vector 𝑓𝑣𝑖 and feasibility distance ‖𝑓𝑣𝑖‖    if  ‖𝑓𝑣𝑖‖>𝛼  then     𝑁𝐼𝑁𝐹=𝑁𝐼𝑁𝐹+1     for every variable 𝑥𝑗  do      𝑛𝑗←𝑛𝑗+1, 𝑠𝑗←𝑠𝑗+𝑓𝑣𝑖𝑗  If  𝑁𝐼𝑁𝐹=0  then   exit successfully  for every variable 𝑥𝑗:  do   if  𝑛𝑗≠0  then     𝑡𝑗=𝑠𝑗/𝑛𝑗    else     𝑡𝑗=0   if  ‖𝑡‖≤𝛽  then    exit unsuccessfully   𝑥=𝑥+𝑡, steps = steps + 1

Algorithm 1 

Original constraint consensus algorithm.

The feasibility vectors are then combined to create a single consensus vector which reflects the movements suggested by all the feasibility vectors. The consensus vector, 𝑡, is created by averaging the nonzero components of the feasibility vectors. Each component of 𝑡 is given by 𝑡_𝑗=𝑠_𝑗/𝑛_𝑗 (the average of the 𝑗th components of the violated feasibility vectors) with 𝑠_𝑗 being the sum of the 𝑗th components of the feasibility vectors for each constraint that is violated at the point 𝑥₀, and 𝑛_𝑗 being the number of constraints that are violated (not near feasible) at point 𝑥₀. The consensus vector is then added to 𝑥₀ to get a new iterate 𝑥₁, and the algorithm is repeated until the number of violated constraints (NINF) at the current iterate is zero. The algorithm will also stop looping if the consensus vector becomes too short and not enough progress towards the feasible region is being made quickly enough—if the movement tolerance 𝛽 is reached.

Ibrahim et al. gave several modifications of the Original method in [9]. While the Original method treats all eligible feasibility vectors equally, the variations they develop place emphasis on feasibility vectors that have a stronger influence on the consensus vector. They generate two different types of improved constraint consensus algorithms. They generally found a direction-based algorithm, called DBmax, to be the most successful among all the variations, see Algorithm 2.

INPUTS: set of 𝑞 constraints (1.1), an initial point 𝑥, a feasibility distance tolerance 𝛼, a movement tolerance 𝛽, maximum number of iterations 𝜇. while steps <𝜇  do  𝑁𝐼𝑁𝐹=0, for all 𝑗∶𝑛+𝑗=0,  𝑛−𝑗=0,  𝑠+𝑗=0,  𝑠−𝑗=0  for every constraint 𝑖  do   if constraint 𝑖 is violated then    Calculate feasibility vector 𝑓𝑣𝑖 and feasibility distance ‖𝑓𝑣𝑖‖    if  ‖𝑓𝑣𝑖‖>𝛼  then     𝑁𝐼𝑁𝐹=𝑁𝐼𝑁𝐹+1     for every variable 𝑥𝑗  do      if  𝑓𝑣𝑖𝑗>0  then       𝑛+𝑗←𝑛+𝑗+1       if  𝑓𝑣𝑖𝑗>𝑠+𝑗  then        𝑠+𝑗←𝑓𝑣𝑖𝑗      else if  𝑓𝑣𝑖𝑗<0  then       𝑛−𝑗←𝑛−𝑗+1       if  𝑓𝑣𝑖𝑗<𝑠−𝑗  then        𝑠−𝑗←𝑓𝑣𝑖𝑗  if  𝑁𝐼𝑁𝐹=0  then   exit successfully  for every variable 𝑥𝑗: do   if  𝑛+=𝑛−  then    𝑡𝑗=(𝑠+𝑗+𝑠−𝑗)/2   else if  𝑛+𝑗>𝑛−𝑗  then    𝑡𝑗=𝑠+𝑗   else    𝑡𝑗=𝑠−𝑗  if  ‖𝑡‖≤𝛽  then   exit unsuccessfully  𝑥=𝑥+𝑡, steps = steps + 1

Algorithm 2 

DBmax constraint consensus algorithm.

DBmax looks at the 𝑗th component of every valid feasibility vector and takes the maximum value for the sign with the most votes, which then becomes the 𝑗th entry of the consensus vector. If there are equal number of positive and negative 𝑗th components, 𝑡_𝑗 becomes the average of the most positive and most negative 𝑗th component of the valid feasibility vectors. Let 𝑠⁺_𝑗 be the value of the most positive 𝑗th component of the feasibility vectors, 𝑠⁻_𝑗 be the value of the most negative 𝑗th component of the feasibility vectors, 𝑛⁺_𝑗 be the number of violated constraints that vote for a positive movement for the variable 𝑥_𝑗, and 𝑛⁻_𝑗 be the number of violated constraints that vote for a negative movement for the variable 𝑥_𝑗.

For a more specific example, we consider a system of 3 SOCs in ℝ², pictured in Figure 1. We will refer to this system when demonstrating concepts throughout this paper.

Figure 1 

The feasible region 𝑆 for Example 2.1.

Example 2.1. A system of 𝑞=3 second-order cones (SOCs) with 𝑛=2 variables:
(1)  𝑐^𝑇₁𝑥+𝑑₁−‖𝐴₁𝑥+𝑏₁‖≥0,
(2)  𝑐^𝑇₂𝑥+𝑑₂−‖𝐴₂𝑥+𝑏₂‖≥0,
(3)  𝑐^𝑇₃𝑥+𝑑₃−‖𝐴₃𝑥+𝑏₃‖≥0,
where 𝐴₁=6−723,𝑏₁=1−7,𝑐₁=58,𝑑₁𝐴=8,₂=−433−8,𝑏₂=−2−3,𝑐₂=4−8,𝑑₂𝐴=8,₃=−2−22−4,𝑏₃=−9−10,𝑐₃=3−4,𝑑₃=6. (2.1) We define the function 𝑓(𝑥)=𝑐^𝑇𝑥+𝑑−‖𝐴𝑥+𝑏‖. (2.2) Note that 𝑓(𝑥)=0 gives the boundary of the cone, 𝑓(𝑥) is nonnegative inside the feasible region of the cone and negative outside of the cone. We can see in Figure 2, an example of the contours of 𝑓(𝑥) for Cone (1) of Example 2.1.

(a)

(b)

Figure 2 

The contours and graph of 𝑓(𝑥) for cone (1) of Example 2.1.

The following theorem is well known. An elementary proof can be given using the triangle inequality as shown. It will be used to discuss the convergence of the Original and DBmax methods.

Theorem 2.2. 𝑓(𝑥) is concave over ℝ^𝑛.

Proof. For convenience, we will consider two separate functions, 𝑓₁(𝑥)=𝑐^𝑇𝑥+𝑑 and 𝑓₂(𝑥)=−‖𝐴𝑥+𝑏‖. The sum of two concave functions is also concave, so it suffices to show that 𝑓₁(𝑥) and 𝑓₂(𝑥) are both concave. 𝑓₁(𝑥) is linear, so it is concave. In order to show 𝑓₂(𝑥) is concave in ℝ^𝑛, it suffices to prove that 𝑓₂(𝑡𝑥+(1−𝑡)𝑦)≥𝑡𝑓₂(𝑥)+(1−𝑡)𝑓₂(𝑦) for 0≤𝑡≤1 and 𝑥,𝑦∈ℝ^𝑛.
By the triangle inequality, we have that 𝑡()+()≤‖𝑡(+)⟺()()𝑏+(⟺𝐴()𝑦()𝑏+(⟺𝐴()𝑦+(𝐴()𝑦−(.𝐴𝑥+𝑏1−𝑡𝐴𝑦+𝑏𝐴𝑥+𝑏)‖(1−𝑡𝐴𝑦+𝑏𝑡𝐴𝑥+𝑡𝑏+1−𝑡𝐴𝑦+1−𝑡≤𝑡‖(𝐴𝑥+𝑏)‖1−𝑡)𝐴𝑦+𝑏𝑡𝑥+1−𝑡+𝑡𝑏+1−𝑡≤𝑡‖(𝐴𝑥+𝑏)‖1−𝑡)𝐴𝑦+𝑏𝑡𝑥+1−𝑡+𝑏≤𝑡‖(𝐴𝑥+𝑏)‖1−𝑡)𝐴𝑦+𝑏⟺−𝑡𝑥+1−𝑡+𝑏≥−𝑡‖(𝐴𝑥+𝑏)‖1−𝑡)𝐴𝑦+𝑏 (2.3) This means that 𝑓₂(𝑡𝑥+(1−𝑡)𝑦)≥𝑡𝑓₂(𝑥)+(1−𝑡)𝑓₂(𝑦), so 𝑓₂(𝑥) is concave. The sum of two concave functions is also concave, so 𝑓₁(𝑥)+𝑓₂(𝑥) is concave. Therefore 𝑓(𝑥) is also concave.

Projection algorithms such as the Original and DBmax constraint consensus methods have been proven to converge, when the functions 𝑓_𝑖(𝑥) are concave [3, 10, 11]. So, Theorem 2.2 guarantees convergence in the case of SOCs. Theorem 2.2 shows that the feasible region 𝑆 of our system (1.1) is convex.

The main task in adapting the consensus methods to SOCs is computing the gradient ∇𝑓(𝑥) of 𝑓(𝑥)=𝑐^𝑇𝑥+𝑑−‖𝐴𝑥+𝑏‖. It is given by (𝑥)𝐴∇𝑓=𝑐−^𝑇(𝐴𝑥+𝑏)‖𝐴𝑥+𝑏‖. (2.4)

When calculating the gradient, there exist two potential problems. First of all, there are times when the gradient may fail to exist. When 𝐴𝑥+𝑏=0 the gradient is undefined. For example, in our first cone, the solution to 𝐴𝑥+𝑏=0 is 𝑥=(0.4375,−0.625). As shown in Figure 3, this point happens to be within the feasible region of our cone. This will not always be the case, but the probability of picking a solution to 𝐴𝑥+𝑏=0 within our algorithm is very slim. Another potential problem is when the gradient is zero, the feasibility vector will be undefined. If ∇𝑓(𝑥)=0 or undefined at an iteration, the algorithm fails and one should pick a different starting point. For cone (1) our example there does not exist a point where the gradient is zero, as shown below(𝑥)𝐴∇𝑓=0,𝑐−^𝑇(𝐴𝑥+𝑏)‖‖=00,𝐴𝑥+𝑏36𝑥₂−40𝑥₁+402𝑥₁+3𝑥₂+1²+7𝑥₂−6𝑥₁+7²+58−58𝑥₂−36𝑥₁+522𝑥₁+3𝑥₂+1²+7𝑥₂−6𝑥₁+7²=00. (2.5) The only possible solution to this system happens to be 𝑥₁=0.4375 and 𝑥₂=−0.625, which is where the gradient is undefined. This means that there is no solution to ∇𝑓(𝑥)=0 for this particular example.

Figure 3 

The gradient for cone (1) of Example 2.1 is undefined at the indicated point.

We know from Theorem 2.2 that 𝑓(𝑥) is concave. A nice result is that if 𝑓(𝑥) is strictly concave, then the only potential place for the gradient to be zero occurs inside of the feasible region, as shown in Theorem 2.3.

Theorem 2.3. Suppose 𝑓(𝑥) is strictly concave. If ∇𝑓(𝑥)=0, then 𝑥 is inside of the feasible region of 𝑓(𝑥)≥0.

Proof. If the gradient is zero, 𝑓(𝑥) must be at a critical point. Since 𝑓(𝑥) is strictly concave, the only potential critical point will be the maximum of 𝑓(𝑥). Since 𝑓(𝑥)<0 outside the feasible region and 𝑓(𝑥)≥0 inside the feasible region, it follows that the maximum of 𝑓(𝑥) must also be nonnegative. Therefore the gradient can only be zero inside of the feasible region.

As a consequence of this result, for strictly concave constraints, the gradient will only be zero inside of the feasible region, and so the feasibility vector exists at all iterates of our algorithms.

Let 𝑔(𝛾)=𝑓(𝑥_𝑘+𝛾𝑠_𝑘), where 𝑠_𝑘 is the feasibility vector at 𝑥_𝑘. Figure 4 is an example of 𝑔(𝛾). The graph of 𝑔(𝛾) is a slice of the graph of 𝑓(𝑥). It follows from Theorem 2.2 that 𝑔(𝛾) is concave in 𝛾 over ℝ.

Figure 4 

𝑔(𝛾) on the interval [0,3] for cone (1) of Example 2.1 at 𝑥𝑘=(−7,6).

The following results show that in each iteration of the algorithm, we move closer to the boundary of the feasible region.

Theorem 2.4. Suppose the line {𝑥_𝑘+𝛾𝑠_𝑘∣𝛾∈ℝ} intersects the boundary of {𝑥∣𝑓(𝑥)≥0} at 𝛾^∗. Then 𝑔(𝛾) is nondecreasing over [0,𝛾^∗].

Proof. 𝑔(𝛾) is negative over [0,𝛾^∗) and is zero at 𝛾^∗. Since 𝑔(𝛾) is concave over ℝ, then 𝑔(𝛾) is increasing over [0,𝛾^∗].

For concave 𝑓(𝑥), approximations of the feasibility vector will most often fall short of reaching the boundary of 𝑓(𝑥)≥0. As such the consensus vector will also fall short of the boundary. It may be desirable to increase the length of the consensus vector as much as possible towards the boundary. From Figure 5, we can see that for all points that lie outside of the feasible region of the cone, the direction of the gradient points is the direction of the boundary. However, this is not the case for all SOCs. As seen in Figure 6, some iterates may not have a gradient that points directly at the feasible region. Hence, there is a limit to how much to increase the length of the consensus vector in general. This point is noted when we discuss the backtracking technique later.

Figure 5 

The gradient vector field of 𝑓(𝑥) for cone (1) of Example 2.1.

Figure 6 

Gradient field of cone (2) of Example 2.1. Many starting points do not have a gradient that points directly at the feasible region.

3. The Case of Convex Quadratic Constraints

In this section, we study how the discussions in the previous section apply to the special case of convex quadratic constraints (CQCs). This study will be limited to results that are different and relevant to CQCs.

We consider a system of CQCs: 𝑐^𝑇_𝑖𝑥+𝑑_𝑖−𝐴_𝑖𝑥+𝑏_𝑖^𝑇𝐴_𝑖𝑥+𝑏_𝑖≥0𝑖=1,2,…,𝑞, (3.1) where 𝑥∈ℝ^𝑛, 𝐴_𝑖 is an 𝑚_𝑖×𝑛 matrix, 𝑏_𝑖 is a vector in ℝ^𝑚𝑖, 𝑐_𝑖 is a vector in ℝ^𝑛, and 𝑑_𝑖 is a scalar.

For ease of presentation, consider the single CQC𝑐^𝑇𝑥+𝑑−(𝐴𝑥+𝑏)^𝑇(𝐴𝑥+𝑏)≥0. (3.2) It is nice to note that a CQC is also SOC. To see this, note that𝑐^𝑇𝑥+𝑑−(𝐴𝑥+𝑏)^𝑇(𝐴𝑥+𝑏)≥0⟺𝑥^𝑇𝑃𝑥+2𝑞^𝑇𝑥+𝑟≤0, (3.3) where 𝑃=𝐴^𝑇𝐴,𝑞=𝐴^𝑇1𝑏−2𝑐,𝑟=𝑏^𝑇𝑏−𝑑. (3.4) For simplicity and without a loss of generalization, we assume that 𝑃 is positive definite. So, 𝑃^1/2 exists and is invertible. Then, the CQC (3.2) is equivalent to𝑃^1/2𝑥+𝑃^−1/2𝑞²−𝑞^𝑇𝑃⁻¹𝑞−𝑟≤0. (3.5) This is equivalent to the SOC𝑞^𝑇𝑃⁻¹𝑞−𝑟^1/2−𝑃^1/2𝑥+𝑃^−1/2𝑞≥0. (3.6) The above result implies that all our results on concavity and convergence on SOCs also extend to the CQC system (3.1). It suffices to compute the gradient ∇𝑓(𝑥) and Hessian 𝐻_𝑓(𝑥) of 𝑓(𝑥)=𝑐^𝑇𝑥+𝑑−(𝐴𝑥+𝑏)^𝑇(𝐴𝑥+𝑏). They are given by ∇𝑓(𝑥)=𝑐−2𝐴^𝑇𝐴𝑥−2𝐴^𝑇𝐻𝑏,_𝑓(𝑥)=−2𝐴^𝑇𝐴. (3.7) The feasibility vector does not exist when the gradient is zero. Note that the gradient ∇𝑓(𝑥)=0 is given by the solution to the linear system2𝐴^𝑇𝐴𝑥=𝑐−2𝐴^𝑇𝑏. (3.8) To use as an illustration, consider the following example.

Example 3.1. A convex quadratic constraint (CQC) with 𝑛=2 variables.
𝑓₁(𝑥)∶=𝑐^𝑇₁𝑥+𝑑₁−(𝐴₁𝑥+𝑏₁)^𝑇(𝐴₁𝑥+𝑏₁)≥0, where 𝐴₁=−516−10,𝑏₁=0−2,𝑐₁=6−8,𝑑₁=8. (3.9)
For Example 3.1, the linear system (3.8) is 122−130𝑥−130202𝑥¹₂=16−34 (3.10) with the solution 𝑥₁=−27/176 and 𝑥₂=−47/176, which is in the feasible region. See Figure 7.

Figure 7 

The solution for the system (3.10) over 𝑓1(𝑥)≥0 of Example 3.1 is given by the intersection of the two lines at 𝑥=(−27/176,−47/176).

Since the Hessian 𝐻_𝑓(𝑥)=−2𝐴^𝑇𝐴, 𝑓(𝑥) is strictly convex if 𝐴^𝑇𝐴 is positive definite. So, similar to Theorem 2.3, if 𝐴^𝑇𝐴 is positive definite, then the gradient can only be zero at a point inside the feasible region. This means, in this case, the feasibility vector exists at all iterates of the consensus algorithms.

4. New Variation of the Constraint Consensus Methods

In this section, we propose a modification to the Original and DBmax constraint consensus methods. It extends the length of the consensus vector at each iteration of the methods with the goal of finding a point in the interior of the feasible region 𝑆 of our system (1.1).

The Backtracking Line Search Technique (Algorithm 3) is a step in the constraint consensus methods (Algorithms 1 and 2) that attempts to extend the length the consensus vector 𝑡 to a closer near-feasible point. It goes to a point where the number of satisfied constraints does not decrease. The backtracking technique seeks to provide rapid convergence by inexpensively testing to see if we can increase the size of the consensus vector. Starting by computing the consensus vector in the usual way (either using the Original method or DBmax), it tests to see if it can successfully increase the size of the consensus vector, in the hope of getting closer to the feasible region in less iterations and time. The technique has the added benefit that it is possible for the final point to be actually in the interior instead of being a near-feasible point.

INPUTS: set of 𝑞 constraints (1.1) or (3.1), a point 𝑥, the consensus vector 𝑡 Compute the binary word 𝑏 at 𝑥 count = 0 while  count<3  do  𝛼=1+0.5count  𝑥temp=𝑥+𝛼∗𝑡  Compute the binary word 𝑏comp at 𝑥temp  if  (𝑏comp)≤sum(𝑏)  then   𝑥=𝑥temp; Exit Function  else   count = count+1 if count = loop then  𝑥=𝑥+𝑡

Algorithm 3 

Backtracking line search technique.

The Backtracking Line Search Technique uses the concept of a binary word.

Definition 4.1. Define 𝛿(𝑥)=𝛿₁(𝑥),…,𝛿_𝑞(𝑥) to be the binary word for the set of constraints (1.1) or (3.1), given by 𝛿_𝑖(𝑥)=0if𝑥is𝑓_𝑖(𝑥)≥0(𝑖thconstraintissatisfiedat𝑥),1,otherwise. (4.1)

If 𝛿(𝑥)=[0⋯0], then 𝑥 is a feasible point. We can easily check the feasibility of 𝑥 by taking the sum of the components of the binary word. When the sum is 0, we know that 𝑥 is feasible and we can allow the algorithm to exit. We can define an equivalence relation on ℝ^𝑛, where 𝑥₁ is related to 𝑥₂ if the binary word of 𝑥₁ is the same as the binary word of 𝑥₂. The equivalence classes form a partition of ℝ^𝑛.

We start by computing the consensus vector 𝑡. Then we scale the consensus vector by an initial scalar factor. We add this consensus vector to 𝑥. Using this new point, we count the number of constraints that are violated with the binary word. We consider the new consensus vector to be successful if the resulting point violates the same number or fewer constraints than the previous point did. If this is not the case, the algorithm backtracks and tries a smaller increase in the consensus vector. If the point still has not found an improvement in the number of satisfied constraints after backtracking three times, then it returns to the starting consensus vector. In our tests, we scale the consensus vector by 2, 1.5, 1.25, and if none of those are satisfactory then we return to a step size of 1. Algorithm 3 could easily be varied to use other step sizes, based on the type of constraint or problem.

In Figure 8, we can see a comparison of the Original method and Original method with backtracking applied to Example 2.1. Given the same initial starting point of (−8,6), the Original method with backtracking reached feasibility in fewer iterations and less time. Furthermore, the final point obtained by using the Original method with backtracking was actually feasible for all three constraints, as opposed to being near feasible. The final point obtained by the Original method was only feasible for the second constraint, and within a feasibility distance tolerance 0.01 for the other two constraints.

(a)

(b)

Figure 8 

Comparison of Original method and Original method with backtracking applied to Example 2.1.

5. Numerical Experiments

In this section, we use numerical experiments to test the Original method and DBmax method with and without the backtracking line search technique on random test problems.

For each SOC test problem, we first generate integer values 𝑞,  𝑚_𝑖=𝑚, and 𝑛 uniformly in the intervals [1,50], [1,10], and [2,10], respectively. Then, a random point is chosen with the values of each entry being in uniform in [−10,10]. After this, 𝑞 SOCs are generated using 𝑚 and 𝑛, where each entry of 𝐴, 𝑏, 𝑐, and 𝑑 is uniform in the interval [−10,10]. We check to ensure that all the 𝑞 SOCs are satisfied at the random point. Table 1 lists the SOC test problems. The CQC test problems are generated like the SOC test problems. However, the 𝑞 CQCs are generated so that each is satisfied at the origin and the entries of 𝑏 are uniform in [−0.5,0.5]. The CQC test problems are given in Table 1.

Table 1 

Test problems.

For each test problem and method, we chose a random infeasible point 𝑥₀ as our starting point with entries uniform in [−100,100]. We set our feasibility distance tolerance to be 𝛼=0.01 and our movement tolerance to be 𝛽=0.001. We chose our maximum number of iterations to be 𝜇=500. All codes were written in MATLAB^© 7.9.0 and ran on Dell OptiPlex GX280.

5.1. Second-Order Cones

Tables 3 and 4 give the results for the SOC test problems. If the number (Iter) of iterations is 500, it means the method did not converge. The last column shows whether or not the method converged to an interior feasible point. Table 2 gives the success rate over all the 25 problems in finding an interior feasible point and average time per problem (where converged). Both the Original method and DBmax method failed in Problem 6 and Problem 13.

Table 2 

Success rates and average times on SOCs.

Table 3 

SOCs: comparison of Original method and Original method with backtracking.

Table 4 

SOCs: comparison of DBmax method and DBmax method with backtracking.

As expected, on average the DBmax method took fewer iterations and less time than the Original method. Also, note that DBmax found an interior feasible point in Problem 24. It is interesting to note that the backtracking technique showed general improvement in iterations and processing time in the methods over all the test problems, where there was convergence.

On average the Original method with backtracking took less time than the Original method. Similarly, the DBmax method with backtracking took less time than the DBmax method. The Original method with backtracking has a strict feasibility success rate of 64% while the DBmax with backtracking has a success of 84%.

5.2. Convex Quadratic Constraints

Tables 6 and 7 give the results for the CQC test problems. Table 5 gives the success rate over all the 25 problems in finding an interior feasible point and average time per problem (where converged). Again, as expected, on average the DBmax method took less number of iterations and less time than the Original method.

Table 5 

Success rates and average times on CQCs.

Table 6 

CQCs: comparison of the Original method and Original method with backtracking.

Table 7 

CQCs: comparison of DBmax method and DBmax method with backtracking.

As can be seen from the tables, applying the backtracking technique to the Original method dramatically reduces the number of iterations and time. In addition, the Original method with backtracking has a strict feasibility success rate of 44%. With DBmax, the backtracking technique did not work well. There is significant increase in both iterations and time to reach feasibility and 11 out of the 25 problems diverged. This is most likely due to the fact that the consensus vector in DBmax is already long. This means that an extension of the consensus vector might move it too far and bring it right over the feasible region. We suspect that this is due to the nature of the CQC constraints, which tend to form bounded feasible regions.

6. Conclusion

We study the Chinneck's Original constraint consensus and DBmax methods as they apply to second-order cones (SOCs) and the special case of convex quadratic constraints (CQCs) to find near-feasible points. We also present a new backtracking line search technique that increases the size of the consensus vector with the goal of finding interior feasible points.

Given a set of SOCs, we adapt the Original and DBmax constraint consensus methods to find near-feasible points. These methods alone rarely can find an interior feasible point with SOCs. We develop a backtracking line search technique to find interior feasible points. We test the methods both with and without the backtracking technique over a variety of test problems and compare them with the time and number of iterations it takes to converge.

Before applying backtracking, the method known to reach feasibility in the least amount of time, with the fewest number of iterations was consistently DBmax. The Original with backtracking method is still not as fast as DBmax. However, the Original with backtracking reaches feasibility in fewer number of iterations than DBmax and is able to find interior feasible points in most of the test problems. The Original with backtracking method works well with SOCs and with CQCs. But, while DBmax with backtracking method works very well with SOCs, it does not work well with CQCs. It might be that with CQCs, the consensus vector in the backtracking step is increased too far.

Overall and considering both SOCs and CQCs, we find the backtracking line search to be most successful in reducing time and iterations needed to reach the feasible region when applied to the Original consensus method. As mentioned before, DBmax with backtracking method is not successful with CQCs. In the future it would be interesting to try more variations of the backtracking line search technique. We could compare the effectiveness of different initial step sizes and reductions in the backtracking technique, especially with DBmax when applied to CQCs. It would also be nice to investigate the effect of the backtracking technique on the other variants of the Original method given in [9].

Acknowledgment

The work of A. Weigandt and K. Tuthill was supported by the Department of Mathematics and Statistics at Northern Arizona University under the REU program of the National Science Foundation in Summer 2010.