A Trust-Region-Based BFGS Method with Line Search Technique for Symmetric Nonlinear Equations

Abstract

A trust-region-based BFGS method is proposed for solving symmetric nonlinear equations. In this given algorithm, if the trial step is unsuccessful, the linesearch technique will be used instead of repeatedly solving the subproblem of the normal trust-region method. We establish the global and superlinear convergence of the method under suitable conditions. Numerical results show that the given method is competitive to the normal trust region method.

1. Introduction

Consider the following system of nonlinear equations:

𝑔(𝑥)=0,𝑥∈ℜ^𝑛, (1.1) where 𝑔∶ℜ^𝑛→ℜ^𝑛 is continuously differentiable, and the Jacobian ∇𝑔(𝑥) of 𝑔 is symmetric for all 𝑥∈ℜ^𝑛. Let 𝜗 be the norm function defined by 𝜗(𝑥)=1/2‖𝑔(𝑥)‖². Then the nonlinear equations (1.1) is equivalent to the following global optimization problem:

min𝜗(𝑥),𝑥∈ℜ^𝑛. (1.2)

There are two ways for nonlinear equations by numerical methods. One is the line search method and the other is the trust region method. For the line search method, the following iterative formula is often used to solve (1.1):

𝑥_𝑘+1=𝑥_𝑘+𝛼_𝑘𝑑_𝑘, (1.3) where 𝑥_𝑘 is the 𝑘th iteration point, 𝛼_𝑘 is a steplength, and 𝑑_𝑘 is search direction. To begin, we briefly review some methods for (1.1) by line search technique. First, we give some techniques for 𝛼_𝑘. Brown and Saad [1] proposed the following line search method to obtain the stepsize 𝛼_𝑘∶

𝜗(𝑥_𝑘+𝛼_𝑘𝑑_𝑘)−𝜗(𝑥_𝑘)≤𝜎𝛼_𝑘∇𝜗(𝑥_𝑘)^𝑇𝑑_𝑘, (1.4) where 𝜎∈(0,1). Based on this technique, Zhu [2] gave the nonmonotone line search technique:

𝜗(𝑥_𝑘+𝛼_𝑘𝑑_𝑘)−𝜗(𝑥_𝑙(𝑘))≤𝜎𝛼_𝑘∇𝜗(𝑥_𝑘)^𝑇𝑑_𝑘, (1.5)‖𝜗(𝑥_𝑙(𝑘))‖=max_{0≤𝑗≤𝑚(𝑘)}{‖𝜗(𝑥_𝑘−𝑗)‖},𝑚(0)=0 and 𝑚(𝑘)=min{𝑚(𝑘−1)+1,𝑀},𝑘≥1, and 𝑀 is a nonnegative integer. From these two techniques (1.4) and (1.5), it is easy to see that the Jacobian matrix ∇𝑔_𝑘 must be computed at every iteration, which will increase the workload especially for large-scale problems or this matrix is expensive to calculate. Considering these points, we [3] presented a new backtracking inexact technique to obtain the stepsize 𝛼_𝑘:

‖𝑔(𝑥_𝑘+𝛼_𝑘𝑑_𝑘)‖²≤‖𝑔(𝑥_𝑘)‖²+𝛿𝛼²_𝑘𝑔^𝑇_𝑘𝑑_𝑘, (1.6) where 𝛿∈(0,1),𝑔_𝑘=𝑔(𝑥_𝑘), and 𝑑_𝑘 is a solution of the system of linear (1.15). We established the global convergence and the superlinear convergence of this method. The numerical results showed that the new line search technique is more effective than the normal methods. Li and Fukashima [4] proposed an approximate monotone line search technique to obtain the step-size 𝛼_𝑘 satisfying

𝜗(𝑥_𝑘+𝛼_𝑘𝑑_𝑘)−𝜗(𝑥_𝑘)≤−𝛿₁‖𝛼_𝑘𝑑_𝑘‖²−𝛿₂‖𝛼_𝑘𝑔_𝑘‖²+𝜖_𝑘‖𝑔(𝑥_𝑘)‖², (1.7) where 𝛿₁>0 and 𝛿₂>0 are positive constants, 𝛼_𝑘=𝑟^𝑖𝑘,𝑟∈(0,1),𝑖_𝑘 is the smallest nonnegative integer 𝑖 such that (1.7), and 𝜖_𝑘 satisfies

^∞_𝑘=0𝜖_𝑘<∞. (1.8) Combining the line search (1.7) with one special BFGS update formula, they got some better results (see [4]). Inspired by their idea, Wei [5] and Yuan [6–8] presented several approximate methods. Further work can be found in [9].

Second, we present some techniques for 𝑑_𝑘. One of the most effective methods is Newton method. It normally requires a fewest number of function evaluations, and it is very good at handling ill-conditioning. However, its efficiency largely depends on the possibility of solving a linear system efficiently which arises when computing the search 𝑑_𝑘 in each iteration:

∇𝑔(𝑥_𝑘)𝑑_𝑘=−𝑔(𝑥_𝑘). (1.9) Moreover, the exact solution of the system (1.9) could be too burdensome, or it is not necessary when 𝑥_𝑘 is far from a solution [10]. Inexact Newton methods [2, 3, 10] represent the basic approach underlying most of the Newton-type large-scale algorithms. At each iteration, the current estimate of the solution is updated by approximately solving the linear system (1.9) using an iterative algorithm. The inner iteration is typically “truncated” before the solution to the linear system is obtained. Griewank [11] firstly proposed the Broyden's rank one method for nonlinear equations and obtained the global convergence. At present, a lot of algorithms have been proposed for solving these two problems (1.1) and (1.2)(see [12–22] etc.).

Trust region method is a kind of important and efficient methods in the area of nonlinear optimization. This method can be traced back to the works of Levenberg [17] and Marquardt [18] on nonlinear least-squares problems and the work of Goldfeld et al. [23] for unconstrained optimization. Powell [24] was the first to establish the convergence result of trust region method for unconstrained optimization. Fletcher [25, 26] firstly proposed trust region algorithms for linearly constrained optimization problems and nonsmooth optimization problems, respectively. This method has been studied by many authors [15, 27–31] and has been applied to equality constrained problems [32–34]. Byrd et al. [35], Fan [36], Powell and Yuan [37], Vardi [38], Yuan [39, 40], Yuan et al. [41], and Zhang and Zhu [42] proposed various trust region algorithms for constrained optimization problems and established the convergence. Fan [36], Yuan [39], and Zhang [43] presented the trust region algorithms for nonlinear equations and got some results.

The normal trust-region subproblem for nonlinear equations is to find the trial step 𝑑_𝑘 such that

min𝑞^∗_𝑘(𝑑)=𝑑^𝑇∇𝑔(𝑥_𝑘)𝑔(𝑥_𝑘1)+2𝑑^𝑇∇𝑔(𝑥_𝑘)^𝑇∇𝑔(𝑥_𝑘)𝑑s.t.‖𝑑‖≤Δ_𝑘, (1.10) where Δ_𝑘>0 is a scalar called the trust region radium. Define the predicted descent of the objective function 𝑔(𝑥) at 𝑘th iteration by

𝑃red^∗_𝑘=𝑞^∗_𝑘(0)−𝑞^∗_𝑘(𝑑_𝑘), (1.11) the actual descent of 𝑔(𝑥) by

𝐴red^∗_𝑘=𝜗(𝑥_𝑘)−𝜗(𝑥_𝑘+𝑑_𝑘), (1.12) and the ratio of actual descent to predicted descent:

𝑟^∗_𝑘=𝐴red^∗_𝑘𝑃red^∗_𝑘. (1.13) For the normal trust region algorithm, if 𝑟^∗_𝑘≥𝜌 (𝜌∈(0,1), this case is called a successful iteration), the next iteration is 𝑥_𝑘+1=𝑥_𝑘+𝑑_𝑘, and go to the next step; otherwise reduce the trust region radium Δ_𝑘 and solve this subproblem (1.10) repeatedly. Sometimes, we must do this work many times and compute the Jacobian matrix ∇𝑔(𝑥_𝑘) and ∇𝑔(𝑥_𝑘)^𝑇∇𝑔(𝑥_𝑘) at every time, which obviously increases the work time and workload, especially for large-scale problems. Even more detrimental, the trust region subproblem is not very easy (see [36, 39] etc.) to be solved for most of the practical problems.

In order to alleviate the above bad situation that traditional algorithms have to compute Jacobian matrix ∇𝑔(𝑥_𝑘) and ∇𝑔(𝑥_𝑘)^𝑇∇𝑔(𝑥_𝑘) at each and every iteration while repeatedly resolving the trust region subproblem, in this paper, we would like to rewrite the following trust-region subproblem as

min𝑞_𝑘(𝑑)=𝑔(𝑥_𝑘)^𝑇1𝑑+2𝑑^𝑇𝐵_𝑘𝑑s.t.‖𝑑‖≤Δ_𝑘, (1.14) where matrix 𝐵_𝑘 is the approximation to the Jacobian matrix of 𝑔(𝑥) at 𝑥_𝑘. Due to the boundness of the region {𝑑∣‖𝑑‖≤Δ_𝑘}, (1.14) has a solution regardless of 𝐵^′_𝑘𝑠 definiteness (see [43]). This implies that it is valid to adopt a BFGS update formula to generate 𝐵_𝑘 for trust region methods and the BFGS update is presented as follows:

𝐵_𝑘+1=𝐵_𝑘+𝑦_𝑘𝑦^𝑇_𝑘𝑠^𝑇_𝑘𝑦_𝑘−𝐵_𝑘𝑠_𝑘𝑠^𝑇_𝑘𝐵_𝑘𝑠^𝑇_𝑘𝐵_𝑘𝑠_𝑘, (1.15) where 𝑦_𝑘=𝑔_𝑘+1−𝑔_𝑘,𝑠_𝑘=𝑥_𝑘+1−𝑥_𝑘. Define the predicted descent of the objective function 𝑔(𝑥) at 𝑘th iteration by

𝑃red_𝑘=𝑞_𝑘(0)−𝑞_𝑘(𝑑_𝑘), (1.16) the actual descent of 𝑔(𝑥) by

𝐴red_𝑘=‖𝑔(𝑥_𝑘)‖²−‖𝑔(𝑥_𝑘+𝑑_𝑘)‖², (1.17) and the ratio of actual descent to predicted descent:

𝑟_𝑘=‖𝑔(𝑥_𝑘)‖²−‖𝑔(𝑥_𝑘+𝑑_𝑘)‖²𝑞_𝑘(0)−𝑞_𝑘(𝑑_𝑘). (1.18) If 𝑟_𝑘≥𝜌 (𝜌∈(0,1), called a successful iteration ), the next iteration is 𝑥_𝑘+1=𝑥_𝑘+𝑑_𝑘. Otherwise, we use a search technique to obtain the steplength 𝜆_𝑘 and let the next iteration be 𝑥_𝑘+1=𝑥_𝑘+𝜆_𝑘𝑑_𝑘. Motivated by the idea of the paper [4], we propose the following linesearch technique to obtain 𝜆_𝑘:

‖𝑔(𝑥_𝑘+𝜆_𝑘𝑑_𝑘)‖²−‖𝑔_𝑘‖²≤−𝜎₁‖𝜆_𝑘𝑔_𝑘‖²−𝜎₂‖𝜆_𝑘𝑑_𝑘‖²+𝜎₃𝜆_𝑘𝑑^𝑇_𝑘𝑔_𝑘, (1.19) where 𝜎₁, 𝜎₂, and 𝜎₃ are some positive constants. In Section 3, we will show (1.19) is well-defined. Here and throughout this paper, ‖⋅‖ denotes the Euclidian norm of vectors or its induced matrix norm. 𝑔(𝑥_𝑘) is replaced by 𝑔_𝑘.

In the next section, the proposed algorithm for solving (1.1) is given. The global and superlinear convergence of the presented algorithm are stated in Section 3 and Section 4, respectively. The numerical results of the method are reported in Section 5.

2. Algorithms

Algorithm 2.1. Initial: choose 𝜌, 𝑟∈(0,1), 0<𝜏₁<𝜏₂<1<𝜏₃, 𝜎₁,𝜎₂,𝜎₃>0,Δ_min>0, 𝑥₀∈ℜ^𝑛. Let 𝑘∶=0;Step 1: Let Δ_𝑘=Δ_min;Step 2: If ‖𝑔_𝑘‖=0, stop. Otherwise go to Step 3;Step 3: Solve the subproblem (1.14) with Δ=Δ_𝑘 to get 𝑑_𝑘;Step 4: If 𝑟_𝑘≜‖𝑔(𝑥_𝑘)‖²−‖𝑔(𝑥_𝑘+𝑑_𝑘)‖²𝑞_𝑘(0)−𝑞_𝑘(𝑑_𝑘)<𝜌, (2.1) Go to Step 5; Otherwise Let 𝑥_𝑘+1=𝑥_𝑘+𝑑_𝑘,Δ_𝑘+1∈[‖𝑑_𝑘‖,𝜏₃‖𝑑_𝑘‖], and go to Step 6; Step 5: Let 𝑘 be the smallest nonnegative integer 𝑖 such that (1.19) holds for 𝜆=𝑟^𝑖. Let 𝜆_𝑘=𝑟^𝑖𝑘 and 𝑥_𝑘+1=𝑥_𝑘+𝜆_𝑘𝑑_𝑘, Δ_𝑘+1∈[𝜏₁‖𝑑_𝑘‖,𝜏₂‖𝑑_𝑘‖];Step 6: Update 𝐵_𝑘 to get 𝐵_𝑘+1 by (1.15). Let 𝑘∶=𝑘+1. Go to Step 2. Here we also give a normal trust-region method for (1.1) and call it Algorithm 2.2.

Algorithm 2.2 (the normal Trust-Region Algorithm [44]). Initial: Given a starting point 𝑥₀∈ℜ^𝑛,Δ₀>0 is the initial trust region radium, an upper bound of trust region radius Δ^′,0<Δ₀≤Δ^′. Set 0<𝜇<1,0<𝜂₁<𝜂₂<1<𝜂₃,𝑘∶=0.Step 1: If ‖𝑔_𝑘‖=0, stop. Otherwise, go to Step 2.Step 2: Solve the trust-region subproblem (1.10) to obtain 𝑑_𝑘.Step 3: Let 𝑟_𝑘=𝜗(𝑥_𝑘)−𝜗(𝑥_𝑘+𝑑_𝑘)𝑞^∗_𝑘(0)−𝑞^∗_𝑘(𝑑_𝑘), (2.2) if 𝑟_𝑘<𝜂₁, set Δ_𝑘+1=𝜂₁Δ_𝑘; If 𝑟_𝑘>𝜂₂ and ‖𝑑_𝑘‖=Δ_𝑘, let Δ_𝑘+1=min{𝜂₃Δ_𝑘,Δ^′}; Otherwise, let Δ_𝑘+1=Δ_𝑘.Step 4: If 𝑟_𝑘>𝜇, let 𝑥_𝑘+1=𝑥_𝑘+𝑑_𝑘 and go to Step 5; otherwise, let 𝑥_𝑘+1=𝑥_𝑘, go to Step 2.Step 5: Set 𝑘∶=𝑘+1. Go to Step 1.

Remark 2.3. By 𝑦_𝑘=𝑔_𝑘+1−𝑔_𝑘, we have the following approximate relations: 𝑦_𝑘=𝑔_𝑘+1−𝑔_𝑘≈∇𝑔_𝑘+1𝑠_𝑘. (2.3) Since 𝐵_𝑘+1 satisfies the secant equation 𝐵_𝑘+1𝑠_𝑘=𝑦_𝑘 and ∇𝑔_𝑘+1 is symmetric, we have approximately 𝐵_𝑘+1𝑠_𝑘≈∇𝑔_𝑘+1𝑠_𝑘=∇𝑔^𝑇_𝑘+1𝑠_𝑘. (2.4) This means that 𝐵_𝑘+1 approximates ∇𝑔_𝑘+1 along direction 𝑠_𝑘.

3. The Global Convergence

In this section, we will establish the global convergence of Algorithm 2.1. Let Ω be the level set defined by

Ω={𝑥∣‖𝑔(𝑥)‖≤‖𝑔(𝑥₀)‖}, (3.1) which is bounded.

Assumption 1. (A) 𝑔 is continuously differentiable on an open convex set Ω₁ containing Ω.
(B) The Jaconbian of 𝑔 is symmetric and bounded on Ω₁ and there exists a positive constant 𝑀 such that ‖∇𝑔(𝑥)‖≤𝑀∀𝑥∈Ω₁. (3.2)
(C) ∇𝑔 is positive definite on Ω₁; that is, there is a constant 𝑚>0 such that 𝑚‖𝑑‖²≤𝑑^𝑇∇𝑔(𝑥)𝑑∀𝑥∈Ω₁,𝑑∈ℜ^𝑛. (3.3)
(D) 𝜗(𝑥) is differentiable and its gradient satisfies ‖∇𝜗(𝑥)−∇𝜗(𝑦)‖≤𝐿‖𝑥−𝑦‖,∀𝑥,𝑦∈Ω₁, (3.4) where 𝐿 is the Lipschitz constant. By Assumptions 1(A) and 1(B), it is not difficult to get the following inequality: ‖𝑦_𝑘‖≤𝑀‖𝑠_𝑘‖. (3.5) According to Assumptions 1(A) and 1(C), we have 𝑠^𝑇_𝑘𝑦_𝑘=𝑠^𝑇_𝑘∇𝑔(𝜉)𝑠_𝑘≥𝑚‖𝑠_𝑘‖², (3.6) where 𝜉=𝑥_𝑘+𝜗₀(𝑥_𝑘+1−𝑥_𝑘),𝜗₀∈(0,1), which means that the update matrix 𝐵_𝑘 is always positive definite. By (3.5) and (3.6), we have 𝑠^𝑇_𝑘𝑦_𝑘‖𝑠_𝑘‖²≥𝑚,‖𝑦_𝑘‖²𝑠^𝑇_𝑘𝑦_𝑘≤𝑀²𝑚. (3.7)

Lemma 3.1 ([see Theorem 2.1 in [45]]). Suppose that Assumption 1 holds. Let 𝐵_𝑘 be updated by BFGS formula (1.15) and let 𝐵₀ be symmetric and positive definite. For any 𝑘≥0,𝑠_𝑘 and 𝑦_𝑘 satisfy (3.7). Then there exist positive constants 𝛽₁,𝛽₂, and 𝛽₃ such that, for any positive integer 𝑘𝛽₁‖𝑑_𝑘‖²≤𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘≤𝛽₂‖𝑑_𝑘‖²,𝛽₁‖𝑑_𝑘‖≤‖𝐵_𝑘𝑑_𝑘‖≤𝛽₃‖𝑑_𝑘‖ (3.8) hold for at least ⌈𝑘/2⌉ value of 𝑘∈{1,2,…,𝑘}.

Considering the subproblem (1.14), we give the following assumption similar to (1.14). Similar to [2], the following assumption is needed.

Assumption 2. 𝐵_𝑘 is a good approximation to ∇𝑔_𝑘, that is, ‖(∇𝑔_𝑘−𝐵_𝑘)𝑑_𝑘‖≤𝜀₀‖𝑔_𝑘‖, (3.9) and 𝑑_𝑘 satisfies ‖𝑔_𝑘+𝐵_𝑘𝑑_𝑘‖≤𝜀₁‖𝑔_𝑘‖, (3.10) where 𝜀₀∈(0,1) is a small quantity, and 𝜀₁>0,𝜀₀+𝜀₁∈(0,1).

Lemma 3.2. Let Assumption 2 hold. Then 𝑑_𝑘 is descent direction for 𝜗(𝑥) at 𝑥_𝑘, that is, ∇𝜗(𝑥_𝑘)^𝑇𝑑_𝑘<0. (3.11)

Proof. Let 𝑟_𝑘 be the residual associated with 𝑑_𝑘 so that 𝑔_𝑘+𝐵_𝑘𝑑_𝑘=𝑟_𝑘: ∇𝜗(𝑥_𝑘)^𝑇𝑑_𝑘=𝑔(𝑥_𝑘)^𝑇∇𝑔(𝑥_𝑘)𝑑_𝑘=𝑔(𝑥_𝑘)^𝑇[(∇𝑔(𝑥_𝑘)−𝐵_𝑘)𝑑_𝑘+(𝑟_𝑘−𝑔(𝑥_𝑘))]=𝑔(𝑥_𝑘)^𝑇(∇𝑔(𝑥_𝑘)−𝐵_𝑘)𝑑_𝑘+𝑔(𝑥_𝑘)^𝑇𝑟_𝑘−𝑔(𝑥_𝑘)^𝑇𝑔(𝑥_𝑘). (3.12) So we have ∇𝜗(𝑥_𝑘)^𝑇𝑑_𝑘+‖𝑔(𝑥_𝑘)‖²=𝑔(𝑥_𝑘)^𝑇(∇𝑔(𝑥_𝑘)−𝐵_𝑘)𝑑_𝑘+𝑔(𝑥_𝑘)^𝑇𝑟_𝑘. (3.13) Therefore, taking the norm in the right-hand side of the above equality, we have that from Assumption 2∇𝜗(𝑥_𝑘)^𝑇𝑑_𝑘≤‖𝑔(𝑥_𝑘)‖‖(∇𝑔(𝑥_𝑘)−𝐵_𝑘)𝑑_𝑘‖+‖𝑔(𝑥_𝑘)‖‖𝑟_𝑘‖−‖𝑔(𝑥_𝑘)‖²≤−(1−𝜀₀−𝜀₁)‖𝑔(𝑥_𝑘)‖². (3.14) Hence, for 𝜀₀+𝜀₁∈(0,1), the lemma is satisfied.

According to the above lemma, it is easy to deduce that the norm function 𝜗(𝑥) is descent, which means that ‖𝑔_𝑘+1‖≤‖𝑔_𝑘‖ is true.

Lemma 3.3. Let {𝑥_𝑘} be generated by Algorithm 2.1 and suppose that Assumption 2 holds. Then {𝑥_𝑘}⊂Ω. Moreover, {‖𝑔_𝑘‖} converges.

Proof. By Lemma 3.2, we have ‖𝑔_𝑘+1‖≤‖𝑔_𝑘‖. Then we conclude from Lemma 3.3 in [46] that {‖𝑔_𝑘‖} converges. Moreover, we have for all 𝑘‖𝑔_𝑘+1‖≤‖𝑔_𝑘‖≤‖𝑔_𝑘−1‖≤⋯≤‖𝑔(𝑥₀)‖. (3.15) This implies that {𝑥_𝑘}⊂Ω.

Lemma 3.4. Let Assumption 1 hold. Then the following inequalities 𝑔^𝑇_𝑘𝑑_𝑘≤−𝛽₁‖𝑑_𝑘‖²,‖𝑔_𝑘‖²≥𝛽²₁‖𝑑_𝑘‖²−1𝛽₁‖𝑔_𝑘‖²≤𝑔^𝑇_𝑘𝑑_𝑘 (3.16)(3.17) hold.

Proof. Since the update matrix 𝐵_𝑘 is positive definite. Then, problem (1.14) has a unique solution 𝑑_𝑘, which together with some multiplier 𝛼_𝑘≥0 satisfies the following equations: 𝐵_𝑘𝑑_𝑘+𝛼_𝑘𝑑_𝑘=−𝑔_𝑘,𝛼_𝑘(‖𝑑_𝑘‖−Δ_𝑘)=0. (3.18) From (3.18), we can obtain 𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘+𝑔^𝑇_𝑘𝑑_𝑘=−𝛼_𝑘‖𝑑_𝑘‖²𝛼≤0,_𝑘=−𝑔^𝑇_𝑘𝑑_𝑘−𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘‖𝑑_𝑘‖². (3.19)(3.20) By (3.19) and (3.8), we get (3.16), which also imply that the inequality (3.17) holds.

The next lemma will show that (1.19) is reasonable, and then Algorithm 2.1 is well defined.

Lemma 3.5. Let Assumptions 1(D) and 2 hold. Then there exists a step-size 𝜆_𝑘 such that (1.19) in a finite number of backtracking steps.

Proof. From Lemma 3.8 in [1] we have that in a finite number of backtracking steps, 𝜆_𝑘 must satisfy ‖𝑔(𝑥_𝑘+𝛼_𝑘𝑑_𝑘)‖²−‖𝑔(𝑥_𝑘)‖²≤𝛿𝜆_𝑘𝑔(𝑥_𝑘)^𝑇∇𝑔(𝑥_𝑘)𝑑_𝑘,𝛿∈(0,1). (3.21) By (3.12) and (3.14), let 𝛽₀=(1−𝜀₀−𝜀₁), and we have 𝑔(𝑥_𝑘)^𝑇∇𝑔(𝑥_𝑘)𝑑_𝑘≤−𝛽₀‖𝑔_𝑘‖²𝛽=−⁰3‖𝑔_𝑘‖²−𝛽₀3‖𝑔_𝑘‖²−𝛽₀3‖𝑔_𝑘‖²𝛽≤−⁰3‖𝑔_𝑘‖²−𝛽₀3𝛽²₁‖𝑑_𝑘‖²+𝛽₀3𝛽₁𝑔^𝑇_𝑘𝑑_𝑘, (3.22) where the last inequality follows (3.16) and (3.17). By 𝜆_𝑘≤1, let 𝜎₁∈(0,(𝛽₀/3)𝛿),𝜎₂∈(0,(𝛽₀/3)𝛽²₁𝛿),𝜎₃∈(0,(𝛽₀/3)𝛽₁𝛿), then we obtain (1.19). The proof is complete.

Lemma 3.6. Let {𝑥_𝑘} be generated by the Algorithm 2.1. Suppose that Assumptions 1 and 2 hold. Then one has ^∞_𝑘=0(−𝑔^𝑇_𝑘𝑑_𝑘)<∞,^∞_𝑘=0𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘<∞. (3.23) In particular, one has lim_𝑘→∞(−𝑔^𝑇_𝑘𝑑_𝑘)=0,lim_𝑘→∞𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘=0. (3.24)

Proof. By (3.8) and (3.19), we have 𝑞_𝑘(𝑑_𝑘)=𝑔^𝑇_𝑘𝑑_𝑘+12𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘≤12𝑔^𝑇_𝑘𝑑_𝑘1≤−2𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘. (3.25) From Step 4 of Algorithm 2.1, if 𝑟_𝑘≥𝜌 is true, we get ‖𝑔(𝑥_𝑘+1)‖²−‖𝑔(𝑥_𝑘)‖²≤𝑞_𝑘(𝑑_𝑘1)≤2𝑔^𝑇_𝑘𝑑_𝑘1≤−2𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘, (3.26) otherwise, if 𝑟_𝑘<𝜌 is true, by Step 5 of Algorithm 2.1, (3.8), and (3.26), we can obtain ‖𝑔(𝑥_𝑘+1)‖²−‖𝑔(𝑥_𝑘)‖²≤−𝜎₁‖𝜆_𝑘𝑔_𝑘‖²−𝜎₂‖𝜆_𝑘𝑑_𝑘‖²+𝜎₃𝜆_𝑘𝑑^𝑇_𝑘𝑔_𝑘≤𝜎₃𝜆_𝑘𝑑^𝑇_𝑘𝑔_𝑘≤−𝜎₃𝜆_𝑘𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘. (3.27) By Lemma 3.5, we know that (1.19) can be satisfied in a finite number of backtracking steps, which means that there exists a constant 𝜆^∗∈(0,1) satisfying 𝜆^∗≤𝜆_𝑘 for all 𝑘. By (3.26) and (3.27), we have ‖𝑔(𝑥_𝑘+1)‖²−‖𝑔(𝑥_𝑘)‖²≤𝜌₁𝑔^𝑇_𝑘𝑑_𝑘≤−𝜌₁𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘≤−𝜌₁𝛽₁‖𝑑_𝑘‖²<0, (3.28) where 𝜌₁=min{1/2,𝜎₃𝜆^∗}. According to (3.28), we get ^∞_𝑘=0𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘≤^∞_𝑘=0(−𝑔^𝑇_𝑘𝑑_𝑘1)≤𝜌₁^∞_𝑘=0(‖𝑔(𝑥_𝑘)‖²−‖𝑔(𝑥_𝑘+1)‖²)=1𝜌₁lim^𝑁_{𝑁→∞𝑘=0}(‖𝑔(𝑥_𝑘)‖²−‖𝑔(𝑥_𝑘+1)‖²)=1𝜌₁lim_𝑁→∞(‖𝑔(𝑥₀)‖²−‖𝑔(𝑥_𝑁+1)‖²), (3.29) and by Lemma 3.3, we know that {‖𝑔_𝑘‖} is convergent. Therefore, we deduce that (3.23) holds. According to (3.23), it is easy to deduce (3.24). The proof is complete.

Lemma 3.7. Suppose that Assumptions 1 and 2 hold. There are positive constants 𝑏₁≤𝑏₂,𝑎𝑛𝑑𝑏₃ such that for any 𝑘, if ‖𝑑_𝑘‖≠Δ_min, then the following inequalities hold: 𝑏₁‖𝑔_𝑘‖≤‖𝑑_𝑘‖≤𝑏₂‖𝑔_𝑘‖,𝛼_𝑘≤𝑏₃. (3.30)

Proof. We will prove this lemma in the following two cases.Case 1 (‖𝑑_𝑘‖<Δ_𝑘). By (3.18), we have 𝛼_𝑘=0 and 𝐵_𝑘𝑑_𝑘=−𝑔_𝑘. Together with (3.8) and (3.19), we get 𝛽₁‖𝑑_𝑘‖²≤𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘=−𝑑^𝑇_𝑘𝑔_𝑘≤‖𝑑_𝑘‖‖𝑔_𝑘‖,‖−𝑔_𝑘‖=‖𝑔_𝑘‖=‖𝐵_𝑘𝑑_𝑘‖≤𝛽₃‖𝑑_𝑘‖. (3.31) Then (3.30) holds with 𝑏₁=1/𝛽₃≤𝑏₂=1/𝛽₁ and 𝑏₃=0.Case 2 (‖𝑑_𝑘‖=Δ_𝑘). From (3.19) and (3.8), we have 𝛽₁‖𝑑_𝑘‖²≤𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘≤−𝑔^𝑇_𝑘𝑑_𝑘≤‖𝑔_𝑘‖‖𝑑_𝑘‖. (3.32) Then, we get ‖𝑑_𝑘‖≤1/𝛽₁‖𝑔_𝑘‖. By (3.10) and (3.8), it is easy to deduce that (1−𝜀₁)‖𝑔_𝑘‖≤‖𝐵_𝑘𝑑_𝑘‖≤𝛽₃‖𝑑_𝑘‖. (3.33) So we obtain ‖𝑑_𝑘‖≥(1−𝜀₁)/𝛽₃‖𝑔_𝑘‖. Using (3.20), we have 𝛼_𝑘=−𝑔^𝑇_𝑘𝑑_𝑘−𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘‖𝑑_𝑘‖²≤‖𝑔_𝑘‖‖𝑑_𝑘‖≤𝛽₃1−𝜀₁. (3.34) Therefore, (3.30) holds. The proof is complete.

In the next theorem, we establish the global convergence of Algorithm 2.1.

Theorem 3.8. Let {𝑥_𝑘} be generated by Algorithm 2.1 and the conditions in Assumptions 1 and 2 hold. Then one has lim_𝑘→∞‖𝑔_𝑘‖=0. (3.35)

Proof. By Lemma 3.6, we have lim_𝑘→∞−𝑔^𝑇_𝑘𝑑_𝑘=lim_𝑘→∞𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘=0. (3.36) Combining (3.8) and (3.36), we get lim_𝑘→∞‖𝑑_𝑘‖=0. (3.37) Together with (3.30), we obtain (3.35). The proof is complete.

4. The Superlinear Convergence Analysis

In this section, we will present the superlinear convergence of Algorithm 2.1.

Assumption 3. ∇𝑔 is Hölder continuous at 𝑥^∗; that is, for every 𝑥 in a neighborhood of 𝑥^∗, there are positive constants 𝑀₁ and 𝛾 such that ‖∇𝑔(𝑥)−∇𝑔(𝑥^∗)‖≤𝑀₁‖𝑥−𝑥^∗‖^𝛾, (4.1) where 𝑥^∗ stands for the unique solution of (1.1) in Ω₁.

Lemma 4.1. Let {𝑥_𝑘} be generated by Algorithm 2.1 and the conditions in Assumptions 1 and 2 hold. Then, for any fixed 𝛾>0, one has ^∞_𝑘=0‖𝑥_𝑘−𝑥^∗‖^𝛾<∞. (4.2) Moreover, one has ^∞_𝑘=0𝜒_𝑘(𝛾)<∞, (4.3) where 𝜒_𝑘(𝛾)=max{‖𝑥_𝑘−𝑥^∗‖^𝛾,‖𝑥_𝑘+1−𝑥^∗‖^𝛾}.

Proof. Using Assumption 1, we can have the following inequality: 𝑚‖𝑥−𝑥^∗‖≤‖𝑔(𝑥)‖=‖𝑔(𝑥)−𝑔(𝑥^∗)‖≤𝑀‖𝑥−𝑥^∗‖,𝑥∈Ω₁. (4.4) By (3.8) and (3.30), we have −𝛽₂‖𝑑_𝑘‖²≤−𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘≤−𝛽₁‖𝑑_𝑘‖²,−𝑏²₂‖𝑔_𝑘‖²≤−‖𝑑_𝑘‖²≤−𝑏²₁‖𝑔_𝑘‖². (4.5) Together with (3.28), we get ‖𝑔_𝑘+1‖²−‖𝑔_𝑘‖²≤𝜌₁𝑔^𝑇_𝑘𝑑_𝑘≤−𝜌₁𝑑^𝑇_𝑘𝐵_𝑘𝑑_𝑘≤−𝜌₁𝛽₁‖𝑑_𝑘‖²≤−𝜌₁𝛽₁𝑏²₁‖𝑔_𝑘‖², (4.6) and let 𝜌₀=min{𝜌₁𝛽₁𝑏²₁,𝜌}∈(0,1). Suppose that there exists a positive integer 𝑘₀, as 𝑘≥𝑘₀, (3.8) holds. Then we obtain ‖𝑔_𝑘+1‖²≤‖𝑔_𝑘‖²−𝜌₀‖𝑔_𝑘‖²=(1−𝜌₀)‖𝑔_𝑘‖²≤⋯≤(1−𝜌₀)^{𝑘−𝑘0+1}‖𝑔_𝑘0‖²=𝑐₀𝑐^𝑘₁, (4.7) where 𝑐₀=(1−𝜌₀)^1−𝑘0‖𝑔₀‖²,𝑐₁=(1−𝜌₀)∈(0,1). This together with (4.4) shows that ‖𝑥_𝑘+1−𝑥^∗‖²≤𝑚⁻²𝑐₀𝑐^𝑘₁ holds for all 𝑘 large enough. Therefore, for any 𝛾, we have (4.2). Notice that 𝜒_𝑘(𝛾)≤‖𝑥_𝑘−𝑥^∗‖^𝛾+‖𝑥_𝑘+1−𝑥^∗‖^𝛾; from (4.2), we can get (4.3).

Lemma 4.2. Let Assumptions 1, 2, and 3 hold. Then, for all 𝑘 sufficiently large, there exists a positive constant 𝑀₂ such that ‖𝑦_𝑘−∇𝑔(𝑥^∗)𝑠_𝑘‖≤𝑀₂𝜒_𝑘‖𝑠_𝑘‖, (4.8) where 𝜒_𝑘=max{‖𝑥_𝑘−𝑥^∗‖^𝛾,‖𝑥_𝑘+1−𝑥^∗‖^𝛾}.

Proof. From Theorem 3.8 and (4.4), it is not difficult to get 𝑥_𝑘→𝑥^∗. Then (4.1) holds for all 𝑘 large enough. Using the mean value theorem, for all 𝑘 sufficiently large, we have ‖𝑦_𝑘−∇𝑔(𝑥^∗)𝑠_𝑘‖=‖∇𝑔(𝑥_𝑘+𝑡₀(𝑥_𝑘+1−𝑥_𝑘))𝑠_𝑘−∇𝑔(𝑥^∗)𝑠_𝑘‖≤‖∇𝑔(𝑥_𝑘+𝑡₀(𝑥_𝑘+1−𝑥_𝑘))−∇𝑔(𝑥^∗)‖‖𝑠_𝑘‖≤𝑀₁‖𝑥_𝑘+𝑡₀(𝑥_𝑘+1−𝑥_𝑘)−𝑥^∗‖^𝛾‖𝑠_𝑘‖≤𝑀₂𝜒_𝑘‖𝑠_𝑘‖, (4.9) where 𝑀₂=𝑀₁(2𝑡₀+1),𝑡₀∈(0,1). Therefore, the inequality of (4.8) holds.

Lemma 4.3. Let Assumptions 1, 2, and 3 hold and let {𝑥_𝑘} be generated by Algorithm 2.1. Denote 𝑄=∇𝑔(𝑥^∗)^−1/2, 𝐻_𝑘=𝐵_𝑘⁻¹. Then, for all large 𝑘, there are positive constants 𝑒_𝑖, 𝑖=1,2,3,4, and 𝜂∈(0,1) such that ‖𝐵_𝑘+1−∇𝑔(𝑥^∗)‖_𝑄,𝐹≤(1+𝑒₁𝜒_𝑘)‖𝐵_𝑘−∇𝑔(𝑥^∗)‖_𝑄,𝐹+𝑒₂𝜒_𝑘,‖𝐻_𝑘+1−∇𝑔(𝑥^∗)⁻¹‖_{𝑄−1,𝐹}≤(1−𝜂𝜛²_𝑘+𝑒₃𝜒_𝑘)‖𝐻_𝑘−∇𝑔(𝑥^∗)⁻¹‖_{𝑄−1,𝐹}+𝑒₄𝜒_𝑘, (4.10)(4.11) where ‖𝐴‖_𝑄,𝐹=‖𝑄^𝑇𝐴𝑄‖_𝐹, ‖⋅‖_𝐹 is the Frobenius norm of a matrix and 𝜛_𝑘 is defined as follows: 𝜛_𝑘=‖𝑄⁻¹(𝐻_𝑘−∇𝑔(𝑥^∗)⁻¹)𝑦_𝑘‖‖𝐻_𝑘−∇𝑔(𝑥^∗)⁻¹‖_{𝑄−1,𝐹}‖𝑄𝑦_𝑘‖. (4.12) In particular, {‖𝐵_𝑘‖}_𝐹 and {‖𝐻_𝑘‖}_𝐹 are bounded.

Proof. From (1.15), we have ‖𝐵_𝑘+1−∇𝑔(𝑥^∗)‖_𝑄,𝐹=‖𝐵_𝑘−∇𝑔(𝑥^∗𝐵)+^𝑘𝑠_𝑘𝑠^𝑇_𝑘𝐵_𝑘𝑠^𝑇_𝑘𝐵_𝑘𝑠_𝑘+𝑦_𝑘𝑦^𝑇_𝑘𝑠^𝑇_𝑘𝑦_𝑘‖_𝑄,𝐹≤(1+𝑒₁𝜏_𝑘)‖𝐵_𝑘−∇𝑔(𝑥^∗)‖_𝑄,𝐹+𝑒₂𝜒_𝑘, (4.13) where the last inequality follows the inequality (49) of [47]. Hence, (4.10) holds. By (4.8), in a way similar to that of [46], we can prove that (4.11) holds and ‖𝐵_𝑘‖ and ‖𝐻_𝑘‖ are bounded. The proof is complete.

Lemma 4.4. Let {𝑥_𝑘} be generated by Algorithm 2.1 and the conditions in Assumptions 1, 2 and 3 hold. Then lim_𝑘→∞‖(𝐵_𝑘−∇𝑔(𝑥^∗))𝑠_𝑘‖‖𝑠_𝑘‖=0, (4.14) where 𝑠_𝑘=𝑥_𝑘+1−𝑥_𝑘.

Proof. In a similar way to [46], it is not difficult to obtain lim_𝑘→∞‖𝑄⁻¹(𝐻_𝑘−∇𝑔(𝑥^∗)⁻¹)𝑦_𝑘‖‖𝑄𝑦_𝑘‖=0. (4.15) On the other hand, we have ‖𝑄⁻¹(𝐻_𝑘−∇𝑔(𝑥^∗)⁻¹)𝑦_𝑘‖=‖𝑄⁻¹𝐻_𝑘(∇𝑔(𝑥^∗)−𝐵_𝑘)∇𝑔(𝑥^∗)⁻¹𝑦_𝑘‖≥‖𝑄⁻¹𝐻_𝑘(∇𝑔(𝑥^∗)−𝐵_𝑘)𝑠_𝑘‖−‖𝑄⁻¹𝐻_𝑘(∇𝑔(𝑥^∗)−𝐵_𝑘)(𝑠_𝑘−∇𝑔(𝑥^∗)⁻¹𝑦_𝑘)‖≥‖𝑄⁻¹𝐻_𝑘(∇𝑔(𝑥^∗)−𝐵_𝑘)𝑠_𝑘‖−‖𝑄⁻¹‖‖𝐻_𝑘‖(‖∇𝑔(𝑥^∗)‖+‖𝐵_𝑘‖)‖∇𝑔(𝑥^∗)⁻¹(𝑦_𝑘−∇𝑔(𝑥^∗)𝑠_𝑘)‖≥‖𝑄⁻¹𝐻_𝑘(∇𝑔(𝑥^∗)−𝐵_𝑘)𝑠_𝑘‖−𝑀₂𝜒_𝑘‖𝑄⁻¹‖‖𝐻_𝑘‖(‖∇𝑔(𝑥^∗)‖+‖𝐵_𝑘‖)‖∇𝑔(𝑥^∗)⁻¹‖‖𝑠_𝑘‖=‖𝑄⁻¹𝐻_𝑘(∇𝑔(𝑥^∗)−𝐵_𝑘)𝑠_𝑘‖−𝑜(‖𝑠_𝑘‖), (4.16) where the last inequality follows from (4.8). We know that {‖𝐵_𝑘‖} and {‖𝐻_𝑘‖} are bounded, and {𝐻_𝑘} is positive definite. By (3.5), we get ‖𝑄𝑦_𝑘‖≤𝑀‖𝑄‖‖𝑠_𝑘‖. (4.17) Combining (4.15) and (4.17), we conclude that (4.14) holds. The proof is complete.

Theorem 4.5. Let the conditions in Assumptions 1, 2 and 3 hold. If 𝜀₁→0 in (3.10). Then the sequence {𝑥_𝑘} generated by Algorithm 2.1 converges to 𝑥^∗ superlinearly for 𝜆_𝑘=1.

Proof. For all 𝑥_𝑘∈Ω₁, we get ‖𝑔_𝑘+1‖‖𝑑_𝑘‖=‖𝑔_𝑘+𝐵_𝑘𝑑_𝑘+(∇𝑔_𝑘−𝐵_𝑘)𝑑_𝑘+𝑂(‖𝑑_𝑘‖²)‖‖𝑑_𝑘‖≤‖𝑔_𝑘+𝐵_𝑘𝑑_𝑘‖‖𝑔_𝑘‖‖𝑔_𝑘‖‖𝑑_𝑘‖+‖(∇𝑔_𝑘−𝐵_𝑘)𝑑_𝑘‖‖𝑑_𝑘‖+𝑂(‖𝑑_𝑘‖²)‖𝑑_𝑘‖≤𝜀₁‖𝑔_𝑘‖‖𝑑_𝑘‖+‖(∇𝑔_𝑘−𝐵_𝑘)𝑑_𝑘‖‖𝑑_𝑘‖+𝑂(‖𝑑_𝑘‖), (4.18) where the last inequality follows (3.10). By (3.5), we have ‖𝑔_𝑘‖≤‖𝑔_𝑘+1−𝑔_𝑘‖+‖𝑔_𝑘+1‖≤𝑀‖𝑑_𝑘‖+‖𝑔_𝑘+1‖. (4.19) Dividing both sides by ‖𝑑_𝑘‖, we get ‖𝑔_𝑘‖‖𝑑_𝑘‖≤𝑀+‖𝑔_𝑘+1‖‖𝑑_𝑘‖. (4.20) Substituting this into (4.18), we can obtain ‖𝑔_𝑘+1‖‖𝑑_𝑘‖≤𝜀₁(𝑀+‖𝑔_𝑘+1‖‖𝑑_𝑘‖)+‖(∇𝑔_𝑘−𝐵_𝑘)𝑑_𝑘‖‖𝑑_𝑘‖+𝑂(‖𝑑_𝑘‖), (4.21) which means that ‖𝑔_𝑘+1‖‖𝑑_𝑘‖≤(𝑀𝜀₁+‖(∇𝑔_𝑘−𝐵_𝑘)𝑑_𝑘‖/‖𝑑_𝑘‖+𝑂(‖𝑑_𝑘‖))(1−𝜀₁). (4.22) Since 𝜀₁→0, and ‖𝑑_𝑘‖→0 as 𝑘→∞, by (4.14) and (3.10), we have lim_𝑘→∞‖𝑔_𝑘+1‖‖𝑑_𝑘‖=0. (4.23) Using (3.16), we get lim_𝑘→∞‖𝑔_𝑘+1‖‖𝑔_𝑘‖=0. (4.24) Considering (4.4), we have lim_𝑘→∞‖𝑥_𝑘+𝑑_𝑘−𝑥^∗‖‖𝑥_𝑘−𝑥^∗‖=0. (4.25) Therefore, we get the result of the superlinear convergence.

5. Numerical Results

In this section, we test the proposed BFGS trust-region method on symmetric nonlinear equations and compare it with Algorithm 2.2. The following problems with various sizes will be solved.

Problem 1. The discretized two-point boundary value problem like the problem in [48] is 1𝑔(𝑥)≜𝐴𝑥+(𝑛+1)²𝐹(𝑥)=0, (5.1) where 𝐴 is the 𝑛×𝑛 tridiagonal matrix given by 𝐴=8−1−18−1−18−1⋱⋱⋱⋱⋱−1−18, (5.2) and 𝐹(𝑥)=(𝐹₁(𝑥),𝐹₂(𝑥),…,𝐹_𝑛(𝑥))^𝑇with 𝐹_𝑖(𝑥)=sin𝑥_𝑖−1,𝑖=1,2,…,𝑛.

Problem 2. Unconstrained optimization problem is min𝑓(𝑥),𝑥∈𝑅^𝑛, (5.3) with Engval function [49] 𝑓∶𝑅^𝑛→𝑅 defined by 𝑓(𝑥)=^𝑛_𝑖=2[(𝑥²_𝑖−1+𝑥²_𝑖)²−4𝑥_𝑖−1+3]. (5.4) The related symmetric nonlinear equation is 1𝑔(𝑥)≜4∇𝑓(𝑥)=0, (5.5) where 𝑔(𝑥)=(𝑔₁(𝑥),𝑔₂(𝑥),…,𝑔_𝑛(𝑥))^𝑇 with 𝑔₁(𝑥)=𝑥₁(𝑥²₁+𝑥²₂𝑔)−1,_𝑖(𝑥)=𝑥_𝑖(𝑥²_𝑖−1+2𝑥²_𝑖+𝑥²_𝑖+1𝑔)−1,𝑖=2,3,…,𝑛−1,_𝑛(𝑥)=𝑥_𝑛(𝑥²_𝑛−1+𝑥²_𝑛). (5.6) In the experiments, the parameters in Algorithm 2.1 were chosen as 𝜏₁=0.5,𝜏₂=0.9,𝜏₃=3,𝑟=0.1,Δ_min=‖𝑔₀‖, 𝐵₀=𝐼,𝜌=0.25, 𝜎₁=𝜎₂=10⁻⁵, and 𝜎₃=0.9. We obtain 𝑑_𝑘 from subproblem (1.14) by the well-known Dogleg method. The parameters in Algorithm 2.2 were chosen as Δ^′=Δ₀=‖𝑔₀‖,𝜂₁=0.25,𝜂₂=0.75,𝜇=0.01, and 𝜂₃=2. Since the matrices ∇𝑔(𝑥_𝑘)^𝑇∇𝑔(𝑥_𝑘) will be singular, we solve (1.10) by 𝐸𝑥𝑡𝑟𝑒𝑚𝑒𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑎𝑡𝑖𝑜𝑛𝑤𝑖𝑡ℎ2−𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛𝑆𝑢𝑏𝑠𝑝𝑎𝑐𝑒𝑀𝑒𝑡ℎ𝑜𝑑 to obtain 𝑑_𝑘. The program was coded in 𝑀𝐴𝑇𝐿𝐴𝐵6.5.1. We stopped the iteration when the condition ‖𝑔(𝑥)‖≤10⁻⁶ was satisfied. If the iteration number is larger than one thousand, we also stop this program and this method is considered to be failed. For Algorithm 2.1, Tables 1(a) and 1(b) and Tables 2(a) and 2(b) show the performance of the method need to solve Problem 1 and Problem 2, respectively. For Algorithm 2.2, Tables 1(c) and 1(d) and Tables 2(c) and 2(d) show the performance of the normal trust region method need to solve Problem 1 and Problem 2, respectively. The columns of the tables have the following meaning:Dim: the dimension of the problem,NI: the total number of iterations,NG: the number of the function evaluations,EG: the norm of the function evaluations.

Table 1 

Test Results For Problem 1.

Table 2 

Test Results For Problem 2.

From Tables 1(a)–2(d), it is not difficult to see that the proposed method performs better than the normal method does. Furthermore, the performance of Algorithm 2.1 hardly changes with the dimension increasing. Overall, the given method is competitive to the normal trust region method.

6. Discussion

We give a trust-region-based BFGS method and establish its convergent results in this paper. The numerical results show that this method is promising. In fact, this problem (1.1) can come from unconstrained optimization problem and an equality constrained optimization problem (for details see [4]). There are some other practical problems, such as the saddle point problem, the discretized two-point boundary value problem, and the discretized elliptic boundary value problem, take the form of (1.1) with symmetric Jacobian (see, e.g., Chapter 1 in [50]). This presented method can also extend to solve the normal nonlinear equations.

Acknowledgments

The authrs are very grateful to anonymous referees and the editors for their valuable suggestions and comments, which improve their paper greatly. This work is supported by China NSF Grands 10761001 and the Scientific Research Foundation of Guangxi University (Grant no. X081082).