An Efficient Ensemble Learning Method for Gene Microarray Classification

Pseudocode 1

The RotBoost pseudocode.

Input

 (i) 𝐿: a training set, 𝐿={(𝑥_𝑖,𝑦_𝑖)}^𝑁_𝑖=1=𝑋𝑌 where 𝑋 is an 𝑁×𝑛 matrix containing the input values and 𝑌 is an

  N-dimensional column vector containing the class labels.

 (ii) 𝐾: number of attribute subsets (or 𝑀: number of input attributes contained in each subset).

 (iii) 𝐶: a base learning.

 (iv) 𝑆: number of iterations for Rotation Forest.

 (v) 𝑇: number of iterations for AdaBoost.

 (vi) 𝑥: a data point to be classified.

Training Phase

 For 𝑠=1,2,…,𝑆

  (1) use the steps similar to those in Rotation Forest to compute the Rotation matrix, say, 𝑅^𝑎_𝑠 and let 𝐿^𝑎=[𝑋𝑅^𝑎_𝑡𝑌] be the

   training set for classifier 𝐶_𝑠.

  (2) Initialize the weight distribution over 𝐿^𝑎 as 𝐷₁(𝑖)=1/𝑁(𝑖=1,2,…,𝑁).

  (3) For 𝑡=1,…,𝑇

   (a) According to distribution 𝐷_𝑡 perform N extractions randomly from 𝐿^𝑎 with replacement to compose a new set 𝐿^𝑎_𝑡.

   (b) Apply 𝑤 to 𝐿^𝑎_𝑡 to train a classifier 𝐶^𝑎_𝑡 and then compute the error of 𝐶^𝑎_𝑡 as

          𝜀_𝑡=Pr_{𝑖∼𝐷𝑡}𝐶^𝑎_𝑡𝑥_𝑖≠𝑦_𝑖=∑^𝑁_𝑖=1𝐼(𝐶^𝑎_𝑡(𝑥_𝑖)≠ 𝑦_𝑖))𝐷_𝑡(𝑖)

   (c) If 𝜀_𝑡>0.5 then set 𝐷_𝑡(𝑖)=1/𝑁(𝑖=1,2,…,𝑁). and go to step (a); if 𝜀_𝑡=0, then set 𝜀_𝑡=10⁻¹⁰ to continue

   the following iterations.

   (d) Choose 𝛼_𝑡=1/2ln1−𝜀_𝑡/𝜀_𝑡

   (e) Update the distribution 𝐷_𝑡 over 𝐿^𝑎 as:

          𝐷_𝑡+1𝐷(𝑖)=^𝑡(𝑖)𝑍_𝑡×𝑒^−𝛼𝑡,if𝐶^𝑎_𝑡𝑥_𝑖=𝑦_𝑖,𝑒^𝛼𝑡,if𝐶^𝑎_𝑡𝑥_𝑖≠𝑦_𝑖,

    where 𝑍_𝑡 is a normalization factor being chosen so that 𝐷_𝑡+1 is a probability distribution over 𝐿^𝑎.

    Endfor

  (4) Let 𝐶_𝑠(𝑥)=argmax_𝑦∈𝜙 ∑^𝑇_𝑡=1𝛼_𝑡𝐼(𝐶^𝑎_𝑡(𝑥)=𝑦).

 Endfor

Output

   (i) The class label for x predicted by the final ensemble 𝐶^∗ as

          𝐶^∗(𝑥)=argmax^𝑆_𝑦∈𝜙∑_𝑠=1𝐼𝐶_𝑠(𝑥)=𝑦,

    where I(·) is an indicator function.