Ranking Biomedical Annotations with Annotator’s Semantic Relevancy

Algorithm 1

Frequent pattern.

input: ℜ={(𝑜_𝑖,𝑢_𝑖,cr)(𝑖=1,…,𝑛,𝑗=1,…,𝑚)},(𝑜_𝑖,𝑢_𝑖,cr) means that user 𝑢_𝑖 annotated biomedical entity 𝑜_𝑖 with correctness

     rate cr(cr>0.6);𝑜_𝑗 also refer to its RDF graphs;

    𝜀, predefined threshold of frequency;

    𝜆, group number of correctness rates defined by user

output: Ω, a set of frequent patterns.

(1) classify ℜ into different sets of ℜ₁={(𝑜_𝑗,𝑢,cr)}, in each set, annotations are all submitted by user 𝑢;

(2) for each ℜ₁

 (2.1) 𝑘=𝜆;ℜ^′₁=ℜ₁;

 (2.2) cluster elements in ℜ^′₁ into a set of groups ∑={ℜ₁₁^cr1,ℜ₁₂^cr2,…,ℜ_1𝑘^cr𝑘} according to cr with 𝑘-mean,

     and cluster center is the correctness of the group, for example, cr1 is correctness of ℜ₁₁;

 (2.3) for each ℜ_1𝑖(1≤𝑖≤𝑘)   //find frequent patterns for given annotator with given correctness

  (2.3.1) 𝛼^′′=𝛼^′={Pattern Path 𝑝𝑝^𝑓∣𝑝𝑝 belong to an entity 𝑜∈ℜ_1𝑖 and 𝑓=|𝑝𝑝|/(|𝑃|+1)>𝜀}, here, |𝑝𝑝| is count of 𝑝𝑝

     in ℜ_1𝑖 and |𝑝| is count of all Pattern Paths in ℜ)  // set of frequent pattern paths

  (2.3.2) for ∀𝑝𝑝_𝑖 ∈𝛼^′(1≤𝑖≤|𝛼^′|), 𝑝𝑝_𝑖 can be a Pattern Path or a sub RDF graph. //find frequent conjugate items

     { for ∀𝑝𝑝_𝑗∈𝛼^′′(1≤𝑗≤|𝛼^′′|)

      {  If 𝑝𝑝_𝑖 and 𝑝𝑝_𝑗 are conjugate and 𝑝𝑝_𝑗⊄𝑝𝑝_𝑖∧𝑝𝑝_𝑗⊄𝑝𝑝_𝑖 and 𝑓=|𝑝𝑝_𝑖∧𝑝𝑝_𝑗|/(|𝑃|+1)>𝜀(|𝑝𝑝_𝑖∧𝑝𝑝_𝑗| is the

       conjunct appearance of 𝑝𝑝_𝑖 and 𝑝𝑝_𝑗 in Δ^′). Then

        {  merge 𝑝𝑝_𝑖 and 𝑝𝑝_𝑗 into a sub RDF graph 𝑔, and 𝑓 is the frequency of 𝑔;𝛼^′=𝑎^′∪𝑔;}}

      If exists one graph 𝑔∈𝛼^′ including 𝑝𝑝_𝑖, then remove 𝑝𝑝_𝑖 from 𝛼^′;}

  (2.3.3) Repeat Step (2.3.2) untill 𝛼^′ doesn’t change;

  (2.3.4) 𝑃^cr𝑖_𝑢=𝛼^′; 𝜔_1𝑖={𝑜∣𝑜∈ℜ_1𝑖∧¬∃𝑝(𝑝∈𝑎^′∧𝑝 matches a RDF path of 𝑜)}

 (2.4) 𝑝_𝑢=𝑝_𝑢∪𝑝^cr1_𝑢∪⋯∪𝑝^cr𝑘_𝑢; 𝜔=𝜔₁₁∪𝜔₁₂∪⋯∪𝜔_1𝑘;

 (2.5) For any two pattern 𝑔∈𝑃^cr𝑖_𝑢,𝑔^′∈𝑃^cr𝑗_𝑢(𝑖≠𝑗), If (𝑔=𝑔^′), then  //merge same pattern with different cr

   { remove 𝑔,𝑔^′ from 𝑝_𝑢;

    𝑝_𝑢=𝑝_𝑢∪𝑔^cr𝑔_𝑢(cr𝑔=(𝑛1∗cr𝑔+𝑛2∗cr𝑗)/(𝑛1+𝑛2);𝑛1 is number of entities matching g in ℜ_1𝑖; 𝑛2 is number of

      entities matching 𝑔^′ in ℜ_1𝑗) }

  (2.6) Ω=Ω∪𝑝_𝑢;

  (2.7) if (𝑘>1 and 𝜔≠𝜙) {ℜ^′₁={(𝑜,𝑢,cr)∣𝑜∈𝜔};𝑘=𝜆⧵2; go to (2.2);}

(3) circularly merge frequent patterns in Ω with Rule 1 and Rule 2 presented in this section until Ω doesn’t change;

(4) return Ω;