Prediction Performance Of Current Methods

Up to this point, we overviewed useful features for predicting PPI sites and how existing methods combine those features. Next, to understand the current status of PPI methods, we will discuss the prediciton accuracy. We will review two recent benchmark studies, because they assessed the performance of most of the methods discussed in the previous section (Zhou, Qin 2007;Huang, Schroeder 2008).The prediction performance is commonly measured in terms of the sensitivity (the proportion of corectly predicted residues at PPI sites relative to the acutal residues at PPI sites) and the specificity (the proportion of the corectly predicted PPI site residues relative to the total number of predicted PPI site residues). Since the sensitivity and the specificty is tradeoff, usually the specificity is plotted relative to the sensitivity.

Zhou and Qin prepared two benchmark datasets; one dataset consists of 35 proteins in the enzyme and inhibitor category from ZDOCK protein-protein docking benchmark dataset (Mintseris et al. 2005) and another one consists of 25 docking prediction targets in CAPRI (Critical Assessment of Prediction of Interactions) prediction contest (http://www.ebi.ac.uk/msd-srv/capri/). The CAPRI dataset contains eight proteins which are not in the enzyme/inhibitor category, i.e. antibody/antigen or other immune system complexes. Six webservers in Table 1, namely, cons-PPISP, Promate, PINUP, PPI-Pred, SPPIDER, and Meta-PPISP are benchmarked. On the 35 enzyme dataset, PPI-Pred, SPIDDER, cons-PPISP, Promate, PINUP, and meta-PPISP showed the specificity of 27, 33, 36, 36, 48, and 50%, respectively, at the sensitivity of 50%. These methods peformed uniformly worse on another dataset of 25 CAPRI targets, probably because proteins in the CAPRI dataset is more diverse. At the sensitivity of 30%, the methods showed the specificity of 23, 25, 26, 26, 28, and 31%, respectively (the methods are placed in the same order). At the sensitivity of 50%, the specificity of the methods ranges roughly between 15 to 28%. Note that the purpose of showing the actual numbers is to have a general idea of the performace of currently avaialble methods but not to rank webservers. Acutally a fair comparison of existing methods is difficult, since some of the tested proteins may be used in tuning settings of the methods. What we can conclude from this benchmark study is that current prediction methods show the specificity of around 20 to 50% at the sensitivity of 50% and that a better performance can be achieved by taking a meta-server approach. A comparison of the benchmark performances (measured by sensitivity verses specificity curves) for each PPI site prediction method can be found in the review by Zhou and Qin.

In attempt to formulate an alternative meta-serer solution, Huang and Schroeder developed MetaPPI which combines five individual PPI site prediction methods, namely, PPI-Pred, SPIDDER, Promate, PINUP, and PPISP. They reported success rates for these five methods in comparison to their MetaPPI method on 62 protein-protein complexes. The protein complexes are calssified into two datasets, one consists of 20 complexes classified as enzyme-inhibitors and another one consists of the remaining 42 complexes that were classified as other types complexes. In their work, the success rate for a dataset is defined as the percentage of protein complexes whose PPI sites are predicted with a sensitivity of over 50% (note that this is different from the average specificity at the sensitivity of 50% , which are reported in the review by Zhou and Qin).

For the dataset of enzyme-inhibitors, the success rate of PPI-Pred, SPIDDER, Promate, PINUP, PPISP, and MetaPPI were 45, 23, 36, 52, 55, and 70%, respectively. While for the category of other types of complexes, the perfomance of the methods were performed with 28, 10, 13, 15, 25, and 44%, respectively. Consistent to Zhou and Qin's benchmark, enzyme-ihibitor complexes can be generally predicted with greater success than other types of complexes.

The performance shown here might not seem very good. However, encouragingly, Zhou and Qin's work showed that a PPI prediction with 40-45% of the sensitivity and the specificity can improve ranking of near-native docking decoys compared to the original scores of the ZDOCK docking program (Chen et al. 2003), thus useful for protein docking prediction.

Was this article helpful?

0 0
How To Bolster Your Immune System

How To Bolster Your Immune System

All Natural Immune Boosters Proven To Fight Infection, Disease And More. Discover A Natural, Safe Effective Way To Boost Your Immune System Using Ingredients From Your Kitchen Cupboard. The only common sense, no holds barred guide to hit the market today no gimmicks, no pills, just old fashioned common sense remedies to cure colds, influenza, viral infections and more.

Get My Free Audio Book


Post a comment