Apycom Software Home
MHCBench  

Evaluation of MHC Binding Peptide Prediction Algorithms

Home Evaluation Help Developers
About
Threshold Independent [TIE]
Threshold Dependent [TDE]
How to use
FAQs
Team
Contact

Datasets Results References Links

 

EVALUATION RESULTS

Below are the results of evaluations performed using MHCBench. The data sets used in evaluations are available on this server. The developers may download these data sets and can evaluate their algorithm.




Evaluation with Set1: All Non Redundant Peptides form data bases and literature

AlgorithmEVALUATION PARAMETERS
Threshold Dependent*Threshold IndependentFor Detailed Results
Category Reference SensSpecPPVNPVAccuCCROC
MotifRammennsee et al., 199569.1671.2183.7751.8869.810.37900.707[Details]
Max et al., 199355.9166.5678.2341.2759.290.20930.649[Details]
Sette et al., 199352.5973.3780.9341.8759.190.24330.650[Details]
Hammer et al., 199364.1258.2076.7243.0262.240.21000.638[Details]
Chicz et al., 199328.8282.6678.1235.0945.920.12320.542[Details]
MatrixMarshal et al., 199572.1972.1484.7754.6972.170.41830.787[Details]
Struniolo et al., 199973.0567.4982.8453.8371.290.38560.777[Details]
Hammer et al., 199469.4570.5983.5451.8269.810.37620.774[Details]
Brusic et al., 199864.2765.3379.9345.9764.600.27680.707[Details]
Southwood et al., 199866.7166.2580.9448.0966.570.30940.700[Details]
Borras-Cuesta et al., 200061.8260.3777.0242.3961.360.20750.664[Details]
ANNBrusic et al., 199870.7570.5983.7959.9070.700.38940.771[Details]

* The Parameters is calculated at threshold where sensitivity is nearly equal to specificity, only for matrices and ANN.

Evaluation with Set2: Natural Peptides selected form set1

AlgorithmEVALUATION PARAMETERS
Threshold Dependent*Threshold IndependentFor Detailed Results
Category Reference SensSpecPPVNPVAccuCCROC
MotifRammennsee et al., 199564.3071.9274.9260.6967.610.35420.704[Details]
Max et al., 199350.3967.1266.6750.9157.650.17550.619[Details]
Sette et al., 199351.9770.5569.7252.9660.030.22600.630[Details]
Hammer et al., 199363.2558.2266.3954.8461.070.21350.621[Details]
Chicz et al., 199338.8580.8272.5550.3257.060.21210.609[Details]
MatrixMarshal et al., 199567.9867.4773.1661.7667.760.35180.753[Details]
Struniolo et al., 199966.9367.1272.5560.8767.010.33780.738[Details]
Hammer et al., 199465.6265.7571.4359.4465.650.31120.743[Details]
Brusic et al., 199864.5764.0470.0958.0764.340.28380.695[Details]
Southwood et al., 199861.4261.9967.8355.1861.660.23210.651[Details]
Borras-Cuesta et al., 200058.2760.2765.6852.5459.140.18380.643[Details]
ANNBrusic et al., 199865.6265.7571.4349.4465.680.31120.685[Details]

* The Parameters is calculated at threshold where sensitivity is nearly equal to specificity, only for matrices and ANN.

Evaluation with Set3a: Non-homologous Peptides selected form set1

AlgorithmEVALUATION PARAMETERS
Threshold Dependent*Threshold IndependentFor Detailed Results
Category Reference SensSpecPPVNPVAccuCCROC
MotifRammennsee et al., 199560.8674.6484.5052.6065.930.34280.681[Details]
Max et al., 199349.8768.2072.9444.1856.610.17590.604[Details]
Sette et al., 199351.4770.5175.0045.8158.470.21390.632[Details]
Hammer et al., 199362.4758.5372.1447.5761.020.20340.605[Details]
Chicz et al., 199334.8582.9577.8442.5552.540.19060.566[Details]
MatrixMarshal et al., 199569.1769.1279.3856.6069.150.37120.753[Details]
Struniolo et al., 199969.1768.2078.9058.2768.810.36260.734[Details]
Hammer et al., 199467.2966.3677.4754.1466.950.32610.737[Details]
Brusic et al., 199864.0864.9875.8751.2764.410.28080.699[Details]
Southwood et al., 199864.3463.5975.2450.9264.070.27030.677[Details]
Borras-Cuesta et al., 200062.4763.5974.6849.6462.880.25180.685[Details]
ANNBrusic et al., 199864.6164.9876.0351.6564.750.28610.693[Details]

* The Parameters is calculated at threshold where sensitivity is nearly equal to specificity, only for matrices and ANN.

Evaluation with Set3b: Non-homologous Peptides selected form set2

AlgorithmEVALUATION PARAMETERS
Threshold Dependent*Threshold IndependentFor Detailed Results
Category Reference SensSpecPPVNPVAccuCCROC
MotifRammennsee et al., 199560.5773.1574.4558.9666.090.3356----1[Details]
Max et al., 199350.9068.5267.6251.9358.590.19480.628[Details]
Sette et al., 199353.4170.3769.9553.9060.810.23810.655[Details]
Hammer et al., 199368.8757.4166.3055.8661.620.22220.616[Details]
Chicz et al., 199339.7882.4174.5051.4558.380.23990.622[Details]
MatrixMarshal et al., 199568.4668.9874.0362.8768.690.37170.761[Details]
Struniolo et al., 199967.0367.5972.7661.3467.270.34360.751[Details]
Hammer et al., 199467.7465.7471.8661.2166.870.33280.751[Details]
Brusic et al., 199863.4464.3569.6957.6863.840.27580.711[Details]
Southwood et al., 199863.4463.4369.1457.3263.430.26660.684[Details]
Borras-Cuesta et al., 200060.5761.5767.0654.7361.010.21970.657[Details]
ANNBrusic et al., 199865.5965.7471.2159.6665.660.31100.713[Details]

* The Parameters is calculated at threshold where sensitivity is nearly equal to specificity, only for matrices and ANN.

Evaluation with Set4a: Randomly selected equal number of HLA-DRB1*0401 binders and non-binders form set1

AlgorithmEVALUATION PARAMETERS
Threshold Dependent*Threshold IndependentFor Detailed Results
Category Reference SensSpecPPVNPVAccuCCROC
MotifRammennsee et al., 199569.6671.2170.7570.1270.430.40870.711[Details]
Max et al., 199354.1866.5661.8459.2360.370.20900.637[Details]
Sette et al., 199347.3773.3764.0258.2360.370.21480.624[Details]
Hammer et al., 199364.4058.2060.6462.0561.300.22640.633[Details]
Chicz et al., 199328.4882.6662.1653.6155.570.13260.535[Details]
MatrixMarshal et al., 199571.2171.2171.2171.2171.210.42410.778[Details]
Struniolo et al., 199971.5267.4968.7570.3269.500.39040.764[Details]
Hammer et al., 199466.2568.4267.7266.9767.340.34680.760[Details]
Brusic et al., 199866.2565.9466.0566.1566.100.32200.711[Details]
Southwood et al., 199864.7164.7164.7164.7164.710.29410.680[Details]
Borras-Cuesta et al., 200058.8260.3759.7559.4559.600.19200.653[Details]
ANNBrusic et al., 199870.2869.9770.0670.1970.120.40250.724[Details]

* The Parameters is calculated at threshold where sensitivity is nearly equal to specificity, only for matrices and ANN.

Evaluation with Set4b: Randomly selected equal number of HLA-DRB1*0401 binders and non-binders form set2

AlgorithmEVALUATION PARAMETERS
Threshold Dependent*Threshold IndependentFor Detailed Results
Category Reference SensSpecPPVNPVAccuCCROC
MotifRammennsee et al., 199568.1571.9270.8269.3170.030.4010-----[Details]
Max et al., 199351.0367.1260.8257.8259.080.18390.619[Details]
Sette et al., 199352.4070.5564.0259.7161.470.23330.633[Details]
Hammer et al., 199362.3358.2259.8760.7160.270.20570.617[Details]
Chicz et al., 199338.7080.8266.8656.8759.760.21520.608[Details]
MatrixMarshal et al., 199568.1568.8468.6268.3768.490.36990.767[Details]
Struniolo et al., 199967.4767.8167.7067.5867.640.35270.751[Details]
Hammer et al., 199466.7865.7866.1066.4466.270.32540.751[Details]
Brusic et al., 199863.7064.0463.9263.8263.870.27740.692[Details]
Southwood et al., 199863.3663.7063.5763.4863.500.27050.661[Details]
Borras-Cuesta et al., 200059.5957.5358.3958.7458.560.17130.649[Details]
ANNBrusic et al., 199864.0464.3864.2664.1664.210.28420.672[Details]

* The Parameters is calculated at threshold where sensitivity is nearly equal to specificity, only for matrices and ANN.

Evaluation with Set5a: Recent Ligands selected accroding to set1 criteion

AlgorithmEVALUATION PARAMETERS
Threshold Dependent*Threshold IndependentFor Detailed Results
Category Reference SensSpecPPVNPVAccuCCROC
MotifRammennsee et al., 199555.7161.7068.4248.3358.120.17080.601[Details]
Max et al., 199347.1463.8366.0044.7853.850.10870.607[Details]
Sette et al., 199321.4378.7260.0040.2244.440.00180.551[Details]
Hammer et al., 199358.5751.0664.0645.2855.560.09490.537[Details]
Chicz et al., 199314.2991.4971.4341.7545.300.08720.466[Details]
MatrixMarshal et al., 199565.7159.6770.7753.8563.250.24950.662[Details]
Struniolo et al., 199967.1455.3269.1253.0662.390.22320.670[Details]
Hammer et al., 199461.4346.8163.2444.9055.560.08190.658[Details]
Brusic et al., 199860.0061.7070.0050.8860.680.21290.603[Details]
Southwood et al., 199860.0053.1965.6247.1757.260.12990.585[Details]
Borras-Cuesta et al., 200042.8648.9455.5636.5145.300.00000.421[Details]
ANNBrusic et al., 199861.4372.3476.7955.7465.810.33140.695[Details]

* The Parameters is calculated at threshold where sensitivity is nearly equal to specificity, only for matrices and ANN.

Evaluation with Set5b: Recent Ligands selected accroding to set2 criteion

AlgorithmEVALUATION PARAMETERS
Threshold Dependent*Threshold IndependentFor Detailed Results
Category Reference SensSpecPPVNPVAccuCCROC
MotifRammennsee et al., 199566.6740.5459.2648.3955.290.0742-*---[Details]
Max et al., 199325.0081.0863.1645.4549.410.07240.621[Details]
Sette et al., 199331.2572.9760.0045.0049.410.04600.568[Details]
Hammer et al., 199341.6764.8660.6146.1551.760.06640.512[Details]
Chicz et al., 199320.8389.1971.4346.4850.590.13400.533[Details]
MatrixMarshal et al., 199562.5072.9775.0060.5067.060.35240.706[Details]
Struniolo et al., 199960.4259.4665.9153.6660.000.19720.677[Details]
Hammer et al., 199450.0056.7660.0046.6752.940.06710.619[Details]
Brusic et al., 199860.4256.7664.4452.5058.820.17060.626[Details]
Southwood et al., 199850.0059.4661.5447.8354.120.09410.691[Details]
Borras-Cuesta et al., 200056.2540.5455.1041.6749.410.00000.439[Details]
ANNBrusic et al., 199860.4267.5770.7356.8263.530.27770.714[Details]

* The Parameters is calculated at threshold where sensitivity is nearly equal to specificity, only for matrices and ANN.

 

Home TIE TDE How to use FAQs Datasets Results References Links Contact