Large-scale validation of methods for cytotoxic T-lymphocyte epitope prediction

By M. Larsen, Claus Lundegaard, Kasper Lamberth, Søren Buus, Ole Lund, Morten Nielsen. BMC Bioinformatics, 2007

925 citations recorded by the source corpus.

BackgroundReliable predictions of Cytotoxic T lymphocyte (CTL) epitopes are essential for rational vaccine design. Most importantly, they can minimize the experimental effort needed to identify epitopes. NetCTL is a web-based tool designed for predicting human CTL epitopes in any given protein. It does so by integrating predictions of proteasomal cleavage, TAP transport efficiency, and MHC class I affinity. At least four other methods have been developed recently that likewise attempt to predict CTL epitopes: EpiJen, MAPPP, MHC-pathway, and WAPP. In order to compare the performance of prediction methods, objective benchmarks and standardized performance measures are needed. Here, we develop such large-scale benchmark and corresponding performance measures and report the performance of an updated version 1.2 of NetCTL in comparison with the four other methods.ResultsWe define a number of performance measures that can handle the different types of output data from the five methods. We use two evaluation datasets consisting of known HIV CTL epitopes and their source proteins. The source proteins are split into all possible 9 mers and except for annotated epitopes; all other 9 mers are considered non-epitopes. In the RANK measure, we compare two methods at a time and count how often each of the methods rank the epitope highest. In another measure, we find the specificity of the methods at three predefined sensitivity values. Lastly, for each method, we calculate the percentage of known epitopes that rank within the 5% peptides with the highest predicted score.ConclusionNetCTL-1.2 is demonstrated to have a higher predictive performance than EpiJen, MAPPP, MHC-pathway, and WAPP on all performance measures. The higher performance of NetCTL-1.2 as compared to EpiJen and MHC-pathway is, however, not statistically significant on all measures. In the large-scale benchmark calculation consisting of 216 known HIV epitopes covering all 12 recognized HLA supertypes, the NetCTL-1.2 method was shown to have a sensitivity among the 5% top-scoring peptides above 0.72. On this dataset, the best of the other methods achieved a sensitivity of 0.64. The NetCTL-1.2 method is available at http://www.cbs.dtu.dk/services/NetCTL.All used datasets are available at http://www.cbs.dtu.dk/suppl/immunology/CTL-1.2.php.

Subfields: Immunology, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/b058b434c016147e99e9070da369a1f7875f9262

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Immunology

1. Bjoern Peters
2. Morten Nielsen
3. Alessandro Sette
4. Ole Lund
5. J. Greenbaum
6. Søren Buus
7. Peng Wang
8. Yohan Kim

In Molecular Biology

1. Morten Nielsen
2. Bjoern Peters
3. Alessandro Sette
4. Ole Lund
5. J. Greenbaum
6. Claus Lundegaard
7. Massimo Andreatta
8. Peng Wang

Citations in the corpus

• An overview of bioinformatics tools for epitope prediction: Implications on vaccine development (2015, background): “Predictions of the MHC class I pathway can be improved by predictions of proteasomal cleavage, TAP transport efficiency, and MHC class I binding affinity [48,49].”
• Influenza Research Database: An integrated bioinformatics resource for influenza research and surveillance (2012, methodology): “Novel data are generated and continually updated using both published and custom bioinformatics algorithms including predicted cytotoxic T-cell epitopes using NetCTL, 33 sequence variation statistics using custom entropy-based algorithms, predicted protein domains and motifs using InterProScan, 34 a”
• Properties of MHC Class I Presented Peptides That Enhance Immunogenicity (2013, methodology): “2 [10], the best performing predictor according to a large benchmark study [8].”
• ViPR: an open bioinformatics database and analysis resource for virology research (2011, methodology): “Selecting a gene or protein from the image or table loads the Gene/Protein Details page (Supplementary Figure S1), which contains derived data consisting of predicted immune epitopes determined using the NetCTL algorithm (14), protein domains and motifs predicted using InterProScan (15), molecular w”
• Immune epitope database analysis resource (IEDB-AR) (2008, methodology): “In the new release, alternative approaches were implemented: NetChop (9) uses novel sequence encoding methods to predict proteasomal cleavage, and NetCTL (10,11) combines these with predictions of MHC binding and TAP transport.”
• Exploring dengue genome to construct a multi-epitope based subunit vaccine by utilizing immunoinformatics approach to battle against dengue infection (2017, background): “CTL epitope prediction was restricted by 12 MHC class-I super types 29 .”