The Need for Guidelines in Publication of Peptide and Protein Identification Data

By Steven A. Carr, R. Aebersold, M. Baldwin, Alma Burlingame, Karl R. Clauser, Alexey I. Nesvizhskii. Molecular & Cellular Proteomics, 2004

476 citations recorded by the source corpus.

Over the past few years, the number and size of proteomic datasets composed of mass spectrometry-derived protein identifications reported in the literature have grown dramatically. This is a direct result of the widespread availability of instruments, methods, and easy-to-use software for collecting large amounts of data and for converting the observed peptide and fragment-ion masses to peptide and then protein identities. In particular, the analysis of samples containing large numbers of proteins by multidimensional liquid chromatography (LC/LC) coupled on-line with tandem mass spectrometry (MS/MS) is now a common component of many biological projects. Clearly it is in the interest of the scientific community to make such data readily available. However, the publication of large proteomic datasets poses new and significant challenges for authors, reviewers, and readers as universally accepted and widely available computational tools for validation of the published results are not yet available (1). In an effort to ensure that high-quality, significant data are entering the proteomics literature, Molecular & Cellular Proteomics (MCP) is introducing guidelines for authors planning to submit manuscripts containing large numbers of proteins identified primarily by LC-MS/MS. The need for these guidelines is driven in part by the fact that a significant but undefined number of the proteins being reported as “identified” in proteomics articles are likely to be false positives (2). These incorrect matches probably result most often from the use of low-quality peptide MS/MS data to search the database. However, even high-quality data can produce invalid identifications if, for example, the actual peptide sequence is not in the database being searched. Many different algorithms are being used for peptide and protein assignment (e.g. MSTag, Mascot, SEQUEST, SpectrumMill, Sonar, etc.), and each has unique rules for scoring to move the most probable peptide assignment to the top of the “hit” list. In addition, new filtering criteria are being developed that, when layered onto the results from the above algorithms, help to eliminate a certain additional percentage of false positives (3, 4). It is very important that the users of these tools, our authors, have at least a working understanding of how the algorithm they use works. However, even the judicious use of scoring, threshold parameters, and additional filtering criteria for search engines, while serving the very important purpose of reducing the number of misassigned peptides and proteins, does not eliminate the problem. It is almost always possible to match a MS/MS spectrum to a peptide in the database; the difficult part is validating that the match is correct. This is not to imply that the situation is bleak. In fact, most assignments of proteins made with high-quality data and using more than a single peptide to identify a protein are likely correct. Furthermore, improved methods are being developed at a rapid pace. Recently, application of statistical methods to validate peptide assignments to MS/MS spectra of peptides has been shown to be a promising approach, and a number of groups are working in this area (2, 5–11). However, these programs are only beginning to be widely used and they are not universally accepted. MCP fully supports continued development and testing of such programs and will publish new search and filtering approaches to make them widely available to the proteomics community. However, in the absence of accepted standards and widely available tools that operate on such standards, there are guidelines that the journal can formulate that would help ensure the publication of highquality information and to assist readers in being able to make their own assessment of the validity of the assignments in manuscripts. Thus, we introduce the initial set of such guidelines in this issue of the journal. The rationale and purpose of each of these is described below. The first (and obvious) guideline is to obtain sufficient information from authors to document what search engine was used and how peptide and protein assignments were made using that software. Guideline 2 defines how peptides should be counted toward the identification of a protein. We do not at this time attempt to deal with the related issue of what constitutes a “unique” peptide with respect to the proteins identified. For From the ‡Broad Institute, Cambridge, MA 02139; ¶Institute for Systems Biology, Seattle, WA 98103; University of California, San Francisco, CA 94143; and **Millennium Pharmaceuticals, Cambridge, MA 02139 Received, April 7, 2004 Published, MCP Papers in Press, April 8, 2004, DOI 10.1074/mcp.T400006-MCP200 1 The abbreviations used are: LC/LC, multidimensional liquid chromatography; MS/MS, tandem mass spectrometry; MCP, Molecular & Cellular Proteomics. Editorial

Subfields: Molecular Biology, Spectroscopy

View Paper: https://www.semanticscholar.org/paper/5916ac593420f111c4b30ac13cae9610518ebe1a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Spectroscopy

Citations in the corpus

Protein biomarker discovery and validation: the long and uncertain path to clinical utility (2006, background): “The pitfalls of inappropriate or misguided use of database search algorithms have been discussed, and guidelines are being develope”
A HUPO test sample study reveals common problems in mass spectrometry-based proteomics (2009, unknown): “Test samples were distributed to participating laboratories, who were instructed to (i) identify the 20 human proteins, (ii) report the details of the identifications (protein name, NCBI gi number, sequence coverage, number of peptides and number of tandem MS spectra) following the criteria of ref”
Proteogenomics: concepts, applications, and computational strategies (2014, background): “In the early days, many data sets with very high FDR were published, prompting calls to establish robust data analysis and publication guidelines 93 .”
Fast and Accurate Protein False Discovery Rates on Large-Scale Proteomics Data Sets with Percolator 3.0 (2016, methodology): “We therefore compared four straightforward and scalable protein inference methods: using the best-scoring peptide, the two-peptide rule [8,9], the product of peptide-level posterior error probabilities (PEPs), and Fisher ’ s method for combining independent p values.”
Spectral Probabilities and Generating Functions of Tandem Mass Spectra: a Strike Against Decoy Databases (2008, background): “Indeed, the Proteomics Publication Guidelines [3,4] recommend searching in decoy databases to determine the statistical significance of peptide identifications (this is in contrast to genomics searches that do not employ decoy databases).”
A survey of computational methods and error rate estimation procedures for peptide and protein identification in shotgun proteomics (2010, background): “The high rate of false positives in early proteomic publications was so alarming to the scientific community that it lead to the establishment of specific data analysis guidelines by the Editorial Boards of the leading proteomic journals [15].”
A practical recipe for stable isotope labeling by amino acids in cell culture (SILAC) (2006, methodology): “Validate peptide and protein identification using standard proteomics guideline”
Interpretation of Shotgun Proteomic Data (2005, background): “The problem is further complicated when no statistical analysis is performed to determine the validity of peptide and protein identifications (10, 29).”
The minimum information about a proteomics experiment (MIAPE) (2007, background): “Other journals, such as Molecular and Cellular Proteomics”
Immunoaffinity profiling of tyrosine phosphorylation in cancer cells (2005, background): “In proteomics, protein identification is facilitated by the availability of guidelines for the publication of protein and peptide identification result”
iProphet: Multi-level Integrative Analysis of Shotgun Proteomic Data Improves Peptide and Protein Identification Rates and Error Estimates* (2011, methodology): “Without a uniformly applied confidence measure it was difficult and unreliable to combine and compare data sets (4, 5).”