Towards a Big Data Curated Benchmark of Inter-project Code Clones

By Jeffrey Svajlenko, Judith F. Islam, Iman Keivanloo, Chanchal Kumar Roy, Mohammad Mamun Mia. 2014

391 citations recorded by the source corpus.

Subfields: Information Systems, Software, Evaluation & Benchmarks, Code Generation & Program Synthesis

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

1. Yanqiao Zhu
2. Yichen Xu
3. Nan Duan
4. Daya Guo
5. Yue Wang
6. Ge Li
7. Steven C. H. Hoi
8. Miltiadis Allamanis

In Software

1. Yue Wang
2. Steven C. H. Hoi
3. Neel Sundaresan
4. Daya Guo
5. Nan Duan
6. Alexey Svyatkovskiy
7. Jian Yin
8. Yanqiao Zhu

In Evaluation & Benchmarks

1. Hongyu Zhang
2. Ge Li
3. M. Zhou
4. Akhilesh Deepak Gotmare
5. Xin Xia
6. Jian Yin
7. Neel Sundaresan
8. Hailong Sun

In Code Generation & Program Synthesis

1. Ge Li
2. Hongyu Zhang
3. Charles Sutton
4. Zhi Jin
5. Hailong Sun
6. Xu Wang
7. Xudong Liu
8. Jian Zhang

Citations in the corpus

• Program Synthesis with Large Language Models (2021, methodology): “…have been proposed for software engineering tasks including code completion [Raychev et al., 2016, Allamanis and Sutton, 2013b], clone detection [Svajlenko et al., 2014], code search [Husain et al., 2019], predicting readable names to describe functions [Allamanis et al., 2016], and generating…”
• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021, methodology): “CodeXGLUE includes eight previously proposed datasets — BigCloneBench [65], POJ-104 [46], Devign [91], PY150 [57], Github Java Corpus [1], Bugs2Fix [69], CONCODE [34], and CodeSearchNet [32]— but also newly introduced datasets that are marked in Table 1.”
• GraphCodeBERT: Pre-training Code Representations with Data Flow (2020, methodology): “We conduct experiments on the BigCloneBench dataset (Svajlenko et al., 2014) and report results in the Table 2.”
• NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness (2024, methodology): “Prior works have considered code-comprehension tasks like Clone-detection (Svajlenko et al., 2014; Lu et al., 2021), Defect-detection (Zhou et al., 2019; Li et al., 2021; Lu et al., 2021), Code-explaination (Muennighoff et al., 2023; Leinonen et al., 2023), and Question Answering over Code (Huang…”
• CodeT5+: Open Code Large Language Models for Code Understanding and Generation (2023, methodology): “The benchmark is curated from the BigClone Benchmark dataset [Svajlenko et al., 2014] For finetuning, we set the learning rate as 2e-5 and finetune the model for 2 epochs.”
• Detecting Code Clones with Graph Neural Network and Flow-Augmented Abstract Syntax Tree (2020, methodology): “In this paper, we build FA-AST for Java programs and evaluate FA-AST and graph neural networks on two code clone datasets: Google Code Jam dataset collected by [6] and the widely used clone detection benchmark BigCloneBench [9].”
• A Novel Neural Source Code Representation Based on Abstract Syntax Tree (2019, methodology): “For BCB, the similarity of clone pairs is defined as the average result of line-based and token-based metrics [47].”
• Neural Detection of Semantic Code Clones Via Tree-Based Convolution (2019, methodology): “Figure 1 shows an example of a Type-4 clone from BigCloneBench [8].”
• Contrastive Code Representation Learning (2020, neutral): “Benchmarks exist like BigCloneBench (Sva-jlenko et al., 2014), but to the best of our knowledge, there is no benchmark for the JavaScript.”
• A Survey on Pretrained Language Models for Neural Code Intelligence (2022, background): “Clone detection (Svajlenko et al., 2014) involves identifying plagiarism between two code snippets by measuring their semantic similarity.”
• UniXcoder: Unified Cross-Modal Pre-training for Code Representation (2022, methodology): “We conduct experiments on POJ-104 (Mou et al., 2016) and Big-CloneBench (Svajlenko et al., 2014) datasets.”
• DeepSim: deep learning code functional similarity (2018, methodology): “The total number of clone pairs obtained from the downloaded database is slightly different from that reported in [53].”
• SourcererCC: Scaling Code Clone Detection to Big-Code (2015, methodology): “So we developed an efficient clone validation strategy based on code functionality and built Big-CloneBench [30], a Big Data clone benchmark containing 8 million validated clones within and between 25,000 open-source projects.”
• Recall Before Rerank: Benchmarking Deep Learning Models for Large-Scale Code-to-Code Retrieval (2026, unknown): “Our primary research questions are as follows: RQ For RQ 2 , we show in Section 6 that embedding rate is scale-critical (e.g., Big-CloneBench’s 6M methods [32]), making IR methods or lossy embedding compression [19,25][7, §13.7] more practical for large corpora.”