Directory of MSR Datasets

GitHub OSS Governance File Dataset

• Download metadata
• Download FAIR analysis
• Download annotations

Yibo Yan, Seth Frey, Amy Zhang, V. Filkov, Likang Yin

DOI: 10.1109/MSR59073.2023.00089

Publication year: 2023

Topic 12 - Terms: project, source, code, development, clone, set, bug, github, os, datasets

Abstract: Open-source Software (OSS) has become a valuable resource in both industry and academia over the last few decades. Despite the innovative structures they develop to support the projects, OSS projec... Abstract: Open-source Software (OSS) has become a valuable resource in both industry and academia over the last few decades. Despite the innovative structures they develop to support the projects, OSS projects and their communities have complex needs and face risks such as getting abandoned. To manage the internal social dynamics and community evolution, OSS developer communities have started relying on written governance documents that assign roles and responsibilities to different community actors.To facilitate the study of the impact and effectiveness of formal governance documents on OSS projects and communities, we present a longitudinal dataset of 710 GitHub-hosted OSS projects with GOVERNANCE.MD governance files. This dataset includes all commits made to the repository, all issues and comments created on GitHub, and all revisions made to the governance file. We hope its availability will foster more research interest in studying how OSS communities govern their projects and the impact of governance files on communities. Read more

Defectors: A Large, Diverse Python Dataset for Defect Prediction

• Download metadata
• Download FAIR analysis
• Download annotations

Parvez Mahbub, Ohiduzzaman Shuvo, M. M. Rahman

DOI: 10.1109/MSR59073.2023.00085

Publication year: 2023

Topic 1 - Terms: project, code, source, developer, study, api, model, system, technique, type

Abstract: Defect prediction has been a popular research topic where machine learning (ML) and deep learning (DL) have found numerous applications. However, these ML/DL-based defect prediction models are ofte... Abstract: Defect prediction has been a popular research topic where machine learning (ML) and deep learning (DL) have found numerous applications. However, these ML/DL-based defect prediction models are often limited by the quality and size of their datasets. In this paper, we present Defectors, a large dataset for just-in-time and line-level defect prediction. Defectors consists of ≈ 213K source code files (≈ 93K defective and ≈ 120K defect- free) that span across 24 popular Python projects. These projects come from 18 different domains, including machine learning, automation, and internet-of-things. Such a scale and diversity make Defectors a suitable dataset for training ML/DL models, especially transformer models that require large and diverse datasets. We also foresee several application areas of our dataset including defect prediction and defect explanation. Read more

LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluations

• Download metadata
• Download FAIR analysis
• Download annotations

Catherine Tony, Markus Mutas, Nicolás E. Díaz Ferreyra, R. Scandariato

DOI: 10.1109/MSR59073.2023.00084

Publication year: 2023

Topic 3 - Terms: code, review, project, source, present, change, set, github, repository, quality

Abstract: Large Language Models (LLMs) like Codex are powerful tools for performing code completion and code generation tasks as they are trained on billions of lines of code from publicly available sources.... Abstract: Large Language Models (LLMs) like Codex are powerful tools for performing code completion and code generation tasks as they are trained on billions of lines of code from publicly available sources. Moreover, these models are capable of generating code snippets from Natural Language (NL) descriptions by learning languages and programming practices from public GitHub repositories. Although LLMs promise an effortless NL-driven deployment of software applications, the security of the code they generate has not been extensively investigated nor documented. In this work, we present LLMSecEval, a dataset containing 150 NL prompts that can be leveraged for assessing the security performance of such models. Such prompts are NL descriptions of code snippets prone to various security vulnerabilities listed in MITRE’s Top 25 Common Weakness Enumeration (CWE) ranking. Each prompt in our dataset comes with a secure implementation example to facilitate comparative evaluations against code produced by LLMs. As a practical application, we show how LLMSecEval can be used for evaluating the security of snippets automatically generated from NL descriptions. Read more

Snapshot Testing Dataset

• Download metadata
• Download FAIR analysis
• Download annotations

Emily Bui, H. Rocha

DOI: 10.1109/MSR59073.2023.00081

Publication year: 2023

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Snapshot testing is a form of software testing that is focused on visual components by highlighting any code changes when compared to a previously stored state. This quick and simple method of test... Abstract: Snapshot testing is a form of software testing that is focused on visual components by highlighting any code changes when compared to a previously stored state. This quick and simple method of testing is growing popular among the industry with companies such as Spotify and Robinhood. Despite its growing popularity, snapshot testing is barely explored in academia. In this paper, we use GitHub API to collect a dataset of 686 repositories tagged with Jest, a popular testing framework capable of snapshot testing. From those repositories, we found 4,604 snapshot files and 11,367 test files. The top-10 repositories represent 20% of all snapshot files in the dataset, even though it is only 3% of the size. We acknowledge that improvements can be made in the dataset but due to the lack of data on snapshot testing, we believe the current dataset is useful in helping researchers to study this topic. Read more

PyMigBench: A Benchmark for Python Library Migration

• Download metadata
• Download FAIR analysis
• Download annotations

Mohayeminul Islam, Ajay Kumar Jha, Sarah Nadi, Ildar Akhmetov

DOI: 10.1109/MSR59073.2023.00075

Publication year: 2023

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: Developers heavily rely on Application Programming Interfaces (APIs) from libraries to build their projects. However, libraries might become obsolete, or new libraries with better APIs might become... Abstract: Developers heavily rely on Application Programming Interfaces (APIs) from libraries to build their projects. However, libraries might become obsolete, or new libraries with better APIs might become available. In such cases, developers replace the used libraries with alternative libraries, a process known as library migration. Since manually migrating between libraries is tedious and error prone, there has been a lot of effort towards automated library migration. However, most of the current research on automated library migration focuses on Java libraries, and even more so on version migrations of the same library. Despite the increasing popularity of Python, limited research has investigated migration between Python libraries. To provide the necessary data for advancing the development of Python library migration tools, this paper contributes PyMigBench, a benchmark of real Python library migrations.PyMigBench contains 59 analogous library pairs and 75 real migrations with migration-related code changes in 161 Python files across 57 client repositories. Read more

A Dataset of Bot and Human Activities in GitHub

• Download metadata
• Download FAIR analysis
• Download annotations

Natarajan Chidambaram, Alexandre Decan, T. Mens

DOI: 10.1109/MSR59073.2023.00070

Publication year: 2023

Topic 4 - Terms: code, bug, repository, github, tool, study, activity, source, technique, apps

Abstract: Software repositories hosted on GitHub frequently use development bots to automate repetitive, effort intensive and error-prone tasks. To understand and study how these bots are used, state-of-the-... Abstract: Software repositories hosted on GitHub frequently use development bots to automate repetitive, effort intensive and error-prone tasks. To understand and study how these bots are used, state-of-the-art bot identification tools have been developed to detect bots based on their comments in commits, issues and pull requests. Given that bots can be involved in many other activity types, there is a need to consider more activities that they are carrying out in the software repositories they are involved in. We therefore propose a curated dataset of such activities carried out by bots and humans involved in GitHub repositories. The dataset was constructed by identifying 24 high-level activity types that could be extracted from 15 lower-level event types that were queried from GitHub’s event stream API for all considered bots and humans. The proposed dataset contains around 834K activities performed by 385 bots and 616 humans involved in GitHub repositories, during an observation period ranging from 25 November 2022 to 9 March 2023. By analysing the activity patterns of bots and humans, this dataset could lead to better bot identification tools and empirical studies on how bots play a role in collaborative software development. Read more

DACOS—A Manually Annotated Dataset of Code Smells

• Download metadata
• Download FAIR analysis
• Download annotations

Himesh Nandani, M. Saad, Tushar Sharma

DOI: 10.1109/MSR59073.2023.00067

Publication year: 2023

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Researchers apply machine-learning techniques for code smell detection to counter the subjectivity of many code smells. Such approaches need a large, manually annotated dataset for training and ben... Abstract: Researchers apply machine-learning techniques for code smell detection to counter the subjectivity of many code smells. Such approaches need a large, manually annotated dataset for training and benchmarking. Existing literature offers a few datasets; however, they are small in size and, more importantly, do not focus on the subjective code snippets. In this paper, we present DACOS, a manually annotated dataset containing 10, 267 annotations for 5, 192 code snippets. The dataset targets three kinds of code smells at different granularity–multifaceted abstraction, complex method, and long parameter list. The dataset is created in two phases. The first phase helps us identify the code snippets that are potentially subjective by determining the thresholds of metrics used to detect a smell. The second phase collects annotations for potentially subjective snippets. We also offer an extended dataset DACOSX that includes definitely benign and definitely smelly snippets by using the thresholds identified in the first phase. We have developed TAGMAN, a web application to help annotators view and mark the snippets one-by-one and record the provided annotations. We make the datasets and the web application accessible publicly. This dataset will help researchers working on smell detection techniques to build relevant and context-aware machine-learning models. Read more

PENTACET data - 23 Million Contextual Code Comments and 250,000 SATD comments

• Download metadata
• Download FAIR analysis
• Download annotations

Murali Sridharan, Leevi Rantala, M. Mäntylä

DOI: 10.1109/MSR59073.2023.00063

Publication year: 2023

Topic 13 - Terms: source, system, evolution, code, build, set, project, developer, comment, information

Abstract: Most Self-Admitted Technical Debt (SATD) research utilizes explicit SATD features such as ‘TODO’ and ‘FIXME’ for SATD detection. A closer look reveals several SATD research uses simple SATD (‘Easy ... Abstract: Most Self-Admitted Technical Debt (SATD) research utilizes explicit SATD features such as ‘TODO’ and ‘FIXME’ for SATD detection. A closer look reveals several SATD research uses simple SATD (‘Easy to Find’) code comments without contextual data (preceding and succeeding source code context). This work addresses this gap through PENTACET (or 5C dataset) data. PENTACET is a large Curated Contextual Code Comments per Contributor and the most extensive SATD data. We mine 9,096 Open Source Software Java projects totaling over 400 million LOC. The outcome is a dataset with 23 million code comments, preceding and succeeding source code context for each comment, and more than 250,000 SATD comments, including both ‘Easy to Find’ and ‘Hard to Find’ SATD. We believe PENTACET data will further SATD research using Artificial Intelligence techniques. Read more

DocMine: A Software Documentation-Related Dataset of 950 GitHub Repositories

• Download metadata
• Download FAIR analysis
• Download annotations

Akhila Sri Manasa Venigalla, S. Chimalakonda

DOI: 10.1109/MSR59073.2023.00062

Publication year: 2023

Topic 10 - Terms: repository, issue, information, system, source, project, github, file, evolution, developer

Abstract: Software documentation is one of the critical aspects of a software project, that could support multiple tasks throughout the software development life-cycle. There is extensive research on underst... Abstract: Software documentation is one of the critical aspects of a software project, that could support multiple tasks throughout the software development life-cycle. There is extensive research on understanding issues and challenges with existing documentation, which is typically available as readme files. In projects that support collaborative development, such as those on GitHub, other software artifacts such as commits, pull requests and issues, apart from the conventional readme files, wikis and source code comments, also contain useful information, that supports in understanding, using, extending and maintaining the project. However, we are not aware of any dataset that explicitly focuses on documentation-related information in multiple software artifacts such as readme files, commits and pull requests across a repository. To address this concern and to facilitate further research in software documentation, we present DocMine, as a dataset of documentation-related information, extracted from around 1.35M software artifacts in 950 GitHub repositories, spanning across four different programming languages. The dataset along with its documentation is made available in CSV and .sql formats at - https://doi.org/10.5281/zenodo.5195084. Read more

SecretBench: A Dataset of Software Secrets

• Download metadata
• Download FAIR analysis
• Download annotations

S. Basak, L. Neil, Bradley Reaves, Laurie A. Williams

DOI: 10.1109/MSR59073.2023.00053

Publication year: 2023

Topic 5 - Terms: github, project, source, vulnerability, apps, development, tool, repository, present, information

Abstract: According to GitGuardian’s monitoring of public GitHub repositories, the exposure of secrets (API keys and other credentials) increased two-fold in 2021 compared to 2020, totaling more than six mil... Abstract: According to GitGuardian’s monitoring of public GitHub repositories, the exposure of secrets (API keys and other credentials) increased two-fold in 2021 compared to 2020, totaling more than six million secrets. However, no benchmark dataset is publicly available for researchers and tool developers to evaluate secret detection tools that produce many false positive warnings. The goal of our paper is to aid researchers and tool developers in evaluating and improving secret detection tools by curating a benchmark dataset of secrets through a systematic collection of secrets from open-source repositories. We present a labeled dataset of source codes containing 97,479 secrets (of which 15,084 are true secrets) of various secret types extracted from 818 public GitHub repositories. The dataset covers 49 programming languages and 311 file types. Read more

HasBugs - Handpicked Haskell Bugs

• Download metadata
• Download FAIR analysis
• Download annotations

Leonhard Applis, Annibale Panichella

DOI: 10.1109/MSR59073.2023.00040

Publication year: 2023

Topic 2 - Terms: source, test, bug, provide, code, analysis, community, method, study, package

Abstract: We present HasBugs, an extensible and manually-curated dataset of real-world 25 Haskell Bugs from 6 open source repositories. We provide a faulty, tested, and fixed version of each bug in our datas... Abstract: We present HasBugs, an extensible and manually-curated dataset of real-world 25 Haskell Bugs from 6 open source repositories. We provide a faulty, tested, and fixed version of each bug in our dataset with reproduction packages, description, and bug context. For technical users, the dataset is meant to either help researchers adapt techniques from other programming languages to Haskell or to provide a human-verified gold standard for tools evaluation and enable future reproducibility. We also see applicability for qualitative research, e.g., by analysis of bug lifecycles and comparison to other languages. We provide a companion website for easy access and overview under https://ciselab.github.io/HasBugs/. Read more

Semantically-enriched Jira Issue Tracking Data

• Download metadata
• Download FAIR analysis
• Download annotations

Themistoklis G. Diamantopoulos, Dimitrios Nastos, A. Symeonidis

DOI: 10.1109/MSR59073.2023.00039

Publication year: 2023

Topic 1 - Terms: project, code, source, developer, study, api, model, system, technique, type

Abstract: Current state of practice dictates that software developers host their projects online and employ project management systems to monitor the development of product features, keep track of bugs, and ... Abstract: Current state of practice dictates that software developers host their projects online and employ project management systems to monitor the development of product features, keep track of bugs, and prioritize task assignments. The data stored in these systems, if their semantics are extracted effectively, can be used to answer several interesting questions, such as finding who is the most suitable developer for a task, what the priority of a task should be, or even what is the actual workload of the software team. To support researchers and practitioners that work towards these directions, we have built a system that crawls data from the Jira management system, performs topic modeling on the data to extract useful semantics and stores them in a practical database schema. We have used our system to retrieve and analyze 656 projects of the Apache Software Foundation, comprising data from more than a million Jira issues. Read more

microSecEnD: A Dataset of Security-Enriched Dataflow Diagrams for Microservice Applications

• Download metadata
• Download FAIR analysis
• Download annotations

S. Schneider, Tufan Özen, Michael Chen, R. Scandariato

DOI: 10.1109/MSR59073.2023.00030

Publication year: 2023

Topic 11 - Terms: source, defect, present, developer, android, test, security, development, project, datasets

Abstract: Dataflow diagrams (DFDs) are useful resources in securing applications since they show a software system’s architecture and allow assessing architectural security and weaknesses. Enriching them wit... Abstract: Dataflow diagrams (DFDs) are useful resources in securing applications since they show a software system’s architecture and allow assessing architectural security and weaknesses. Enriching them with annotations about implemented security features further strengthens this ability. This is especially true for microservice applications, as their most pressing security concerns stem from their separation into multiple services. Researchers need data to work on these issues and enhance microservices’ architectural security. In this work, we present microSecEnD, a dataset of 17 manually created DFDs that are extensively annotated with information on implemented security features. We provide traceability for all model items. Further, a mapping to a list of 17 architectural security best-practices is provided. Finally, for each best-practice that an application violates, we present a model variant that does adhere to it. Read more

GIRT-Data: Sampling GitHub Issue Report Templates

• Download metadata
• Download FAIR analysis
• Download annotations

Nafiseh Nikeghbal, Amir Hossein Kargaran, A. Heydarnoori, Hinrich Schutze

DOI: 10.1109/MSR59073.2023.00026

Publication year: 2023

Topic 9 - Terms: repository, bug, source, project, code, information, method, developer, system, present

Abstract: GitHub’s issue reports provide developers with valuable information that is essential to the evolution of a software development project. Contributors can use these reports to perform software engi... Abstract: GitHub’s issue reports provide developers with valuable information that is essential to the evolution of a software development project. Contributors can use these reports to perform software engineering tasks like submitting bugs, requesting features, and collaborating on ideas. In the initial versions of issue reports, there was no standard way of using them. As a result, the quality of issue reports varied widely. To improve the quality of issue reports, GitHub introduced issue report templates (IRTs), which pre-fill issue descriptions when a new issue is opened. An IRT usually contains greeting contributors, describing project guidelines, and collecting relevant information. However, despite of effectiveness of this feature which was introduced in 2016, only nearly 5% of GitHub repositories (with more than 10 stars) utilize it. There are currently few articles on IRTs, and the available ones only consider a small number of repositories.In this work, we introduce GIRT-DATA, the first and largest dataset of IRTs in both YAML and Markdown format. This dataset and its corresponding open-source crawler tool are intended to support research in this area and to encourage more developers to use IRTs in their repositories. The stable version of the dataset contains 1,084,300 repositories and 50,032 of them support IRTs. The stable version of the dataset and crawler is available here: https://github.com/kargaranamir/girt-data Read more

NICHE: A Curated Dataset of Engineered Machine Learning Projects in Python

• Download metadata
• Download FAIR analysis
• Download annotations

Ratnadira Widyasari, Zhou Yang, Ferdian Thung, Sheng Qin Sim, Fiona Wee, Camellia Lok, Jack Phan, Haodi Qi, Constance Tan, Qijin Tay, David Lo

DOI: 10.1109/MSR59073.2023.00022

Publication year: 2023

Topic 12 - Terms: project, source, code, development, clone, set, bug, github, os, datasets

Abstract: Machine learning (ML) has gained much attention and has been incorporated into our daily lives. While there are numerous publicly available ML projects on open source platforms such as GitHub, ther... Abstract: Machine learning (ML) has gained much attention and has been incorporated into our daily lives. While there are numerous publicly available ML projects on open source platforms such as GitHub, there have been limited attempts in filtering those projects to curate ML projects of high quality. The limited availability of such a high-quality dataset poses an obstacle to understanding ML projects. To help clear this obstacle, we present NICHE, a manually labelled dataset consisting of 572 ML projects. Based on the evidence of good software engineering practices, we label 441 of these projects as engineered and 131 as non-engineered. This dataset can help researchers understand the practices that are adopted in high-quality ML projects. It can also be used as a benchmark for classifiers designed to identify engineered ML projects. Read more

PTMTorrent: A Dataset for Mining Open-source Pre-trained Model Packages

• Download metadata
• Download FAIR analysis
• Download annotations

Wenxin Jiang, Nicholas Synovic, Purvish Jajal, Taylor R. Schorlemmer, Arav Tewari, Bhavesh Pareek, G. Thiruvathukal, James C. Davis

DOI: 10.1109/MSR59073.2023.00021

Publication year: 2023

Topic 8 - Terms: study, project, model, development, based, repository, source, issue, code, tool

Abstract: Due to the cost of developing and training deep learning models from scratch, machine learning engineers have begun to reuse pre-trained models (PTMs) and fine-tune them for downstream tasks. PTM r... Abstract: Due to the cost of developing and training deep learning models from scratch, machine learning engineers have begun to reuse pre-trained models (PTMs) and fine-tune them for downstream tasks. PTM registries known as “model hubs” support engineers in distributing and reusing deep learning models. PTM packages include pre-trained weights, documentation, model architectures, datasets, and metadata. Mining the information in PTM packages will enable the discovery of engineering phenomena and tools to support software engineers. However, accessing this information is difficult — there are many PTM registries, and both the registries and the individual packages may have rate limiting for accessing the data.We present an open-source dataset, PTMTorrent, to facilitate the evaluation and understanding of PTM packages. This paper describes the creation, structure, usage, and limitations of the dataset. The dataset includes a snapshot of 5 model hubs and a total of 15,913 PTM packages. These packages are represented in a uniform data schema for cross-hub mining. We describe prior uses of this data and suggest research opportunities for mining using our dataset.The PTMTorrent dataset (v1) is available at: https://app.globus.org/file-manager?origin_id=55e17a6e-9d8f-11ed-a2a2-8383522b48d9&origin_path=%2F%7E%2F.Our dataset generation tools are available on GitHub: https://doi.org/10.5281/zenodo.7570357 Read more

DeepScenario: An Open Driving Scenario Dataset for Autonomous Driving System Testing

• Download metadata
• Download FAIR analysis
• Download annotations

Chengjie Lu, T. Yue, Sajid Ali

DOI: 10.1109/MSR59073.2023.00020

Publication year: 2023

Topic 11 - Terms: source, defect, present, developer, android, test, security, development, project, datasets

Abstract: With the rapid development of autonomous driving systems (ADSs), testing ADSs under various environmental conditions has become a key method to ensure the successful deployment of ADS in the real w... Abstract: With the rapid development of autonomous driving systems (ADSs), testing ADSs under various environmental conditions has become a key method to ensure the successful deployment of ADS in the real world. However, it is impossible to test all the scenarios due to the inherent complexity and uncertainty of ADSs and the driving tasks. Further, testing of ADSs is expensive regarding time and computational resources. Therefore, a large-scale driving scenario dataset consisting of various driving conditions is needed. To this end, we present an open driving scenario dataset DeepScenario, containing over 30K executable driving scenarios, which are collected by 2880 test executions of three driving scenario generation strategies. Each scenario in the dataset is labeled with six attributes characterizing test results. We further show the attribute statistics and distribution of driving scenarios. For example, there are 1050 collision scenarios, in 917 scenarios there were collisions with other vehicles, 105 and 28 with pedestrians and static obstacles, respectively. Target users include ADS developers who need to validate their systems under various environmental conditions. Read more

EqBench: A Dataset of Equivalent and Non-equivalent Program Pairs

• Download metadata
• Download FAIR analysis
• Download annotations

Sahar Badihi, Yi Li, J. Rubin

DOI: 10.1109/MSR52588.2021.00084

Publication year: 2021

Topic 12 - Terms: project, source, code, development, clone, set, bug, github, os, datasets

Abstract: Equivalence checking techniques help establish whether two versions of a program exhibit the same behavior. The majority of popular techniques for formally proving/refuting equivalence are evaluate... Abstract: Equivalence checking techniques help establish whether two versions of a program exhibit the same behavior. The majority of popular techniques for formally proving/refuting equivalence are evaluated on small and simplistic benchmarks, omitting "difficult" programming constructs, such as non-linear arithmetic, loops, floating-point arithmetic, and string and array manipulation. This hinders efficient evaluation of these techniques and the ability to establish their practical applicability in real scenarios. This paper addresses this gap by contributing EqBench – the largest and most comprehensive benchmark for equivalence checking analysis, which contains 147 equivalent and 125 non-equivalent cases, in both C and Java languages. We believe EqBench can facilitate a more realistic evaluation of equivalence checking techniques, assessing their individual strength and weaknesses. EqBench is publicly available at: https://osf.io/93s5b/. Read more

GE526: A Dataset of Open-Source Game Engines

• Download metadata
• Download FAIR analysis
• Download annotations

Dheeraj Vagavolu, Vartika Agrahari, S. Chimalakonda, Akhila Sri Manasa Venigalla

DOI: 10.1109/MSR52588.2021.00083

Publication year: 2021

Topic 7 - Terms: bug, project, repository, report, developer, code, study, single, source, activity

Abstract: Game engines, are frameworks that provide a platform for developers to build games with an interface tailored to handle the complexity of game development. Though there is extensive empirical resea... Abstract: Game engines, are frameworks that provide a platform for developers to build games with an interface tailored to handle the complexity of game development. Though there is extensive empirical research on software frameworks, there is a need for empirical studies on game engines, as they differ from traditional software frameworks. Thus, to aid research and development in the area of game engines, we present GE526, a curated dataset of 526 game engine repositories mined from GitHub, which can help researchers to analyze game engines in terms of the release cycles, code quality, API usability and so on. To the best of our knowledge, we are not aware of any curated dataset that caters to game engines in the literature. The dataset contains metadata of all the mined repositories, including 582,079 commits, 20,138 pull requests, 30,287 issues reports and 2,111 releases. The dataset along with its documentation is made available at - https://bit.ly/3pyexnc. Read more

Andror2: A Dataset of Manually-Reproduced Bug Reports for Android apps

• Download metadata
• Download FAIR analysis
• Download annotations

Tyler Wendland, Jingyang Sun, Junayed Mahmud, S M Hasan Mansur, Steven Huang, Kevin Moran, J. Rubin, M. Fazzini

DOI: 10.1109/MSR52588.2021.00082

Publication year: 2021

Topic 7 - Terms: bug, project, repository, report, developer, code, study, single, source, activity

Abstract: Software maintenance constitutes a large portion of the software development lifecycle. To carry out maintenance tasks, developers often need to understand and reproduce bug reports. As such, there... Abstract: Software maintenance constitutes a large portion of the software development lifecycle. To carry out maintenance tasks, developers often need to understand and reproduce bug reports. As such, there has been increasing research activity coalescing around the notion of automating various activities related to bug reporting. A sizable portion of this research interest has focused on the domain of mobile apps. However, as research around mobile app bug reporting progresses, there is a clear need for a manually vetted and reproducible set of real-world bug reports that can serve as a benchmark for future work. This paper presents AndroR2: a dataset of 90 manually reproduced bug reports for Android apps listed on Google Play and hosted on GitHub, systematically collected via an in-depth analysis of 459 reports extracted from the GitHub issue tracker. For each reproduced report, AndroR2 includes the original bug report, an apk file for the buggy version of the app, an executable reproduction script, and metadata regarding the quality of the reproduction steps associated with the original report. We believe that the AndroR2 dataset can be used to facilitate research in automatically analyzing, understanding, reproducing, localizing, and fixing bugs for mobile applications as well as other software maintenance activities more broadly. Read more

Apache Software Foundation Incubator Project Sustainability Dataset

• Download metadata
• Download FAIR analysis
• Download annotations

Likang Yin, Zhiyuan Zhang, Qi Xuan, V. Filkov

DOI: 10.1109/MSR52588.2021.00081

Publication year: 2021

Topic 12 - Terms: project, source, code, development, clone, set, bug, github, os, datasets

Abstract: Open Source Software success and sustainability is critically important for the digital infrastructure as OSS is used broadly and yet 83+% of such projects fail. To increase chances of success many... Abstract: Open Source Software success and sustainability is critically important for the digital infrastructure as OSS is used broadly and yet 83+% of such projects fail. To increase chances of success many projects join established software communities, e.g. the Apache Software Foundation (ASF), with clearly established rules and support. Specifically at ASF, projects that strive to join ASF and are at a nascent development stage are digitally housed in the ASF incubator (ASFI), which provides a mature governance environment and expert help toward long-term sustainability. Projects in ASFI eventually conclude their incubation by graduating, if successful on the path to sustainability. Otherwise, they get retired. In ASF, digital traces of developer activities for projects in ASFI are publicly available, together with monthly project status.Here we present a longitudinal dataset of developer coding and communication activities of 269 projects from the Apache Software Foundation Incubator (ASFI). Each project in ASFI is evaluated while in incubation and is eventually "graduated" or "retired", a label indicating the project sustainability promise with respect to their technical development and community diversity. This extrinsically labeled dataset offers heretofore unavailable sustainability data of OSS project development under ASF regulations and governance. We hope its availability will foster more research interest in studying sustainability in OSS projects. Read more

QScored: A Large Dataset of Code Smells and Quality Metrics

• Download metadata
• Download FAIR analysis
• Download annotations

Tushar Sharma, Marouane Kessentini

DOI: 10.1109/MSR52588.2021.00080

Publication year: 2021

Topic 3 - Terms: code, review, project, source, present, change, set, github, repository, quality

Abstract: Code quality aspects such as code smells and code quality metrics are widely used in exploratory and empirical software engineering research. In such studies, researchers spend a substantial amount... Abstract: Code quality aspects such as code smells and code quality metrics are widely used in exploratory and empirical software engineering research. In such studies, researchers spend a substantial amount of time and effort to not only select the appropriate subject systems but also to analyze them to collect the required code quality information. In this paper, we present QScored dataset; the dataset contains code quality information of more than 86 thousand C# and Java GitHub repositories containing more than 1.1 billion lines of code. The code quality information contains seven kinds of detected architecture smells, 20 kinds of design smells, eleven kinds of implementation smells, and 27 commonly used code quality metrics computed at project, package, class, and method levels. Availability of the dataset will facilitate empirical studies involving code quality aspects by making the information readily available for a large number of active GitHub repositories. Read more

ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference

• Download metadata
• Download FAIR analysis
• Download annotations

A. Mir, Evaldas Latoskinas, Georgios Gousios

DOI: 10.1109/MSR52588.2021.00079

Publication year: 2021

Topic 1 - Terms: project, code, source, developer, study, api, model, system, technique, type

Abstract: In this paper, we present ManyTypes4Py, a large Python dataset for machine learning (ML)-based type inference. The dataset contains a total of 5,382 Python projects with more than 869K type annotat... Abstract: In this paper, we present ManyTypes4Py, a large Python dataset for machine learning (ML)-based type inference. The dataset contains a total of 5,382 Python projects with more than 869K type annotations. Duplicate source code files were removed to eliminate the negative effect of the duplication bias. To facilitate training and evaluation of ML models, the dataset was split into training, validation and test sets by files. To extract type information from abstract syntax trees (ASTs), a light-weight static analyzer pipeline is developed and accompanied with the dataset. Using this pipeline, the collected Python projects were analyzed and the results of the AST analysis were stored in JSON-formatted files. The ManyTypes4Py dataset is shared on zenodo and its tools are publicly available on GitHub. Read more

Andromeda: A Dataset of Ansible Galaxy Roles and Their Evolution

• Download metadata
• Download FAIR analysis
• Download annotations

R. Opdebeeck, Ahmed Zerouali, Coen De Roover

DOI: 10.1109/MSR52588.2021.00078

Publication year: 2021

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: Cloud-native applications increasingly provision infrastructure resources programmatically through Infrastructure as Code (IaC) scripts. These scripts have in turn become the subject of empirical s... Abstract: Cloud-native applications increasingly provision infrastructure resources programmatically through Infrastructure as Code (IaC) scripts. These scripts have in turn become the subject of empirical software engineering research. However, an often-overlooked part are the software ecosystems that have grown around the IaC languages. For example, Galaxy is an ecosystem for the popular Ansible IaC language. Galaxy features a large number of so-called "roles", which are reusable collections of Ansible code akin to libraries for general-purpose languages. In contrast to, and despite their similarities, such IaC ecosystems have enjoyed far less attention in the literature than library ecosystems for general-purpose languages.In this data showcase paper, we present Andromeda, the first dataset capturing the Ansible Galaxy ecosystem, its roles, and their evolution. Andromeda provides structural representations of more than 125 000 role versions, and upwards of 800 000 concrete changes between such versions extracted from the underlying git repositories. Andromeda aims to provide an extensive view of the contributor side of the Galaxy ecosystem, which we hope will stimulate additional research on IaC ecosystems. Read more

Search4Code: Code Search Intent Classification Using Weak Supervision

• Download metadata
• Download FAIR analysis
• Download annotations

N. Rao, Chetan Bansal, Joe Guan

DOI: 10.1109/MSR52588.2021.00077

Publication year: 2021

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Developers use search for various tasks such as finding code, documentation, debugging information, etc. In particular, web search is heavily used by developers for finding code examples and snippe... Abstract: Developers use search for various tasks such as finding code, documentation, debugging information, etc. In particular, web search is heavily used by developers for finding code examples and snippets during the coding process. Recently, natural language based code search has been an active area of research. However, the lack of real-world large-scale datasets is a significant bottleneck. In this work, we propose a weak supervision based approach for detecting code search intent in search queries for C# and Java programming languages. We evaluate the approach against several baselines on a real-world dataset comprised of over 1 million queries mined from Bing web search engine and show that the CNN based model can achieve an accuracy of 77% and 76% for C# and Java respectively. Furthermore, we are also releasing Search4Code, the first large-scale real-world dataset of code search queries mined from Bing web search engine. We hope that the dataset will aid future research on code search. Read more

AndroCT: Ten Years of App Call Traces in Android

• Download metadata
• Download FAIR analysis
• Download annotations

Wen Li, Xiaoqin Fu, Haipeng Cai

DOI: 10.1109/MSR52588.2021.00076

Publication year: 2021

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Data-driven approaches have proven to be promising in mobile software analysis, yet these approaches rely on sizable and quality datasets. For Android app analysis in particular, there have been se... Abstract: Data-driven approaches have proven to be promising in mobile software analysis, yet these approaches rely on sizable and quality datasets. For Android app analysis in particular, there have been several well-known datasets that are widely used by the community. However, there is still a lack of such datasets that represent the run-time behaviors of apps— existing datasets are largely static, whereas run-time datasets are essential for data-driven dynamic and hybrid analysis of apps. In this paper, we present AndroCT, a large-scale dataset on the run-time traces of function calls in 35,974 benign and malicious Android apps from ten historical years (2010 through 2019). These call traces were produced by running each sample app against automatically generated test inputs for ten minutes. Moreover, each app was exercised both on an emulator and a real device, and the traces were separately curated. AndroCT has been used to build a novel dynamic profile of Android apps that has enabled several effective techniques and informative empirical studies concerning Android app security. We describe what this dataset includes, how it was created and stored, and how it has been used in past and would be used in the future. Read more

The Wonderless Dataset for Serverless Computing

• Download metadata
• Download FAIR analysis
• Download annotations

Nafise Eskandani, G. Salvaneschi

DOI: 10.1109/MSR52588.2021.00075

Publication year: 2021

Topic 9 - Terms: repository, bug, source, project, code, information, method, developer, system, present

Abstract: Function as a Service (FaaS) has grown in popularity in recent years, with an increasing number of applications following the Serverless computing model. Serverless computing supports out of the bo... Abstract: Function as a Service (FaaS) has grown in popularity in recent years, with an increasing number of applications following the Serverless computing model. Serverless computing supports out of the box autoscaling in a pay-as-you-go manner, letting developers focus on the application logic rather than worrying about resource management. With the increasing adoption of the this model, researchers have started studying a wide variety of aspects of Serverless computing, including communication, security, performance, and cost optimization. Yet, we still know very little of how Serverless computing is used in practice.In this paper, we introduce Wonderless, a novel dataset of open-source Serverless applications. Wonderless consists of 1,877 real-world Serverless applications extracted from GitHub, and it can be used as a data source for further research in the Serverless ecosystem, such as performance evaluation and software mining. To the best of our knowledge, Wonderless is currently the most diverse and largest dataset for research on Serverless computing. Read more

Sampling Projects in GitHub for MSR Studies

• Download metadata
• Download FAIR analysis
• Download annotations

Ozren Dabić, Emad Aghajani, G. Bavota

DOI: 10.1109/MSR52588.2021.00074

Publication year: 2021

Topic 8 - Terms: study, project, model, development, based, repository, source, issue, code, tool

Abstract: Almost every Mining Software Repositories (MSR) study requires, as first step, the selection of the subject software repositories. These repositories are usually collected from hosting services lik... Abstract: Almost every Mining Software Repositories (MSR) study requires, as first step, the selection of the subject software repositories. These repositories are usually collected from hosting services like GitHub using specific selection criteria dictated by the study goal. For example, a study related to licensing might be interested in selecting projects explicitly declaring a license. Once the selection criteria have been defined, utilities such as the GitHub APIs can be used to "query" the hosting service. However, researchers have to deal with usage limitations imposed by these APIs and a lack of required information. For example, the GitHub search APIs allow 30 requests per minute and, when searching repositories, only provide limited information (e.g., the number of commits in a repository is not included). To support researchers in sampling projects from GitHub, we present GHS (GitHub Search), a dataset containing 25 characteristics (e.g., number of commits, license, etc.) of 735,669 repositories written in 10 programming languages. The set of characteristics has been derived by looking for frequently used project selection criteria in MSR studies and the dataset is continuously updated to (i) always provide fresh data about the existing projects, and (ii) increase the number of indexed projects. The GHS dataset can be queried through a web application we built that allows to set many combinations of selection criteria needed for a study and download the information of matching repositories: https://seart-ghs.si.usi.ch. Read more

A Traceability Dataset for Open Source Systems

• Download metadata
• Download FAIR analysis
• Download annotations

Mouna Hammoudi, Christoph Mayr-Dorn, A. Mashkoor, Alexander Egyed

DOI: 10.1109/MSR52588.2021.00073

Publication year: 2021

Topic 9 - Terms: repository, bug, source, project, code, information, method, developer, system, present

Abstract: Software engineers use requirement-to-method trace matrices to indicate the methods implementing different system requirements. Requirement-to-method trace matrices pinpoint the exact method implem... Abstract: Software engineers use requirement-to-method trace matrices to indicate the methods implementing different system requirements. Requirement-to-method trace matrices pinpoint the exact method implementing each requirement, which facilitates software maintenance and bug fixing. The code structure of a system can be used to make predictions about requirement-to-method traces. In this paper, we present a data set documenting the requirement-to-method traces as well as the code structure (methods, variables, etc.) for four open source systems. The code structure was obtained by parsing the systems under consideration and extracting the methods, variables, etc. The requirement-to-method trace matrices were obtained by resorting to students as well as to the original developers of the systems who provided us with the list of requirement-to-method traces. Read more

KGTorrent: A Dataset of Python Jupyter Notebooks from Kaggle

• Download metadata
• Download FAIR analysis
• Download annotations

L. Quaranta, Fabio Calefato, F. Lanubile

DOI: 10.1109/MSR52588.2021.00072

Publication year: 2021

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: Computational notebooks have become the tool of choice for many data scientists and practitioners for performing analyses and disseminating results. Despite their increasing popularity, the researc... Abstract: Computational notebooks have become the tool of choice for many data scientists and practitioners for performing analyses and disseminating results. Despite their increasing popularity, the research community cannot yet count on a large, curated dataset of computational notebooks. In this paper, we fill this gap by introducing KGTorrent, a dataset of Python Jupyter notebooks with rich metadata retrieved from Kaggle, a platform hosting data science competitions for learners and practitioners with any levels of expertise. We describe how we built KGTorrent, and provide instructions on how to use it and refresh the collection to keep it up to date. Our vision is that the research community will use KGTorrent to study how data scientists, especially practitioners, use Jupyter Notebook in the wild and identify potential shortcomings to inform the design of its future extensions. Read more

Duets: A Dataset of Reproducible Pairs of Java Library-Clients

• Download metadata
• Download FAIR analysis
• Download annotations

Thomas Durieux, César Soto-Valero, B. Baudry

DOI: 10.1109/MSR52588.2021.00071

Publication year: 2021

Topic 1 - Terms: project, code, source, developer, study, api, model, system, technique, type

Abstract: Software engineering researchers look for software artifacts to study their characteristics or to evaluate new techniques. In this paper, we introduce Duets, a new dataset of software libraries and... Abstract: Software engineering researchers look for software artifacts to study their characteristics or to evaluate new techniques. In this paper, we introduce Duets, a new dataset of software libraries and their clients. This dataset can be exploited to gain many different insights, such as API usage, usage inputs, or novel observations about the test suites of clients and libraries. Duets is meant to support both static and dynamic analysis. This means that the libraries and the clients compile correctly, they are executable and their test suites pass. The dataset is composed of open-source projects that have more than five stars on GitHub. The final dataset contains 395 libraries and 2,874 clients. Additionally, we provide the raw data that we use to create this dataset, such as 34,560 pom.xml files or the complete file list from 34,560 projects. This dataset can be used to study how libraries are used by their clients or as a list of software projects that successfully build. The client’s test suite can be used as an additional verification step for code transformation techniques that modify the libraries. Read more

Denchmark: A Bug Benchmark of Deep Learning-related Software

• Download metadata
• Download FAIR analysis
• Download annotations

Misoo Kim, Youngkyoung Kim, Eunseok Lee

DOI: 10.1109/MSR52588.2021.00070

Publication year: 2021

Topic 8 - Terms: study, project, model, development, based, repository, source, issue, code, tool

Abstract: A growing interest in deep learning (DL) has instigated a concomitant rise in DL-related software (DLSW). Therefore, the importance of DLSW quality has emerged as a vital issue. Simultaneously, res... Abstract: A growing interest in deep learning (DL) has instigated a concomitant rise in DL-related software (DLSW). Therefore, the importance of DLSW quality has emerged as a vital issue. Simultaneously, researchers have found DLSW more complicated than traditional SW and more difficult to debug owing to the black-box nature of DL. These studies indicate the necessity of automatic debugging techniques for DLSW. Although several validated debugging techniques exist for general SW, no such techniques exist for DLSW. There is no standard bug benchmark to validate these automatic debugging techniques. In this study, we introduce a novel bug benchmark for DLSW, Denchmark, consisting of 4,577 bug reports from 193 popular DLSW projects, collected through a systematic dataset construction process. These DLSW projects are further classified into eight categories: framework, platform, engine, compiler, tool, library, DL-based application, and others. All bug reports in Denchmark contain rich textual information and links with bug-fixing commits, as well as three levels of buggy entities, such as files, methods, and lines. Our dataset aims to provide an invaluable starting point for the automatic debugging techniques of DLSW. Read more

AndroidCompass: A Dataset of Android Compatibility Checks in Code Repositories

• Download metadata
• Download FAIR analysis
• Download annotations

Sebastian Nielebock, Paul Blockhaus, J. Krüger, F. Ortmeier

DOI: 10.1109/MSR52588.2021.00069

Publication year: 2021

Topic 11 - Terms: source, defect, present, developer, android, test, security, development, project, datasets

Abstract: Many developers and organizations implement apps for Android, the most widely used operating system for mobile devices. Common problems developers face are the various hardware devices, customized ... Abstract: Many developers and organizations implement apps for Android, the most widely used operating system for mobile devices. Common problems developers face are the various hardware devices, customized Android variants, and frequent updates, forcing them to implement workarounds for the different versions and variants of Android APIs used in practice. In this paper, we contribute the Android Compatibility checkS dataSet (AndroidCompass) that comprises changes to compatibility checks developers use to enforce workarounds for specific Android versions in their apps. We extracted 80,324 changes to compatibility checks from 1,394 apps by analyzing the version histories of 2,399 projects from the F-Droid catalog. With AndroidCompass, we aim to provide data on when and how developers introduced or evolved workarounds to handle Android incompatibilities. We hope that AndroidCompass fosters research to deal with version incompatibilities, address potential design flaws, identify security concerns, and help derive solutions for other developers, among others—helping researchers to develop and evaluate novel techniques, and Android app as well as operating-system developers in engineering their software. Read more

An Empirical History of Permission Requests and Mistakes in Open Source Android Apps

• Download metadata
• Download FAIR analysis
• Download annotations

Gian Luca Scoccia, Anthony S Peruma, Virginia Pujols, Ben Christians, Daniel E. Krutz

DOI: 10.1109/MSR.2019.00090

Publication year: 2019

Topic 9 - Terms: repository, bug, source, project, code, information, method, developer, system, present

Abstract: Android applications (apps) rely upon proper permission usage to ensure that the user's privacy and security are adequately protected. Unfortunately, developers frequently misuse app permissions in... Abstract: Android applications (apps) rely upon proper permission usage to ensure that the user's privacy and security are adequately protected. Unfortunately, developers frequently misuse app permissions in a variety of ways ranging from using too many permissions to not correctly adhering to Android's defined permission guidelines. The implications of these permissionissues (possible permission problems) can range from harming the user's perception of the app to significantly impacting their privacy and security. An imperative component to creating more secure apps that better protect a user's privacy is an improved understanding of how and when these issues are being introduced and repaired. While there are existing permissions-analysis tools and Android datasets, there are no available datasets that contain a large-scale empirical history of permission changes and mistakes. This limitation inhibits both developers and researchers from empirically studying and constructing a holistic understanding of permission-related issues. To address this shortfall with existing resources, we created a dataset of permission-based changes and permission-issues in open source Android apps. Our unique dataset contains information from 2,002 apps with commits from 10,601 unique committers, totaling 789,577 commits. We accomplished this by mining app repositories from F-Droid, extracting their version and commit histories, and analyzing this information using two permission analysis tools. Our work creates the foundation for future research in permission decisions and mistakes. Complete project details and data is available on our project website: https://mobilepermissions.github.io Read more

RapidRelease - A Dataset of Projects and Issues on Github with Rapid Releases

• Download metadata
• Download FAIR analysis
• Download annotations

Saket Joshi, S. Chimalakonda

DOI: 10.1109/MSR.2019.00088

Publication year: 2019

Topic 12 - Terms: project, source, code, development, clone, set, bug, github, os, datasets

Abstract: In the recent years, there has been a surge in the adoption of agile development model and continuous integration (CI) in software development. Recent trends have reduced average release cycle leng... Abstract: In the recent years, there has been a surge in the adoption of agile development model and continuous integration (CI) in software development. Recent trends have reduced average release cycle lengths to as low as 1-2 weeks, leading to an extensive number of studies in release engineering. Open-source development (OSD) has also witnessed a rapid increase in release rates, however, no large dataset of open-source projects exists which features high release rates. In this paper, we introduce the RapidRelease dataset, a data showcase of high release frequency open-source projects. The dataset hosts 994 projects from Github, with over 2 million issue reports. To the best of our knowledge, this is the first dataset that can facilitate researchers to empirically study release engineering and agile software development in open-source projects with rapid releases. Read more

A Benchmark of Data Loss Bugs for Android Apps

• Download metadata
• Download FAIR analysis
• Download annotations

O. Riganelli, M. Mobilio, D. Micucci, L. Mariani

DOI: 10.1109/MSR.2019.00087

Publication year: 2019

Topic 5 - Terms: github, project, source, vulnerability, apps, development, tool, repository, present, information

Abstract: Android apps must be able to deal with both stop events, which require immediately stopping the execution of the app without losing state information, and start events, which require resuming the e... Abstract: Android apps must be able to deal with both stop events, which require immediately stopping the execution of the app without losing state information, and start events, which require resuming the execution of the app at the same point it was stopped. Support to these kinds of events must be explicitly implemented by developers who unfortunately often fail to implement the proper logic for saving and restoring the state of an app. As a consequence apps can lose data when moved to background and then back to foreground (e.g., to answer a call) or when the screen is simply rotated. These faults can be the cause of annoying usability issues and unexpected crashes. This paper presents a public benchmark of 110 data loss faults in Android apps that we systematically collected to facilitate research and experimentation with these problems. The benchmark is available on GitLab and includes the faulty apps, the fixed apps (when available), the test cases to automatically reproduce the problems, and additional information that may help researchers in their tasks. Read more

Boa Meets Python: A Boa Dataset of Data Science Software in Python Language

• Download metadata
• Download FAIR analysis
• Download annotations

Sumon Biswas, Md Johirul Islam, Yijia Huang, Hridesh Rajan

DOI: 10.1109/MSR.2019.00086

Publication year: 2019

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: The popularity of Python programming language has surged in recent years due to its increasing usage in Data Science. The availability of Python repositories in Github presents an opportunity for m... Abstract: The popularity of Python programming language has surged in recent years due to its increasing usage in Data Science. The availability of Python repositories in Github presents an opportunity for mining software repository research, e.g., suggesting the best practices in developing Data Science applications, identifying bug-patterns, recommending code enhancements, etc. To enable this research, we have created a new dataset that includes 1,558 mature Github projects that develop Python software for Data Science tasks. By analyzing the metadata and code, we have included the projects in our dataset which use a diverse set of machine learning libraries and managed by a variety of users and organizations. The dataset is made publicly available through Boa infrastructure both as a collection of raw projects as well as in a processed form that could be used for performing large scale analysis using Boa language. We also present two initial applications to demonstrate the potential of the dataset that could be leveraged by the community. Read more

SeSaMe: A Data Set of Semantically Similar Java Methods

• Download metadata
• Download FAIR analysis
• Download annotations

Marius Kamp, Patrick Kreutzer, M. Philippsen

DOI: 10.1109/MSR.2019.00079

Publication year: 2019

Topic 12 - Terms: project, source, code, development, clone, set, bug, github, os, datasets

Abstract: In the past, techniques for detecting similarly behaving code fragments were often only evaluated with small, artificial oracles or with code originating from programming competitions. Such code fr... Abstract: In the past, techniques for detecting similarly behaving code fragments were often only evaluated with small, artificial oracles or with code originating from programming competitions. Such code fragments differ largely from production codes. To enable more realistic evaluations, this paper presents SeSaMe, a data set of method pairs that are classified according to their semantic similarity. We applied text similarity measures on JavaDoc comments mined from 11 open source repositories and manually classified a selection of 857 pairs. Read more

RmvDroid: Towards A Reliable Android Malware Dataset with App Metadata

• Download metadata
• Download FAIR analysis
• Download annotations

Haoyu Wang, Junjun Si, Hao Li, Yao Guo

DOI: 10.1109/MSR.2019.00067

Publication year: 2019

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: A large number of research studies have been focused on detecting Android malware in recent years. As a result, a reliable and large-scale malware dataset is essential to build effective malware cl... Abstract: A large number of research studies have been focused on detecting Android malware in recent years. As a result, a reliable and large-scale malware dataset is essential to build effective malware classifiers and evaluate the performance of different detection techniques. Although several Android malware benchmarks have been widely used in our research community, these benchmarks face several major limitations. First, most of the existing datasets are outdated and cannot reflect current malware evolution trends. Second, most of them only rely on VirusTotal to label the ground truth of malware, while some anti-virus engines on VirusTotal may not always report reliable results. Third, all of them only contain the apps themselves (apks), while other important app information (e.g., app description, user rating, and app installs) is missing, which greatly limits the usage scenarios of these datasets. In this paper, we have created a reliable Android malware dataset based on Google Play's app maintenance results over several years. We first created four snapshots of Google Play in 2014, 2015, 2017 and 2018 respectively. Then we use VirusTotal to label apps with possible sensitive behaviors, and monitor these apps on Google Play to see whether Google has removed them or not. Based on this approach, we have created a malware dataset containing 9,133 samples that belong to 56 malware families with high confidence. We believe this dataset will boost a series of research studies including Android malware detection and classification, mining apps for anomalies, and app store mining, etc. Read more

A Dataset of Non-Functional Bugs

• Download metadata
• Download FAIR analysis
• Download annotations

Aida Radu, Sarah Nadi

DOI: 10.1109/MSR.2019.00066

Publication year: 2019

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: While several researchers have published bug data sets in the past, there has been less focus on bugs related to non-functional requirements. Non-functional requirements describe the quality attrib... Abstract: While several researchers have published bug data sets in the past, there has been less focus on bugs related to non-functional requirements. Non-functional requirements describe the quality attributes of a program. In this work, we introduce NFBugs, a data set of 133 non-functional bug fixes collected from 65 open-source projects written in Java and Python. NFBugs can be used to support code recommender systems focusing on non-functional properties. Read more

A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software

• Download metadata
• Download FAIR analysis
• Download annotations

Serena Elisa Ponta, H. Plate, A. Sabetta, M. Bezzi, Cédric Dangremont

DOI: 10.1109/MSR.2019.00064

Publication year: 2019

Topic 5 - Terms: github, project, source, vulnerability, apps, development, tool, repository, present, information

Abstract: Advancing our understanding of software vulnerabilities, automating their identification, the analysis of their impact, and ultimately their mitigation is necessary to enable the development of sof... Abstract: Advancing our understanding of software vulnerabilities, automating their identification, the analysis of their impact, and ultimately their mitigation is necessary to enable the development of software that is more secure. While operating a vulnerability assessment tool, which we developed, and that is currently used by hundreds of development units at SAP, we manually collected and curated a dataset of vulnerabilities of open-source software, and the commits fixing them. The data were obtained both from the National Vulnerability Database (NVD), and from project-specific web resources, which we monitor on a continuous basis. From that data, we extracted a dataset that maps 624 publicly disclosed vulnerabilities affecting 205 distinct opensource Java projects, used in SAP products or internal tools, onto the 1282 commits that fix them. Out of 624 vulnerabilities, 29 do not have a CVE (Common Vulnerability and Exposure) identifier at all, and 46, which do have such identifier assigned by a numbering authority, are not available in the NVD yet. The dataset is released under an open-source license, together with supporting scripts that allow researchers to automatically retrieve the actual content of the commits from the corresponding repositories, and to augment the attributes available for each instance. Moreover, these scripts allow to complement the dataset with additional instances that are not security fixes (which is useful, for example, in machine learning applications). Our dataset has been successfully used to train classifiers that could automatically identify security-relevant commits in code repositories. The release of this dataset and the supporting code as open-source will allow future research to be based on data of industrial relevance; it also represents a concrete step towards making the maintenance of this dataset a shared effort involving open-source communities, academia, and the industry. Read more

The Maven Dependency Graph: A Temporal Graph-Based Representation of Maven Central

• Download metadata
• Download FAIR analysis
• Download annotations

Amine Benelallam, Nicolas Harrand, César Soto-Valero, B. Baudry, Olivier Barais

DOI: 10.1109/MSR.2019.00060

Publication year: 2019

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: The Maven Central Repository provides an extraordinary source of data to understand complex architecture and evolution phenomena among Java applications. As of September 6, 2018, this repository in... Abstract: The Maven Central Repository provides an extraordinary source of data to understand complex architecture and evolution phenomena among Java applications. As of September 6, 2018, this repository includes 2.8M artifacts (compiled piece of code implemented in a JVM-based language), each of which is characterized with metadata such as exact version, date of upload and list of dependencies towards other artifacts. Today, one who wants to analyze the complete ecosystem of Maven artifacts and their dependencies faces two key challenges: (i) this is a huge data set; and (ii) dependency relationships among artifacts are not modeled explicitly and cannot be queried. In this paper, we present the Maven Dependency Graph. This open source data set provides two contributions: a snapshot of the whole Maven Central taken on September 6, 2018, stored in a graph database in which we explicitly model all dependencies; an open source infrastructure to query this huge dataset. Read more

A Panel Data Set of Cryptocurrency Development Activity on GitHub

• Download metadata
• Download FAIR analysis
• Download annotations

Rijnard van Tonder, Asher Trockman, Claire Le Goues

DOI: 10.1109/MSR.2019.00037

Publication year: 2019

Topic 5 - Terms: github, project, source, vulnerability, apps, development, tool, repository, present, information

Abstract: Cryptocurrencies are a significant development in recent years, featuring in global news, the financial sector, and academic research. They also hold a significant presence in open source developme... Abstract: Cryptocurrencies are a significant development in recent years, featuring in global news, the financial sector, and academic research. They also hold a significant presence in open source development, comprising some of the most popular repositories on GitHub. Their openly developed software artifacts thus present a unique and exclusive avenue to quantitatively observe human activity, effort, and software growth for cryptocurrencies. Our data set marks the first concentrated effort toward high-fidelity panel data of cryptocurrency development for a wide range of metrics. The data set is foremost a quantitative measure of developer activity for budding open source cryptocurrency development. We collect metrics like daily commits, contributors, lines of code changes, stars, forks, and subscribers. We also include financial data for each cryptocurrency: the daily price and market capitalization. The data set includes data for 236 cryptocurrencies for 380 days (roughly January 2018 to January 2019). We discuss particularly interesting research opportunities for this combination of data, and release new tooling to enable continuing data collection for future research opportunities as development and application of cryptocurrencies mature. Read more

GreenSource: A Large-Scale Collection of Android Code, Tests and Energy Metrics

• Download metadata
• Download FAIR analysis
• Download annotations

Rui Rua, Marco Couto, J. Saraiva

DOI: 10.1109/MSR.2019.00035

Publication year: 2019

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: This paper presents the GreenSource infrastructure: a large body of open source code, executable Android applications, and curated dataset containing energy code metrics. The dataset contains energ... Abstract: This paper presents the GreenSource infrastructure: a large body of open source code, executable Android applications, and curated dataset containing energy code metrics. The dataset contains energy metrics obtained by both static analysing the applications' source code and by executing them with available test inputs. To automate the execution of the applications we developed the AnaDroid tool which instruments its code, compiles and executes it with test inputs in any Android device, while collecting energy metrics. GreenSource includes all Android applications included in the MUSE Java source code repository, while AnaDroid implements all Android's energy greedy features described in the literature, GreenSource aims at characterizing energy consumption in the Android ecosystem, providing both Android developers and researchers a setting to reason about energy efficient Android software development. Read more

GreenHub Farmer: Real-World Data for Android Energy Mining

• Download metadata
• Download FAIR analysis
• Download annotations

Hugo Matalonga, Bruno Cabral, F. C. Filho, Marco Couto, Rui Pereira, S. Sousa, J. Fernandes

DOI: 10.1109/MSR.2019.00034

Publication year: 2019

Topic 8 - Terms: study, project, model, development, based, repository, source, issue, code, tool

Abstract: As mobile devices are supporting more and more of our daily activities, it is vital to widen their battery up-time as much as possible. In fact, according to the Wall Street Journal, 9/10 users suf... Abstract: As mobile devices are supporting more and more of our daily activities, it is vital to widen their battery up-time as much as possible. In fact, according to the Wall Street Journal, 9/10 users suffer from low battery anxiety. The goal of our work is to understand how Android usage, apps, operating systems, hardware and user habits influence battery lifespan. Our strategy is to collect anonymous raw data from devices all over the world, through a mobile app, build and analyze a large-scale dataset containing real-world, day-to-day data, representative of user practices. So far, the dataset we collected includes 12 million+ (anonymous) data samples, across 900+ device brands and 5.000+ models. And, it keeps growing. The data we collect, which is publicly available and by different channels, is sufficiently heterogeneous for supporting studies with a wide range of focuses and research goals, thus opening the opportunity to inform and reshape user habits, and even influence the development of both hardware and software for mobile devices. Read more

The Software Heritage Graph Dataset: Public Software Development Under One Roof

• Download metadata
• Download FAIR analysis
• Download annotations

Antoine Pietri, D. Spinellis, Stefano Zacchiroli

DOI: 10.1109/MSR.2019.00030

Publication year: 2019

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Software Heritage is the largest existing public archive of software source code and accompanying development history: it currently spans more than five billion unique source code files and one bil... Abstract: Software Heritage is the largest existing public archive of software source code and accompanying development history: it currently spans more than five billion unique source code files and one billion unique commits, coming from more than 80 million software projects. This paper introduces the Software Heritage graph dataset: a fully-deduplicated Merkle DAG representation of the Software Heritage archive. The dataset links together file content identifiers, source code directories, Version Control System (VCS) commits tracking evolution over time, up to the full states of VCS repositories as observed by Software Heritage during periodic crawls. The dataset's contents come from major development forges (including GitHub and GitLab), FOSS distributions (e.g., Debian), and language-specific package managers (e.g., PyPI). Crawling information is also included, providing timestamps about when and where all archived source code artifacts have been observed in the wild. The Software Heritage graph dataset is available in multiple formats, including downloadable CSV dumps and Apache Parquet files for local use, as well as a public instance on Amazon Athena interactive query service for ready-to-use powerful analytical processing. Source code file contents are cross-referenced at the graph leaves, and can be retrieved through individual requests using the Software Heritage archive API. Read more

A Data Set of Program Invariants and Error Paths

• Download metadata
• Download FAIR analysis
• Download annotations

Dirk Beyer

DOI: 10.1109/MSR.2019.00026

Publication year: 2019

Topic 1 - Terms: project, code, source, developer, study, api, model, system, technique, type

Abstract: The analysis of correctness proofs and counterexamples of program source code is an important way to gain insights into methods that could make it easier in the future to find invariants to prove a... Abstract: The analysis of correctness proofs and counterexamples of program source code is an important way to gain insights into methods that could make it easier in the future to find invariants to prove a program correct or to find bugs. The availability of high-quality data is often a limiting factor for researchers who want to study real program invariants and real bugs. The described data set provides a large collection of concrete verification results, which can be used in research projects as data source or for evaluation purposes. Each result is made available as verification witness, which represents either program invariants that were used to prove the program correct (correctness witness) or an error path to replay the actual bug (violation witness). The verification results are taken from actual verification runs on 10522 verification problems, using the 31 verification tools that participated in the 8th edition of the International Competition on Software Verification (SV-COMP). The collection contains a total of 125720 verification witnesses together with various meta data and a map to relate a witness to the C program that it originates from. Data set is available at: https://doi.org/10.5281/zenodo.2559175 Read more

A Dataset of Parametric Cryptographic Misuses

• Download metadata
• Download FAIR analysis
• Download annotations

A. Wickert, Michael Reif, Michael Eichberg, Anam Dodhy, M. Mezini

DOI: 10.1109/MSR.2019.00023

Publication year: 2019

Topic 4 - Terms: code, bug, repository, github, tool, study, activity, source, technique, apps

Abstract: Cryptographic APIs (Crypto APIs) provide the foundations for the development of secure applications. Unfortunately, most applications do not use Crypto APIs securely and end up being insecure, e.g.... Abstract: Cryptographic APIs (Crypto APIs) provide the foundations for the development of secure applications. Unfortunately, most applications do not use Crypto APIs securely and end up being insecure, e.g., by the usage of an outdated algorithm, a constant initialization vector, or an inappropriate hashing algorithm. Two different studies [1], [2] have recently shown that 88% to 95% of those applications using Crypto APIs are insecure due to misuses. To facilitate further research on these kinds of misuses, we created a collection of 201 misuses found in real-world applications along with a classification of those misuses. In the provided dataset, each misuse consists of the corresponding open-source project, the project's build information, a description of the misuse, and the misuse's location. Further, we integrated our dataset into MUBench [3], a benchmark for API misuse detection. Our dataset provides a foundation for research on Crypto API misuses. For example, it can be used to evaluate the precision and recall of detection tools, as a foundation for studies related to Crypto API misuses, or as a training set. Read more

Cleaning StackOverflow for Machine Translation

• Download metadata
• Download FAIR analysis
• Download annotations

Musfiqur Rahman, Peter C. Rigby, Dharani Palani, T. Nguyen

DOI: 10.1109/MSR.2019.00021

Publication year: 2019

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Generating source code API sequences from an English query using Machine Translation (MT) has gained much interest in recent years. For any kind of MT, the model needs to be trained on a parallel c... Abstract: Generating source code API sequences from an English query using Machine Translation (MT) has gained much interest in recent years. For any kind of MT, the model needs to be trained on a parallel corpus. In this paper we clean StackOverflow, one of the most popular online discussion forums for programmers, to generate a parallel English-Code corpus from Android posts. We contrast three data cleaning approaches: standard NLP, title only, and software task extraction. We evaluate the quality of the each corpus for MT. To provide indicators of how useful each corpus will be for machine translation, we provide researchers with measurements of the corpus size, percentage of unique tokens, and per-word maximum likelihood alignment entropy. We have used these corpus cleaning approaches to translate between English and Code [22, 23], to compare existing SMT approaches from word mapping to neural networks [24], and to re-examine the "natural software" hypothesis [29]. After cleaning and aligning the data, we create a simple maximum likelihood MT model to show that English words in the corpus map to a small number of specific code elements. This model provides a basis for the success of using StackOverflow for search and other tasks in the software engineering literature and paves the way for MT. Our scripts and corpora are publicly available on GitHub [1] as well as at https://search.datacite.org/works/10.5281/zenodo.2558551. Read more

Semantic Source Code Models Using Identifier Embeddings

• Download metadata
• Download FAIR analysis
• Download annotations

V. Efstathiou, D. Spinellis

DOI: 10.1109/MSR.2019.00015

Publication year: 2019

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: The emergence of online open source repositories in the recent years has led to an explosion in the volume of openly available source code, coupled with metadata that relate to a variety of softwar... Abstract: The emergence of online open source repositories in the recent years has led to an explosion in the volume of openly available source code, coupled with metadata that relate to a variety of software development activities. As an effect, in line with recent advances in machine learning research, software maintenance activities are switching from symbolic formal methods to data–driven methods. In this context, the rich semantics hidden in source code identifiers provide opportunities for building semantic representations of code which can assist tasks of code search and reuse. To this end, we deliver in the form of pretrained vector space models, distributed code representations for six popular programming languages, namely, Java, Python, PHP, C, C++, and C#. The models are produced using fastText, a state–of–the–art library for learning word representations. Each model is trained on data from a single programming language; the code mined for producing all models amounts to over 13.000 repositories. We indicate dissimilarities between natural language and source code, as well as variations in coding conventions in between the different programming languages we processed. We describe how these heterogeneities guided the data preprocessing decisions we took and the selection of the training parameters in the released models. Finally, we propose potential applications of the models and discuss limitations of the models. Read more

A Data Set of OCL Expressions on GitHub

• Download metadata
• Download FAIR analysis
• Download annotations

Jeroen Noten, J. Mengerink, Alexander Serebrenik

DOI: 10.1109/MSR.2017.52

Publication year: 2017

Topic 8 - Terms: study, project, model, development, based, repository, source, issue, code, tool

Abstract: In model driven engineering (MDE), meta-models are the central artifacts. As a complement, the Object Constraint Language (OCL) is a language used to express constraints and operations on meta-mode... Abstract: In model driven engineering (MDE), meta-models are the central artifacts. As a complement, the Object Constraint Language (OCL) is a language used to express constraints and operations on meta-models. The Eclipse Modeling Framework (EMF) provides an implementation of OCL, enabling OCL-annotated meta-models. Existing empirical studies of the OCL have been conducted on small collections of data. To facilitate empirical research into the OCL on a larger scale, we present the first publicly available data set of OCL expressions. The data set contains 9188 OCL expressions originating from 504 EMF meta-models in 245 systematically selected GitHub repositories. Both the original meta-models and the generated abstract syntax trees are included, allowing for a variety of empirical studies of the OCL. To illustrate the applicability of this data set in practice, we performed three case studies. Read more

Rediscovery Datasets: Connecting Duplicate Reports

• Download metadata
• Download FAIR analysis
• Download annotations

Mefta Sadat, A. Bener, A. Miranskyy

DOI: 10.1109/MSR.2017.50

Publication year: 2017

Topic 11 - Terms: source, defect, present, developer, android, test, security, development, project, datasets

Abstract: The same defect can be rediscovered by multiple clients, causing unplanned outages and leading to reduced customer satisfaction. In the case of popular open source software, high volume of defects ... Abstract: The same defect can be rediscovered by multiple clients, causing unplanned outages and leading to reduced customer satisfaction. In the case of popular open source software, high volume of defects is reported on a regular basis. A large number of these reports are actually duplicates / rediscoveries of each other. Researchers have analyzed the factors related to the content of duplicate defect reports in the past. However, some of the other potentially important factors, such as the inter-relationships among duplicate defect reports, are not readily available in defect tracking systems such as Bugzilla. This information may speed up bug fixing, enable efficient triaging, improve customer profiles, etc. In this paper, we present three defect rediscovery datasets mined from Bugzilla. The datasets capture data for three groups of open source software projects: Apache, Eclipse, and KDE. The datasets contain information about approximately 914 thousands of defect reports over a period of 18 years (1999-2017) to capture the inter-relationships among duplicate defects. We believe that sharing these data with the community will help researchers and practitioners to better understand the nature of defect rediscovery and enhance the analysis of defect reports. Read more

A Dataset for Dynamic Discovery of Semantic Changes in Version Controlled Software Histories

• Download metadata
• Download FAIR analysis
• Download annotations

Chenguang Zhu, Yi Li, J. Rubin, M. Chechik

DOI: 10.1109/MSR.2017.49

Publication year: 2017

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Over the last few years, researchers proposed several semantic history slicing approaches that identify the set of semantically-related commits implementing a particular software functionality. How... Abstract: Over the last few years, researchers proposed several semantic history slicing approaches that identify the set of semantically-related commits implementing a particular software functionality. However, there is no comprehensive benchmark for evaluating these approaches, making it difficult to assess their capabilities. This paper presents a dataset of 81 semantic change data collected from 8 real-world projects. The dataset is created for benchmarking semantic history slicing techniques. We provide details on the data collection process and the storage format. We also discuss usage and possible extensions of the dataset. Read more

An Extensive Dataset of UML Models in GitHub

• Download metadata
• Download FAIR analysis
• Download annotations

G. Robles, Truong Ho-Quang, R. Hebig, M. Chaudron, M. A. Fernández

DOI: 10.1109/MSR.2017.48

Publication year: 2017

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: The Unified Modeling Language (UML) is widely taught in academia and has good acceptance in industry. However, there is not an ample dataset of UML diagrams publicly available. Our aim is to offer ... Abstract: The Unified Modeling Language (UML) is widely taught in academia and has good acceptance in industry. However, there is not an ample dataset of UML diagrams publicly available. Our aim is to offer a dataset of UML files, together with meta-data of the software projects where the UML files belong to. Therefore, we have systematically mined over 12 million GitHub projects to find UML files in them. We present a semi-automated approach to collect UML stored in images, .xmi, and .uml files. We offer a dataset with over 93,000 UML diagrams from over 24,000 projects in GitHub. Read more

Continuous Defect Prediction: The Idea and a Related Dataset

• Download metadata
• Download FAIR analysis
• Download annotations

L. Madeyski, M. Kawalerowicz

DOI: 10.1109/MSR.2017.46

Publication year: 2017

Topic 10 - Terms: repository, issue, information, system, source, project, github, file, evolution, developer

Abstract: We would like to present the idea of our Continuous Defect Prediction (CDP) research and a related dataset that we created and share. Our dataset is currently a set of more than 11 million data row... Abstract: We would like to present the idea of our Continuous Defect Prediction (CDP) research and a related dataset that we created and share. Our dataset is currently a set of more than 11 million data rows, representing files involved in Continuous Integration (CI) builds, that synthesize the results of CI builds with data we mine from software repositories. Our dataset embraces 1265 software projects, 30,022 distinct commit authors and several software process metrics that in earlier research appeared to be useful in software defect prediction. In this particular dataset we use TravisTorrent as the source of CI data. TravisTorrent synthesizes commit level information from the Travis CI server and GitHub open-source projects repositories. We extend this data to a file change level and calculate the software process metrics that may be used, for example, as features to predict risky software changes that could break the build if committed to a repository with CI enabled. Read more

A Dataset of Scratch Programs: Scraped, Shaped and Scored

• Download metadata
• Download FAIR analysis
• Download annotations

Efthimia Aivaloglou, F. Hermans, J. Moreno-León, G. Robles

DOI: 10.1109/MSR.2017.45

Publication year: 2017

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: Scratch is increasingly popular, both as an introductory programming language and as a research target in the computing education research field. In this paper, we present a dataset of 250K recent ... Abstract: Scratch is increasingly popular, both as an introductory programming language and as a research target in the computing education research field. In this paper, we present a dataset of 250K recent Scratch projects from 100K different authors scraped from the Scratch project repository. We processed the projects' source code and metadata to encode them into a database that facilitates querying and further analysis. We further evaluated the projects in terms of programming skills and mastery, and included the project scoring results. The dataset enables the analysis of the source code of Scratch projects, of their quality characteristics, and of the programming skills that their authors exhibit. The dataset can be used for empirical research in software engineering and computing education. Read more

Software Evolution and Quality Data from Controlled, Multiple, Industrial Case Studies

• Download metadata
• Download FAIR analysis
• Download annotations

A. Yamashita, S. Amirhossein Abtahizadeh, Foutse Khomh, Yann-Gaël Guéhéneuc

DOI: 10.1109/MSR.2017.44

Publication year: 2017

Topic 1 - Terms: project, code, source, developer, study, api, model, system, technique, type

Abstract: A main difficulty to study the evolution and quality of real-life software systems is the effect of moderator factors, such as: programming skill, type of maintenance task, and learning effect. Exp... Abstract: A main difficulty to study the evolution and quality of real-life software systems is the effect of moderator factors, such as: programming skill, type of maintenance task, and learning effect. Experimenters must account for moderator factors to identify the relationships between the variables of interest. In practice, controlling for moderator factors in realistic (industrial) settings is expensive and rather difficult. The data presented in this paper has two particularities: First, it involves six professional developers and four real-life, industrial systems. Second, it was obtained from controlled, multiple case studies where the moderator variables: programming skill, maintenance task, and learning effect were controlled for. This data set is relevant to experimenters studying evolution and quality of real-life systems, in particular those interested in studying industrial systems and replicating empirical studies. Read more

A Dataset of Open-Source Android Applications

• Download metadata
• Download FAIR analysis
• Download annotations

Daniel E. Krutz, Mehdi Mirakhorli, Samuel A. Malachowsky, Andres Ruiz, Jacob Peterson, Andrew Filipski, Jared Smith

DOI: 10.1109/MSR.2015.79

Publication year: 2015

Topic 5 - Terms: github, project, source, vulnerability, apps, development, tool, repository, present, information

Abstract: Android has grown to be the world's most popular mobile platform with apps that are capable of doing everything from checking sports scores to purchasing stocks. In order to assist researchers and ... Abstract: Android has grown to be the world's most popular mobile platform with apps that are capable of doing everything from checking sports scores to purchasing stocks. In order to assist researchers and developers in better understanding the development process as well as the current state of the apps themselves, we present a large dataset of analyzed open-source Android applications and provide a brief analysis of the data, demonstrating potential usefulness. This dataset contains 1,179 applications, including 4,416 different versions of these apps and 435,680 total commits. Furthermore, for each app we include the analytical results obtained from several static analysis tools including Androguard, Sonar, and Stowaway. In order to better support the community in conducting research on the security characteristics of the apps, our large analytical dataset comes with the detailed information including various versions of AndroidManifest.xml files and synthesized information such as permissions, intents, and minimum SDK. We collected 13,036 commits of the manifest files and recorded over 69,707 total permissions used. The results and a brief set of analytics are presented on our website: http://androsec.rit.edu. Read more

A Dataset of High Impact Bugs: Manually-Classified Issue Reports

• Download metadata
• Download FAIR analysis
• Download annotations

M. Ohira, Yutaro Kashiwa, Yosuke Yamatani, Hayato Yoshiyuki, Yoshiya Maeda, Nachai Limsettho, K. Fujino, Hideaki Hata, Akinori Ihara, Ken-ichi Matsumoto

DOI: 10.1109/MSR.2015.78

Publication year: 2015

Topic 7 - Terms: bug, project, repository, report, developer, code, study, single, source, activity

Abstract: The importance of supporting test and maintenance activities in software development has been increasing, since recent software systems have become large and complex. Although in the field of Minin... Abstract: The importance of supporting test and maintenance activities in software development has been increasing, since recent software systems have become large and complex. Although in the field of Mining Software Repositories (MSR) there are many promising approaches to predicting, localizing, and triaging bugs, most of them do not consider impacts of each bug on users and developers but rather treat all bugs with equal weighting, excepting a few studies on high impact bugs including security, performance, blocking, and so forth. To make MSR techniques more actionable and effective in practice, we need deeper understandings of high impact bugs. In this paper we introduced our dataset of high impact bugs which was created by manually reviewing four thousand issue reports in four open source projects (Ambari, Camel, Derby and Wicket). Read more

A Data Set for Social Diversity Studies of GitHub Teams

• Download metadata
• Download FAIR analysis
• Download annotations

Bogdan Vasilescu, Alexander Serebrenik, V. Filkov

DOI: 10.1109/MSR.2015.77

Publication year: 2015

Topic 3 - Terms: code, review, project, source, present, change, set, github, repository, quality

Abstract: Like any other team oriented activity, the software development process is effected by social diversity in the programmer teams. The effect of team diversity can be significant, but also complex, e... Abstract: Like any other team oriented activity, the software development process is effected by social diversity in the programmer teams. The effect of team diversity can be significant, but also complex, especially in decentralized teams. Discerning the precise contribution of diversity on teams' effectiveness requires quantitative studies of large data sets. Here we present for the first time a large data set of social diversity attributes of programmers in GitHub teams. Using alias resolution, location data, and gender inference techniques, we collected a team social diversity data set of 23,493 GitHub projects. We illustrate how the data set can be used in practice with a series of case studies, and we hope its availability will foster more interest in studying diversity issues in software teams. Read more

Generating the Blueprints of the Java Ecosystem

• Download metadata
• Download FAIR analysis
• Download annotations

Vassilios Karakoidas, Dimitris Mitropoulos, P. Louridas, Georgios Gousios, D. Spinellis

DOI: 10.1109/MSR.2015.76

Publication year: 2015

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: Examining a large number of software artifacts can provide the research community with data regarding quality and design. We present a dataset obtained by statically analyzing 22730 jar files taken... Abstract: Examining a large number of software artifacts can provide the research community with data regarding quality and design. We present a dataset obtained by statically analyzing 22730 jar files taken from the Maven central archive, which is the de-facto application library repository for the Java ecosystem. For our analysis we used three popular static analysis tools that calculate metrics regarding object-oriented design, program size, and package design. The dataset contains the metrics results that every tool reports for every selected jar of the ecosystem. Our dataset can be used to produce interesting research results, such as measure the domain-specific language usage. Read more

A Dataset for API Usage

• Download metadata
• Download FAIR analysis
• Download annotations

A. Sawant, Alberto Bacchelli

DOI: 10.1109/MSR.2015.75

Publication year: 2015

Topic 1 - Terms: project, code, source, developer, study, api, model, system, technique, type

Abstract: An Application Programming Interface (API) provides a specific set of functionalities to a developer. The main aim of an API is to encourage the reuse of already existing functionality. There has b... Abstract: An Application Programming Interface (API) provides a specific set of functionalities to a developer. The main aim of an API is to encourage the reuse of already existing functionality. There has been some work done into API popularity trends, API evolution and API usage. For all the aforementioned research avenues there has been a need to mine the usage of an API in order to perform any kind of analysis. Each one of the approaches that has been employed in the past involved a certain degree of inaccuracy as there was no type check that takes place. We introduce an approach that takes type information into account while mining API method invocations and annotation usages. This approach accurately makes a connection between a method invocation and the class of the API to which the method belongs to. We try collecting as many usages of an API as possible, this is achieved by targeting projects hosted on GitHub. Additionally, we look at the history of every project to collect the usage of an API from earliest version onwards. By making such a large and rich dataset public, we hope to stimulate some more research in the field of APIs with the aid of accurate API usage samples. Read more

An Architectural Evolution Dataset

• Download metadata
• Download FAIR analysis
• Download annotations

M. Wermelinger, Y. Yu

DOI: 10.1109/MSR.2015.74

Publication year: 2015

Topic 7 - Terms: bug, project, repository, report, developer, code, study, single, source, activity

Abstract: A good evolution process and a good architecture can greatly support the maintainability of long-lived, large software systems. We present AREVOL, a dataset for the empirical study of architectural... Abstract: A good evolution process and a good architecture can greatly support the maintainability of long-lived, large software systems. We present AREVOL, a dataset for the empirical study of architectural evolution. The dataset comprises two popular systems from the same domain and using the same component model, to make comparative studies possible. Besides the original component metadata, AREVOL includes scripts to obtain simplified models that nevertheless support rich studies of architectural evolution, as the authors' previous work has shown. Read more

The Firefox Temporal Defect Dataset

• Download metadata
• Download FAIR analysis
• Download annotations

Mayy Habayeb, A. Miranskyy, Syed Shariyar Murtaza, Leotis Buchanan, A. Bener

DOI: 10.1109/MSR.2015.73

Publication year: 2015

Topic 7 - Terms: bug, project, repository, report, developer, code, study, single, source, activity

Abstract: The bug tracking repositories of software projects capture initial defect (bug) reports and the history of interactions among developers, testers, and customers. Extracting and mining information f... Abstract: The bug tracking repositories of software projects capture initial defect (bug) reports and the history of interactions among developers, testers, and customers. Extracting and mining information from these repositories is time consuming and daunting. Researchers have focused mostly on analyzing the frequency of the occurrence of defects and their attributes (e.g., The number of comments and lines of code changed, count of developers). However, the counting process eliminates information about the temporal alignment of events leading to changes in the attributes count. Software quality teams could plan and prioritize their work more efficiently if they were aware of these temporal sequences and knew their frequency of occurrence. In this paper, we introduce a novel dataset mined from the Fire fox bug repository (Bugzilla) which contains information about the temporal alignment of developer interactions. Our dataset covers eight years of data from the Fire fox project on activities throughout the project's lifecycle. Some of these activities have not been reported in frequency-based or other temporal datasets. The dataset we mined from the Fire fox project contains new activities, such as reporter experience, file exchange events, code-review process activities, and setting of milestones. We believe that this new dataset will improve analysis of bug reports and enable mining of temporal relationships so that practitioners can enhance their bug-fixing process. Read more

A Novel Industry Grade Dataset for Fault Prediction Based on Model-Driven Developed Automotive Embedded Software

• Download metadata
• Download FAIR analysis
• Download annotations

Harald Altinger, S. Siegl, Y. Dajsuren, F. Wotawa

DOI: 10.1109/MSR.2015.72

Publication year: 2015

Topic 8 - Terms: study, project, model, development, based, repository, source, issue, code, tool

Abstract: In this paper, we present a novel industry dataset on static software and change metrics for Matlab/Simulink models and their corresponding auto-generated C source code. The data set comprises data... Abstract: In this paper, we present a novel industry dataset on static software and change metrics for Matlab/Simulink models and their corresponding auto-generated C source code. The data set comprises data of three automotive projects developed and tested accordingly to industry standards and restrictive software development guidelines. We present some background information of the projects, the development process and the issue tracking as well as the creation steps of the dataset and the used tools during development. A specific highlight of the dataset is a low measurement error on change metrics because of the used issue tracking and commit policies. Read more

Dataset of Developer-Labeled Commit Messages

• Download metadata
• Download FAIR analysis
• Download annotations

Andreas Mauczka, Florian Brosch, Christian Schanes, T. Grechenig

DOI: 10.1109/MSR.2015.71

Publication year: 2015

Topic 3 - Terms: code, review, project, source, present, change, set, github, repository, quality

Abstract: Current research on change classification centers around automated and semi-automated approaches which are based on evaluation by either the researchers themselves or external experts. In most case... Abstract: Current research on change classification centers around automated and semi-automated approaches which are based on evaluation by either the researchers themselves or external experts. In most cases, the persons evaluating the effectiveness of the classification schemes are not the authors of the original changes and therefore can only make assumptions about the intent of the changes. To support validation of existing labeling mechanisms and to provide a training set for future approaches, we present a survey of source code changes that were labeled by their original authors. Seven developers from six different project applied three existing classification schemes from current literature to enrich their own changes with meta-information, so the intent of the changes becomes more evident. The final data set consists of 967 classified changes and is available as an SQLite database as part of the MSR data set. Read more

Fuse: A Reproducible, Extendable, Internet-Scale Corpus of Spreadsheets

• Download metadata
• Download FAIR analysis
• Download annotations

Titus Barik, Kevin Lubick, Justin Smith, John Slankas, E. Murphy-Hill

DOI: 10.1109/MSR.2015.70

Publication year: 2015

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Spreadsheets are perhaps the most ubiquitous form of end-user programming software. This paper describes a corpus, called Fuse, containing 2,127,284 URLs that return spreadsheets (and their HTTP se... Abstract: Spreadsheets are perhaps the most ubiquitous form of end-user programming software. This paper describes a corpus, called Fuse, containing 2,127,284 URLs that return spreadsheets (and their HTTP server responses), and 249,376 unique spreadsheets, contained within a public web archive of over 26.83 billion pages. Obtained using nearly 60,000 hours of computation, the resulting corpus exhibits several useful properties over prior spreadsheet corpora, including reproducibility and extendability. Our corpus is unencumbered by any license agreements, available to all, and intended for wide usage by end-user software engineering researchers. In this paper, we detail the data and the spreadsheet extraction process, describe the data schema, and discuss the trade-offs of Fuse with other corpora. Read more

Landfill: An Open Dataset of Code Smells with Public Evaluation

• Download metadata
• Download FAIR analysis
• Download annotations

Fabio Palomba, Dario Di Nucci, Michele Tufano, G. Bavota, Rocco Oliveto, D. Poshyvanyk, A. D. Lucia

DOI: 10.1109/MSR.2015.69

Publication year: 2015

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: Code smells are symptoms of poor design and implementation choices that may hinder code comprehension and possibly increase change- and fault-proneness of source code. Several techniques have been ... Abstract: Code smells are symptoms of poor design and implementation choices that may hinder code comprehension and possibly increase change- and fault-proneness of source code. Several techniques have been proposed in the literature for detecting code smells. These techniques are generally evaluated by comparing their accuracy on a set of detected candidate code smells against a manually-produced oracle. Unfortunately, such comprehensive sets of annotated code smells are not available in the literature with only few exceptions. In this paper we contribute (i) a dataset of 243 instances of five types of code smells identified from 20 open source software projects, (ii) a systematic procedure for validating code smell datasets, (iii) LANDFILL, a Web-based platform for sharing code smell datasets, and (iv) a set of APIs for programmatically accessing LANDFILL's contents. Anyone can contribute to Landfill by (i) improving existing datasets (e.g., Adding missing instances of code smells, flagging possibly incorrectly classified instances), and (ii) sharing and posting new datasets. Landfill is available at www.sesa.unisa.it/landfill/, while the video demonstrating its features in action is available at http://www.sesa.unisa.it/tools/landfill.jsp. Read more

The MetricsGrimoire Database Collection

• Download metadata
• Download FAIR analysis
• Download annotations

Jesus M. Gonzalez-Barahona, G. Robles, Daniel Izquierdo-Cortazar

DOI: 10.1109/MSR.2015.68

Publication year: 2015

Topic 10 - Terms: repository, issue, information, system, source, project, github, file, evolution, developer

Abstract: The Metrics Grimoire system is composed by a set of tools designed to retrieve data from repositories related to software development. Their aim is to produce organized databases suitable for easy ... Abstract: The Metrics Grimoire system is composed by a set of tools designed to retrieve data from repositories related to software development. Their aim is to produce organized databases suitable for easy querying with research and industrial purposes. The data in those databases have a similar structure, to easy cross-database studies, and can be enriched with information such as linkage of the multiple identities of actors, or their affiliation. This paper presents the general structure of those databases, and a collection of up-to-date database dumps that are publicly available. They correspond to two well-known projects, Open Stack, and Eclipse, including data from source code management repositories, issue tracking systems, mailing lists, and code review systems. Read more

StORMeD: Stack Overflow Ready Made Data

• Download metadata
• Download FAIR analysis
• Download annotations

Luca Ponzanelli, Andrea Mocci, Michele Lanza

DOI: 10.1109/MSR.2015.67

Publication year: 2015

Topic 13 - Terms: source, system, evolution, code, build, set, project, developer, comment, information

Abstract: Stack Overflow is the de facto Question and Answer (Q&A) website for developers, and it has been used in many approaches by software engineering researchers to mine useful data. However, the conten... Abstract: Stack Overflow is the de facto Question and Answer (Q&A) website for developers, and it has been used in many approaches by software engineering researchers to mine useful data. However, the contents of a Stack Overflow discussion are inherently heterogeneous, mixing natural language, source code, stack traces and configuration files in XML or JSON format. We constructed a full island grammar capable of modeling the set of 700,000 Stack Overflow discussions talking about Java, building a heterogeneous abstract syntax tree (H-AST) of each post (question, answer or comment) in a discussion. The resulting dataset models every Stack Overflow discussion, providing a full H-AST for each type of structured fragment (i.e., JSON, XML, Java, Stack traces), and complementing this information with a set of basic meta-information like term frequency to enable natural language analyses. Our dataset allows the end-user to perform combined analyses of the Stack Overflow by visiting the H-AST of a discussion. Read more

A Dataset of the Activity of the Git Super-repository of Linux in 2012

• Download metadata
• Download FAIR analysis
• Download annotations

D. Germán, Bram Adams, A. Hassan

DOI: 10.1109/MSR.2015.66

Publication year: 2015

Topic 10 - Terms: repository, issue, information, system, source, project, github, file, evolution, developer

Abstract: This dataset documents the activity in the public portion of the git Super-repository of the Linux kernel during 2012. In a distributed version control system, such as git, the Super-repository is ... Abstract: This dataset documents the activity in the public portion of the git Super-repository of the Linux kernel during 2012. In a distributed version control system, such as git, the Super-repository is the collection of all the repositories (repos) used for development. In such a Super-repository, some repos will be accessible only by their owners (they are private, and are located in places that are unreachable to other users) while others are available to other members of the team. The latter public repositories are used as avenues through which commits flow from one developer to another. During the last six weeks of 2011, we proceeded to automatically discover the public portion of the Super-repository of Linux. Then, in 2012, every 3 hrs, each of these public repositories was queried to see what new commits it had and what commits had disappeared from it using a process we call continuous mining. This resulted in the identification of 533,513 different commits across 451 different public repositories and how they propagated through the Linux Super-repository, including the repository of Linus Torvalds (i.e., The main repository of the Linux kernel). This information could help us understand how kernel contributors use git, how they collaborate and how commits are integrated into the Linux kernel and into the repositories of organizations that distribute the kernel. Read more

The Debsources Dataset: Two Decades of Debian Source Code Metadata

• Download metadata
• Download FAIR analysis
• Download annotations

Stefano Zacchiroli

DOI: 10.1109/MSR.2015.65

Publication year: 2015

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: We present the Debsources Dataset: distribution metadata and source code metrics spanning two decades of Free and Open Source Software (FOSS) history, seen through the lens of the Debian distributi... Abstract: We present the Debsources Dataset: distribution metadata and source code metrics spanning two decades of Free and Open Source Software (FOSS) history, seen through the lens of the Debian distribution. Debsources is a software platform used to gather, search, and publish on the Web the full source code of the Debian operating system, as well as measures about it. A notable public instance of Debsources is available at http://sources.debian.net, it includes both current and historical releases of Debian. Plugins to compute popular source code metrics (lines of code, defined symbols, disk usage) and other derived data (e.g., Checksums) have been written, integrated, and run on all the source code available on sources.debian.net. The Debsources Dataset is a PostgreSQL database dump of sources.debian.net metadata, as of February 10th, 2015. The dataset contains both Debian-specific metadata -- e.g., which software packages are available in which release, which source code file belong to which package, release dates, etc. -- and source code information gathered by running Debsources plugins. The Debsources Dataset offer a very long-term historical view of the macro-level evolution and constitution of FOSS through the lens of popular, representative FOSS projects of their times. Read more

A Repository with 44 Years of Unix Evolution

• Download metadata
• Download FAIR analysis
• Download annotations

D. Spinellis

DOI: 10.1109/MSR.2015.64

Publication year: 2015

Topic 10 - Terms: repository, issue, information, system, source, project, github, file, evolution, developer

Abstract: The evolution of the Unix operating system is made available as a version-control repository, covering the period from its inception in 1972 as a five thousand line kernel, to 2015 as a widely-used... Abstract: The evolution of the Unix operating system is made available as a version-control repository, covering the period from its inception in 1972 as a five thousand line kernel, to 2015 as a widely-used 26 million line system. The repository contains 659 thousand commits and 2306 merges. The repository employs the commonly used Git system for its storage, and is hosted on the popular GitHub archive. It has been created by synthesizing with custom software 24 snapshots of systems developed at Bell Labs, Berkeley University, and the 386BSD team, two legacy repositories, and the modern repository of the open source FreeBSD system. In total, 850 individual contributors are identified, the early ones through primary research. The data set can be used for empirical research in software engineering, information systems, and software archaeology. Read more

INVocD: Identifier name vocabulary dataset

• Download metadata
• Download FAIR analysis
• Download annotations

Simon Butler, M. Wermelinger, Y. Yu, H. Sharp

DOI: 10.1109/MSR.2013.6624056

Publication year: 2013

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: INVocD is a database of the identifier name declarations and vocabulary found in 60 FLOSS Java projects where the source code structure is recorded and the identifier name vocabulary is made direct... Abstract: INVocD is a database of the identifier name declarations and vocabulary found in 60 FLOSS Java projects where the source code structure is recorded and the identifier name vocabulary is made directly available, offering advantages for identifier name research over conventional source code models. The database has been used to support a range of research projects from identifier name analysis to concept location, and provides many opportunities to researchers. INVocD may be downloaded from http://oro.open.ac.uk/36992. Read more

A dataset for evaluating identifier splitters

• Download metadata
• Download FAIR analysis
• Download annotations

D. Binkley, Dawn J Lawrie, L. Pollock, Emily Hill, K. Vijay-Shanker

DOI: 10.1109/MSR.2013.6624055

Publication year: 2013

Topic 1 - Terms: project, code, source, developer, study, api, model, system, technique, type

Abstract: Software engineering and evolution techniques have recently started to exploit the natural language information in source code. A key step in doing so is splitting identifiers into their constituen... Abstract: Software engineering and evolution techniques have recently started to exploit the natural language information in source code. A key step in doing so is splitting identifiers into their constituent words. While simple in concept, identifier splitting raises several challenging issues, leading to a range of splitting techniques. Consequently, the research community would benefit from a dataset (i.e., a gold set) that facilitates comparative studies of identifier splitting techniques. A gold set of 2,663 split identifiers was constructed from 8,522 individual human splitting judgements and can be obtained from www.cs.loyola.edu/~binkley/ludiso. This set's construction and observations aimed at its effective use are described. Read more

A historical dataset of software engineering conferences

• Download metadata
• Download FAIR analysis
• Download annotations

Bogdan Vasilescu, Alexander Serebrenik, T. Mens

DOI: 10.1109/MSR.2013.6624051

Publication year: 2013

Topic 9 - Terms: repository, bug, source, project, code, information, method, developer, system, present

Abstract: The Mining Software Repositories community typically focuses on data from software configuration management tools, mailing lists, and bug tracking repositories to uncover interesting and actionable... Abstract: The Mining Software Repositories community typically focuses on data from software configuration management tools, mailing lists, and bug tracking repositories to uncover interesting and actionable information about the evolution of software systems. However, the techniques employed and the challenges faced when mining are not restricted to these types of repositories. In this paper, we present an atypical dataset of software engineering conferences, containing historical data about the accepted papers and the composition of programme committees for eleven well-established conferences. The dataset (published on Github at https://github.com/tue-mdse/conferenceMetrics) can be used, e.g., by conference steering committees or programme committee chairs to assess their selection process and compare against other conferences in the field, or by prospective authors to decide in which conferences to publish. Read more

An unabridged source code dataset for research in software reuse

• Download metadata
• Download FAIR analysis
• Download annotations

Werner Janjic, Oliver Hummel, M. Schumacher, C. Atkinson

DOI: 10.1109/MSR.2013.6624047

Publication year: 2013

Topic 13 - Terms: source, system, evolution, code, build, set, project, developer, comment, information

Abstract: This paper describes a large, unabridged data-set of Java source code gathered and shared as part of the Merobase Component Finder project of the Software-Engineering Group at the University of Man... Abstract: This paper describes a large, unabridged data-set of Java source code gathered and shared as part of the Merobase Component Finder project of the Software-Engineering Group at the University of Mannheim. It consists of the complete index used to drive the search engine, www.merobase.com, the vast majority1 of the source code modules accessible through it, and a tool that enables researchers to efficiently browse the collected data. We describe the techniques used to collect, format and store the data set, as well as the core capabilities of the Merobase search engine such as classic keyword-based, interface-based and test-driven search. This data-set, which represents one of the largest searchable collections of source and binary modules available online, has been recently made available for download and use in further research projects. All files are available at http://merobase.informatik.uni-mannheim.de/sources/. Read more

Apache-affiliated Twitter screen names: A dataset

• Download metadata
• Download FAIR analysis
• Download annotations

Megan Squire

DOI: 10.1109/MSR.2013.6624043

Publication year: 2013

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: This paper describes a new dataset containing Twitter screen names for members of the projects affiliated with the Apache Software Foundation (ASF). The dataset includes the confirmed Twitter scree... Abstract: This paper describes a new dataset containing Twitter screen names for members of the projects affiliated with the Apache Software Foundation (ASF). The dataset includes the confirmed Twitter screen names, as well as the real name as listed on Twitter, and the user identification as used within the Apache organization. The paper also describes the process used to collect and clean this data, and shows some sample queries for learning how to use the data. The dataset has been donated to the FLOSSmole project and is available for download (https://code.google.com/p/flossmole/downloads/detail?name=apacheTwitter20 13-Jan.zip) or direct querying via a database client. Read more

Project roles in the Apache Software Foundation: A dataset

• Download metadata
• Download FAIR analysis
• Download annotations

Megan Squire

DOI: 10.1109/MSR.2013.6624042

Publication year: 2013

Topic 0 - Terms: code, source, project, library, tool, android, metric, bug, test, analysis

Abstract: This paper outlines the steps in the creation and maintenance of a new dataset listing leaders of the various projects of the Apache Software Foundation (ASF). Included in this dataset are differen... Abstract: This paper outlines the steps in the creation and maintenance of a new dataset listing leaders of the various projects of the Apache Software Foundation (ASF). Included in this dataset are different levels of committers to the various ASF project code bases, as well as regular and emeritus members of the ASF, and directors and officers of the ASF. The dataset has been donated to the FLOSSmole project under an open source license, and is available for download (https://code.google.com/p/flossmole/downloads/detail?name=apachePeople2013-Jan.zip), or for direct querying via a database client. Read more

The GHTorent dataset and tool suite

• Download metadata
• Download FAIR analysis
• Download annotations

Georgios Gousios

DOI: 10.1109/MSR.2013.6624034

Publication year: 2013

Topic 5 - Terms: github, project, source, vulnerability, apps, development, tool, repository, present, information

Abstract: During the last few years, GitHub has emerged as a popular project hosting, mirroring and collaboration platform. GitHub provides an extensive REST API, which enables researchers to retrieve high-q... Abstract: During the last few years, GitHub has emerged as a popular project hosting, mirroring and collaboration platform. GitHub provides an extensive REST API, which enables researchers to retrieve high-quality, interconnected data. The GHTorent project has been collecting data for all public projects available on Github for more than a year. In this paper, we present the dataset details and construction process and outline the challenges and research opportunities emerging from it. Read more

A network of Rails a graph dataset of Ruby on Rails and associated projects

• Download metadata
• Download FAIR analysis
• Download annotations

Patrick Wagstrom, C. Jergensen, A. Sarma

DOI: 10.1109/MSR.2013.6624033

Publication year: 2013

Topic 12 - Terms: project, source, code, development, clone, set, bug, github, os, datasets

Abstract: Software projects, whether open source, proprietary, or a combination thereof, rarely exist in isolation. Rather, most projects build on a network of people and ideas from dozens, hundreds, or even... Abstract: Software projects, whether open source, proprietary, or a combination thereof, rarely exist in isolation. Rather, most projects build on a network of people and ideas from dozens, hundreds, or even thousands of other projects. Using the GitHub APIs it is possible to extract these relationships for millions of users and projects. In this paper we present a dataset of a large network of open source projects centered around Ruby on Rails. This dataset provides insight into the relationships between Ruby on Rails and an ecosystem involving 1116 projects. To facilitate understanding of this data in the context of relationships between projects, users, and their activities, it is provided as a graph database suitable for assessing network properties of the community and individuals within those communities and can be found at https://github.com/pridkett/gitminer-data-rails. Read more

A historical dataset for the Gnome ecosystem

• Download metadata
• Download FAIR analysis
• Download annotations

M. Goeminne, Maëlick Claes, T. Mens

DOI: 10.1109/MSR.2013.6624032

Publication year: 2013

Topic 9 - Terms: repository, bug, source, project, code, information, method, developer, system, present

Abstract: We present a dataset of the open source software ecosystem Gnome from a social point of view. We have collected historical data about the contributors to all Gnome projects stored on git.gnome.org,... Abstract: We present a dataset of the open source software ecosystem Gnome from a social point of view. We have collected historical data about the contributors to all Gnome projects stored on git.gnome.org, taking into account the problem of identity matching, and associating different activity types to the contributors. This type of information is very useful to complement the traditional, source-code related information one can obtain by mining and analyzing the actual source code. The dataset can be obtained at https://bitbucket.org/mgoeminne/sgl-flossmetric-dbmerge. Read more

The Maven repository dataset of metrics, changes, and dependencies

• Download metadata
• Download FAIR analysis
• Download annotations

S. Raemaekers, A. Deursen, Joost Visser

DOI: 10.1109/MSR.2013.6624031

Publication year: 2013

Topic 9 - Terms: repository, bug, source, project, code, information, method, developer, system, present

Abstract: We present the Maven Dependency Dataset (MDD), containing metrics, changes and dependencies of 148,253 jar files. Metrics and changes have been calculated at the level of individual methods, classe... Abstract: We present the Maven Dependency Dataset (MDD), containing metrics, changes and dependencies of 148,253 jar files. Metrics and changes have been calculated at the level of individual methods, classes and packages of multiple library versions. A complete call graph is also presented which includes call, inheritance, containment and historical relationships between all units of the entire repository. In this paper, we describe our dataset and the methodology used to obtain it. We present different conceptual views of MDD and we also describe limitations and data quality issues that researchers using this data should be aware of. Read more

The Eclipse and Mozilla defect tracking dataset: A genuine dataset for mining bug information

• Download metadata
• Download FAIR analysis
• Download annotations

Ahmed Lamkanfi, Javier Pérez, S. Demeyer

DOI: 10.1109/MSR.2013.6624028

Publication year: 2013

Topic 7 - Terms: bug, project, repository, report, developer, code, study, single, source, activity

Abstract: The analysis of bug reports is an important subfield within the mining software repositories community. It explores the rich data available in defect tracking systems to uncover interesting and act... Abstract: The analysis of bug reports is an important subfield within the mining software repositories community. It explores the rich data available in defect tracking systems to uncover interesting and actionable information about the bug triaging process. While bug data is readily accessible from systems like Bugzilla and JIRA, a common database schema and a curated dataset could significantly enhance future research because it allows for easier replication. Consequently, in this paper we propose the Eclipse and Mozilla Defect Tracking Dataset, a representative database of bug data, filtered to contain only genuine defects (i.e., no feature requests) and designed to cover the whole bug-triage life cycle (i.e., store all intermediate actions). We have used this dataset ourselves for predicting bug severity, for studying bug-fixing time and for identifying erroneously assigned components. Sharing these data with the rest of the community will allow for reproducibility, validation and comparison of the results obtained in bug-report analyses and experiments. Read more

Apache commits: Social network dataset

• Download metadata
• Download FAIR analysis
• Download annotations

Alexander C. MacLean, C. Knutson

DOI: 10.1109/MSR.2013.6624020

Publication year: 2013

Topic 6 - Terms: code, repository, source, project, datasets, file, analysis, developer, language, github

Abstract: Building non-trivial software is a social endeavor. Therefore, understanding the social network of developers is key to the study of software development organizations. We present a graph represent... Abstract: Building non-trivial software is a social endeavor. Therefore, understanding the social network of developers is key to the study of software development organizations. We present a graph representation of the commit behavior of developers within the Apache Software Foundation for 2010 and 2011. Relationships between developers in the network represent collaborative commit behavior. Several similarity and summary metrics have been pre-calculated. The data, along with the tools that were used to create it and some further discussion, can be found at: http://sequoia.cs.byu.edu/lab/?page=artifacts/apacheGraphs. Read more

A dataset from change history to support evaluation of software maintenance tasks

• Download metadata
• Download FAIR analysis
• Download annotations

Bogdan Dit, Andrew Holtzhauer, D. Poshyvanyk, Huzefa H. Kagdi

DOI: 10.1109/MSR.2013.6624019

Publication year: 2013

Topic 12 - Terms: project, source, code, development, clone, set, bug, github, os, datasets

Abstract: Approaches that support software maintenance need to be evaluated and compared against existing ones, in order to demonstrate their usefulness in practice. However, oftentimes the lack of well-esta... Abstract: Approaches that support software maintenance need to be evaluated and compared against existing ones, in order to demonstrate their usefulness in practice. However, oftentimes the lack of well-established sets of benchmarks leads to situations where these approaches are evaluated using different datasets, which results in biased comparisons. In this data paper we describe and make publicly available a set of benchmarks from six Java applications, which can be used in the evaluation of various software engineering (SE) tasks, such as feature location and impact analysis. These datasets consist of textual description of change requests, the locations in the source code where they were implemented, and execution traces. Four of the benchmarks were already used in several SE research papers, and two of them are new. In addition, we describe in detail the methodology used for generating these benchmarks and provide a suite of tools in order to encourage other researchers to validate our datasets and generate new benchmarks for other subject software systems. Our online appendix: http://www.cs.wm.edu/semeru/data/msr13/. Read more

Who does what during a code review? Datasets of OSS peer review repositories

• Download metadata
• Download FAIR analysis
• Download annotations

Kazuki Hamasaki, R. Kula, Norihiro Yoshida, Ana Erika Camargo Cruz, Kenji Fujiwara, Hajimu Iida

DOI: 10.1109/MSR.2013.6624003

Publication year: 2013

Topic 3 - Terms: code, review, project, source, present, change, set, github, repository, quality