Readings

The papers below represent an important, but an incomplete selection of works in the scope of the module. Some, but not all of them will be discussed.

Not interested in presenting any of the papers, or enthusiastic about a topic or paper not represented in the list? Reach out to rigger@nus.edu.sg with suggestions.

Differential Testing

• Data-Oriented Differential Testing of Object-Relational Mapping Systems (ICSE ‘21) [GitHub] [YouTube]

• Finding bugs in Gremlin-based graph database systems via Randomized differential testing (ISSTA ‘21) [GitHub] [YouTube]

• DiffStream: Differential Output Testing for Stream Processing Programs (OOPSLA ‘20) [GitHub] [YouTube]

• APOLLO: Automatic Detection and Diagnosis of Performance Regressions in Database Systems (VLDB ‘19) [GitHub] [YouTube],

Metamorphic Testing

• Metamorphic testing of Datalog engines (ESEC/FSE ‘21) [GitHub] [YouTube]

• Finding bugs in database systems via query partitioning (OOPSLA ‘20) [GitHub] [YouTube]

• Detecting optimization bugs in database engines via non-optimizing reference engine construction (ESEC/FSE ‘20) [GitHub] [YouTube]

• Automatic Detection of Performance Bugs in Database Systems using Equivalent Queries (ICSE ‘22)

Isolation-level Testing

Recommended primarily for presenters with knowledge of isolation levels.

• Elle: inferring isolation anomalies from experimental observations (VLDB ‘20) [GitHub] [YouTube]

• Cobra: Making Transactional Key-Value Stores Verifiably Serializable (OSDI ‘20) [YouTube]

• Seeing is Believing: A Client-Centric Specification of Database Isolation (PODC ‘17)

Other Testing and Test Oracle Approaches

• Testing Database Engines via Pivoted Query Synthesis (OSDI ‘20) [GitHub] [YouTube]

• Torturing Databases for Fun and Profit (OSDI ‘14)

Test Case Generation & Fuzzing

• Detecting Logical Bugs of DBMS with Coverage-based Guidance (Usenix Security ‘22) [GitHub]

• SQUIRREL: Testing Database Management Systems with Language Validity and Coverage Feedback (CCS ‘20)

• 🆕 BigFuzz: Efficient Fuzz Testing for Data Analytics Using Framework Abstraction (ASE ‘20) [YouTube]

• Search-based test data generation for SQL queries (ICSE ‘18) [GitHub] [YouTube]

• Query-aware Test Generation Using a Relational Constraint Solver (ASE ‘08)

• 🆕 Dynamic test input generation for database applications (ISSTA ‘07)

• QAGen: generating query-aware test databases (SIGMOD ‘07)

• Flexible Database Generators (VLDB ‘05)

Debugging and IDEs

• Debugging Database Queries: A Survey of Tools, Techniques, and Users (CHI ‘20) [YouTube]

• SQLCheck: Automated Detection and Diagnosis of SQL Anti-Patterns (SIGMOD ‘20) [GitHub] [YouTube]

• View-Centric Performance Optimization for Database-Backed Web Applications (ICSE ‘19)

• 🆕 Explaining Wrong Queries Using Small Examples (SIGMOD ‘19)

• 🆕 QFix: Diagnosing Errors through Query Histories (SIGMOD ‘17)

• 🆕 DBSherlock: A Performance Diagnostic Tool for Transactional Databases (SIGMOD ‘16)

• CHECKCELL: Data Debugging for Spreadsheets (OOPSLA ‘14)

• 🆕 Observing SQL queries in their natural habitat (TODS ‘13)

• How to ConQueR Why-Not Questions (SIGMOD ‘10)

Formal Methods and Verification

Recommended primarily for presenters with a background in PL semantics.

• Synthesizing Analytical SQL Queries from Computation Demonstration (PLDI ‘22) [Zenodo] [YouTube]

• Information-Flow Control for Database-backed Applications (EuroS&P ‘19)

• Cypher: An Evolving Query Language for Property Graphs (SIGMOD ‘18)

• 🆕 HoTTSQL: Proving Query Rewrites with Univalent SQL Semantics (PLDI ‘17)

• A formal semantics of SQL queries, its validation, and applications (VLDB ‘17)

• Optimal Security-Aware Query Processing (VLDB ‘14)

• A Coq Formalization of the Relational Data Model (ESOP ‘14)

• Static and Dynamic Semantics of NoSQL Languages (POPL ‘13)

• Toward a verified relational database management system (POPL ‘10)

Compilation & Optimization

• 🆕 One WITH RECURSIVE is Worth Many GOTOs (SIGMOD ‘21)

• 🆕 Evaluating End-to-End Optimization for Data Analytics Applications in Weld (VLDB ‘19)

• 🆕 AIDA - Abstraction for Advanced In-Database Analytics (VLDB ‘18)

• 🆕 Froid: Optimization of Imperative Programs in a Relational Database (VLDB ‘17)

• 🆕 Language-Agnostic Integrated Queries in a Managed Polyglot Runtime (VLDB ‘14)

Empirical Studies

• Managing data constraints in database-backed web applications (ICSE ‘20) [YouTube]

• How not to structure your database-backed web applications: a study of performance bugs in the wild (ICSE’18)

• How Good Are Query Optimizers, Really? (VLDB ‘15)

Miscellaneous

• Data Generation for Testing and Grading SQL Queries (VLDB ‘15)

• Query reverse engineering (VLDB ‘14)

• Query by output (SIGMOD ‘09)

• Reverse Query Processing (ICDE ‘07)