Efficient in-memory processing of SQL queries with JIT compilation

Abstract

Database systems are vital to the modern world. The conventional approach to SQL query execution is to convert a SQL query into a plan tree of relational algebra operators and then interpret them over each tuple. This method has an advantage when the bottleneck is disk I/O. However, modern advances in hardware have led to faster storage systems and large main memory capacities. With in-memory query processing, the mentioned traditional approach based becomes a performance bottleneck by consuming a significant portion of query execution time.

Therefore, the thesis introduces a compilation-based in-memory database system. It leverages the advantages of intermediate representation code generation for scan, filter, group-by, sort-by, aggregation, and join operations of SQL queries with Just-In-Time compilation using the Multi-level Intermediate Representation framework. Evaluation shows that compared with a conventional database system (PostgreSQL) and a high-level language (C++) code generating query processor, our proposed system performs significantly better.