Parsing a string into an AST using “normal” functions

Next, we use builders to parse a SQL query from a string, by parsing a SelectClause from a string, which is segmented into key components. The process involves normalizing the input, ensuring that keywords are uppercase, as TypeScript’s type system is case-sensitive. This step is crucial because while databases often ignore case sensitivity, our type inference relies on consistent casing to function correctly.

Once the string is normalized, we split it into tokens and begin parsing, focusing on identifying terminal keywords like SELECT. We process these tokens until we reach a keyword that signifies the end of a section. One challenge here is handling subqueries, where keywords like WHERE might appear within nested queries. To manage this, we count open and closed parentheses to determine whether we’re within a subquery or the main query, ensuring accurate parsing.

Finally, we reconstruct the query by joining tokens and splitting column specifiers, checking for aliasing with patterns like X AS Y or T.X AS Y. This allows us to shape the Abstract Syntax Tree (AST) correctly. While the process is straightforward, it sets the stage for more complex parsing tasks we’ll tackle later, using TypeScript to handle the intricacies of SQL query parsing efficiently.

Validating an AST against a schema to ensure the types are correct

Now that we’ve built the ability to create a query AST with our builders and parse one through functions, the final piece is ensuring these queries are valid against our database schema. Simply having a syntactically correct SQL query doesn’t guarantee it will run, especially if the referenced tables or columns don’t exist. To address this, we focus on type-based validation, using TypeScript’s type system to catch errors at compile time.

We primarily rely on the Invalid type to provide feedback during validation. This process involves a series of conditional types that verify if a query’s components, like table references and columns, are correctly defined within the database schema. If a validation check fails, it returns an Invalid type, triggering TypeScript compilation errors. This approach allows us to catch issues such as missing tables or incorrect columns before they become runtime problems.

One challenge we encounter is ensuring the proper order of elements in SQL statements, especially as we prepare for handling more complex features like ORDER BY clauses. While order isn’t critical for our current validations, it’s something we’ll need to address as we expand our capabilities. For now, our type-based validation effectively ensures that the basic structure of SQL queries aligns with the database schema, setting the stage for more advanced query validation in the future.

Interpreting raw strings as a SQL AST

We have now covered the process of parsing strings to build an AST in code. The general approach involved normalizing the string to maintain consistency, tokenizing it into manageable components, reading through these tokens to identify key structural markers, and recursively processing the tokens to build the AST. We also explored how to implement these steps using TypeScript’s type system, including recursive calls, template string types, and type inference to verify the structural integrity of the parsed SQL.

One challenge we encountered was counting parentheses to correctly handle subqueries within the parsing process. TypeScript’s type system doesn’t support counting in a conventional sense, so we had to devise a workaround. By leveraging TypeScript’s array handling, we can simulate counting by incrementing or decrementing based on the presence of opening and closing parentheses. This approach allows us to accurately track the depth of nested queries, ensuring that our parser processes only the relevant sections of the SQL string.

This counting method is essential as it forms the backbone of our parsing logic, allowing us to string together the concepts we’ve developed so far. With this technique, we can manage the complexities of subqueries and proceed with building a robust SQL parser that operates entirely within TypeScript’s type system, ensuring both correctness and compile-time validation.

Putting it all together

Now that we have all the necessary tools, it’s time to bring everything together to build our simple SELECT parser. Before diving in, there’s an important utility we need to understand that’s responsible for splitting strings into chunks while being aware of parentheses. This utility uses string template typing to identify patterns and count matching parentheses, ensuring safe splits. Although there are edge cases with invalid groupings, we assume the queries provided are generally well-formed, reducing the need for extensive error handling.

To prepare the string for parsing, we first normalize it by manually handling specific characters—like removing unnecessary whitespace or identifying key identifiers such as parentheses and quotes. This normalization process ensures that the query has a predictable and standardized shape, making the subsequent parsing steps more straightforward. Once normalized, we can use specific utility types to parse the query, starting with the ParseSQL type, which focuses on SELECT statements.