Today, I’m excited to release the next major update to expltools, a bundle that includes the static analysis tool explcheck for the expl3 programming language. This update finishes the semantic analysis, completing four out of five planned processing steps.

What is semantic analysis?

Syntactic analysis is the stage where expl3 function calls identified during syntactic analysis (as discussed in my previous blog post) are classified as statements.

For example, consider the following expl3 code snippet:

\tl_new:N
  \l_example_tl
\tl_set:Nn
  \l_example_tl
  { foo~bar }

In this example, semantic analysis successfully recognizes two statements:

1. A variable \l_example_tl is declared.
2. The variable \l_example_tl is defined as “foo bar”.

Additionally, it warns that the variable \l_example_tl seems unused.

The semantic analysis is the most complex processing step thus far and took four months to complete, as detailed in two of my previous blog posts.

What’s next? Flow analysis

In the following months, I plan to implement flow analysis. This next step aims to determine the data and control flows in expl3 programs and answer advanced questions, such as:

• “Is there any dead code that is never executed?”
• “Is a boolean expression fully-expandable?”
• “Do we always open a stream before accessing it and close it afterwards?”

“I’m in over my head!”

Unlike the semantic analysis, which is mostly context-free and can be conducted even when large parts of the expl3 program are not understood, the flow analysis requires near-complete understanding of the program, which we usually only have for the simplest of programs at this point.

I foresaw this problem in the project proposal and suggested the following approach:

The initial version of the linter may not have a sufficient understanding of expl3 code to support proper flow analysis. Instead, the initial version of the linter may need to use pseudo-flow-analysis that would check for simple cases of the warnings and errors from flow analysis. Future versions of the linter should improve their code understanding to the point where proper flow analysis can be performed.

However, this solution would require building a separate processing step (pseudo-flow-analysis) that would need to be fully replaced later, which doesn’t seem economical.

Instead, the ticket witiko/expltools#124 introduced a new mechanism that allows a processing step to say “I’m in over my head!” and stop the processing early. This way, only code that is well-understood will be processed by the flow analysis, whereas more complex code will stop at the semantic analysis.

Get involved!

Your feedback is invaluable. Feel free to contribute or share your thoughts by visiting the project repository. Stay tuned for more updates as explcheck continues to evolve!