Static analysis of expl3 programs (8¾): Semantic analysis for functions and what's taking so long

Today, I’m excited to announce another update to expltools, the bundle that includes the static analysis tool explcheck for the expl3 programming language. This release significantly extends support for the semantic analysis processing step.

In this post, I’ll explain what this means for expl3 programmers, what functionality is still missing, and provide a realistic estimate for when full support for semantic (and flow) analysis can be expected. I’ll also discuss how this impacts the project roadmap.

Additionally, I’d like to tease a work-in-progress article titled Expltools: Development tools for expl3 programmers, forthcoming in TUGboat 46(2) or (3), along with my upcoming talk this July at TUG 2025 in India.

Finally, as a follow-up to the previous two posts, in which I described how I collaborated with package authors to fix issues detected by explcheck, I’ll provide updates on the status of those tickets.

What are the new features?

As discussed in my previous post, the initial semantic analysis implementation laid the foundation but did not yet report any new issues. The latest release changes that, introducing support for six issues from Section 4.1 of Warnings and errors for the expl3 analysis tool:

W401: Unused private function
W402: Unused private function variant
T403: Function variant of incompatible type
E404: Protected predicate function
E405: Function variant for an undefined function
W407: Multiply defined function variant

This accounts for roughly one quarter of all issues planned for the semantic analysis.

What is taking so long?

The original project proposal assumed that each of the five processing steps would take about two months to implement, with a two-month buffer for unforeseen work. This assumption held for the first three steps:

Preprocessing: 2 months (Oct – Nov 2024)
Lexical analysis: 1.5 months (Dec – mid Jan 2025)
Housekeeping and bugfixes: 1.5 months (mid Jan – Feb)
Syntactic analysis: 1 month (March)

By contrast, semantic analysis has already taken two full months, and we’re only a quarter of the way through its planned issues—despite the fact that I’ve spent significantly more time on the project this month than in any of the previous seven.

Why the delay? The short answer is: semantic analysis is hard. This step is responsible for approximately 36% of all planned issues—nearly double what would be expected if the issues were evenly distributed.

Moreover, it’s the first step where all accumulated knowledge must be combined to detect higher-level problems. While previous steps focused on parsing and data collection, semantic analysis is where the real analysis begins.

It may even be more challenging than the final flow analysis step, where full implementation may not be feasible due to gaps in knowledge, as discussed in Section 4.3 of the proposal.

If we weight the remaining steps by the number of planned issues (halving the expected effort for flow analysis) and fit the work into the original 10-month timeframe, we arrive at this estimate:

Semantic analysis: 4 months (April – July, concluding around TUG 2025)
Flow analysis: 2 months (Aug – Sep)

However, this may still be optimistic. If it takes two months to complete just one quarter of semantic analysis issues, a pessimistic projection would suggest a total of eight months—pushing the final release of explcheck to January 2026.

Working with the community

Since March, I’ve been actively reaching out to package maintainers and developers to report issues identified by explcheck. Out of eleven reported issues, seven have been resolved:

As of today’s release, no additional errors have been found in TeX Live packages with public repositories, only warnings.

Thank you for your attention and stay tuned for more updates as explcheck continues to evolve!

Written on May 29, 2025
Last updated on May 29, 2025