AST Transformations and Transpiler Internals
An in-depth exploration of ForgeWP's AST-powered compiler engine: Babel and TypeScript syntax trees, static expression evaluation, dual-host transpilation to PHP and React.createElement, and AST-driven island extraction.
The ForgeWP compiler does not rely on brute-force string substitution or regular expressions to translate themes. Converting modern React 19, TypeScript, and JSX into native WordPress PHP templates and Gutenberg editor bundles requires deep syntactic understanding of the source code.
To accomplish this, ForgeWP uses an Abstract Syntax Tree (AST) pipeline. An AST is a tree representation of source code where every variable declaration, JSX element, type annotation, and function call is represented as a distinct node with precise character boundaries.
This architecture ensures that nested expressions, generic TypeScript parameters, and complex conditional layouts compile cleanly into valid PHP and JavaScript without syntax errors or runtime overhead.
Why an AST Engine Replaced Regular Expressions
Early prototype compilers often attempt to transform code using regular expressions or naive bracket matching. In real-world projects, text-based scanning inevitably breaks down when facing idiomatic modern TypeScript and JSX:
1. The Generic Cast Boundary Problem
When a component uses a TypeScript type assertion such as as Partial<Props>, a naive regex matching <WpEditable[^>]*> treats the closing angle bracket > inside Partial<Props> as the end of the HTML tag. This prematurely cuts the match in half, leaving unstripped tags in the output and corrupting the template.
2. Nested Ternaries and Logical Operators
Transforming conditional JSX (condition && <Element /> or isExpanded ? <Open /> : <Closed />) into WordPress PHP if blocks requires knowing the exact boundaries of both the condition expression and the JSX branches. Brackets and quotes inside strings or callback parameters break regex-based scanners.
3. Precise Byte-Range Splicing
By using an AST, ForgeWP obtains exact [start, end) byte offsets for every syntax construct. The compiler can remove TypeScript type annotations or extract interactive subtrees with mathematical precision, without modifying unrelated code nearby.
Note
ForgeWP employs a dual-parser design: @babel/parser (configured with JSX and TypeScript plugins) powers block scanning, static evaluation, and code generation, while the native TypeScript Compiler API (typescript) powers type-aware interactivity inspection and island splitting.
The Shared AST Entry Point (ast-parser.js)
All block scanning, sanitization, and transpilation modules share a single, standardized Babel parser entry point defined in packages/compiler/lib/blocks/ast-parser.js.
Typ1import { parse } from '@babel/parser';
2import _traverse from '@babel/traverse';
3
4export const traverse = _traverse.default || _traverse;
5
6const PARSE_OPTIONS = {
7 sourceType: 'module',
8 plugins: ['jsx', 'typescript'],
9 allowReturnOutsideFunction: true,
10 allowAwaitOutsideFunction: true,
11 allowSuperOutsideMethod: true,
12 errorRecovery: false,
13};
14
15export function parseSource(code) {
16 return parse(code, PARSE_OPTIONS);
17}
18
19export function tryParseSource(code) {
20 try {
21 return parseSource(code);
22 } catch {
23 return null;
24 }
25}Parser Options Rationale
Option Setting Architectural Purpose sourceType'module'Parses ES module imports, exports, and top-level statements. plugins['jsx', 'typescript']Enables native parsing of TSX files without requiring separate pre-compilation passes. allowReturnOutsideFunctiontrueAllows parsing standalone code fragments (such as component render bodies extracted during intermediate compiler steps) without throwing syntax errors. errorRecoveryfalseEnsures that malformed syntax immediately surfaces during compilation rather than producing corrupt PHP output. |
|---|
Static AST Evaluation (evaluateStaticNode)
When ForgeWP scans components to discover Gutenberg blocks, attribute defaults, or editable field schemas, it needs to evaluate data structures declared in code.
Using JavaScript's eval() or new Function() for this purpose is dangerous and unreliable: it can execute untrusted code at build time and breaks when encountering imported identifiers or browser globals.
Instead, ForgeWP's scanner.js implements a pure-data recursive AST evaluator (evaluateStaticNode). This function walks Babel AST nodes directly and resolves their values without executing code:
Typ1function evaluateStaticNode(node, bindings = {}, callables = {}) {
2 switch (node.type) {
3 case 'StringLiteral':
4 case 'NumericLiteral':
5 case 'BooleanLiteral':
6 return node.value;
7 case 'NullLiteral':
8 return null;
9 case 'Identifier':
10 if (node.name === 'undefined') return undefined;
11 if (Object.prototype.hasOwnProperty.call(bindings, node.name))
12 return bindings[node.name];
13 throw new Error(`unresolvable identifier "${node.name}"`);
14 case 'TemplateLiteral':
15 if (node.expressions.length === 0) return node.quasis[0].value.cooked;
16 throw new Error('template literal with dynamic expressions');
17 case 'TSAsExpression':
18 case 'TSSatisfiesExpression':
19 case 'ParenthesizedExpression':
20 return evaluateStaticNode(node.expression, bindings, callables);
21 case 'ArrayExpression':
22 return node.elements.map(el => evaluateStaticNode(el, bindings, callables));
23 case 'ObjectExpression': {
24 const obj = {};
25 for (const prop of node.properties) {
26 if (prop.type === 'ObjectProperty' && !prop.computed) {
27 const key = prop.key.type === 'Identifier' ? prop.key.name : String(prop.key.value);
28 obj[key] = evaluateStaticNode(prop.value, bindings, callables);
29 }
30 }
31 return obj;
32 }
33 case 'CallExpression': {
34 if (node.callee.type === 'Identifier' && callables[node.callee.name]) {
35 const args = node.arguments.map(a => evaluateStaticNode(a, bindings, callables));
36 return callables[node.callee.name](...args);
37 }
38 throw new Error(`call to unsupported helper: ${node.callee.name}`);
39 }
40 default:
41 throw new Error(`unsupported static node type: ${node.type}`);
42 }
43}What Static Evaluation Resolves Safely
- Literal primitives: strings, numbers, booleans, and
null. - Object and array definitions, including nested structures and object spread (
...defaults). - TypeScript wrappers:
as const,satisfies, and type assertions are transparently unwrapped to their underlying expressions. - Whitelisted schema helpers: invocations of
defineBlock(),defineEditable(),text(),richText(),image(), andselect().
Tip
Because evaluateStaticNode resolves values purely through AST traversal, schema evaluation cannot cause infinite loops, cannot access the filesystem, and fails predictably with descriptive errors if an author attempts to use runtime calculations inside a static schema.
Source Sanitization and TypeScript Stripping
Gutenberg block edit controls run inside the browser in /wp-admin. Because the browser does not execute raw TypeScript, type annotations must be stripped.
Rather than running a heavy multi-step compilation pass that transforms the entire file structure, source-sanitize.js uses AST byte-range splicing to surgically remove TypeScript-only syntax while preserving the developer's original code layout:
Typ1export function stripTypeScriptSyntax(code) {
2 const ast = tryParseSource(code);
3 if (ast) {
4 try {
5 const ranges = collectTsStripRanges(ast, code);
6 return spliceRanges(code, ranges);
7 } catch {
8 // Fallback path if parsing fails
9 }
10 }
11 return stripTypeScriptSyntaxLegacy(code);
12}Spliced Syntax Constructs
The AST visitor traverses the code and identifies byte ranges for:
TSInterfaceDeclarationandTSTypeAliasDeclarationstatements.TSTypeAnnotationnodes attached to variables, function parameters, and return types.- Type assertions:
TSAsExpression,TSSatisfiesExpression, and non-null assertions (!). - Generic type argument lists attached to function invocations:
useState<string>()becomesuseState(). - Type-only import statements:
import type { Props } from './types'.
The collected ranges are sorted in reverse offset order and spliced out of the source string. Because the byte offsets come directly from the AST parser, adjacent operators, string literals, and JSX attributes remain completely untouched.
The PHP Transpiler Engine (php-transpiler.js)
The core transpiler module (packages/compiler/lib/blocks/php-transpiler.js) translates recognized React component subtrees into standard WordPress PHP.
1. Loop Transpilation (transpileLoops)
When an author maps over an array in JSX (items.map((item, index) => <li key={index}>{item.title}</li>)), transpileLoops parses the call expression and emits native PHP foreach statements:
Array Mapping Transpilation
TSX1<ul className="space-y-3">
2 {features.map((feature, idx) => (
3 <li key={idx} className="flex items-center gap-2">
4 <CheckIcon className="w-5 h-5 text-emerald-500" />
5 <span>{feature.name}</span>
6 </li>
7 ))}
8</ul>
PHP1<ul class="space-y-3">
2 <?php if (!empty($attributes['features']) && is_array($attributes['features'])) : ?>
3 <?php foreach ($attributes['features'] as $idx => $feature) : ?>
4 <li class="flex items-center gap-2">
5 <svg class="w-5 h-5 text-emerald-500" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
6 <polyline points="20 6 9 17 4 12"></polyline>
7 </svg>
8 <span><?php echo esc_html($feature['name'] ?? ''); ?></span>
9 </li>
10 <?php endforeach; ?>
11 <?php endif; ?>
12</ul>2. Conditionals and Ternaries (transpileConditionals & transpileTernaries)
Short-circuit logical expressions and ternary operators compile into idiomatic PHP conditional blocks:
Conditional Rendering Transpilation
TSX1{badge && (
2 <span className="badge">{badge}</span>
3)}
4{isPublished ? (
5 <p className="status-live">Live</p>
6) : (
7 <p className="status-draft">Draft</p>
8)}1<?php if (!empty($attributes['badge'])) : ?>
2 <span class="badge"><?php echo esc_html($attributes['badge']); ?></span>
3<?php endif; ?>
4
5<?php if (!empty($attributes['isPublished'])) : ?>
6 <p class="status-live">Live</p>
7<?php else : ?>
8 <p class="status-draft">Draft</p>
9<?php endif; ?>3. JavaScript Expression Translation (translateJsExpressionToPhp)
Expressions inside JSX curly braces ({}) are translated according to specific syntax rules:
- Variable lookups (
attributes.title) translate to associative array accesses ($attributes['title']). - Optional chaining (
post?.meta?.subtitle) translates to safe null-coalescing operations (($post['meta']['subtitle'] ?? '')). - String concatenation (
"Total: " + count) translates to PHP concatenation ('Total: ' . $count). - Equality checks (
status === 'active') translate to PHP comparisons ($status === 'active').
4. Stripping WpEditable Wrappers
In the Gutenberg block editor, editable areas require <WpEditable name="..." /> tags to provide inline typing and attribute updating. During server-side PHP template generation, stripWpEditableBlocks unrolls these wrappers, rendering only the underlying escaped field value or raw HTML without editor attributes.
The Editor Transpiler Engine (editor-transpiler.js)
While php-transpiler.js generates backend PHP files, editor-transpiler.js prepares components to execute inside Gutenberg's block editor interface (/wp-admin).
JSX to React.createElement Conversion
Gutenberg bundles a pre-compiled version of React within window.wp.element. generateReactCreateElement translates JSX trees into standard createElement function calls:
Jav1// Input: <section className="hero"><h1 className="title">{attributes.heading}</h1></section>
2
3// Output emitted for Gutenberg editor bundle:
4createElement("section", { className: "hero" },
5 createElement("h1", { className: "title" }, attributes.heading)
6)Inlining Icons into Native SVGs
Modern frontend themes often import icons from libraries like lucide-react or @radix-ui/react-icons. If these icon components were bundled directly into the block editor script, editor bundles would balloon by hundreds of kilobytes.
Instead, resolveIconToCreateElement resolves the icon name at compile time against ForgeWP's icon cache, reads the optimized SVG markup, and converts it into a lightweight, inline createElement("svg", ...) tree:
Typ1export function resolveIconToCreateElement(componentName, packageName, themeRoot, classNameStr) {
2 const svgHtml = resolveIconToSvgHtml(componentName, packageName, themeRoot, classNameStr);
3 if (!svgHtml) return null;
4 return svgHtmlToCreateElement(svgHtml, classNameStr);
5}This transformation normalizes HTML/SVG attributes to standard React props (stroke-width becomes strokeWidth, class becomes className, and fill-rule becomes fillRule), ensuring seamless rendering inside the block editor iframe.
AST-Driven Island Splitting (island-split.js)
When authoring a complex page template, developers frequently combine static presentation elements (headers, text blocks, grids) with small interactive islands (a newsletter popup, an accordion FAQ, or a search input).
Shipping the entire page template as a client-side React bundle would destroy page speed and defeat the Zero-PHP philosophy.
ForgeWP's island-split.js uses TypeScript's compiler AST to solve this through automated island extraction:
The Splitting Workflow
- Candidate Discovery: The compiler walks
src/app,src/components, andsrc/blocks. - Scope Analysis:
collectAllBoundNamesDeep()identifies all variable declarations, parameters, and imports available in the component scope. - Free Identifier Resolution: For an interactive subtree (e.g. an interactive modal inside a static hero), the AST walker determines which free identifiers (props or local variables) the subtree references.
- Island File Generation: The extracted subtree is written to a dedicated file inside
.forgewp-islands/(e.g.,.forgewp-islands/HeroInteractive.tsx), exporting an isolated component that accepts those identified variables as props. - Parent Shell Replacement: In the parent component, the interactive subtree is replaced with:
TSX1<Hydrate island="HeroInteractive" props={{ count, title }}>
2 <HeroInteractive count={count} title={title} />
3 </Hydrate>- Virtual Memory Overrides (
contentOverrides): Rather than modifying the developer's source file on disk,is-interactive.jsregisters an in-memory override. When downstream compiler passes read the component source, they see the static shell with the<Hydrate>boundary, while the original file remains clean in the developer's workspace.
Diagnostic Telemetry and Fallback Tracing
To ensure that the compilation pipeline maintains strict AST integrity across framework updates, ForgeWP includes built-in fallback tracing telemetry in ast-parser.js.
Tracking Fallbacks with FORGEWP_TRACE_FALLBACKS
Running an export with the FORGEWP_TRACE_FALLBACKS=1 environment variable enables real-time tracking of any code path that falls back to legacy text scanners:
BasFORGEWP_TRACE_FALLBACKS=1 pnpm forgewp exportWhen compilation finishes, the compiler prints a diagnostics report to stderr:
[fallback-trace] --- legacy fallback hits ---
(none — every code path stayed on the AST engine)If any hit counts are recorded (e.g., php-transpiler:extractJsx-would-have-tried-legacy: 2), it indicates that a non-standard syntax pattern was encountered, providing immediate insight into edge cases that require parser refinement.
Strict Mode Schema Validation
When building for production in CI/CD pipelines, use the --strict flag:
Baspnpm forgewp export --validate --strictIn strict mode, flushEditableIssues() treats any unresolved editable binding, mismatched field name, or unparseable schema helper as a hard build error, preventing invalid themes from reaching production servers.
Summary of Compiler AST Modules
Module Location Primary Responsibility ast-parser.jslib/blocks/Unified @babel/parser configuration, AST traversal, and fallback telemetry.scanner.jslib/blocks/Static AST evaluation ( evaluateStaticNode) of block schemas and editable field contracts.source-sanitize.jslib/blocks/Byte-range splicing of TypeScript syntax and module-level helper function extraction. php-transpiler.jslib/blocks/Transforms JSX into WordPress PHP, loops into foreach, and expressions into PHP variables.editor-transpiler.jslib/blocks/Transforms JSX into React.createElement() and Lucide icons into normalized SVGs for Gutenberg.is-interactive.jslib/hydration/TypeScript AST analysis detecting hooks, DOM events, and client animation libraries. island-split.jslib/hydration/AST-driven extraction of interactive subtrees into .forgewp-islands/ chunks. |
|---|
Next Steps
Now that you have explored the compiler architecture, translation contracts, and low-level AST transpiler internals, proceed to the React API Reference to review the complete type specifications, signatures, and prop contracts for all @forgewp/react hooks and primitives.