From 3eeb7eee9013e2c034fb54e32395707364f93412 Mon Sep 17 00:00:00 2001 From: Jan-Niclas Loosen Date: Sat, 26 Sep 2026 18:14:49 +0200 Subject: [PATCH] refactor: split TextReplacer into HtmlTokens and StoredForms (M1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - HtmlTokens (static, with the S45 docblock): SPLIT, RAW_OPEN, tokenize(), structure(), closes(), unterminated() moved unchanged; new RAW_ELEMENTS list. - StoredForms: of() (was TextReplacer::forms), mentions(), unicodeEscaped() – the DB prefilter forms; TextReplacer::mentions() delegates, the ability's candidate query uses StoredForms::of(). - TextReplacer keeps problems(), apply() and the splice/safe() verification; safe() still runs the tokenizer and HtmlContext as two independent scans, in the same order. HtmlTokensTest asserts RAW_ELEMENTS === HtmlContext::RAW and that RAW_OPEN opens exactly those elements. TextReplacerTest and HtmlContextTest unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01At3V6SjwbMe6eMzVGW3Hs7 --- src/Integrations/Divi/Compose/HtmlContext.php | 4 +- src/Integrations/Divi/Compose/HtmlTokens.php | 134 ++++++++++++++ src/Integrations/Divi/Compose/StoredForms.php | 58 ++++++ .../Divi/Compose/TextReplacer.php | 165 +----------------- .../Divi/Pages/DiviReplaceTextAbility.php | 5 +- .../Divi/Compose/HtmlTokensTest.php | 34 ++++ 6 files changed, 239 insertions(+), 161 deletions(-) create mode 100644 src/Integrations/Divi/Compose/HtmlTokens.php create mode 100644 src/Integrations/Divi/Compose/StoredForms.php create mode 100644 tests/Unit/Integrations/Divi/Compose/HtmlTokensTest.php diff --git a/src/Integrations/Divi/Compose/HtmlContext.php b/src/Integrations/Divi/Compose/HtmlContext.php index 9301494..ca581bc 100644 --- a/src/Integrations/Divi/Compose/HtmlContext.php +++ b/src/Integrations/Divi/Compose/HtmlContext.php @@ -6,7 +6,7 @@ namespace LIT\MCP\Integrations\Divi\Compose; /** * An independent second scan of an HTML fragment (ruling S45): a small state machine after the - * HTML tokenizer rules, written separately from TextReplacer's regex tokenizer so a mistake in + * HTML tokenizer rules, written separately from the regex tokenizer HtmlTokens so a mistake in * one is caught by the other. It lists the byte spans that are NOT text: tags (with quoted * attribute values, where ">" does not end the tag), comments, bogus comments and the bodies of * raw-text elements (script, style, …). Where it is unsure it errs towards "not text": an @@ -15,7 +15,7 @@ namespace LIT\MCP\Integrations\Divi\Compose; */ final class HtmlContext { - /** Elements whose body the browser does not parse as text/markup. */ + /** Elements whose body the browser does not parse as text/markup (= HtmlTokens::RAW_ELEMENTS). */ public const RAW = ['script', 'style', 'xmp', 'iframe', 'noembed', 'noframes', 'noscript', 'plaintext']; /** @return list [start, end) byte spans that are markup or raw bodies. */ diff --git a/src/Integrations/Divi/Compose/HtmlTokens.php b/src/Integrations/Divi/Compose/HtmlTokens.php new file mode 100644 index 0000000..83a8700 --- /dev/null +++ b/src/Integrations/Divi/Compose/HtmlTokens.php @@ -0,0 +1,134 @@ +" inside a quoted attribute value does not end the tag), comments, + * entities and $variable(…)$ references ("token"), the bodies of raw-text elements such as + *