feat(extract): opt-in --html-as-code to route .html through the AST path (#1230)#2096
Open
Yyunozor wants to merge 1 commit into
Open
feat(extract): opt-in --html-as-code to route .html through the AST path (#1230)#2096Yyunozor wants to merge 1 commit into
Yyunozor wants to merge 1 commit into
Conversation
…ath (Graphify-Labs#1230) .html is always classified as DOCUMENT (detect.DOC_EXTENSIONS), so it only goes through the semantic (LLM) path, and is simply skipped under --code-only. For a project whose .html carries real logic (Angular templates, generated dashboards, meaningful inline <script>), that logic never enters the graph. Adds a --html-as-code flag to `graphify extract`, off by default: - detect.classify_file() takes an html_as_code param; when set, .html classifies as CODE instead of DOCUMENT. - New extract_html() mirrors extract_vue/extract_svelte/extract_astro: mask everything outside inline <script> bodies (preserving newlines so source_location stays correct in the host file), then parse the masked source with the plain JS grammar. No new dependency. - <script src="..."> (external) and non-JS type="..." blocks are blanked, not parsed. - The flag persists into update/watch/hook rebuilds via the existing .graphify_build.json mechanism, with .html explicitly excluded from the unrelated doc-AST-fallback path in watch.py when the flag is off (that path's `_get_extractor(p) is not None` check would otherwise silently pick up .html once it has a registered extractor, regardless of the flag). Full suite: 3395/3381 passed vs. parent (delta = the 14 new tests), same single pre-existing unrelated failure both sides. ruff clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1230.
Problem
.htmlis always classified asDOCUMENT(detect.py'sDOC_EXTENSIONS), so it only goes through the semantic (LLM) extraction path — never the local AST path, even under--code-only, where it's simply skipped. For a project whose.htmlcarries real logic (Angular templates, generated dashboards, anything with meaningful inline<script>), that logic is invisible to the graph:The workaround suggested in-thread (
.graphifyignorethe.htmlfiles, @LuisPeregrina) just removes them from the graph entirely rather than making them useful.Fix
Opt-in
--html-as-codeflag ongraphify extract, off by default (zero behavior change for existing users):detect.classify_file()takes anhtml_as_codeparam; when set,.htmlclassifies asCODEinstead ofDOCUMENT, checked before theDOC_EXTENSIONS/paper-heuristic branch so an opted-in file isn't second-guessed by the paper sniffer.extract_html()mirrors the existingextract_vue/extract_svelte/extract_astroextractors: mask everything outside inline<script>bodies (preserving newlines, sosource_locationstays correct in the host file), then feed the masked source to the plain JS grammar. No new dependency — reuses the masking technique already shipping for three other markup+JS host languages, instead of addingtree-sitter-html.<script src="...">(external, no local body) and non-JStype="..."blocks (e.g.application/json) are blanked, not parsed — they contribute nothing rather than producing garbage nodes.update/watch/hook rebuilds via the existing.graphify_build.jsonmechanism (same pattern as--exclude/--no-gitignore,--excludepatterns only apply to the initial scan — update/watch/hook rebuilds silently re-include excluded paths #1886/Allow extracting git-ignored code: add--no-gitignoreopt-out #1971), and is threaded so.html's registered extractor can't leak into the unrelated doc-AST-fallback path inwatch.pywhen the flag is off.Validation
<script>, externalsrc=, non-JStype=, multiple blocks, call-graph + line numbers), the opt-in contract (off → unchanged, on → extracted), persistence acrossupdate.e32c9f4): 3395/3381 passed, 1/1 failed (same pre-existing, unrelated: missing optionalbuilddev dependency), 137/137 skipped — delta is exactly the 14 added tests.ruff checkclean.callsedges under--html-as-code— e.g.updateCamera()callingmatPerspective()— with line numbers matching the source.Scope
Deliberately minimal: JS extraction, not full HTML/DOM structure. The CSS-out-of-scope position from discussion #1625 is untouched. Structural HTML parsing is a natural follow-up on this foundation, not bundled here.