Skip to content

[codex] Add plain text fallback for unsupported extensions#581

Merged
buger merged 2 commits into
mainfrom
codex/plain-text-extension-fallback-580
Jun 27, 2026
Merged

[codex] Add plain text fallback for unsupported extensions#581
buger merged 2 commits into
mainfrom
codex/plain-text-extension-fallback-580

Conversation

@buger

@buger buger commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Fixes #580.

Summary

  • Added plain-text fallback handling for unsupported extensions in symbols and query instead of failing or silently skipping useful text.
  • Added standard text treatment for .1, .5, .txt, .conf, .tex, .sh, and .json.
  • Added --strict to keep the old skip/error behavior and repeatable --text-extension EXT to force additional extensions through text search.
  • Added a shared text-search file guard used by regular search, filename matches, query, and symbol extraction. It keeps hard-denied binary/archive/media/font/database/object extensions out of text search, rejects NUL-containing files, rejects non-regular files, skips symlinks/junctions, and enforces the existing 1 MiB read limit consistently.
  • Documented the new flags and behavior in the CLI docs.

Test and Stability Work

  • Added focused tests for symbols/query fallback, strict mode, standard text extensions, explicit language behavior, custom text extensions, hard-denied extension override attempts, oversized text files, and file-list filtering.
  • Replaced slow nested cargo run test harness usage with the compiled test binary path and removed stale target/debug/probe assumptions.
  • Fixed pre-existing test failures uncovered by the full suite: stale outline assertions, JSON/XML output expectations, filename required-term handling, exact/excluded query ranking and filename false positives, deterministic fixture setup, LSP runtime handling, LSP comprehensive isolation/bootstrap, Python symbol fallback, C# tree-sitter ABI pin, and stale doctests.

Validation

  • cargo fmt --check
  • cargo check --tests
  • cargo test
  • cargo test --doc
  • Focused guard/fallback checks: cargo test file_guard --lib, cargo test symbols_ --lib, cargo test --test query_command_tests query_ -- --nocapture, cargo test file_list_ --lib
  • Focused regression checks: cargo test --test integration_tests -- --nocapture, cargo test --test filename_search_tests -- --nocapture, cargo test --test required_terms_filename_tests -- --nocapture, cargo test --test json_format_tests -- --nocapture
  • Stress loop: cargo test --test quoted_term_with_negative_keyword_tests -- --nocapture x5
  • Anti-pattern scan for nested cargo run/hardcoded target binaries in tests returned no matches.
  • Leftover LSP/server process scan returned no matches.

@buger buger marked this pull request as ready for review June 27, 2026 05:58
@buger buger merged commit 5349938 into main Jun 27, 2026
1 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support unsupported file extensions as plain text (fallback mode) instead of warning + empty

1 participant