Skip to content

Disable parquet vectored reads to fix gcs-connector incompatibility#5954

Merged
kellen merged 1 commit into
mainfrom
kellen/disable-vectored
Jun 25, 2026
Merged

Disable parquet vectored reads to fix gcs-connector incompatibility#5954
kellen merged 1 commit into
mainfrom
kellen/disable-vectored

Conversation

@kellen

@kellen kellen commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Disables parquet-hadoop vectored reads via parquet.hadoop.vectored.io.enabled=false in core-site.xml
  • gcs-connector 3.1.16 calls VectoredReadUtils.validateRangeRequest(FileRange) expecting a void return, but hadoop-common 3.4.3 changed the return type to FileRange, causing NoSuchMethodError at runtime when parquet-hadoop attempts vectored reads through GoogleHadoopFileSystem
  • This affects the legacy HadoopFormatIO read path (used when UseSplittableDoFn=false)

Test plan

🤖 Generated with Claude Code

gcs-connector 3.1.16 calls VectoredReadUtils.validateRangeRequest
expecting a void return, but hadoop-common 3.4.3 changed the return
type to FileRange, causing NoSuchMethodError when parquet-hadoop
attempts vectored reads through GoogleHadoopFileSystem.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.95%. Comparing base (b6aad90) to head (4015c41).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5954      +/-   ##
==========================================
- Coverage   61.95%   61.95%   -0.01%     
==========================================
  Files         323      323              
  Lines       11727    11727              
  Branches      821      821              
==========================================
- Hits         7266     7265       -1     
- Misses       4461     4462       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kellen kellen merged commit dbf5b5c into main Jun 25, 2026
12 checks passed
@kellen kellen deleted the kellen/disable-vectored branch June 25, 2026 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant