Skip to content

fix: array_any_value returns NULL for empty list elements#23775

Open
bjchambers wants to merge 1 commit into
apache:mainfrom
bjchambers:fix/array-any-value-empty-list
Open

fix: array_any_value returns NULL for empty list elements#23775
bjchambers wants to merge 1 commit into
apache:mainfrom
bjchambers:fix/array-any-value-empty-list

Conversation

@bjchambers

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

array_any_value reads the wrong value (or panics) when its input list column
contains a non-null empty (length-0) element.

general_array_any_value guards null and all-null elements, but a non-null
empty element falls through to the no-nulls branch, which unconditionally reads
values[start]:

  • interior empty list → reads values[start], i.e. the next element's
    value (silently wrong data)
  • trailing empty list (start == values.len()) → out-of-bounds slice →
    panic range end index N out of range for slice of length N-1

The panic is easy to trigger in practice when the array_any_value output flows
into a hash RepartitionExec (e.g. the value is used as an equi-join key):
repartitioning slices batches so an empty element can land at the end of a
values buffer, tripping the out-of-bounds read on a spawned task.

What changes are included in this PR?

Guard the empty case explicitly in general_array_any_value — an empty list has
no value to take, so the result is NULL.

Sibling functions in extract.rs were audited and are already safe:
array_element bounds-checks the index against len; array_slice /
array_pop_front / array_pop_back guard len == 0.

Are these changes tested?

Yes:

  • Kernel-level regression tests for general_array_any_value: an interior empty
    element (previously wrong value) and a trailing empty element (previously
    panic).
  • array_any_value.slt cases covering List and LargeList with interior and
    trailing empty elements.

Are there any user-facing changes?

array_any_value now returns NULL for an empty list element instead of
returning the next element's value or panicking the query. No API changes.

…ements

general_array_any_value handled null and all-null list elements, but a
non-null *empty* (length-0) element fell through to the no-nulls branch,
which unconditionally read values[start]. That returned the next element's
value for an interior empty list (silently wrong data), and read out of
bounds when start == values.len() (a trailing empty element), panicking with
"range end index N out of range for slice of length N-1".

The panic surfaced when the array_any_value output flowed into a hash
RepartitionExec (e.g. used as an equi-join key): batches got sliced so an
empty element landed at the end of a values buffer, tripping the
out-of-bounds read on a spawned task.

Guard the empty case explicitly: an empty list has no value to take, so the
result is NULL. Sibling functions in this file are already safe
(array_element bounds-checks the index against len; array_slice / pop_front
/ pop_back guard len == 0).

Regression tests added at the kernel level (interior + trailing empty) and
as sqllogictest cases.

Signed-off-by: Ben Chambers <bchambers@apache.org>
@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Jul 21, 2026
@codecov-commenter

codecov-commenter commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.65517% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.71%. Comparing base (eef1017) to head (ed69900).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/functions-nested/src/extract.rs 89.65% 0 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23775      +/-   ##
==========================================
- Coverage   80.71%   80.71%   -0.01%     
==========================================
  Files        1089     1089              
  Lines      368748   368777      +29     
  Branches   368748   368777      +29     
==========================================
+ Hits       297633   297651      +18     
- Misses      53372    53375       +3     
- Partials    17743    17751       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

array_any_value returns wrong value / panics for empty list elements

2 participants