Skip to content

perf: optimize parse_string_to_decimal (30-40% faster)#4916

Open
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/parse_string_to_decimal-datafusion-comet-20260714-040039
Open

perf: optimize parse_string_to_decimal (30-40% faster)#4916
andygrove wants to merge 1 commit into
apache:mainfrom
andygrove:auto-opt/parse_string_to_decimal-datafusion-comet-20260714-040039

Conversation

@andygrove

@andygrove andygrove commented Jul 14, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A

Rationale for this change

Optimize existing expression.

What changes are included in this PR?

Replaced six redundant scans plus two str::parse calls with a single validating byte pass and direct i128 digit accumulation, gated the seven inf/nan string comparisons behind a first-byte check, and swapped 10_i128.pow for a const power-of-ten table.

How are these changes tested?

Existing tests.

Benchmark (criterion):

  • decimal_38_10: 41.516% faster (base 265533ns -> cand 155293ns)
  • decimal_18_2: 36.846% faster (base 302661ns -> cand 191142ns)
  • decimal_38_10: 40.979% faster (base 266661ns -> cand 157386ns)
  • decimal_18_2: 36.564% faster (base 305466ns -> cand 193776ns)
  • decimal_38_10: 40.66% faster (base 264940ns -> cand 157214ns)
  • decimal_18_2: 36.607% faster (base 303296ns -> cand 192269ns)

Full criterion output:

cast_string_to_decimal/legacy/decimal_38_10
                        time:   [156.99 µs 157.35 µs 157.96 µs]
                        change: [−41.202% −40.979% −40.792%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
  6 (6.00%) high severe
cast_string_to_decimal/legacy/decimal_18_2
                        time:   [193.54 µs 193.64 µs 193.77 µs]
                        change: [−36.658% −36.564% −36.469%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) low severe
  1 (1.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe

cast_string_to_decimal/ansi/decimal_38_10
                        time:   [155.19 µs 155.26 µs 155.35 µs]
                        change: [−41.605% −41.516% −41.425%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) low severe
  3 (3.00%) low mild
  5 (5.00%) high mild
  2 (2.00%) high severe
cast_string_to_decimal/ansi/decimal_18_2
                        time:   [191.08 µs 191.15 µs 191.25 µs]
                        change: [−36.908% −36.846% −36.785%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe

cast_string_to_decimal/try/decimal_38_10
                        time:   [156.94 µs 157.14 µs 157.42 µs]
                        change: [−40.772% −40.660% −40.556%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) low mild
  3 (3.00%) high mild
  5 (5.00%) high severe
cast_string_to_decimal/try/decimal_18_2
                        time:   [192.19 µs 192.30 µs 192.46 µs]
                        change: [−36.697% −36.607% −36.524%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

@andygrove andygrove changed the title perf: optimize parse_string_to_decimal in datafusion-comet-spark-expr perf: optimize parse_string_to_decimal (30-40% faster) Jul 14, 2026
@andygrove andygrove marked this pull request as ready for review July 14, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant