Prevent out-of-memory crashes using line length limits #2902
Conversation
einfachIrgendwer0815
force-pushed
the
feature/line_length_limit
branch
from
e11770d to
2f0faa2
Compare
einfachIrgendwer0815
force-pushed
the
feature/line_length_limit
branch
from
2f0faa2 to
55bebbb
Compare
einfachIrgendwer0815
force-pushed
the
feature/line_length_limit
branch
from
55bebbb to
d0bb0ba
Compare
einfachIrgendwer0815
force-pushed
the
feature/line_length_limit
branch
from
d0bb0ba to
8c072fd
Compare
Discards long lines to prevent out-of-memory events.
einfachIrgendwer0815
force-pushed
the
feature/line_length_limit
branch
from
8c072fd to
4e5eacc
Compare
Enselic
left a comment
Enselic
left a comment
There was a problem hiding this comment.
Hi! A took a quick look and had some drive-by comments.
| Line length (in bytes) at which the line will be ignored. Zero disables this limit. | ||
| Default: 64 kB | ||
|
|
||
| --hard-line-limit <BYTES> |
There was a problem hiding this comment.
In what scenario do we prefer aborting bat over ignoring the line?
There was a problem hiding this comment.
This is helpful for files (especially pseudo-files) that have very long lines or do not contain any LF characters at all. One example given in #636 is /dev/zero. Let us assume there is the soft limit but not the hard limit. When reading such a file, bat would hang forever while wasting CPU time.
So the soft limit prevents out-of-memory events, the hard limit additionally prevents hanging infinitely.
Admittedly, the use case is tiny. Removing the hard limit is probably fine, since oom crashes are already prevented by the soft limit alone.
| escape sequences unless the syntax highlighting language is plain text. Possible values: | ||
| auto, always, *never*. | ||
|
|
||
| --disable-line-limits |
There was a problem hiding this comment.
The fewer options the better IMO. So I don't think we need this one, since I don't think we need two settings (see comment below).
| --disable-line-limits | ||
| Disables all line limits. Short for `--soft-line-limit 0 --hard-line-limit 0`. | ||
|
|
||
| --soft-line-limit <BYTES> |
There was a problem hiding this comment.
I'm thinking about the name for this. What name for this setting do other editors use?
| │ File: sample.rs | ||
| ───────┼──────────────────────────────────────────────────────────────────────── | ||
| 1 │ /// A rectangle. First line is changed to prevent a regression of #1869 | ||
| 2 │ struct Rectangle { |
There was a problem hiding this comment.
Can we prevent formatting to change in this file (and others?)
There was a problem hiding this comment.
The inserted spaces are there to leave room for the ! that is placed in front of replaced lines, so that the formatting does not look off in those cases. Without these placeholder spaces the formatting would stay as it was, however, the grid would look weird for replaced lines.
It would look something like this:
51 │ normal content
52 ! │ <line too long>
53 │ normal content
The other alternative is to not add the ! altogether:
51 │ normal content
52 │ <line too long>
53 │ normal content
But, especially without color, it would be somewhat easy to overlook that a line was replaced.
Enselic
added
the
waiting-on-author
2 participants
Fixes #636 with the solution proposed in this comment.
The limit at which reading continues but content gets discarded is now called "soft limit". Respectively, the limit at which bat will abort is now called "hard limit".
The default values for the soft and hard limits can be overwritten using the
--soft-line-limitand--hard-line-limitcli options.--disable-line-limitswill disable both limits.