proposal: cue: lazier errors

[rogpeppe]

Currently cue.Value.Err returns an error if any field that is part of the value is in error, even if some parts of the value are not in error and can be looked up without error.

I propose that there should be a way to make Err do less work and return an error only when the top level of the value (its scalar value or the set of fields) cannot be computed. Value.Validate could still be used to do a more thorough job.

Specifically, if Value.Err returns nil for some value v:

• v.IncompleteKind does not return bottom
• the fields of v can be iterated over without error if it's a struct
• the indexes of v can be iterated over without error if it's a list

To avoid breaking compatibility, this behavior would be opt-in, either by providing an option when creating the cue.Context, or by providing an option when creating the value. What the spelling of this option is to be decided.

This change has a couple of significant advantages:

• it means that Value.IncompleteKind better reflects the actual operations that can be done on the value. Currently even if it returns cue.Bottom, there are still some operations that are possible, but it's not easy to determine what those are.
• it means that the implementation is free to use a more "compute-on-demand" implementation. Until the caller actually demands a value, the implementation can defer the computation necessary to provide it.

[mvdan]

I support this mainly because, currently, Err and Validate serve a very similar purpose and they are rather confusing. And with CUE leaning into lazy or optimal evaluation, I think having Err be for "is this value directly an error" makes sense.

I guess we don't need to tweak the spec, as it doesn't make any mention of e.g. a struct value being an error (bottom) if one of its regular fields is an error?

[mpvl]

I support this mainly because, currently, Err and Validate serve a very similar purpose and they are rather confusing. And with CUE leaning into lazy or optimal evaluation, I think having Err be for "is this value directly an error" makes sense.

I guess we don't need to tweak the spec, as it doesn't make any mention of e.g. a struct value being an error (bottom) if one of its regular fields is an error?

Actually, that is part of the spec. In general, this is how graph unification is defined: in principle a structure is at err if any of its fields is, recursively. I've been discussion with Rog how we could justify this, but it is a very fundamental change of which I don't fully understand the consequences. There is definitely CUE code out there that relies on this behavior as well.

At the API level we can still make different choices, of course. But it is not as easy as it may seem.

[rogpeppe]

At the API level we can still make different choices

FWIW this proposal is entirely about changing the API-level semantics, not the CUE spec itself, which would be considerably more significant thing to change. Given that cue.Value already provides some of the semantics needed here (for example, it's possible to iterate over the fields of a struct even when a field of that struct holds an error), I'm hoping the change wouldn't be too tricky.

But it is not as easy as it may seem.

Maybe we could expand on this a bit? It would be nice to know what the challenges are here.