Friendlier `unevaluatedProperties` errors

[ikonst]

It would be helpful to bifurcate the "Unevaluated properties are not allowed" error messages over whether the property was expected — defined but failing validation, e.g.

• "Unevaluated properties are not allowed ('foo', 'bar' were not valid under the given schema)"
• "Unevaluated properties are not allowed ('baz' was unexpected)"

A single unevaluatedProperties could yield 0, 1 or 2 errors.

I personally found that aspect of unevaluatedProperties to be confusing when I first stumbled on it ("foo is not a typo, so why is it reported?"). Later I understood it, but my learning path could be shorter and astonishment smaller if we somehow structured the message to suggest this possibility.

--

One way to implement this would be to have find_evaluated_property_keys_by_schema collect tuples of (property_name, is_valid), and then process them within unevaluatedProperties:

-evaluated_keys = find_evaluated_property_keys_by_schema(...)
+res = find_evaluated_property_keys_by_schema(...)
+evaluated_keys = {key for key, is_valid in res if is_valid}
+expected_keys = {key for key, _ in res if key in instance}

Then unevaluated_keys could be split into:

unevaluated_expected_keys = [k for k in unevaluated_keys if k in expected_keys]
unevaluated_unexpected_keys = [k for k in unevaluated_keys if k not in expected_keys]

and each list used to emit the relevant error message.

[Julian]

Another example of this is the simple schema:

from jsonschema import Draft202012Validator
schema = {
    "required": ["foo"],
    "unevaluatedProperties": False,
    "oneOf": [{"required": ["bar"]}],
}
Draft202012Validator(schema).validate({"foo": 37})

which produces the confusing:

Traceback (most recent call last):
  File "/Users/julian/Desktop/uP.py", line 7, in <module>
    Draft202012Validator(schema).validate({"foo": 37})
  File "/Users/julian/Development/jsonschema/jsonschema/validators.py", line 424, in validate
    raise error
jsonschema.exceptions.ValidationError: Unevaluated properties are not allowed ('foo' was unexpected)

Failed validating 'unevaluatedProperties' in schema:
    {'oneOf': [{'required': ['bar']}],
     'required': ['foo'],
     'unevaluatedProperties': False}

On instance:
    {'foo': 37}

which can be especially bad if the oneOf (or applicator in general) produces some very deep error. Here it's probably best_match that needs enhancing to traverse into the applicators for this example.

[ikonst]

IIRC, this issue has disappeared for me once #1075 got merged in.

[Julian]

Ah, good to know -- I'll move my above comment (which was really just using this as a clotheshanger issue) to some other unevaluatedProperty one.

Feel free to file another issue if you have one with unevaluatedProperties.