Migrate from github.com/mailru/easyjson

[RichardoC]

This library hasn't released an updated version in 2 years, and the maintaining company is subject to EU sanctions.

Given this, it might be better to migrate to another library.

This drive by is because the Kubernetes clients are dependant via your library.

Thanks!

[havardelnan]

Falco picked up this 2024-02-26T22:33, im not sure if its a false positive but its a bit suspicious as we have built this almost every day without a problem.

{
    "output": "2024-02-26T22:33:11.165254872+0000: Warning Sensitive file opened for reading by non-trusted program (user=<NA> user_loginuid=-1 program=compile command=compile -o /tmp/go-build779975163/b476/_pkg_.a -trimpath /tmp/go-build779975163/b476=> -p github.com/mailru/easyjson/buffer -lang=go1.12 -complete -buildid -OGlPEiKYrm3GXpAsMZO/-OGlPEiKYrm3GXpAsMZO -goversion go1.22.0 -c=4 -nolocalimports -importcfg /tmp/go-build779975163/b476/importcfg -pack /builds/.../.go/pkg/mod/github.com/mailru/easyjson@v0.7.7/buffer/pool.go file=/etc/shadow parent=<NA> gparent=<NA> ggparent=<NA> gggparent=<NA> container_id=host image=<NA>) k8s.ns=<NA> k8s.pod=<NA> container=host k8s.ns=<NA> k8s.pod=<NA> container=host k8s.ns=<NA> k8s.pod=<NA> container=host k8s.ns=<NA> k8s.pod=<NA> container=host",
    "priority": "Warning",
    "rule": "Read sensitive file untrusted",
    "source": "syscall",
    "tags": [
        "filesystem",
        "mitre_credential_access",
        "mitre_discovery"
    ],
    "time": "2024-02-26T22:33:11.165254872Z",
    "output_fields": {
        "container.id": "host",
        "container.image.repository": null,
        "evt.time.iso8601": 1708986791165254872,
        "fd.name": "/etc/shadow",
        "k8s.ns.name": null,
        "k8s.pod.name": null,
        "proc.aname[2]": null,
        "proc.aname[3]": null,
        "proc.aname[4]": null,
        "proc.cmdline": "compile -o /tmp/go-build779975163/b476/_pkg_.a -trimpath /tmp/go-build779975163/b476=> -p github.com/mailru/easyjson/buffer -lang=go1.12 -complete -buildid -OGlPEiKYrm3GXpAsMZO/-OGlPEiKYrm3GXpAsMZO -goversion go1.22.0 -c=4 -nolocalimports -importcfg /tmp/go-build779975163/b476/importcfg -pack /builds/.../.go/pkg/mod/github.com/mailru/easyjson@v0.7.7/buffer/pool.go",
        "proc.name": "compile",
        "proc.pname": null,
        "user.loginuid": -1,
        "user.name": null
    }
}

[fredbi]

It is indeed desirable to reduce the spreading of dependencies for consumers of this package that don't need them.
I think easyjson is a great tool and I'd like to continue supporting it. But not to spread this dependency everywhere like now.

The general idea I've had to address this is to deprecate entirely the existing API at the swag package level and move it to sub-packages that are their own modules.

This should not break anything and encourage users of this package (including other go-openapi packages and go-swagger generated code) to migrate their import github.com/go-openapi/swag to import github.com/go-openapi/swag/{narrow-focused-package}

At this point, easyjson (and yaml.v3) would be dependencies for JSON and YAML utilities. That's still a lot, but other utilities (like name mangling, type conversions, etc) would no longer require them.

The final stage is to transform the JSON utilities (e.g. ReadJSON, WriteJSON, JSONMapItem, JSONMapSlice) so as to delegate the actual marshalling/unmarshalling to an adapter registered at runtime (e.g. a little like encoding/gob). We can build such an adapter for easyjson (and other cool and fast json libraries btw). swag functions would always register easyjson to work exactily as they do now.

On the other hand, the inner modules for JSON and YAML would no longer support easyjson by default. It would require something like :

import github.com/go-openapi/swag/jsonutils/adapters/easyjson-register

register.Enable()

Several such adapters could be registered and checked for compatibility like we do now.

This option could be added to go-swagger so as to generate code that relies on easyjson the way it used to.

Work plan to achieve this:

1. Refactor swag to deprecate configuration by globals (this prevents proper insulation of modules
   • loading: feat(loading): HTTP or file loader configurable with LoadingOptions #104
   • mangling: refactored mangling utilities #115
2. Refactor swag to move all its current API to a sub-package level - announce the current API as deprecated (it still works the same)
   • convert: Feat/conv package: refactored conversions with generics #108
   • loading: refact loading, jsonutils, yamlutils utililities #111
   • mangling: refactored mangling utilities #115
   • json utils, yaml utils: refact loading, jsonutils, yamlutils utililities #111
   • others: refact: refactored the package into multiple specialized sub-packages #106
3. Upgrade all sub-package to modules (todo: make sure dependabot scans all these go.mod, configure linter to let us put replace clauses in go.mod)
   • split packages as modules #119
4. Introduce the registrable JSON adapters * [ ] easyjson
   • Remove dependency to mailru/easyjson by default #136
   • possibly others (jsoniterators, goccy/go-json, gojay, ...
5. Support this in go-swagger codegen

[barrybarrette]

Referencing the conversation in go-openapi/spec#214, I am also concerned about the security risk. I appreciate that you've already started to work on this. Do you see a clear path to supporting other serializers, possibly even just the stdlib encoding/json as a starting point since it wouldn't add any dependencies?

[fredbi]

Referencing the conversation in go-openapi/spec#214, I am also concerned about the security risk. I appreciate that you've already started to work on this. Do you see a clear path to supporting other serializers, possibly even just the stdlib encoding/json as a starting point since it wouldn't add any dependencies?

Yeah that's precisely the point. So I need a stdlib implementation of the "ordered JSON keys" feature and some kind of Adapter to feed this as well as the "dynamic JSON" feature.

[jordangov]

Happy to see this progressing. Not a go dev, so can't help. But thank you! I think the bigger issue though is its presence in Kubernetes core: https://github.com/kubernetes/kubernetes/blob/master/go.mod#L176

[cameracker]

Hi @fredbi , I really love the plan you have in place.

Would you be interested in some help? And what would that look like? I think I have a good understanding of what you had in mind but is there any other information you'd want to share for a potential contribution?

[soltysh]

@fredbi if you're interested I'm happy to help with the migration off of easyjson, which kubernetes is also tracking as unmaintained, and would love to see it being removed.