Multiple major revs in `hashes` dependency doesn't work the way I expected

[apoelstra]

This library currently supports hashes 0.12 or 0.13. The point was that you can use this library alongside rust-bitcoin 0.30 (which depends on 0.12) or rust-bitcoin 0.31 (which depends on 0.13). You wouldn't want to do this permanently -- it's for the upgrade process between major revs of rust-bitcoin, so that you can update your various deps piecemeal and things will continue to work.

However, cargo will cheerfully allow (and may actually do, by default) you to have rust-secp depending on hashes 0.12 in the same tree as rust-bitcoin depending on hashes 0.13. Then the ThirtyTwoByteHash traits will be incompatible and you'll get compilation errors.

Our intention was "rust-secp256k1 supports whatever version of hashes rust-bitcoin is trying to use" but apparently we can't guide cargo into enforcing this :/.

[Kixunil]

Oh, right that actually makes sense even though it took me a bit to understand. The situation cannot be solved by cargo alone if you have two different bitcoin versions. I see two possible solutions:

• Have a separate micro-crate for the trait and stabilize it practically immediately
• Have versioned hashes feature (bitcoin_hashes_0_12, bitcoin_hashes_0_13), then bitcoin depends on the version of hashes it supports and when you turn on both the trait is implemented for both so everything works.

[apoelstra]

I don't like the idea of having micro-crates because we'd wind up with a ton of them, which may be formally "stable" but probably we'd be constantly creating and abondoning them.

Versioned features might work ... it's a little ugly, but I think it'd solve this particular upgrade path.

[Kixunil]

Well, in this case I also prefer not doing a micro crate since we want to stabilize things ~soon anyway.

[tcharding]

Ha! stablizing secp is literally years away. What if we were to move ThirtyTwoByteHash to hashes and then we can use the semver trick - well at least I think we can. If there are no obvious problems I'll have a go, will wait to see if this gets shot down first though.

[Kixunil]

The trait is not the only reason for the hashes feature. I wish it was. Hashing is actually used (in ECDH IIRC).

stablizing secp is literally years away.

Is it? I thought removing contexts and improving errors is enough. If I'm wrong we'll have to seriously rethink APIs in bitcoin.

[apoelstra]

The trait is not the only reason for the hashes feature. I wish it was. Hashing is actually used (in ECDH IIRC).

Right, but the trait is the only part of hashes exposed in the public API. And it's a weird trait that straddles rust-secp and bitcoin_hashes.

+1 to moving it to bitcoin_hashes and semver tricking it, except that I think the transition will be a big PITA and maybe it's not worth the pain.

Ha! stablizing secp is literally years away.

For rust-secp I think we just have a couple things, as Kix says. For libsecp itself, you're correct that they'll continue to have new versions for years, which we need to keep up with for security reasons but which may break our APIs (meaning we need to avoid exposing them except by feature gates that are clearly marked "unstable"). I think we can manage that.

[Kixunil]

I mean it's impossible to move it to hashes because it'd create a circular dependency. (Also I strongly believe secp is the proper place for it if there's no specialized crate.)

[apoelstra]

I don't think it'd create a circular dependency, but we'd have to keep the impls in rust-secp, which might not actually improve anything as far as version compatibility went.

[Kixunil]

For trait: hashes -> secp
For ECDH: secp -> hashes

[tcharding]

I'll just hack it up and solve this one way or the other :)

[apoelstra]

@Kixunil the trait doesn't require hashes to depend on secp. It's just a trait definition that specifies that you can convert a blob to a [u8; 32] and has a promise about being a cryptographic commitment.

[Kixunil]

Oh, I don't know why I was thinking it does need to. Probably confused by orphan rules or something.

[tcharding]

BOOM! It works. I'll write what I did from memory, it was convoluted so I probably need to write it out and re-do the whole thing before we do it for real. Requires one patch to rust-bitcoin that does not build. Some things I used manifest patching and some steps I pushed branches to github.

1. Move the trait from secp to hashes
2. Do a hashes 0.14.0 release (rust-bitcoin does not build on this patch)
3. Upgrade secp to use hashes 0.14
4. Create branch hashes-0.14.x on rust-bitcoin repo
5. Do a hashes 0.15 release (no changes, just to prove the semver trick works).
6. Do a hashes 0.14.1 release on branch hashes-0.14.x (add semver trick).

We do a lot of bitching about cargo around here, myself included. I must say that this whole process went off without a hitch and cargo behaved well and did what it was told. I didn't swore once.