gcp: implement multipart uploads using resumable upload API #503
Conversation
|
|
||
| #[derive(Debug)] | ||
| struct GCSResumableUpload { | ||
| state: Arc<ResumableState>, |
There was a problem hiding this comment.
Why do we need to use Arc and atomics here, AFAICT there isn't an obvious reason for this?
Edit: Oh I see, it is to try to prevent concurrent put part... NGL kind of feels like we're trying to shoehorn something into an interface that doesn't really fit...
There was a problem hiding this comment.
That's a fair judgement.
There was a problem hiding this comment.
I wonder if we could use https://docs.rs/tokio/latest/tokio/sync/struct.Notify.html to serialise the uploads, I think this would be more in keeping with the design than erroring if polled concurrently
There was a problem hiding this comment.
We could, but i'm not sure if it's in keeping with the design enough to not break something. I was worried about applications that try to do parallel upload and being surprised that most of their tasks are stalled.
2 participants
Which issue does this PR close?
Closes #504
What changes are included in this PR?
Are there any user-facing changes?
Yes, see changes to the client builder.