feat: Transferable

[rhoban13]

In #676 for copying to/from DockerContainer, it was suggested that we should clarify the interface & usage a bit before proceeding. This PR aims to push that conversation forward with some test cases illustrating proposed usages. Although here I created a Transferable object, in most cases there's no need for the caller to explicitly construct a Transferable, just pass the bytes | Path

Proposed use cases:

1. Copy into a container by passing a TransferSpec into the initializer:

DockerContainer(... transferrables=[TransferSpec(source, destination_in_container), ...)

2. Copy into the container via the builder pattern, prior to starting the container:

DockerContainer(...)
     .with_copy_into_container(b"some_bytes", destination_in_container)
     .with_copy_into_container(some_path, destination_in_container)

3. Copy into the container at runtime:

with DockerContainer(...) as container:
    container.copy_into_container(b"some_bytes", destination_in_container)
    container.copy_into_container(some_path, destination_in_container)

[codecov]

Codecov Report

❌ Patch coverage is 93.33333% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@60d21f8). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
core/testcontainers/core/container.py	92.85%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #852   +/-   ##
=======================================
  Coverage        ?   79.68%           
=======================================
  Files           ?       15           
  Lines           ?     1216           
  Branches        ?      193           
=======================================
  Hits            ?      969           
  Misses          ?      204           
  Partials        ?       43           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[wtf-is-flying]

Hi @rhoban13, what is the current status of this feature? What could I do to help?

For what it's worth, I think the suggested interface is clear. I haven't tested it with my use cases yet though.

[rhoban13]

Tagging a few maintainers seeking feedback @terry-docker @alexanderankin @Tranquility2

[alexanderankin]

i think my current priorities are docs, anything that makes this library behave like the others (this PR is illustrating current state not improving it), so far the only new issue from the new release is someone complaining about warning logs - so i think that means that the new release was successful (either that or there are no more users of this library). anecdotally someone reached out on slack asking for a new release but they are only allowed to use it at work a week after it has been released. so there may be new issues in some time.

[alexanderankin]

wait, this adds Transferable - this is great - if we can also fix the immediate starting of the container this almost entirely fixes copying. just have to maybe see about creating folders automatically and i think we may have parity on copying files with the other impls

[rhoban13]

Yes I was just about to update the PR title - my original intent was just some tests, but I found it easier to write those tests using TDD & just implement it at the same time. I think this brings the library a bit closer to feature parity with some others.

I'll take a look at adding some tests around folders if it sounds like you're in agreement with the interface at a high level.

[wtf-is-flying]

Thanks! I'll try to run some tests then and will let you know how they went.

[rhoban13]

if we can also fix the immediate starting of the container

This is independent of this PR, right? I presume you're referring to the fact that we're calling docker run instead of create+start?

[alexanderankin]

yeah this whole API is not going to behave as intended until we start doing create then start

[rhoban13]

Maybe the name file_content here isn't great? Maybe source or copy_source?

[alexanderankin]

In java they all just take transferable and destination I think

[rhoban13]

I refined the initial definition of Transferable to now simply the be the union of those types. I kinda like that the signatures all just contain transferable: Transferable. It was an easy update except for the case of passing args to the initializer, which I changed to more closely mirror the type passed to volumes.

[wtf-is-flying]

I ran some tests with this PR, it went well 🙂.

I used this function to transfer directories:

def transferables_from_directory(
    source_dir: Path, dest_dir: Path, mode: int
) -> list[TransferSpec]:
    transferables: list[TransferSpec] = []
    for dirpath, _, filenames in source_dir.walk():
        for name in filenames:
            origin = dirpath / name
            destination = dest_dir / dirpath.relative_to(source_dir) / name
            transferables.append((origin, str(destination), mode))

    return transferables

[rhoban13]

I updated the implementation so all these methods should work if the transferable Path is a directory as well.