Skip to content

Conversation

@ktsitsi
Copy link
Contributor

@ktsitsi ktsitsi commented Sep 8, 2025

This PR:

  • Changes the build process to install libtiledb core library on an external destination and outside the wheel. This allows auditwheel to pick it up as an external dependency and graft it.
  • Requires fixes on the test_sdist step as it fails. This requires further investigation as the built wheels have been installed on local/machines and EC2 instances and the tests run successfully during the build stage. https://github.com/TileDB-Inc/TileDB-Py/actions/runs/17504381704/job/49724678086#step:4:6929
  • The structure of the wheel is the following after the changes for UNIX:
konstantinostsitsimpikos@Konstantinoss-MacBook-Pro-3 deleteme % ls -l
total 35328
drwxr-xr-x@ 33 konstantinostsitsimpikos  staff      1056 Sep  4  2025 examples
drwxr-xr-x@  5 konstantinostsitsimpikos  staff       160 Sep  4  2025 external
drwxr-xr-x@ 58 konstantinostsitsimpikos  staff      1856 Sep  4  2025 tiledb
-rw-r--r--@  1 konstantinostsitsimpikos  staff  18085008 Sep  4  2025 tiledb-0.34.2-cp310-cp310-manylinux_2_28_x86_64.whl
drwxr-xr-x@  6 konstantinostsitsimpikos  staff       192 Sep  4  2025 tiledb-0.34.2.dist-info
drwxr-xr-x@  3 konstantinostsitsimpikos  staff        96 Sep  4  2025 tiledb.libs
konstantinostsitsimpikos@Konstantinoss-MacBook-Pro-3 deleteme % cd tiledb.libs 
konstantinostsitsimpikos@Konstantinoss-MacBook-Pro-3 tiledb.libs % ls
libtiledb-37432e17.so.2.28

So the libtitledb library is being grafted with a 37432e17 hash.

REMARKS:

  • The build process follows two different paths depending on whether or not we build our wheels on CI utilizing cibuildwheel -> auditwheel or building the wheel through the source distribution files.

Closes CORE-332

@ktsitsi
Copy link
Contributor Author

ktsitsi commented Sep 8, 2025

@kounelisagis @ihnorton could you take a look on the test_sdist that fails even though the build tests are succeeding and the wheel is being built. I tested the wheels on EC2 instances and the tests runs just fine there after the installation.
https://github.com/TileDB-Inc/TileDB-Py/actions/runs/17504381704

@ihnorton
Copy link
Member

ihnorton commented Sep 9, 2025

could you take a look on the test_sdist

The changeset here should only be applied when we are doing a CI build: https://github.com/TileDB-Inc/TileDB-Py/pull/2238/files#diff-64e075a38340d9989b5b947d543f08464f279b09f838b898dc5bf5a887b64603L46-R51

Otherwise when building from the sdist, the resulting (locally-built) wheel will only have the hardcoded tiledb-external RPATH (which doesn't exist).

@ktsitsi ktsitsi force-pushed the kt/unbundle-libtiledb-from-wheel branch from 56d918a to d988169 Compare September 9, 2025 14:08
@ktsitsi ktsitsi force-pushed the kt/unbundle-libtiledb-from-wheel branch 2 times, most recently from a24707b to 24d97b5 Compare September 10, 2025 00:06
@ktsitsi ktsitsi force-pushed the kt/unbundle-libtiledb-from-wheel branch 3 times, most recently from 39a3d82 to da25222 Compare September 10, 2025 01:34
@ktsitsi ktsitsi force-pushed the kt/unbundle-libtiledb-from-wheel branch 4 times, most recently from b84c3f4 to 25f7b22 Compare September 10, 2025 04:54
@ktsitsi ktsitsi force-pushed the kt/unbundle-libtiledb-from-wheel branch from 25f7b22 to 6bcc513 Compare September 10, 2025 05:03
@ktsitsi ktsitsi force-pushed the kt/unbundle-libtiledb-from-wheel branch from 27a20e8 to d586862 Compare September 10, 2025 13:41
@ktsitsi ktsitsi marked this pull request as ready for review September 10, 2025 17:59
Copy link
Contributor

@jp-dark jp-dark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@ktsitsi
Copy link
Contributor Author

ktsitsi commented Sep 15, 2025

@ihnorton @kounelisagis Would you like to make a final check before we merge?

@teo-tsirpanis teo-tsirpanis changed the title Re-enable libtiledb grafting with audithwheel Re-enable libtiledb grafting with auditwheel Sep 16, 2025
@kounelisagis
Copy link
Member

I did a quick pass. The changes seem reasonable but, please, give me 1 more day.

Copy link
Member

@kounelisagis kounelisagis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing locally on macos and ubuntu, it all LGTM. Thanks!!!

@kounelisagis kounelisagis merged commit f706b78 into main Sep 17, 2025
115 of 116 checks passed
@kounelisagis kounelisagis deleted the kt/unbundle-libtiledb-from-wheel branch September 17, 2025 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants