Pitch: Possible improvements to the build experience.

[mikolasstuchlik]

The aim of this post is to pitch some ideas that may ease the usage and maintenance of the project. The document is not a call for an immediate action, but rather pitches for future developments. I would be more than happy to implement any or all of following features.

This project heavily relies on auto-generated code. The lack of support for external build tools on the side of the Swift Package Manager (SPM) makes it impossible to avoid using bash scripts completely.

At the same time, there are options available to shorten or omit some parts of the bash scripts.

Building the project using swift build

Partially related to #5, unfortunately no further developments were observed in the matter of external build tools support in the SPM.

The SPM allows user to list packages which should be imported and linked. This is done using pkgConfig: String? property of the PackageDescription or .systemLibrary target. This feature is used incorrectly by the project and therefore does not work and additional linker and import flags are needed in order to compile the project.

Problem:
Packages in this project regularly uses the pkgConfig argument to specify list of packages which shall be imported. This is however not the intended use of the arguments. As stated in source code, the SPM will search for the <name>.pc file. To my knowledge, there is no way of specifying more than 1 pkgConfig package dependency for a given systemLibrary target.

Possible solution
For example, in case of SwiftGLib, the solution was fairly simple. The gio-unix-2.0 package contains all of the same linker flags and search paths as the glib-2.0 package. Removing the latter will result in successful build using swift build command (in case that the wrapper files were already generated; Non whitelisted flags found warning will be raised for -pthread).
In case of less fortunate composition on packages, the solution may be to split the systemLibrary target.

Implementation of this proposal will end the need for build.sh script.

Paths, name and dependencies retrieving from the Swift Package

Various features mentioned below are built upon this feature.

The SPM provides tools for reading the contents of the Swift Package. Those tools provide information such as name of the package, path to the package, recursive list of dependencies (containing also the name and the path to the dependency) etc. The downside is, that those information are provided as a json. Other output formats lack the information needed.

In order to access the information from a bash script, we would need to ask user to install jq - command line json utility.

In turn, we would no longer need to parse the Package.swift in order to obtain Mod variable.

Adding the gir2swift as a dependency

Since the presence of gir2swift is required in order to build any project using the SwiftGtk, the gir2swift could be mentioned as a dependency in the Package.swift. Including tools as dependencies is not an unheard of practice. For example unrelated projects I have worked on did so with BartyCrouch (localization tool for iOS projects). Using previously mentioned feature, we are able to retrieve exact path for gir2swift package in the file-system.

Implementing this feature will make us able to remove some of the scripts. For example generate-wrapper.sh would be present only in the gir2swift repository and all the other repositories would contain only the gir-to-swift.sh script, which would be responsible for executing gir2swift and post processing.

Decomposition of bash script files and documentation

In my opinion the most important bash script in the repositories is gir-to-swift.sh which does the important job of executing gir2swift with appropriate arguments and post processing files. Those tasks are very important and tailored for the specifics and need of a given part of the project.

Some parts of the script are common for all of the repositories, for example the search for gir files. This feature could be removed from the individual bash scripts in the repositories and only present in the gir2swift repository as a bash function.

Following paragraph builds on premise, that gir2swift gir prerequisites of a package compose of prerequisites and girs of it's dependencies.

Since we would have the ability to know what packages are present during wrapper generation time and the exact tree of dependencies, gir-to-swift bash script would be able to retrieve (compute) prerequisites from other dependencies. Therefore the feature of executing gir2swift would no longer be needed inside of gir-to-swift script and could be implemented in the previously mentioned common generate-wrapper bash script.

In case this feature would be implemented, the only part of bash scripts that would require per-repository maintenance and documentation would be the post processing with awk and sed files.

Summary

This document pitches some ideas that would in my humble opinion result in drastic simplification of build process. In case that all of the features would be implemented, the number of scripts per repository would be reduced to two: gir-to-swift remnant for post processing and generate-documentation. Other scripts that support code generation would be part of the code generation package gir2swift.

[rhx]

This is great work, thanks. Some comments:

• Replacing grep with jq in config.sh to retrieve the module doesn't make it any more lightweight and adds an additional dependency. While it's nice to get SPM to tell us the information needed, SwiftGtk has full control of the structure of its Package.swift so the grep command should not be too much of an issue.
• Simplifying the scripts and making them more beginner-friendly is a great idea. From what I can see, though, it does not appear that SPM will be able to handle everything, so some form of scripting will probably need to remain.
• An alternative (that is somewhat orthogonal to the above) would be to look at cmake, which now has some support for Swift and is more flexible when it comes to running code generators such as gir2swift.

[mikolasstuchlik]

I would like to bring up a status update.

I my forks of the main repositories, I've implemented the "Building the project using swift build" and while it works great on ubuntu (both my machine and the CI), the build fails on macOS due to (what is I believe is) a bug in SPM. At the same time, this modification fixes problems with Sourcekit-LSP which now works under Ubuntu (with no observed degradation).

At the same time I've implemented "Paths, name and dependencies retrieving from the Swift Package", "Adding the gir2swift as a dependency" and "Decomposition of bash script files and documentation". The implementation lacks most of the original features since I wanted to focus only on the features essential for prototyping signal wrappers, but it is certainly possible to implement them in the future with ease. Some unexpected benefits of the new implementation is the ability to specify desired version of gir2swift simply by changing the branch name in the root package which is really helping me in the development of signal wrappers.

[rhx]

Any updates on the current status? I saw that you closed your pull request. Was that due to the macOS issues?

[mikolasstuchlik]

I have opened a PR by mistake (due to behavior of Github when opening a PR on forked repository) and immediately closed it.

Status:
I haven't done much work on this specific issue since my last post, as I focused on other elements of the wrapper. Also I'm currently writing the thesis itself.

Current issues:

• During my work on some unrelated matter, I found out, that LLDB does not work. I haven't created dedicated thread on the Forum, but the issue behaves as described in https://forums.swift.org/t/need-help-with-lldb-vscode-plugin/43047/5 and I explore some clues which could lead me to a proper solution. Support of ALL tools is of the essence for me. If you had any suggestions on how to solve the issue with the LLDB, please let me know.
• The macOS support is the biggiest issue when considering merge to you repos and I plan on resolving the issue when my thesis is written and I have some free.

[mikolasstuchlik]

@rhx

I have been working on the macOS build problems. The solution for the pkg-config issue should be the same as on linux (the implementation I have committed should work on macOS - in theory). However, it does not. The issue manifests itself in the same manner as described in https://forums.swift.org/t/possible-bug-regression-with-regards-to-pc-files/42306.

I wanted to investigate more, but unfortunately, related Swift but report is inaccessible due to Swift JIRA being shut down for maintenance.

If the issue is in fact the same as posted in the thread, the problem will "fix itself" in upcoming Swift 5.4 (which is probably half a year away at the time of writing this post).

The possibility to extract list of required pkg-config packages is already built into the new gir2swift-generation-driver.sh script. I will build on this in an attempt to fix macOS build.

[mikolasstuchlik]

The initial implementation of this pitch is now part of the main repositories.