Change share/ruby-build/*-dev to build a specific revision

[rubys]

Continuing one thread of a discussion started on my weblog. I'll start with a description of the problem (as I see it), and an outline of a possible approach...

rbenv is optimized for enforcing a precisely specified Ruby revision for each application on your system, ensuring your apps have the same environment in development and production

The current *-dev recipies don't do that, instead they build whatever the latest is on the specified branch. Run the same command at a different time, and you could very well get a different revision.

Builds of Ruby from the trunk (or another branch) are identified by their svn revision number. The latest revision can be found by going to the svn repository.

Builds will generally identify the date, branch, and revision number, for example:

$ ruby -v
ruby 2.1.0dev (2013-03-22 trunk 39876) [x86_64-darwin12.3.0]

Having a separate recipe for each revision is clearly impractical. Instead, ruby-build should accept a -r or --revision= argument which provides this information. Perhaps latest would be a reasonable default, or perhaps this argument should be required when dealing with recipes that build from git. Either way, the resulting version shouldn't be named with a generic name, such as 2.1.0-dev, but instead with a specific revision like 2.1.0-r39876.

The only thing that ruby-build needs to do to make this happen is to checkout the corresponding git commit, and to name the output directory differently. Once built, rbenv doesn't care how the version was created, and will happily use revisions named 2.1.0-r39876 and the like.

Unfortunately, checking out the corresponding commit would require a non-shallow git clone, and would be made easier if git-svn were a prereq. git-svn appears to already be installed on my Mac OS X machine without my knowingly requesting it, and is readily available on Linux.

An optimization that can be pursued separately: if the user opts to keep the git clone around, subsequent rbenv-build calls could issue git pull requests to fetch only the updates since the previous build. I do this today with rvm. One thing I chose to do is to copy that directory before running the build ensuring that I have a pristine copy for my next build.

Some numbers for discussion. I'll indicate what OS I obtained each on, but it shouldn't matter:

• rbenv/sources/2.1.0-dev is 180Meg on OS X. If was cloned with --depth=1, and built.
• rvm/src/ruby-head is 55Meg on Ubuntu. It is a full clone, pristine.
• rvm/src/ruby-head-n39875 is 235Meg on Ubuntu. It is a copy of a full clone, and built.

Hopefully that should be enough to start a discussion. :-)

[sferik]

Thanks for the blog post and the suggestion. This idea seems well-aligned with the goals of this project. Do you have any interest in implementing this?

[rubys]

I'm certainly willing. Not being overly familiar with this project, I just would rather not invest a lot of time heading down a wrong direction. And the way I'm proceeding is to express my assumptions out loud to see if there are any pushback. Continuing on that path:

ASSUMPTION

given the stated goals of this project, relatively few people actually use definition files that make use of install_git.

ASSUMPTION

those that make use of such definition files, particularly those that specify options like --keep, are likely to use them again over time. Possibly many times. I certainly do.

The reason why I'm checking these assumptions is that, if true, the right way to proceed is to change install_git. Without knowing the history of decisions made, I can't know what I would be breaking if I did so. If it turns out that there are important use cases which differ from mine, perhaps it would be best to leave install_git alone and to build another shell function that behaves differently.

Which ever way this goes, there will probably need to be something on the order of a full web page describing how to plan for, manage, and periodically prune such builds. I can provide the initial contents of that page too. Depending on how it goes, it might make sense to write a plugin to help with these tasks. We'll see how it goes.

[jeremy]

👍 to your assumptions & conclusion. Thanks for digging in to this workflow!

[rubys]

It turns out the exploring --keep was a red herring / dead end. The build directory is dirty and the only reasons to keep it are orthogonal to this use case, and are spelled out in the README for ruby-build (e.g. gdb and memprof).

The right solution was to use the cache directory if provided, and that turned out to be low hanging fruit: #333

Returning to this use case, I made some progress, and put out a proposed change for discussion: rubys@879e1c5 Issues and next steps are spelled out there.

A third part which would complete this issue would be to add --revision and --commit flags to ruby-build and rbenv-install. Flags that would be only used if the definition includes install_git. This appears to be a relatively straightforward change.

[sstephenson]

#334