Nice! @MSP-Greg. Do you have an example/log file that can demonstrate this working and what the final generated binstub is?

@colby-swandale

demonstrate this working

It does.

what the final generated binstub is?

Below are old and new. C:\ruby25_64 is the location of my normal day-to-day ruby. In a short time, I expect to be using 2.6, and I'll move the current version to an archive directory and/or another drive.

Using the old version, I would need to reset all the paths. Also, if one has many ruby versions (I have several hundred, though they're not all unzipped), I can run any bin file by including the full path in the command line. With the old, that won't work...

Thanks, Greg

New erubis.bat file:

@ECHO OFF
IF NOT "%~f0" == "~f0" GOTO :WinNT
@"%~dp0ruby.exe" "%~dp0erubis" %1 %2 %3 %4 %5 %6 %7 %8 %9
GOTO :EOF
:WinNT
@"%~dp0ruby.exe" "%~dpn0" %*

Old erubis.bat file:

@ECHO OFF
IF NOT "%~f0" == "~f0" GOTO :WinNT
@"C:\ruby25_64\bin\ruby.exe" "C:/ruby25_64/bin/erubis" %1 %2 %3 %4 %5 %6 %7 %8 %9
GOTO :EOF
:WinNT
@"C:\ruby25_64\bin\ruby.exe" "%~dpn0" %*

@hsbt - prefer old or new? Or other ruby committers familiar with Windows... Thanks, Greg

Looking into how rubygems is generating binstubs, maybe we should be implementing a windows version of the shebang method where a setting can be configured to set the executable?

where a setting can be configured to set the executable

I'm sorry, I don't know what you mean by 'to set the executable'...

Apologies, I'm not sure what the Windows terminology is. I mean the the path to ruby. Either it be C:\ruby25_64\bin\ruby.exe or just ruby.exe

I'm not familiar with %~dp0 variables on Windows. Can you show me the documents of Windows specification?

@colby-swandale

I suspect terms are the same. I might say path & file name, or directory & filename.

@hsbt

Can you show me the documents of Windows specification?

I've been at windows for a while (I was coding in DOS days). Never looked, and google does not seem to have anything that I could find, at least on microsoft.com.

Best description of %~dp0:

%~dp0 = __dir__ + '\'

So, like __dir__, %~dp0 doesn't work from a prompt, it only works from within a cmd/bat file

Really, the only difference between the old code vs this PR is that after the PR, the directory is resolved at runtime, vs the old code resolves at gem installation time. Hence, not portable...

Sorry, I should have phrased it as above when I did the PR...

Thanks, Greg

@colby-swandale

Just thought of one thing I haven't looked at. With all the CI & local testing I've done (often on gems I don't use), I've often used their bunder setup and ran bundler with a 'install in this folder' parameter (don't recall what it is). Never had any problems. I also know you can install gems in folders other than the default. I assume doing so does not alter where bin files go, it just changes LOAD_PATH?

@unak

Sorry for the ping. If you have a few minutes, another windows opinion might be helpful.

This PR changes the gem files in the bin folder. Old & new examples above...

Thanks, Greg

@MSP-Greg

I agree the WinNT part, but have a little concern about non WinNT part.

New elbis.bat should be

@ECHO OFF
IF NOT "%~f0" == "~f0" GOTO :WinNT
@"C:\ruby25_64\bin\ruby.exe" "C:/ruby25_64/bin/erubis" %1 %2 %3 %4 %5 %6 %7 %8 %9
GOTO :EOF
:WinNT
@"%~dp0ruby.exe" "%~dpn0" %*

because non WinNT platform (= Win9x) does not recognize "%~" pattern.

BTW, we (ruby itself) don't support Win9x no longer, so simply we may be able to remove non WinNT part.

@unak

You have a seriously good memory. I used original PC's and Apple II's, but I don't remember much.

I just looked at the existing code and replaced the directory strings, but

@ECHO OFF
@"%~dp0ruby.exe" "%~dpn0" %*

is certainly simple. I think I'd concur that win9x support is not needed. I'll update the PR.

For non-windows types, I changed my rackup (puma) file to match the above, as it's probably got the most parameters. Works fine.

Also, the gem bin file is not Win9x compatible, so this change would be matching it...

Thanks, Greg

Oh, I just remembered!
The reason why %~dp0 is not used now is that .bat files generated by rubygems may be installed another directory of ruby.exe's one when user uses --bindir option.
So, @MSP-Greg , you should resolve such case before changing.

@unak

I kind of asked about that earlier,..

If --bindir is not used, are you okay with using %~dp0?

If --bindir is used, maybe there are two options?

1. If --bindir is a sub directory of the normal ruby TOPDIR, one could use relative paths.
2. If not, I assume all path would need to be hardcoded.

Agreed? I'm not sure what the typical use case for specifying --bindir is...

Thanks, Greg

Agreed.

@unak @segiddins

Two more use cases:

1. What should bin file be when using -​-user-install? I think ruby.exe should be simply that, iow, its path should be found from ENV['PATH'].

2. Noticed some odd bin files when using bundler with a --path parameter. I assume that code is still from RubyGems? Need to look at this some more, probably same layout as -​-user-install?

So, we've got --bindir, -​-user-install, and bundler --path. I haven't looked thru the code enough to see how to determine which of the three is in use.

I'm assuming that -​-user-install and bundler --path would normally not be within a ruby folder. Not sure about bindir.

hi! it's been >2 months since the last activity here. can someone let me know what the status of this PR is?

thanks!

Again, what path should be used in the bin files for gems installed with any or all of the following options:
--bindir, -​-user-install, and bundler --path?

I think that the Ruby command should default to the ruby version based on PATH. An normal install should use the Ruby version it's contained in (using %~dp0). Doing so this way leaves both portable.

I think we should default to the equivalent of /usr/bin/env ruby, but I'm not sure that's even the default on non-windows systems at the moment

I thought this was mostly a discussion of the window bin files. The only 'standard' for the Ruby installation location might be from the default RubyInstaller setup, but I don't think that's a good assumption.

If you have any thoughts on 'standard' use cases for this issue, they'd be helpful. I've been using & coding on Windows since DOS days, and I've got hundreds of Ruby builds on my system, as I occasionally need to manually bisect with the builds. I know how I'd like it to work, but that may not translate well to a beginning Ruby/Rails user...

As I see it, the main issue is how to code the path for ruby.exe & and the file it loads. They can be:

1. Absolute (bad idea, non portable)
2. Based on current path of the script file (*.cmd, *.bat, using %~dp0)
3. Left plain, so ENV['PATH'] would be used.

Thanks, Greg

@unak @segiddins

My home development system's motherboard stopped working, so (for the time being), I set up a small notebook for basic work. In the process of doing that, I've got some thoughts that might wrap this up.

Assuming we don't require support of Windows previous to Vista, I now feel the stub should be:

  def windows_stub_script(bindir, bin_file_name)
    # All comparisons should be case insensitive
    if bindir.downcase == RbConfig::CONFIG["bindir"].downcase
      # stub & ruby.exe withing same folder.
      <<-TEXT
@ECHO OFF
@"%~dp0ruby.exe" "%~dpn0" %*
      TEXT
    elsif bindir.downcase.start_with? RbConfig::TOPDIR.downcase
      # stub within ruby folder, but not standard bin.
      require 'pathname'
      from = Pathname.new bindir
      to   = Pathname.new RbConfig::CONFIG["bindir"]
      rel  = to.relative_path_from from
      <<-TEXT
@ECHO OFF
@"%~dp0#{rel}/ruby.exe" "%~dpn0" %*
      TEXT
    else
      # outside ruby folder, maybe -user-install or bundler.
      <<-TEXT
@ECHO OFF
@ruby.exe "%~dpn0" %*
      TEXT
    end
  end

Since both Bundler and --user-install separate gems into 'ruby_version' folders,
I think it's best to asssume that the user's desired Ruby build is in their path.

Thoughts?

Thanks, Greg

@nobu

Sorry for the ping; I thought your input re this PR would be appreciated. BTW, I noticed you've recently made a few windows specfic core commits (thank you).

I've always been bothered by the the .cmd/.bat files (bin stubs) created by RubyGems. I felt they should be portable/reproducible, and I also found them cumbersome to work with. For quite while, I've had scripts to replace them.

Anyway, my main use case was for testing across several builds of a given major.minor version (often trunk.) I install gems using the --user-install option, and set the bin stubs to run with just ruby.exe, assuming that PATH was set to the correct location of the Ruby build. So, this is an install location outside of RbConfig::TOPDIR.

For bin stub locations within RbConfig::TOPDIR, if the location matches RbConfig::CONFIG['bindir'], "%~dp0ruby.exe" is used. If it's a different location, the PR uses Pathname to create a relative path. Both of these leave the whole build portable.

There is one issue that this affects, although the issue is also problematic with the current bin stubs, and that is if one tries to install extension gems outside of RbConfig::TOPDIR for both 32 and 64 bit versions.

Off topic - fat binaries:

Since many gems repos have limited resources, and often more so re Windows, I'm working on PowerShell scripts to build fat binary gems on Appveyor, as all the build tools are already installed, and no one needs to determine how to cross compile (often needing both DevKit and MSYS2). One thing I expect to do is combine both 32 and 64 builds in one gem file, and adjust the load code to account for that. Obviously, the gem is larger, but since many gems' *.so files aren't stripped, once that's done, the increase in size isn't that great...

A good example of the mess that can occur is building EventMachine or Puma, which both build with OpenSSL. EM currently supports Ruby 2.0 thru 2.5. To build it to match the standard Ruby builds, it requires four different OpenSSL packages (two DevKit, two MinGW), and the two builds systems (DevKit & MSYS2 / MinGW). That's just too much for people who normally use MacOS or *nix...

Thanks, Greg

LGTM.
Note that Pathname#relative_path_from raises an ArgumentError when drive letters differ.

☔ The latest upstream changes (presumably #2299) made this pull request unmergeable. Please resolve the merge conflicts.

I discussed this issue with @unak at RubyKaigi 2018. But I and @unak couldn't understand this problem.

@MSP-Greg Can you add reproduction instructions for this problem?

@hsbt

The current method hard codes the ruby.exe path into the bin stub. First of all, that is a portability issue.

I consider this issue (and the code in the PR) to be divided into three categories, dependent on the relationship of the bin stub folder to RbConfig::TOPDIR.

1. No option is used, so bin stubs are placed in RbConfig::CONFIG['bindir']

For portability, RubyInstaller, RubyInstaller2 and ruby-loco builds modify these bin stubs by changing the hard coded path to "%~dp0ruby.exe", which resolves to a ruby.exe file in the current script (*.bar, .cmd) folder. FYI, ~dp0 is similar to __dir__. This makes the installs portable.

2. Gem installed with --bindir option, --bindir is within RbConfig::TOPDIR

The code determines a relative path to RbConfig::CONFIG['bindir'] from --bindir, and uses that for the ruby call.

3. Gem installed with --bindir option, and --bindir is not within RbConfig::TOPDIR, or, --user-install option is used.

Since these bin stub are outside of RbConfig::TOPDIR, the ruby call in the bin stub is simply ruby.exe, allowing whatever build of ruby is in one’s PATH to be run.

Of the above three cases, the first two behave exactly like the current code, but result in portable builds.

The third case is where ‘desired functionality’ comes into play, but only if one has more than one ruby build installed.

Gems with bin stubs behave in different ways. One common behavior is to operate on a standardized file name in the folder it’s launched from (like Gemfile or Rakefile). Since this assumes your path is set to the version of Ruby that you want to run it with, using ruby.exe will work fine.

Another use case is testing with multiple builds/versions of Ruby with the same ABI version. One would often use --user—install so all would have the same gems available. Because of the ’multiple’, one is always setting PATH to switch between them, and hence, ruby.exe in the bin stub works.

I could probably come up with more ‘plus’ cases for category 3, and I could probably come with some ‘minus’ use cases also.

Summing up, if one has only one Ruby build, this PR works the same as the current code. With multiple builds/versions and a bin stub folder outside of RbConfig::TOPDIR, things get messy. I prefer using ruby.exe for the ruby call in these situations.

Thanks, Greg

@hsbt

Sorry, I missed "Can you add reproduction instructions for this problem?"

I hope you agree with the above options 1 & 2 above, as with the PR, the current code does not allow the ruby install to be moved (and PATH adjusted accordingly), as the '--user-install` bin stubs are calling ruby in a different path. The PR code does.

So we're down to repo'ing option 3. First of all, I may often have several console windows open, but I also have a script that allows me to change Ruby versions, it modifies PATH and also sets the window title. So I'm assuming that users are not going to be prefixing rake with a long path. They're going to have their

As to option 3 (typically a --user-install gem), two use cases:

1. Move the ruby install as above.

2. Either:

Install more that one Ruby build of a given ABI version (say 2.5.0 and 2.5.1). then --user-install a gem with a bin stub. It won't work with both, but it should.

Or

Install both 32 & 64 bit builds of a given ABI version, along with binary gems for each. If you switch between the builds by changing your path, the bin stub will work with the PR's code, with the current code, they will not.

I hope this makes sense. Thanks, Greg

I think Gregs proposal is a pretty good solution. This is the only issue for using Ruby in a portable way on Windows right now. It would be nice if this could be fixed.

I agree with larskanis, Gregs solution looks good and would have solved my issues from #1402 as well. Thank you Greg!

I understood. Thanks.

@bundlerbot r+

📌 Commit 974c7ae has been approved by hsbt

⌛ Testing commit 974c7ae with merge 2606d58...

☀️ Test successful - status-travis
Approved by: hsbt
Pushing 2606d58 to master...