Skip to content

Commit 8c78d24

Browse files
segiddinssegiddins
committed
Add comprehensive wheel platform support to RubyGems
Implements wheel-type platforms (whl-{ruby}-{abi}-{platform}) for binary gem support. Key changes: - Add Gem::Platform::Wheel class for wheel platform parsing and matching - Add Gem::Platform::Specific class for structured environment representation - Implement comprehensive wheel tag generation with platform compatibility - Add Linux-specific platform detection (manylinux/musllinux) with ELF parsing - Update platform matching logic to handle wheel vs traditional platform resolution - Maintain full backwards compatibility with existing platform functionality All tests pass with enhanced platform resolution supporting both traditional and wheel platform formats for future binary gem distribution. Add Bundler integration tests for wheel platform support Add comprehensive tests to validate wheel platform gem building and resolution in Bundler: - Test wheel platform gem creation with correct platform naming - Test fallback to ruby gems when wheel platforms don't match - Test multi-tag wheel platform handling (currently skipped) - Test platform resolution priority (currently skipped) - Test lockfile wheel platform recording (currently skipped) Most tests are currently skipped as they require Bundler resolver updates to handle Gem::Platform::Wheel objects. The working test demonstrates that wheel platform gems can be built successfully with full platform names (e.g. wheel_native-1.0.0-whl-rb33-x86_64_linux.gem).
1 parent 9934845 commit 8c78d24

24 files changed

+4391
-118
lines changed

‎Manifest.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -456,6 +456,11 @@ lib/rubygems/package/tar_writer.rb
456456
lib/rubygems/package_task.rb
457457
lib/rubygems/path_support.rb
458458
lib/rubygems/platform.rb
459+
lib/rubygems/platform/elffile.rb
460+
lib/rubygems/platform/manylinux.rb
461+
lib/rubygems/platform/musllinux.rb
462+
lib/rubygems/platform/specific.rb
463+
lib/rubygems/platform/wheel.rb
459464
lib/rubygems/psych_tree.rb
460465
lib/rubygems/query_utils.rb
461466
lib/rubygems/rdoc.rb

‎lib/rubygems/basic_specification.rb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ def base_dir
7272

7373
def contains_requirable_file?(file)
7474
if ignored?
75-
if platform == Gem::Platform::RUBY || Gem::Platform.local === platform
75+
if platform == Gem::Platform::RUBY || Gem::Platform::Specific.local === platform
7676
warn "Ignoring #{full_name} because its extensions are not built. " \
7777
"Try: gem pristine #{name} --version #{version}"
7878
end

‎lib/rubygems/commands/pristine_command.rb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ def execute
125125
end
126126
end
127127

128-
specs = specs.select {|spec| spec.platform == RUBY_ENGINE || Gem::Platform.local === spec.platform || spec.platform == Gem::Platform::RUBY }
128+
specs = specs.select {|spec| spec.platform == RUBY_ENGINE || Gem::Platform::Specific.local === spec.platform || spec.platform == Gem::Platform::RUBY }
129129

130130
if specs.to_a.empty?
131131
raise Gem::Exception,

‎lib/rubygems/platform.rb‎

Lines changed: 134 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@
88
# See `gem help platform` for information on platform matching.
99

1010
class Gem::Platform
11+
require_relative "platform/elffile"
12+
require_relative "platform/manylinux"
13+
require_relative "platform/musllinux"
14+
require_relative "platform/wheel"
15+
require_relative "platform/specific"
16+
1117
@local = nil
1218

1319
attr_accessor :cpu, :os, :version
@@ -49,19 +55,23 @@ def self.match_gem?(platform, gem_name)
4955
raise "Not a string: #{gem_name.inspect}" unless String === gem_name
5056

5157
if REUSE_AS_BINARY_ON_TRUFFLERUBY.include?(gem_name)
52-
match_platforms?(platform, [Gem::Platform::RUBY, Gem::Platform.local])
58+
match_platforms?(platform, [Gem::Platform::RUBY, Gem::Platform::Specific.local])
5359
else
54-
match_platforms?(platform, Gem.platforms)
60+
match_platforms?(platform, Gem.platforms.map {|pl| Specific.local(pl) })
5561
end
5662
end
5763
else
5864
def self.match_gem?(platform, gem_name)
59-
match_platforms?(platform, Gem.platforms)
65+
match_platforms?(platform, Gem.platforms.map {|pl| Specific.local(pl) })
6066
end
6167
end
6268

6369
def self.sort_priority(platform)
64-
platform == Gem::Platform::RUBY ? -1 : 1
70+
case platform
71+
when Gem::Platform::RUBY then -1
72+
when Gem::Platform::Wheel then 2 # Higher priority than traditional platforms
73+
else 1
74+
end
6575
end
6676

6777
def self.installable?(spec)
@@ -78,64 +88,79 @@ def self.new(arch) # :nodoc:
7888
Gem::Platform.local
7989
when Gem::Platform::RUBY, nil, "" then
8090
Gem::Platform::RUBY
91+
when /^whl-/ then
92+
Gem::Platform::Wheel.new(arch)
93+
when Wheel then
94+
Wheel.new(arch)
95+
when Specific then
96+
Specific.new(arch)
97+
when / v:\d+/
98+
Gem::Platform::Specific.parse(arch)
8199
else
82100
super
83101
end
84102
end
85103

86104
def initialize(arch)
87105
case arch
106+
when String then
88107
when Array then
108+
raise "Array #{arch.inspect} is not a valid platform" unless arch.size <= 3
89109
@cpu, @os, @version = arch
90-
when String then
91-
cpu, os = arch.sub(/-+$/, "").split("-", 2)
92-
93-
@cpu = if cpu&.match?(/i\d86/)
94-
"x86"
95-
else
96-
cpu
97-
end
98-
99-
if os.nil?
100-
@cpu = nil
101-
os = cpu
102-
end # legacy jruby
103-
104-
@os, @version = case os
105-
when /aix-?(\d+)?/ then ["aix", $1]
106-
when /cygwin/ then ["cygwin", nil]
107-
when /darwin-?(\d+)?/ then ["darwin", $1]
108-
when "macruby" then ["macruby", nil]
109-
when /^macruby-?(\d+(?:\.\d+)*)?/ then ["macruby", $1]
110-
when /freebsd-?(\d+)?/ then ["freebsd", $1]
111-
when "java", "jruby" then ["java", nil]
112-
when /^java-?(\d+(?:\.\d+)*)?/ then ["java", $1]
113-
when /^dalvik-?(\d+)?$/ then ["dalvik", $1]
114-
when /^dotnet$/ then ["dotnet", nil]
115-
when /^dotnet-?(\d+(?:\.\d+)*)?/ then ["dotnet", $1]
116-
when /linux-?(\w+)?/ then ["linux", $1]
117-
when /mingw32/ then ["mingw32", nil]
118-
when /mingw-?(\w+)?/ then ["mingw", $1]
119-
when /(mswin\d+)(?:[_-](\d+))?/ then
120-
os = $1
121-
version = $2
122-
@cpu = "x86" if @cpu.nil? && os.end_with?("32")
123-
[os, version]
124-
when /netbsdelf/ then ["netbsdelf", nil]
125-
when /openbsd-?(\d+\.\d+)?/ then ["openbsd", $1]
126-
when /solaris-?(\d+\.\d+)?/ then ["solaris", $1]
127-
when /wasi/ then ["wasi", nil]
128-
# test
129-
when /^(\w+_platform)-?(\d+)?/ then [$1, $2]
130-
else ["unknown", nil]
131-
end
132-
when Gem::Platform then
110+
return
111+
when Gem::Platform
133112
@cpu = arch.cpu
134113
@os = arch.os
135114
@version = arch.version
115+
return
136116
else
137117
raise ArgumentError, "invalid argument #{arch.inspect}"
138118
end
119+
120+
cpu, os = arch.sub(/-+$/, "").split("-", 2)
121+
122+
@cpu = if cpu&.match?(/i\d86/)
123+
"x86"
124+
elsif cpu == "dotnet"
125+
os = "dotnet-#{os}"
126+
nil
127+
else
128+
cpu
129+
end
130+
131+
if os.nil?
132+
@cpu = nil
133+
os = cpu
134+
end # legacy jruby
135+
136+
@os, @version = case os
137+
when /aix-?(\d+)?/ then ["aix", $1]
138+
when /cygwin/ then ["cygwin", nil]
139+
when /darwin-?(\d+)?/ then ["darwin", $1]
140+
when "macruby" then ["macruby", nil]
141+
when /^macruby-?(\d+(?:\.\d+)*)?/ then ["macruby", $1]
142+
when /freebsd-?(\d+)?/ then ["freebsd", $1]
143+
when "java", "jruby" then ["java", nil]
144+
when /^java-?(\d+(?:\.\d+)*)?/ then ["java", $1]
145+
when /^dalvik-?(\d+)?$/ then ["dalvik", $1]
146+
when "dotnet" then ["dotnet", nil]
147+
when /^dotnet-?(\d+(?:\.\d+)*)?/ then ["dotnet", $1]
148+
when /linux-?(\w+)?/ then ["linux", $1]
149+
when /mingw32/ then ["mingw32", nil]
150+
when /mingw-?(\w+)?/ then ["mingw", $1]
151+
when /(mswin\d+)(?:[_-](\d+))?/ then
152+
os = $1
153+
version = $2
154+
@cpu = "x86" if @cpu.nil? && os.end_with?("32")
155+
[os, version]
156+
when /netbsdelf/ then ["netbsdelf", nil]
157+
when /openbsd-?(\d+\.\d+)?/ then ["openbsd", $1]
158+
when /solaris-?(\d+\.\d+)?/ then ["solaris", $1]
159+
when /wasi/ then ["wasi", nil]
160+
# test
161+
when /^(\w+_platform)-?(\d+)?/ then [$1, $2]
162+
else ["unknown", nil]
163+
end
139164
end
140165

141166
def to_a
@@ -218,25 +243,9 @@ def normalized_linux_version
218243

219244
def =~(other)
220245
case other
221-
when Gem::Platform then # nop
222-
when String then
223-
# This data is from http://gems.rubyforge.org/gems/yaml on 19 Aug 2007
224-
other = case other
225-
when /^i686-darwin(\d)/ then ["x86", "darwin", $1]
226-
when /^i\d86-linux/ then ["x86", "linux", nil]
227-
when "java", "jruby" then [nil, "java", nil]
228-
when /^dalvik(\d+)?$/ then [nil, "dalvik", $1]
229-
when /dotnet(\-(\d+\.\d+))?/ then ["universal","dotnet", $2]
230-
when /mswin32(\_(\d+))?/ then ["x86", "mswin32", $2]
231-
when /mswin64(\_(\d+))?/ then ["x64", "mswin64", $2]
232-
when "powerpc-darwin" then ["powerpc", "darwin", nil]
233-
when /powerpc-darwin(\d)/ then ["powerpc", "darwin", $1]
234-
when /sparc-solaris2.8/ then ["sparc", "solaris", "2.8"]
235-
when /universal-darwin(\d)/ then ["universal", "darwin", $1]
236-
else other
237-
end
238-
239-
other = Gem::Platform.new other
246+
when Gem::Platform, Gem::Platform::Wheel
247+
when Gem::Platform::Specific then other = other.platform
248+
when String then other = Gem::Platform.new(other)
240249
else
241250
return nil
242251
end
@@ -278,7 +287,15 @@ class << self
278287
# Returns the generic platform for the given platform.
279288

280289
def generic(platform)
281-
return Gem::Platform::RUBY if platform.nil? || platform == Gem::Platform::RUBY
290+
case platform
291+
when NilClass, Gem::Platform::RUBY
292+
return Gem::Platform::RUBY
293+
when Gem::Platform::Wheel
294+
return platform
295+
when Gem::Platform
296+
else
297+
raise ArgumentError, "invalid argument #{platform.inspect}"
298+
end
282299

283300
GENERIC_CACHE[platform] ||= begin
284301
found = GENERICS.find do |match|
@@ -295,6 +312,48 @@ def platform_specificity_match(spec_platform, user_platform)
295312
return -1 if spec_platform == user_platform
296313
return 1_000_000 if spec_platform.nil? || spec_platform == Gem::Platform::RUBY || user_platform == Gem::Platform::RUBY
297314

315+
# Handle Specific user platforms
316+
if user_platform.is_a?(Gem::Platform::Specific)
317+
case spec_platform
318+
when Gem::Platform::Wheel
319+
# Use each_possible_match to find the best match for wheels
320+
# Return negative values to indicate better matches than traditional platforms
321+
index = user_platform.each_possible_match.to_a.index do |abi_tag, platform_tag|
322+
# Check if the wheel matches this generated tag pair
323+
spec_platform.ruby_abi_tag.split(".").include?(abi_tag) && spec_platform.platform_tags.split(".").include?(platform_tag)
324+
end
325+
return(if index == 0
326+
-10
327+
elsif index
328+
index
329+
else
330+
1_000_000
331+
end)
332+
when Gem::Platform
333+
# For traditional platforms with Specific user platforms, use original scoring
334+
user_platform = user_platform.platform
335+
return -1 if spec_platform == user_platform # Better than non-matching wheels but worse than matching wheels
336+
else
337+
raise ArgumentError, "spec_platform must be Gem::Platform or Gem::Platform::Wheel, given #{spec_platform.inspect}"
338+
end
339+
end
340+
341+
# Handle traditional Platform user platforms
342+
case user_platform
343+
when Gem::Platform
344+
# For wheel spec platforms with traditional user platforms, create a Specific user platform
345+
if spec_platform.is_a?(Gem::Platform::Wheel)
346+
specific_user = Gem::Platform::Specific.local(user_platform)
347+
return platform_specificity_match(spec_platform, specific_user)
348+
end
349+
when Gem::Platform::Specific
350+
# TODO: also match on ruby ABI tags!
351+
user_platform = user_platform.platform
352+
return -1 if spec_platform == user_platform
353+
else
354+
raise ArgumentError, "user_platform must be Gem::Platform or Gem::Platform::Specific, given #{user_platform.inspect}"
355+
end
356+
298357
os_match(spec_platform, user_platform) +
299358
cpu_match(spec_platform, user_platform) * 10 +
300359
version_match(spec_platform, user_platform) * 100
@@ -303,34 +362,34 @@ def platform_specificity_match(spec_platform, user_platform)
303362
##
304363
# Sorts and filters the best platform match for the given matching specs and platform.
305364

306-
def sort_and_filter_best_platform_match(matching, platform)
365+
def sort_and_filter_best_platform_match(matching, user_platform)
307366
return matching if matching.one?
308367

309-
exact = matching.select {|spec| spec.platform == platform }
368+
exact = matching.select {|spec| spec.platform == user_platform }
310369
return exact if exact.any?
311370

312-
sorted_matching = sort_best_platform_match(matching, platform)
371+
sorted_matching = sort_best_platform_match(matching, user_platform)
313372
exemplary_spec = sorted_matching.first
314373

315-
sorted_matching.take_while {|spec| same_specificity?(platform, spec, exemplary_spec) && same_deps?(spec, exemplary_spec) }
374+
sorted_matching.take_while {|spec| same_specificity?(user_platform, spec, exemplary_spec) && same_deps?(spec, exemplary_spec) }
316375
end
317376

318377
##
319378
# Sorts the best platform match for the given matching specs and platform.
320379

321-
def sort_best_platform_match(matching, platform)
380+
def sort_best_platform_match(matching, user_platform)
322381
matching.sort_by.with_index do |spec, i|
323382
[
324-
platform_specificity_match(spec.platform, platform),
383+
platform_specificity_match(spec.platform, user_platform),
325384
i, # for stable sort
326385
]
327386
end
328387
end
329388

330389
private
331390

332-
def same_specificity?(platform, spec, exemplary_spec)
333-
platform_specificity_match(spec.platform, platform) == platform_specificity_match(exemplary_spec.platform, platform)
391+
def same_specificity?(user_platform, spec, exemplary_spec)
392+
platform_specificity_match(spec.platform, user_platform) == platform_specificity_match(exemplary_spec.platform, user_platform)
334393
end
335394

336395
def same_deps?(spec, exemplary_spec)

0 commit comments

Comments
 (0)