-
-
Notifications
You must be signed in to change notification settings - Fork 5.7k
Remove idxfloor and shrink age, and maxprobe #48380
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
The only major benefit of |
that makes sense to me. |
…s full idxfloor will usually be <4 so the compute and memory time of tracking this number aren't worth much
…ance of missing a concurrent write due to overflow in age seems acceptable to me (these changes together shrink by 8 bytes)
a5ee881
to
8293543
Compare
Co-authored-by: Kristoffer Carlsson <[email protected]>
I believe we have some basic Dict benchmarks in BaseBenchmarks, so would be good to verify with those before merging |
@nanosoldier |
Your job failed. |
maxprobe could be arbitrarily large, if the hash function is poorly chosen (it just degrades the dict into an array) |
if the hash function is poorly chosen everything else about our |
For julia> struct X; x::Int64; end
julia> Base.hash(x::X) = UInt64(1)
julia> for n = 5000:5000:20000; print("n=",n);
@time (d = Dict(); for i=1:n; d[X(i)] = i; end)
end
n=5000 0.458749 seconds (12.52 M allocations: 213.679 MiB, 30.10% gc time, 0.72% compilation time)
n=10000 1.442519 seconds (50.03 M allocations: 785.996 MiB, 7.78% gc time)
n=15000 3.760717 seconds (112.53 M allocations: 1.699 GiB, 10.08% gc time)
n=20000 9.156206 seconds (200.07 M allocations: 3.070 GiB, 7.67% gc time) |
new(copy(d.slots), copy(d.keys), copy(d.vals), d.ndel, d.count, d.age, | ||
d.idxfloor, d.maxprobe) | ||
d.maxprobe) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess it can be moved to the previous line.
Co-authored-by: Petr Vana <[email protected]>
Now, the error from docs is that |
closing in favor of #59769 |
I believe this frees up 16 bytes for approximately free.
idxfloor
seems like a good idea, but it adds very little performance because with uniformly distributed hashes, on average, the idxfloor will be very low. The other change is to shrinkage
andmaxprobe
by 32 bits because there never should be 4 billion collisions next to each other, and age doesn't need to be 64 bits either.