This CL makes a bunch of changes (mostly to raw_hash_set which underlies flat_hash_set and flat_hash_map). Techniques used: * Extract code that does not depend on the specific hash table type into common (non-inlined) functions. * Place ABSL_ATTRIBUTE_NOINLINE directives judiciously. * Out-of-line some slow paths. Reduces sizes of some large binaries by ~0.5%. Has no significant performance impact on a few performance critical binaries. ## Speed of fleetbench micro-benchmarks Following is a histogram of %-age changes in [fleetbench](https://github.com/google/fleetbench) hot_swissmap_benchmark results. Negative numbers indicate a speedup caused by this change. Statistically insignificant changes are mapped to zero. XXX Also run and merge in cold_swissmap_benchmark Across all 351 benchmarks, the average speedup is 0.38%. The best speedup was -25%, worst slowdown was +6.81%. ``` Count: 351 Average: -0.382764 StdDev: 3.77807 Min: -25 Median: 0.435135 Max: 6.81 --------------------------------------------- [ -25, -10) 16 4.558% 4.558% # [ -9, -8) 2 0.570% 5.128% [ -8, -7) 1 0.285% 5.413% [ -7, -6) 1 0.285% 5.698% [ -6, -5) 2 0.570% 6.268% [ -5, -4) 5 1.425% 7.692% [ -4, -3) 13 3.704% 11.396% # [ -3, -2) 15 4.274% 15.670% # [ -2, -1) 26 7.407% 23.077% ## [ -1, 0) 14 3.989% 27.066% # [ 0, 1) 185 52.707% 79.772% ############ [ 1, 2) 14 3.989% 83.761% # [ 2, 3) 8 2.279% 86.040% # [ 3, 4) 7 1.994% 88.034% [ 4, 5) 32 9.117% 97.151% ## [ 5, 6) 6 1.709% 98.860% [ 6, 7) 4 1.140% 100.000% ``` We looked at the slowdowns and they do not seem worth worrying about. E.g., the worst one was: ``` BM_FindHit_Hot<::absl::node_hash_set,64>/set_size:4096/density:0 2.61ns ± 1% 2.79ns ± 1% +6.81% (p=0.008 n=5+5) ``` ## Detailed changes * Out-of-line slow paths in hash table sampler methods. * Explicitly unregister from sampler instead of from destructor. * Introduced a non-templated CommonFields struct that holds some of the hash table fields (infoz, ctrl, slots, size, capacity). This struct can be passed to new non-templated helpers. The struct is a private base class of raw_hash_set. * Made non-inlined InitializeSlots<> that is only templated on allocator and size/alignment of the slot type so that we can share instantiations across types that have the same size/alignment. * Moved some infrequently called code paths into non-inlined type-erased. functions. Pass a suite of type-specific function pointers to these routines for when they need to operate on slots. * Marked some methods as non-inlined. * Avoid unnecessary reinitialization in destructor. * Introduce UpdateSpine type-erased helper that is called from clear() and rehash(). PiperOrigin-RevId: 491413386 Change-Id: Ia5495c5a6ec73622a785a0d260e406ddb9085a7c
| Name |
Last commit
|
Last Update |
|---|---|---|
| .. | ||
| btree.h | Loading commit data... | |
| btree_container.h | Loading commit data... | |
| common.h | Loading commit data... | |
| common_policy_traits.h | Loading commit data... | |
| common_policy_traits_test.cc | Loading commit data... | |
| compressed_tuple.h | Loading commit data... | |
| compressed_tuple_test.cc | Loading commit data... | |
| container_memory.h | Loading commit data... | |
| container_memory_test.cc | Loading commit data... | |
| counting_allocator.h | Loading commit data... | |
| hash_function_defaults.h | Loading commit data... | |
| hash_function_defaults_test.cc | Loading commit data... | |
| hash_generator_testing.cc | Loading commit data... | |
| hash_generator_testing.h | Loading commit data... | |
| hash_policy_testing.h | Loading commit data... | |
| hash_policy_testing_test.cc | Loading commit data... | |
| hash_policy_traits.h | Loading commit data... | |
| hash_policy_traits_test.cc | Loading commit data... | |
| hashtable_debug.h | Loading commit data... | |
| hashtable_debug_hooks.h | Loading commit data... | |
| hashtablez_sampler.cc | Loading commit data... | |
| hashtablez_sampler.h | Loading commit data... | |
| hashtablez_sampler_force_weak_definition.cc | Loading commit data... | |
| hashtablez_sampler_test.cc | Loading commit data... | |
| inlined_vector.h | Loading commit data... | |
| layout.h | Loading commit data... | |
| layout_benchmark.cc | Loading commit data... | |
| layout_test.cc | Loading commit data... | |
| node_slot_policy.h | Loading commit data... | |
| node_slot_policy_test.cc | Loading commit data... | |
| raw_hash_map.h | Loading commit data... | |
| raw_hash_set.cc | Loading commit data... | |
| raw_hash_set.h | Loading commit data... | |
| raw_hash_set_allocator_test.cc | Loading commit data... | |
| raw_hash_set_benchmark.cc | Loading commit data... | |
| raw_hash_set_probe_benchmark.cc | Loading commit data... | |
| raw_hash_set_test.cc | Loading commit data... | |
| test_instance_tracker.cc | Loading commit data... | |
| test_instance_tracker.h | Loading commit data... | |
| test_instance_tracker_test.cc | Loading commit data... | |
| tracked.h | Loading commit data... | |
| unordered_map_constructor_test.h | Loading commit data... | |
| unordered_map_lookup_test.h | Loading commit data... | |
| unordered_map_members_test.h | Loading commit data... | |
| unordered_map_modifiers_test.h | Loading commit data... | |
| unordered_map_test.cc | Loading commit data... | |
| unordered_set_constructor_test.h | Loading commit data... | |
| unordered_set_lookup_test.h | Loading commit data... | |
| unordered_set_members_test.h | Loading commit data... | |
| unordered_set_modifiers_test.h | Loading commit data... | |
| unordered_set_test.cc | Loading commit data... |