Repository navigation
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces the ClientConfigurationManager to periodically poll GetClientConfiguration in the background and manage client configurations per instance and application profile. It integrates this manager into DataConnectionImpl and adds options to control polling. The review feedback focuses on optimizing performance by using a custom key struct with transparent hashing and equality for config_managers_ to avoid heap allocations on lookup. Additionally, several style guide violations regarding the use of auto hiding domain objects, protobuf messages, or function return types were identified.
| absl::flat_hash_map<std::pair<std::string, std::string>, | ||
| std::shared_ptr<ClientConfigurationManager>> | ||
| config_managers_; |
There was a problem hiding this comment.
Using std::pair<std::string, std::string> as the key type for config_managers_ causes two heap allocations on every lookup (such as on every data operation) because std::string_view arguments must be converted to std::string to construct the key. We can avoid these allocations entirely by defining a custom key struct with transparent hashing and equality, allowing heterogeneous lookup with std::pair<std::string_view, std::string_view>.
struct ConfigManagerKey {
std::string instance_id;
std::string app_profile_id;
struct Hash {
using is_transparent = void;
std::size_t operator()(ConfigManagerKey const& key) const {
return absl::Hash<std::pair<std::string_view, std::string_view>>()(
{key.instance_id, key.app_profile_id});
}
std::size_t operator()(std::pair<std::string_view, std::string_view> const& p) const {
return absl::Hash<std::pair<std::string_view, std::string_view>>()(p);
}
};
struct Equal {
using is_transparent = void;
bool operator()(ConfigManagerKey const& lhs, ConfigManagerKey const& rhs) const {
return lhs.instance_id == rhs.instance_id &&
lhs.app_profile_id == rhs.app_profile_id;
}
bool operator()(ConfigManagerKey const& lhs,
std::pair<std::string_view, std::string_view> const& rhs) const {
return lhs.instance_id == rhs.first &&
lhs.app_profile_id == rhs.second;
}
};
};
absl::flat_hash_map<ConfigManagerKey,
std::shared_ptr<ClientConfigurationManager>,
ConfigManagerKey::Hash,
ConfigManagerKey::Equal>
config_managers_;| absl::flat_hash_map<std::pair<std::string, std::string>, | ||
| std::shared_ptr<ClientConfigurationManager>> | ||
| managers; |
| auto iter = config_managers_.find(std::pair<std::string, std::string>{ | ||
| std::string{instance_name}, std::string{app_profile}}); |
There was a problem hiding this comment.
With the transparent ConfigManagerKey defined in the header, we can perform the lookup using a std::pair<std::string_view, std::string_view> to avoid any heap allocations on the hot path.
auto iter = config_managers_.find(std::make_pair(instance_name, app_profile));References
- When using a map with transparent lookups (e.g.,
absl::flat_hash_map<std::string, ...>), preferfind()overoperator[]to avoid constructing a key object on every call. This is often more performant, especially if cache hits are common.
| std::pair<std::string, std::string> key{std::string{instance_name}, | ||
| std::string{app_profile}}; | ||
| { | ||
| std::shared_lock<std::shared_mutex> lk(config_managers_mu_); | ||
| if (shutting_down_) return nullptr; | ||
| if (auto iter = config_managers_.find(key); | ||
| iter != config_managers_.end()) { | ||
| return iter->second; | ||
| } | ||
| } | ||
|
|
||
| auto stub = stub_manager_->GetStub(instance_name); | ||
| auto candidate = ClientConfigurationManager::Create( | ||
| background_->cq(), std::move(stub), key.first, key.second, options_); | ||
|
|
||
| std::shared_ptr<ClientConfigurationManager> manager; | ||
| bool inserted = false; | ||
| { | ||
| std::unique_lock<std::shared_mutex> lk(config_managers_mu_); | ||
| if (shutting_down_) return nullptr; | ||
| auto res = config_managers_.emplace(std::move(key), std::move(candidate)); | ||
| manager = res.first->second; | ||
| inserted = res.second; | ||
| } |
There was a problem hiding this comment.
Similarly, we can perform the initial lookup using a std::pair<std::string_view, std::string_view> to avoid allocations when the manager already exists. We only construct the allocating ConfigManagerKey when inserting a new manager.
auto key = std::make_pair(instance_name, app_profile);
{
std::shared_lock<std::shared_mutex> lk(config_managers_mu_);
if (shutting_down_) return nullptr;
if (auto iter = config_managers_.find(key);
iter != config_managers_.end()) {
return iter->second;
}
}
auto stub = stub_manager_->GetStub(instance_name);
auto candidate = ClientConfigurationManager::Create(
background_->cq(), std::move(stub), std::string{instance_name},
std::string{app_profile}, options_);
std::shared_ptr<ClientConfigurationManager> manager;
bool inserted = false;
{
std::unique_lock<std::shared_mutex> lk(config_managers_mu_);
if (shutting_down_) return nullptr;
auto res = config_managers_.emplace(
ConfigManagerKey{std::string{instance_name}, std::string{app_profile}},
std::move(candidate));
manager = res.first->second;
inserted = res.second;
}References
- When using a map with transparent lookups (e.g.,
absl::flat_hash_map<std::string, ...>), preferfind()overoperator[]to avoid constructing a key object on every call. This is often more performant, especially if cache hits are common.
| interval = ProtoDurationToMillisCapped(config.polling_interval(), | ||
| kMaxPollingInterval); | ||
| } else { | ||
| auto const defaults = DefaultClientConfiguration(); |
There was a problem hiding this comment.
According to the repository style guide, we should reject auto when it hides domain objects or protobuf messages/fields. Please use the explicit type google::bigtable::v2::ClientConfiguration here.
google::bigtable::v2::ClientConfiguration const defaults =
DefaultClientConfiguration();References
- Reject auto when it hides domain objects, protobuf messages/fields, or function return types. (link)
| config.polling_configuration().validity_duration(), | ||
| kMaxValidityDuration); | ||
| } | ||
| auto const defaults = DefaultClientConfiguration(); |
There was a problem hiding this comment.
According to the repository style guide, we should reject auto when it hides domain objects or protobuf messages/fields. Please use the explicit type google::bigtable::v2::ClientConfiguration here.
google::bigtable::v2::ClientConfiguration const defaults =
DefaultClientConfiguration();References
- Reject auto when it hides domain objects, protobuf messages/fields, or function return types. (link)
| last_error_.reset(); | ||
| last_error_at_.reset(); | ||
|
|
||
| auto effective = ApplyOverrides(*std::move(result)); |
There was a problem hiding this comment.
According to the repository style guide, we should reject auto when it hides domain objects or protobuf messages/fields. Please use the explicit type google::bigtable::v2::ClientConfiguration here.
google::bigtable::v2::ClientConfiguration effective =
ApplyOverrides(*std::move(result));References
- Reject auto when it hides domain objects, protobuf messages/fields, or function return types. (link)
| auto cq = cq_; | ||
| auto options = options_; |
There was a problem hiding this comment.
According to the repository style guide, we should reject auto when it hides domain objects or function return types. Please use explicit types for CompletionQueue and internal::ImmutableOptions here.
CompletionQueue cq = cq_;
internal::ImmutableOptions options = options_;References
- Reject auto when it hides domain objects, protobuf messages/fields, or function return types. (link)
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #16670 +/- ##
==========================================
- Coverage 92.63% 92.63% -0.01%
==========================================
Files 2271 2274 +3
Lines 224833 225827 +994
==========================================
+ Hits 208276 209185 +909
- Misses 16557 16642 +85 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
scotthart
left a comment
There was a problem hiding this comment.
Thanks for the clean ConfigManagerKey transparent lookup and style updates in ad5eaf1d! A few build/CI issues and logic findings from reviewing the current HEAD:
-
Fix
libc++compilation failure inComputeRetryBackoff(google/cloud/bigtable/internal/client_configuration_manager.cc:707-710):- On Linux x86_64 with
-stdlib=libc++(libcxx-pr,msan-pr,ubsan-pr,xsan-pr),std::int64_tislongwhilestd::chrono::milliseconds::repislong long, so(std::max)(std::int64_t{0}, config_.initial_retry_backoff.count())and(std::max)(base_ms, config_.max_retry_backoff.count())fail template deduction. Clamping asstd::chrono::millisecondsbefore calling.count()avoids therepmismatch:std::chrono::milliseconds const base = (std::max)(std::chrono::milliseconds{0}, config_.initial_retry_backoff); std::chrono::milliseconds const max_delay = (std::max)(base, config_.max_retry_backoff); std::int64_t const base_ms = base.count(); std::int64_t const max_ms = max_delay.count();
- On Linux x86_64 with
-
Fix
clang-tidy-pr-Werrorfailures (client_configuration_manager.cc:551,client_configuration_manager.h:225):OnFetchAttemptCompleted(client_configuration_manager.cc:551) exceedsgoogle/cloud/bigtable/.clang-tidy'sreadability-function-cognitive-complexitythreshold (28 > 25); extracting the lockedresult.ok()and error branches into private helpers will bring it below threshold.std::atomic<std::thread::id> delivering_thread{};(client_configuration_manager.h:225) triggersreadability-redundant-member-init. Initializing via the 1-arg constructorstd::atomic<std::thread::id> delivering_thread{std::thread::id{}};satisfiesclang-tidywhile still value-initializing on pre-P0883R2 C++17 standard libraries.
-
AddDiversionLoadsListenerlate registration &config_seq_ == 0when the first poll matchescurrent_config_(client_configuration_manager.cc:454-480,586-591):AddDiversionLoadsListenerchecksif (seq == 0) return;to skip the bootstrap config, but passes/*fire_immediately=*/falsetoAddListener. If a listener is registered after the initial eager poll has already completed (config_seq_ >= 1), it won't receive the currentDiversionLoadsuntil a future poll returns a differentClientConfiguration(or never, ifstop_pollingistrueor subsequent responses are unchanged). Passing/*fire_immediately=*/trueletsif (seq == 0) return;skip the bootstrap state while immediately delivering already-fetched (seq >= 1) loads.- In
OnFetchAttemptCompleted(lines 586-591), if the first successful poll (config_seq_ == 0) produces aneffectiveconfig equal tocurrent_config_(e.g., server returns defaults, orBIGTABLE_CLIENT_CONFIG_OVERRIDEalready merged those values intocurrent_config_),MessageDifferencer::Equals(effective, current_config_)returns early without advancingconfig_seq_from0to1. Guarding that early return withif (config_seq_ > 0 && google::protobuf::util::MessageDifferencer::Equals(effective, current_config_))ensures the first successful poll always advancesconfig_seq_to1.
-
Drain in-flight callback delivery in
UnregisterCallback(client_configuration_manager.cc:404-413):- Unlike
Shutdown(),UnregisterCallbackonly setss->active = falseand erasesidfromlisteners_without waiting ons->delivery_mu. Lockings->delivery_muoutsideself->mu_whens != nullptr && s->delivering_thread.load(std::memory_order_acquire) != std::this_thread::get_id()prevents an in-flight callback on another thread from outlivingUnregisterCallback()while keeping self-unregistration deadlock-free.
- Unlike
Summary
Add
ClientConfigurationManager(google/cloud/bigtable/internal/client_configuration_manager.{h,cc}) to periodically fetch and managegoogle.bigtable.v2.ClientConfigurationper(instance_name, app_profile_id)onCompletionQueue, and hook it intoDataConnectionImpl/MakeDataConnection.Changes
AsyncGetClientConfigurationonBigtableStub:"GetClientConfiguration"togen_async_rpcsingenerator/generator_config.textprotoand regenerateBigtableStuband its decorators (BigtableAuth,BigtableMetadata,BigtableLogging,BigtableRoundRobin,BigtableTracingStub).AsyncGetClientConfigurationforwarding inBigtableChannelRefresh,BigtableRandomTwoLeastUsed, andMockBigtableStub.ClientConfigurationManager:DefaultClientConfiguration(),ResolveDiversionLoads()(supportingglobal_diversion,per_scope_diversion, and fallback tosession_load), and effective polling/validity/retry helpers (1mminimum polling interval clamp, safe validity duration cap, explicit0retry support).CompletionQueuecontinuations so single-threaded backgroundCompletionQueuepools cannot self-deadlock.MessageDifferencer::Equals), honorstop_polling, and fall back toDefaultClientConfiguration()when a poll fails aftervalidity_durationelapses.AddListener,AddSessionPoolConfigurationListener,AddDiversionLoadsListener) with monotonic sequence ordering, per-slice diff suppression,BIGTABLE_CLIENT_CONFIG_OVERRIDEtextproto support, snapshot/history inspection, and cleanShutdown()that cancels pending timers and in-flight RPCs immediately.DataConnectionImpl/MakeDataConnection:experimental::EnableClientConfigurationPollingOption(enabled by default inDefaultDataOptionsunlessBIGTABLE_EMULATOR_HOSTis set; disabled incbt_test_proxy).MakeDataConnection(instances, options), lazily initialize per(instance_name, app_profile_id)on data operations, and shut down all managers cleanly in~DataConnectionImpl().client_configuration_manager_test.cc,data_connection_impl_test.cc,data_connection_test.cc, anddefaults_test.cc.CMakeLists.txt,google_cloud_cpp_bigtable.bzl, andbigtable_client_unit_tests.bzl.Validation
bazel test --test_tag_filters=-integration-test //google/cloud/bigtable/...(89/89 test targets passed)bazel build //google/cloud/bigtable/test_proxy:cbt_test_proxy_main