Skip to content

impl(bigtable): add ClientConfigurationManager and wire into DataConnection - #16670

Open
sushanb wants to merge 3 commits into
googleapis:mainfrom
sushanb:feat/bigtable-client-configuration-manager
Open

sushanb wants to merge 3 commits into
googleapis:mainfrom
sushanb:feat/bigtable-client-configuration-manager

Conversation

@sushanb

@sushanb sushanb commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Add ClientConfigurationManager (google/cloud/bigtable/internal/client_configuration_manager.{h,cc}) to periodically fetch and manage google.bigtable.v2.ClientConfiguration per (instance_name, app_profile_id) on CompletionQueue, and hook it into DataConnectionImpl / MakeDataConnection.

Changes

  • Generate AsyncGetClientConfiguration on BigtableStub:
    • Add "GetClientConfiguration" to gen_async_rpcs in generator/generator_config.textproto and regenerate BigtableStub and its decorators (BigtableAuth, BigtableMetadata, BigtableLogging, BigtableRoundRobin, BigtableTracingStub).
    • Implement AsyncGetClientConfiguration forwarding in BigtableChannelRefresh, BigtableRandomTwoLeastUsed, and MockBigtableStub.
  • Add ClientConfigurationManager:
    • Provide DefaultClientConfiguration(), ResolveDiversionLoads() (supporting global_diversion, per_scope_diversion, and fallback to session_load), and effective polling/validity/retry helpers (1m minimum polling interval clamp, safe validity duration cap, explicit 0 retry support).
    • Drive eager initial fetch, jittered exponential-backoff retries on transient errors, and periodic refreshes asynchronously via CompletionQueue continuations so single-threaded background CompletionQueue pools cannot self-deadlock.
    • Refresh validity windows on unchanged responses while suppressing redundant listener notifications (MessageDifferencer::Equals), honor stop_polling, and fall back to DefaultClientConfiguration() when a poll fails after validity_duration elapses.
    • Provide thread-safe, re-entrant listener registration (AddListener, AddSessionPoolConfigurationListener, AddDiversionLoadsListener) with monotonic sequence ordering, per-slice diff suppression, BIGTABLE_CLIENT_CONFIG_OVERRIDE textproto support, snapshot/history inspection, and clean Shutdown() that cancels pending timers and in-flight RPCs immediately.
  • Hook into DataConnectionImpl / MakeDataConnection:
    • Add experimental::EnableClientConfigurationPollingOption (enabled by default in DefaultDataOptions unless BIGTABLE_EMULATOR_HOST is set; disabled in cbt_test_proxy).
    • Eagerly start managers for instances supplied to MakeDataConnection(instances, options), lazily initialize per (instance_name, app_profile_id) on data operations, and shut down all managers cleanly in ~DataConnectionImpl().
  • Tests & Build:
    • Add unit tests in client_configuration_manager_test.cc, data_connection_impl_test.cc, data_connection_test.cc, and defaults_test.cc.
    • Update CMakeLists.txt, google_cloud_cpp_bigtable.bzl, and bigtable_client_unit_tests.bzl.

Validation

  • bazel test --test_tag_filters=-integration-test //google/cloud/bigtable/... (89/89 test targets passed)
  • bazel build //google/cloud/bigtable/test_proxy:cbt_test_proxy_main

@sushanb
sushanb requested a review from a team as a code owner November 16, 2026 22:23
@product-auto-label product-auto-label Bot added the api: bigtable Issues related to the Bigtable API. label Nov 16, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the ClientConfigurationManager to periodically poll GetClientConfiguration in the background and manage client configurations per instance and application profile. It integrates this manager into DataConnectionImpl and adds options to control polling. The review feedback focuses on optimizing performance by using a custom key struct with transparent hashing and equality for config_managers_ to avoid heap allocations on lookup. Additionally, several style guide violations regarding the use of auto hiding domain objects, protobuf messages, or function return types were identified.

Comment on lines +165 to +167
absl::flat_hash_map<std::pair<std::string, std::string>,
std::shared_ptr<ClientConfigurationManager>>
config_managers_;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using std::pair<std::string, std::string> as the key type for config_managers_ causes two heap allocations on every lookup (such as on every data operation) because std::string_view arguments must be converted to std::string to construct the key. We can avoid these allocations entirely by defining a custom key struct with transparent hashing and equality, allowing heterogeneous lookup with std::pair<std::string_view, std::string_view>.

  struct ConfigManagerKey {
    std::string instance_id;
    std::string app_profile_id;

    struct Hash {
      using is_transparent = void;
      std::size_t operator()(ConfigManagerKey const& key) const {
        return absl::Hash<std::pair<std::string_view, std::string_view>>()(
            {key.instance_id, key.app_profile_id});
      }
      std::size_t operator()(std::pair<std::string_view, std::string_view> const& p) const {
        return absl::Hash<std::pair<std::string_view, std::string_view>>()(p);
      }
    };

    struct Equal {
      using is_transparent = void;
      bool operator()(ConfigManagerKey const& lhs, ConfigManagerKey const& rhs) const {
        return lhs.instance_id == rhs.instance_id &&
               lhs.app_profile_id == rhs.app_profile_id;
      }
      bool operator()(ConfigManagerKey const& lhs,
                      std::pair<std::string_view, std::string_view> const& rhs) const {
        return lhs.instance_id == rhs.first &&
               lhs.app_profile_id == rhs.second;
      }
    };
  };

  absl::flat_hash_map<ConfigManagerKey,
                      std::shared_ptr<ClientConfigurationManager>,
                      ConfigManagerKey::Hash,
                      ConfigManagerKey::Equal>
      config_managers_;

Comment on lines +263 to +265
absl::flat_hash_map<std::pair<std::string, std::string>,
std::shared_ptr<ClientConfigurationManager>>
managers;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

With the transparent ConfigManagerKey defined in the header, we can use decltype(config_managers_) to match the type of config_managers_ cleanly.

  decltype(config_managers_) managers;

Comment on lines +281 to +282
auto iter = config_managers_.find(std::pair<std::string, std::string>{
std::string{instance_name}, std::string{app_profile}});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

With the transparent ConfigManagerKey defined in the header, we can perform the lookup using a std::pair<std::string_view, std::string_view> to avoid any heap allocations on the hot path.

  auto iter = config_managers_.find(std::make_pair(instance_name, app_profile));
References
  1. When using a map with transparent lookups (e.g., absl::flat_hash_map<std::string, ...>), prefer find() over operator[] to avoid constructing a key object on every call. This is often more performant, especially if cache hits are common.

Comment on lines +292 to +315
std::pair<std::string, std::string> key{std::string{instance_name},
std::string{app_profile}};
{
std::shared_lock<std::shared_mutex> lk(config_managers_mu_);
if (shutting_down_) return nullptr;
if (auto iter = config_managers_.find(key);
iter != config_managers_.end()) {
return iter->second;
}
}

auto stub = stub_manager_->GetStub(instance_name);
auto candidate = ClientConfigurationManager::Create(
background_->cq(), std::move(stub), key.first, key.second, options_);

std::shared_ptr<ClientConfigurationManager> manager;
bool inserted = false;
{
std::unique_lock<std::shared_mutex> lk(config_managers_mu_);
if (shutting_down_) return nullptr;
auto res = config_managers_.emplace(std::move(key), std::move(candidate));
manager = res.first->second;
inserted = res.second;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Similarly, we can perform the initial lookup using a std::pair<std::string_view, std::string_view> to avoid allocations when the manager already exists. We only construct the allocating ConfigManagerKey when inserting a new manager.

  auto key = std::make_pair(instance_name, app_profile);
  {
    std::shared_lock<std::shared_mutex> lk(config_managers_mu_);
    if (shutting_down_) return nullptr;
    if (auto iter = config_managers_.find(key);
        iter != config_managers_.end()) {
      return iter->second;
    }
  }

  auto stub = stub_manager_->GetStub(instance_name);
  auto candidate = ClientConfigurationManager::Create(
      background_->cq(), std::move(stub), std::string{instance_name},
      std::string{app_profile}, options_);

  std::shared_ptr<ClientConfigurationManager> manager;
  bool inserted = false;
  {
    std::unique_lock<std::shared_mutex> lk(config_managers_mu_);
    if (shutting_down_) return nullptr;
    auto res = config_managers_.emplace(
        ConfigManagerKey{std::string{instance_name}, std::string{app_profile}},
        std::move(candidate));
    manager = res.first->second;
    inserted = res.second;
  }
References
  1. When using a map with transparent lookups (e.g., absl::flat_hash_map<std::string, ...>), prefer find() over operator[] to avoid constructing a key object on every call. This is often more performant, especially if cache hits are common.

interval = ProtoDurationToMillisCapped(config.polling_interval(),
kMaxPollingInterval);
} else {
auto const defaults = DefaultClientConfiguration();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository style guide, we should reject auto when it hides domain objects or protobuf messages/fields. Please use the explicit type google::bigtable::v2::ClientConfiguration here.

    google::bigtable::v2::ClientConfiguration const defaults =
        DefaultClientConfiguration();
References
  1. Reject auto when it hides domain objects, protobuf messages/fields, or function return types. (link)

config.polling_configuration().validity_duration(),
kMaxValidityDuration);
}
auto const defaults = DefaultClientConfiguration();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository style guide, we should reject auto when it hides domain objects or protobuf messages/fields. Please use the explicit type google::bigtable::v2::ClientConfiguration here.

  google::bigtable::v2::ClientConfiguration const defaults =
      DefaultClientConfiguration();
References
  1. Reject auto when it hides domain objects, protobuf messages/fields, or function return types. (link)

last_error_.reset();
last_error_at_.reset();

auto effective = ApplyOverrides(*std::move(result));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository style guide, we should reject auto when it hides domain objects or protobuf messages/fields. Please use the explicit type google::bigtable::v2::ClientConfiguration here.

      google::bigtable::v2::ClientConfiguration effective =
          ApplyOverrides(*std::move(result));
References
  1. Reject auto when it hides domain objects, protobuf messages/fields, or function return types. (link)

Comment on lines +639 to +640
auto cq = cq_;
auto options = options_;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

According to the repository style guide, we should reject auto when it hides domain objects or function return types. Please use explicit types for CompletionQueue and internal::ImmutableOptions here.

  CompletionQueue cq = cq_;
  internal::ImmutableOptions options = options_;
References
  1. Reject auto when it hides domain objects, protobuf messages/fields, or function return types. (link)

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.43337% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.63%. Comparing base (b77b9a1) to head (4aedbb5).
⚠️ Report is 24 commits behind head on main.

Files with missing lines Patch % Lines
.../bigtable/internal/client_configuration_manager.cc 94.38% 26 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16670      +/-   ##
==========================================
- Coverage   92.63%   92.63%   -0.01%     
==========================================
  Files        2271     2274       +3     
  Lines      224833   225827     +994     
==========================================
+ Hits       208276   209185     +909     
- Misses      16557    16642      +85     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@scotthart scotthart left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clean ConfigManagerKey transparent lookup and style updates in ad5eaf1d! A few build/CI issues and logic findings from reviewing the current HEAD:

  1. Fix libc++ compilation failure in ComputeRetryBackoff (google/cloud/bigtable/internal/client_configuration_manager.cc:707-710):

    • On Linux x86_64 with -stdlib=libc++ (libcxx-pr, msan-pr, ubsan-pr, xsan-pr), std::int64_t is long while std::chrono::milliseconds::rep is long long, so (std::max)(std::int64_t{0}, config_.initial_retry_backoff.count()) and (std::max)(base_ms, config_.max_retry_backoff.count()) fail template deduction. Clamping as std::chrono::milliseconds before calling .count() avoids the rep mismatch:
      std::chrono::milliseconds const base =
          (std::max)(std::chrono::milliseconds{0}, config_.initial_retry_backoff);
      std::chrono::milliseconds const max_delay =
          (std::max)(base, config_.max_retry_backoff);
      std::int64_t const base_ms = base.count();
      std::int64_t const max_ms = max_delay.count();
  2. Fix clang-tidy-pr -Werror failures (client_configuration_manager.cc:551, client_configuration_manager.h:225):

    • OnFetchAttemptCompleted (client_configuration_manager.cc:551) exceeds google/cloud/bigtable/.clang-tidy's readability-function-cognitive-complexity threshold (28 > 25); extracting the locked result.ok() and error branches into private helpers will bring it below threshold.
    • std::atomic<std::thread::id> delivering_thread{}; (client_configuration_manager.h:225) triggers readability-redundant-member-init. Initializing via the 1-arg constructor std::atomic<std::thread::id> delivering_thread{std::thread::id{}}; satisfies clang-tidy while still value-initializing on pre-P0883R2 C++17 standard libraries.
  3. AddDiversionLoadsListener late registration & config_seq_ == 0 when the first poll matches current_config_ (client_configuration_manager.cc:454-480, 586-591):

    • AddDiversionLoadsListener checks if (seq == 0) return; to skip the bootstrap config, but passes /*fire_immediately=*/false to AddListener. If a listener is registered after the initial eager poll has already completed (config_seq_ >= 1), it won't receive the current DiversionLoads until a future poll returns a different ClientConfiguration (or never, if stop_polling is true or subsequent responses are unchanged). Passing /*fire_immediately=*/true lets if (seq == 0) return; skip the bootstrap state while immediately delivering already-fetched (seq >= 1) loads.
    • In OnFetchAttemptCompleted (lines 586-591), if the first successful poll (config_seq_ == 0) produces an effective config equal to current_config_ (e.g., server returns defaults, or BIGTABLE_CLIENT_CONFIG_OVERRIDE already merged those values into current_config_), MessageDifferencer::Equals(effective, current_config_) returns early without advancing config_seq_ from 0 to 1. Guarding that early return with if (config_seq_ > 0 && google::protobuf::util::MessageDifferencer::Equals(effective, current_config_)) ensures the first successful poll always advances config_seq_ to 1.
  4. Drain in-flight callback delivery in UnregisterCallback (client_configuration_manager.cc:404-413):

    • Unlike Shutdown(), UnregisterCallback only sets s->active = false and erases id from listeners_ without waiting on s->delivery_mu. Locking s->delivery_mu outside self->mu_ when s != nullptr && s->delivering_thread.load(std::memory_order_acquire) != std::this_thread::get_id() prevents an in-flight callback on another thread from outliving UnregisterCallback() while keeping self-unregistration deadlock-free.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the Bigtable API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants