Skip to content

perf(web): parse each sync response body once - #2069

Merged
WilliamBergamin merged 1 commit into
mainfrom
perf-single-json-parse
Nov 19, 2026
Merged

WilliamBergamin merged 1 commit into
mainfrom
perf-single-json-parse

Conversation

@WilliamBergamin

@WilliamBergamin WilliamBergamin commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This pull request makes the sync WebClient parse each response body once instead of twice.

  • Cause: with retry handlers set (the default), _perform_urllib_http_request parses a JSON body to build the RetryHttpResponse, then _urllib_api_call and _request_for_pagination parse the same string again.
  • The parsed body is now kept on the internal response dict and reused; the callers fall back to parsing when it is absent (no retry handlers, a non-JSON or binary body, or an error status).
  • Saves one json.loads per call and per page: about 2.3 ms on an ~830 KB conversations.history page (Python 3.15), next to nothing on small bodies.
  • RetryHttpResponse.body and SlackResponse.data now share one dict, as they already do in the async client. No built-in handler reads or edits body; a custom handler that edits it now changes the returned SlackResponse.

Testing

The two call-count tests fail on main (2 parses per response) and pass on this branch:

  • One api_call with the default retry handlers parses the body once.
  • Two pages of pagination parse each page once.
  • A 200 with a non-JSON body still raises SlackApiError with the "non-JSON format" error (passes before and after).

Category

  • slack_sdk.web.WebClient (sync/async) (Web API client)
  • slack_sdk.webhook.WebhookClient (sync/async) (Incoming Webhook, response_url sender)
  • slack_sdk.socket_mode (Socket Mode client)
  • slack_sdk.signature (Request Signature Verifier)
  • slack_sdk.oauth (OAuth Flow Utilities)
  • slack_sdk.models (UI component builders)
  • slack_sdk.scim (SCIM API client)
  • slack_sdk.audit_logs (Audit Logs API client)
  • slack_sdk.rtm_v2 (RTM client)
  • /docs (Documents)
  • /tutorial (PythOnBoardingBot tutorial)
  • tests/integration_tests (Automated tests for this library)

Requirements

  • I've read and understood the Contributing Guidelines and have done my best effort to follow them.
  • I've read and agree to the Code of Conduct.
  • I have signed the Salesforce CLA (required before your PR can be reviewed or merged).
  • I've run python3 -m venv .venv && source .venv/bin/activate && ./scripts/run_validation.sh after making the changes.

🤖 Generated with Claude Code

With retry handlers set (the default), the sync client parsed a JSON
response body for the retry handlers and then parsed it again in
_urllib_api_call and _request_for_pagination. Keep the parsed body on
the internal response dict and reuse it.

Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com>
@codecov

codecov Bot commented Nov 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.71%. Comparing base (1101cb7) to head (d81c106).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2069   +/-   ##
=======================================
  Coverage   84.71%   84.71%           
=======================================
  Files         118      118           
  Lines       14119    14122    +3     
=======================================
+ Hits        11961    11964    +3     
  Misses       2158     2158           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@WilliamBergamin WilliamBergamin self-assigned this Nov 19, 2026
@WilliamBergamin WilliamBergamin added bug M-T: A confirmed bug report. Issues are confirmed when the reproduction steps are documented semver:patch web-client python Pull requests that update Python code labels Nov 19, 2026
@WilliamBergamin
WilliamBergamin marked this pull request as ready for review November 19, 2026 20:03
@WilliamBergamin
WilliamBergamin requested a review from a team as a code owner November 19, 2026 20:03

@mwbrooks mwbrooks left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Thanks @WilliamBergamin! Halving the JSON parsing on big history pages is a nice free win for every sync client.

headers = transport.call_args[1]["args"]["headers"]
self.assertEqual("application/json;charset=utf-8", headers["Content-Type"])

def test_api_calls_parse_the_response_body_once(self):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: Love that these call-count tests fail on main and pass here, it makes the perf claim easy to trust.

@cchensh cchensh left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Each retry gets a fresh resp, so the cached parsed_body always comes from the final attempt, and the non-JSON, binary and error paths are unchanged. A few optional nits:

  1. RetryHttpResponse.body and SlackResponse.data are now the same dict, so a custom retry handler that changes response.body in can_retry and then declines to retry now changes what the caller sees. The docs don't suggest mutating body, but a short comment (or a copy) would make the sharing explicit.
  2. Test gap: there's no test where a handler forces a second attempt with a different JSON body, checking that resp.data comes from the final attempt. That's the main correctness claim. A retry_handlers=[] case would be good too.
  3. test_non_json_response_body_raises_api_error never hits the cache, since the body doesn't start with {. That's fine as a regression guard.
  4. The Returns: docstring of _perform_urllib_http_request should mention the optional parsed_body.

@WilliamBergamin
WilliamBergamin merged commit 4482fe7 into main Nov 19, 2026
19 checks passed
@WilliamBergamin
WilliamBergamin deleted the perf-single-json-parse branch November 19, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug M-T: A confirmed bug report. Issues are confirmed when the reproduction steps are documented python Pull requests that update Python code semver:patch web-client

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants