Repository navigation
Conversation
Excluding administrative or backfill read-write transactions from change streams had Go and Java samples without a Python reference.
|
Here is the summary of changes. You are about to add 1 region tag.
This comment is generated by snippet-bot.
|
There was a problem hiding this comment.
Code Review
This pull request introduces a new sample script and corresponding tests to demonstrate how to run a read-write Spanner transaction that is excluded from change stream capture. The review feedback suggests a more idiomatic Python implementation by returning the statement modification count directly from the transaction callback and run_in_transaction rather than using a mutable list in the outer scope, along with a corresponding update to the unit test mocks.
| row_counts: list[int] = [] | ||
|
|
||
| def insert_and_update_singer(transaction: spanner_v1.Transaction) -> None: | ||
| row_counts.clear() | ||
| inserted = transaction.execute_update( | ||
| "INSERT OR UPDATE Singers (SingerId, FirstName, LastName) " | ||
| "VALUES (101, ' Virginia', 'Watson')" | ||
| ) | ||
| updated = transaction.execute_update( | ||
| "UPDATE Singers SET FirstName = 'Virginia' WHERE SingerId = 101" | ||
| ) | ||
| row_counts.append(int(inserted) + int(updated)) | ||
|
|
||
| try: | ||
| database.run_in_transaction( | ||
| insert_and_update_singer, | ||
| exclude_txn_from_change_streams=True, | ||
| ) | ||
| except exceptions.NotFound as err: | ||
| print( | ||
| f"error: database '{database_id}' on instance '{instance_id}' " | ||
| f"in project '{project_id}' was not found: {err.message}", | ||
| file=sys.stderr, | ||
| ) | ||
| except exceptions.GoogleAPICallError as err: | ||
| print( | ||
| f"error: change-stream excluded transaction failed on database " | ||
| f"'{database_id}' in instance '{instance_id}' for project " | ||
| f"'{project_id}': {err.message}", | ||
| file=sys.stderr, | ||
| ) | ||
| else: | ||
| print(f"Resource ID: {database.name}") | ||
| print("Excluded From Change Streams: True") | ||
| print(f"Modified Statements: {row_counts[-1]}") |
There was a problem hiding this comment.
Instead of using a mutable list (row_counts) in the outer scope and clearing/appending to it inside the transaction callback, you can leverage the fact that database.run_in_transaction returns the return value of the callback function. This is more idiomatic, cleaner, and avoids side-effects on outer-scope variables during transaction retries.
def insert_and_update_singer(transaction: spanner_v1.Transaction) -> int:
inserted = transaction.execute_update(
"INSERT OR UPDATE Singers (SingerId, FirstName, LastName) "
"VALUES (101, ' Virginia', 'Watson')"
)
updated = transaction.execute_update(
"UPDATE Singers SET FirstName = 'Virginia' WHERE SingerId = 101"
)
return int(inserted) + int(updated)
try:
modified_statements = database.run_in_transaction(
insert_and_update_singer,
exclude_txn_from_change_streams=True,
)
except exceptions.NotFound as err:
print(
f"error: database '{database_id}' on instance '{instance_id}' "
f"in project '{project_id}' was not found: {err.message}",
file=sys.stderr,
)
except exceptions.GoogleAPICallError as err:
print(
f"error: change-stream excluded transaction failed on database "
f"'{database_id}' in instance '{instance_id}' for project "
f"'{project_id}': {err.message}",
file=sys.stderr,
)
else:
print(f"Resource ID: {database.name}")
print("Excluded From Change Streams: True")
print(f"Modified Statements: {modified_statements}")There was a problem hiding this comment.
Thanks! This comment refers to an earlier revision of this file; the code here has since been reworked in later commits, so this thread no longer applies to the current diff.
| def run_txn( | ||
| fn: Callable[[mock.MagicMock], None], | ||
| exclude_txn_from_change_streams: bool = False, | ||
| ) -> None: | ||
| assert exclude_txn_from_change_streams is True | ||
| fn(transaction) | ||
|
|
||
| database.run_in_transaction.side_effect = run_txn |
There was a problem hiding this comment.
Update the transaction runner mock to return the result of the callback function, aligning with the updated implementation of set_exclude_txn_from_change_streams.
| def run_txn( | |
| fn: Callable[[mock.MagicMock], None], | |
| exclude_txn_from_change_streams: bool = False, | |
| ) -> None: | |
| assert exclude_txn_from_change_streams is True | |
| fn(transaction) | |
| database.run_in_transaction.side_effect = run_txn | |
| def run_txn( | |
| fn: Callable[[mock.MagicMock], int], | |
| exclude_txn_from_change_streams: bool = False, | |
| ) -> int: | |
| assert exclude_txn_from_change_streams is True | |
| return fn(transaction) | |
| database.run_in_transaction.side_effect = run_txn |
There was a problem hiding this comment.
Thanks! The sample records the count in an enclosing variable rather than returning it from the callback, so the mock doesn't need to return the callback result.
Replaces the outer list cell with a nonlocal integer and omits project identifiers from stdout and stderr.
Prints only runtime resource and query results instead of hardcoded string literals.
…_change_streams Problem: multi-line SQL and DDL statements were written as adjacent string fragments. That style is easy to get wrong (a missing trailing space joins two SQL tokens) and is harder to read than the statement itself, and the samples in this set mixed it with triple-quoted SQL. Solution: write each multi-line statement as a single triple-quoted string, matching the layout already used by the other samples. Only whitespace between SQL tokens changes; the statements are otherwise identical. Files updated: - spanner/change_streams/src/spanner_change_streams/set_exclude_txn_from_change_streams.py
Description
Change stream guides show Go and Java tabs for excluding specific read-write transactions from capture, leaving Python developers without a reference for
exclude_txn_from_change_streams=Trueonrun_in_transaction.Adds
spanner_set_exclude_txn_from_change_streams.Checklist
Testing
Compliance & Style
spanner/, which the default repository owners cover.Post-Approval Actions