Skip to content

Delete partial file when local download_object_range fails (#2223) - #2227

Merged
micafer merged 3 commits into
apache:trunkfrom
raamrutsagarb25-tech:fix-local-range-download-cleanup
Nov 17, 2026
Merged

micafer merged 3 commits into
apache:trunkfrom
raamrutsagarb25-tech:fix-local-range-download-cleanup

Conversation

@raamrutsagarb25-tech

Copy link
Copy Markdown

Changes Title (replace this with a logical title for your changes)

Description

Replace this with the PR description (mention the changes you have made, why
you have made them, provide some background and any references to the provider
documentation if needed, etc.).

For more information on contributing, please see Contributing
section of our documentation.

Status

Replace this: describe the PR status. Examples:

  • work in progress
  • done, ready for review

Checklist (tick everything that applies)

  • Code linting (required, can be done after the PR checks)
  • Documentation
  • Tests
  • ICLA (required for bigger changes)

@raamrutsagarb25-tech

Copy link
Copy Markdown
Author

Hi, I've opened a pull request for this: <#2227 >

download_object_range in the local storage driver now removes the partially written file when reading the source fails and delete_on_failure=True. I also added a test and a changelog entry. Happy to make any changes you'd like.

@micafer micafer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! The implementation looks reasonable, but I think the regression test should cover a failure after some data has already been written.

Currently, the test deletes the source object before starting the download. This causes a FileNotFoundError before any data is yielded, so it doesn't actually verify cleanup of a partially downloaded file.

Could you update the test to use a generator that yields a chunk of data and then raises an exception? This would verify that the partially written destination file is removed when delete_on_failure=True.

It would also be useful to test that the partial file is preserved when delete_on_failure=False.

One minor suggestion: consider handling errors from os.remove() so they don't mask the original download exception.

micafer and others added 2 commits November 17, 2026 16:40
…tream

- Replace the previous test, which deleted the source object before the
  download (raising FileNotFoundError before any data was yielded), with a
  mocked stream that yields a chunk and then raises. This actually exercises
  the partially-written-file cleanup path.
- Add a test asserting the partial file is preserved when
  delete_on_failure=False.
- Guard os.remove() in download_object_range's failure handler so an error
  removing the partial file doesn't mask the original download exception.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.67%. Comparing base (4e06395) to head (8e373f8).

Files with missing lines Patch % Lines
libcloud/storage/drivers/local.py 81.82% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            trunk    #2227   +/-   ##
=======================================
  Coverage   83.67%   83.67%           
=======================================
  Files         352      352           
  Lines       82487    82524   +37     
  Branches     8901     8907    +6     
=======================================
+ Hits        69015    69048   +33     
- Misses      10592    10595    +3     
- Partials     2880     2881    +1     
Files with missing lines Coverage Δ
libcloud/test/storage/test_local.py 93.80% <100.00%> (+0.38%) ⬆️
libcloud/storage/drivers/local.py 78.52% <81.82%> (-0.11%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@micafer micafer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@micafer
micafer enabled auto-merge November 17, 2026 16:01
@micafer
micafer merged commit 635168e into apache:trunk Nov 17, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants