feat: StorageProvider auto-selection on StorageClient API #6

Merged
CoreRasurae merged 1 commit from feat/storage_auto_selection-#5 into master 2025-09-29 20:30:13 +00:00
Member

ISSUES CLOSED: #5

ISSUES CLOSED: #5
feat: StorageProvider auto-selection on StorageClient API
All checks were successful
/ test (push) Successful in 10m57s
c0e8dec04b
ISSUES CLOSED: #5
CoreRasurae 2025-09-24 18:07:12 +00:00
  • added the
    Type
    Feature
    label
  • requested reviews from khird, aditya
CoreRasurae force-pushed feat/storage_auto_selection-#5 from c0e8dec04b to 7caa68abb6 2025-09-24 18:17:40 +00:00 Compare
aditya approved these changes 2025-09-26 10:02:16 +00:00
khird approved these changes 2025-09-29 14:06:59 +00:00
khird left a comment
Member

Many of the code changes here are not complete repetitions, but are still very similar to one another. It might be more maintainable to define a function like:

def _execute_function_on_provider(self, path, funcname, *args, **kwargs):
    # Obtain a suitable provider for the path if it exists
    source_provider = self._helper_find_provider_if_needed(path)

    if source_provider != self.provider:
        try:
            return getattr(source_provider, funcname)(*args, **kwargs)
        except StorageError:
            # Always fall back to the current provider
        finally:
            if source_provider:
                source_provider.close()

    return getattr(self.provider, funcname)(*args, **kwargs)

Then you can call:

  • list(self._execute_func_on_provider('list_files', path, recursive)) in list_files
  • self._execute_func_on_provider('file_exists', path) in file_exists
  • self._execute_func_on_provider('get_file_size', path) in get_file_size

and so on.

Many of the code changes here are not complete repetitions, but are still very similar to one another. It might be more maintainable to define a function like: ``` def _execute_function_on_provider(self, path, funcname, *args, **kwargs): # Obtain a suitable provider for the path if it exists source_provider = self._helper_find_provider_if_needed(path) if source_provider != self.provider: try: return getattr(source_provider, funcname)(*args, **kwargs) except StorageError: # Always fall back to the current provider finally: if source_provider: source_provider.close() return getattr(self.provider, funcname)(*args, **kwargs) ``` Then you can call: - `list(self._execute_func_on_provider('list_files', path, recursive))` in `list_files` - `self._execute_func_on_provider('file_exists', path)` in `file_exists` - `self._execute_func_on_provider('get_file_size', path)` in `get_file_size` and so on.
@ -38,0 +59,4 @@
else:
source_provider = StorageClientFactory.get_provider_for_path(path)
return self.provider if source_provider is None else source_provider
Member

Idiomatic Python might prefer return source_provider or self.provider

Idiomatic Python might prefer `return source_provider or self.provider`
Author
Member

I will take note of those suggestions for later, but i have some urgency on getting this merged, as CleverSwarm Endpoints S3 support depends on this.

  • Regarding the first suggestion yes, it makes the code more maintainable, however the code repetition is not that large and it may be easier to track in debug sessions. If a bug comes up in that particular section i will consider refactoring it your way.

  • As for the second, you are also right, i will probably open an issue with this suggestions.

I will take note of those suggestions for later, but i have some urgency on getting this merged, as CleverSwarm Endpoints S3 support depends on this. - Regarding the first suggestion yes, it makes the code more maintainable, however the code repetition is not that large and it may be easier to track in debug sessions. If a bug comes up in that particular section i will consider refactoring it your way. - As for the second, you are also right, i will probably open an issue with this suggestions.
CoreRasurae marked this conversation as resolved
CoreRasurae deleted branch feat/storage_auto_selection-#5 2025-09-29 20:30:14 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: cleverlibre/clevercloud-storage#6
No description provided.