Better handling for read and writing from/to Pre-Signed URL files #7

New issue

Open

opened 2025-09-26 10:45:19 +00:00 by CoreRasurae · 0 comments

CoreRasurae commented

2025-09-26 10:45:19 +00:00

Member

Tasks to be done:

s3_provider.py:S3StorageProvider constructor has parameters aws_access_key and aws_secret_key has optional but they seem to be mandatory. At least the access key, otherwise we get an exception. Please validate which is really required and set those as required. Also update the Factory and ProviderConfiguration.
Having to specify the sas_token in the constructor is probably AI nonsense, because the SAS_TOKEN only pertains to a given resource (e.g. file), this implies that we need to create a S3 Provider instance per each file that we access with SAS Token. This makes no sense and is not performant.
- Optionally enable setting the SAS Token via a setter method and also a removal method, but not a getter method.
- That solution is still not the best in case of threading. Consider optionally passing a ContextObject to the read_file and write_file, or registering contexts in the providers.
Ensure the tests are passing and that the coverage is still at least 85%

Definition of done:

Show that a single S3StorageProvider can easily interact with Pre-Signed URL file resources, by writing to them, or reading from them.
Show that the coverage is still in the desired range and that the tests are passing.

Currently CleverCloud Storage has a bit cumbersome interaction with S3 Pre-Signed URLs as it requires instantiation of a new S3StorageProvider object for each single Pre-Signed URL resource. This is neither efficient, nor elegant. Tasks to be done: - s3_provider.py:S3StorageProvider constructor has parameters aws_access_key and aws_secret_key has optional but they seem to be mandatory. At least the access key, otherwise we get an exception. Please validate which is really required and set those as required. Also update the Factory and ProviderConfiguration. - Having to specify the sas_token in the constructor is probably AI nonsense, because the SAS_TOKEN only pertains to a given resource (e.g. file), this implies that we need to create a S3 Provider instance per each file that we access with SAS Token. This makes no sense and is not performant. - Optionally enable setting the SAS Token via a setter method and also a removal method, but not a getter method. - That solution is still not the best in case of threading. Consider optionally passing a ContextObject to the read_file and write_file, or registering contexts in the providers. - Ensure the tests are passing and that the coverage is still at least 85% Definition of done: - Show that a single S3StorageProvider can easily interact with Pre-Signed URL file resources, by writing to them, or reading from them. - Show that the coverage is still in the desired range and that the tests are passing.

Rows
Columns