Custom cloud storage provider no longer supported with JSON gui upload removed?

[miranda]

Hi,

We were using a cloud-hybrid setup with a custom cloud storage provider (not AWS, Google, or Microsoft) successfully for the past two years.

Our project is now almost complete (we had planned to finish in the next 3 months) but it seems my ability to use my own cloud storage provider has disappeared?

Our integration still works with data that was previously uploaded via JSON and the cloud integration I had set up in the past, but with the Python SDK I seem to be forced into choosing AWS, Google, or Microsoft as the cloud providers (as something to pass to the IAM_integration parameter). I’m not allowed to use any of these providers due to privacy restrictions on my data.

This is devastating for my project if this functionality has been removed. LabelBox seems to want me to update/confirm my integration (and there is now a logo of AWS next to my ‘untitled integration’ even though it is 100% not AWS supported). Is there any way around choosing one of these providers?

When I run the get_iam_integration code it says my provider is AWS (which is not true, I assume this is some default for unrecognized cloud services?) and that my integration is not valid. This is despite the integration previously working just fine…

I would really appreciate any help. Thank you in advance!

[ptancre]

Hi @miranda ,

We recently stop supporting JSON file import via the UI but no other changes in terms of integration or else was made.
If you have a JSON formatted import you can sent through the SDK.

Reference doc : labelbox-python/examples/basics/data_rows.ipynb at master · Labelbox/labelbox-python · GitHub

If you still have issue even via the SDK, let me know.

Many thanks,
PT

[miranda]

Hi PT,

Thank you. I have been trying to use the SDK as you describe because the JSON upload is now gone, but it doesn’t seem to work with my alternate cloud storage.

I get an error with the IAM_integration parameter when I try to run the SDK; it says my cloud integration isn’t valid.

I cannot seem to validate it without using one of the three main cloud storage providers and all of the instructions only pertain to setting up an integration with one of these providers.

How can I get the SDK to work with my custom cloud storage?

Thanks a lot,

Miranda

[ptancre]

could you try :

dataset = client.create_dataset(name="dataset_name", iam_integration=None)

and then through the UI change to your integration :

You may need to reprocess those data rows in the dataset but could be a viable workaround.

Many thanks,
PT

[miranda]

This works!! Thank you so much for the help.

Best wishes,

Miranda