Export_v2 fails

Python SDK
,
1

When exporting using the new export_v2 method, my query fails after the first is successful. More specifically, at idx=0 the export is successful. After the first export, all future exports fail. For example

for idx, drows in enumerate(data_rows):
        task = lb_project.export_v2(
            params={
                "data_row_details": False, "project_details": True,
                "metadata_fields": False, "attachments": False,
                "media_attributes": False, "media_fields": False,
                "performance_details": False, "label_details": False,
                'labels': False, "projects": True
            },
            filters={'data_row_ids': drows},
        )
        task.wait_till_done(timeout_seconds = 600)
        for data_row in task.result:
              continue
        print(f'Finished Labelbox Export Task... {idx+1}/{len(data_rows)}')

each drow is a list of 500 data_row_ids. I have tried adding sleep timers for upwards of 10 minutes and I still get errors. The error is:

ValueError: Job failed. Errors : None

2

I am using python 3.8 and lablebox api version 3.47.1

3

Hi @samuel, can you please elaborate on why you are seeking to export iteratively rather than creating one list of data row IDs and performing a singular export?

4

I have more than 2000 data rows ids i need to retrieve. Some projects are closer to 20,000. I see that there was error that the max data rows are 2000, so I split them into chunks of 500. I tried chunks of 2000 but the error seems more prevalent with larger chunk sizes

5

Is there any reason not to just omit the data_row_ids filter and export all the data rows in the project together? Are you trying to avoid exporting certain data rows?

6

I figured the filter would be faster since I want small partitions of datasets that can be massive. I will try pulling everything and removing unwanted rows after