Skip to content

All crawlee persistance names should be valid names for Apify platform #1354

@Pijukatel

Description

@Pijukatel

In FileSystemRequestQueueClient there is persistance to kvs with name persist_state_kvs_name=f'__RQ_STATE_{self._metadata.id}',

When used on Apify platform, it will throw an error:

apify_client._errors.ApifyApiError: Invalid value provided in store: name can only contain letters 'a' through 'z', the digits '0' through '9', and the hyphen ('-') but only in the middle of the string (e.g. 'my-value-1')

which is caused by the underscores in the name.

Fix the name to be compatible with the Apify constraints and check all similar names of RecoverableStates in code base

To reproduce, run this on Apify platform, which create file system base request queue, that persists to Apify kvs:

async def main() -> None:
    async with Actor:
        rq = await RequestQueue.open(
            storage_client=FileSystemStorageClient(),
        )

Metadata

Metadata

Assignees

Labels

bugSomething isn't working.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions