Skip to content

DynamoDB paginator fails when partition key is of type number #2932

@twitu

Description

@twitu

Describe the bug
Pagination for dynamodb fails when the partition key is of type number.

This happens because the returned LastEvaluatedKey is of type Decimal and it cannot be json encoded (json.dumps fails). This happens in botocore.paginate.encode it tries to do convert the token to string.

This issue talks about the issue with Decimal types. The gist is that a Number type is returned like this { value: 3} -> { value: { N: Decimal("3")}}. This cannot be converted to string by json.

Steps to reproduce

  1. Create a table, choose Number as type of partition key, choose a sort key (anything is fine)
  2. Add two records with same partition key and different sort key
  3. Run a query similar to the following.
ddb_client = boto3.resource("dynamodb").meta.client
query_paginator = ddb_client.get_paginator("query")

page_iterator = query_paginator.paginate(
    TableName=<TABLE-NAME>,
    KeyConditionExpression=Key(<PK>).eq(<PK-VALUE>),
    ScanIndexForward=False,
    PaginationConfig = {
        MaxItems: 1,
        PageSize: 1,
    }
)

for page in page_iterator:
  for return_value in page["Items"]:
      print(return_value)

Expected behavior
Pagination should succeed. It should handle the Decimal type by itself.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue is a confirmed bug.dynamodbp2This is a standard priority issuepagination

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions