DynamoDBStreams and Partial Batch Failure Handling

Feature Requests
1

Hi folks
I am trying to use partial batch failure handling on a lambda in conjunction with a dynamodbstream. For testing purposes, I am trying to set up a lambda that reports a itemFailure for every third record received via the dynamodbstream (batch size 100). I am struggling to make this work. The lambda does return an object: {“batchItemFailures”: [{itemIdentifier: “03f893b2”}]} (example).

I have configured the eventSource mapping with FunctionResponseTypes: [“ReportBatchItemFailures”] and a DestinationConfig: { OnFailure: { Destination: dlqArn}}

I would like to ask if anybody has got this to work? I am wondering if this is not yet implemented or I have made a mistake in the configuration. I did find a PR for SQS PartialBatchItemFailures, but I do not know if this also includes dynamodbstreams.

Cheers

2

I am seeing the same behavior with Kinesis. Here’s my event config within the serverless function:

Events:
  KinesisEvent:
    Type: Kinesis
    Properties:
      Stream: 
        Fn::Sub: arn:aws:kinesis:${AWS::Region}:${AWS::AccountId}:stream/my-stream
      StartingPosition: TRIM_HORIZON
      BatchSize: 100
      MaximumBatchingWindowInSeconds: 10
      MaximumRetryAttempts: 0
      BisectBatchOnFunctionError: false
      FunctionResponseTypes: 
          - ReportBatchItemFailures
      DestinationConfig: 
        OnFailure: 
          Destination: 
            Fn::GetAtt:
              - MyDeadLetterQueue
              - Arn 
      Enabled: true
3

Hi @butterfire and @eporta :wave:

Thank you for reporting this issue. We are aware of several issues related to Lambda Event Source Mappings and I can confirm that we are lacking test coverage for your specific scenario.

We are working on a complete rewrite of the Lambda Event Source Mapping with better AWS parity, performance, and test coverage. Most tests are already passing and partial batch failure handling has been carefully considered during the re-design. You will soon be able to opt into the new implementation (ETA: in the coming weeks). Stay tuned :chart_with_upwards_trend: :rocket:

PS: Partial batch failure handling is supported for EventBridge Pipes

Best,
Joel

4

Awesome, thanks! I’ll be looking forward to try the new implementation.