Lambda Shipper Test

4.4.2 Lambda Shipper Test

Objective

Validate the log ingestion flow from CloudWatch to the processing system via Lambda Shipper.

Implementation Steps

After completing the configuration, logs are generated to simulate input data from the system.

  1. Create logs in CloudWatch

Create log

  1. Lambda Shipper receives and processes the logs

Shipper run

  1. Messages are pushed to SQS for further processing

SQS message

  1. Verify processing results
  • Log data is stored in S3
  • Data is recorded in DynamoDB
  • Email notifications are successfully sent

DynamoDB result

Workflow Description

  • Logs are generated in the system and sent to Amazon CloudWatch

  • Lambda Shipper is triggered to process logs and push data into Amazon SQS

  • Lambda Processor consumes messages from SQS for further processing

  • Results are stored in Amazon S3 and Amazon DynamoDB

  • Notifications are sent via Amazon SNS (Email Subscription):

    • The email must be subscribed and confirmed
    • If not confirmed, SNS will send a subscription confirmation request
    • Only confirmed emails will receive subsequent notifications

Conclusion

The ingestion pipeline operates reliably, ensuring that logs are correctly transferred from CloudWatch to the processing system.