I’m using an AWS Glue Job to read JSON files from S3 and write the data back into S3 as parquet files. I want to enable bookmarking so that I can run this job continuously and only process new data.
When I create the job, I pass parameters --job-bookmark-option= job-bookmark-enable
But I see this message in the Job logs when I run the Job:
24/12/23 23:53:49 WARN JobBookmark$: Disabling Job Bookmarks for developer environment
Is bookmarking possible in localstack AWS Glue Jobs?