0

I am using 1.17 Flink and I want to print the size of parquets that FileSink creates. I am using forBulkFormat. I need an indication for parquets size that I am uploading to blob storage for monitoring.

FileSink.forBulkFormat(s3SinkPath, new ParquetWriterFactory<>(builder))
                    .withBucketAssigner(new CustomDateTimeBucketAssigner())
                    .withRollingPolicy(new OnCheckpointRollingPolicy())
                    .build(); 


The only solution I found is the link I attached but because some of the fileSink classes declared as internal I can't implement this solution in advance Flink version like 1.17 using this link

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.