Apache Kafka is a very popular distributed messaging system, and it’s now used by one-third of the Fortune 500, including seven of the top 10 global

A Kafka pitfall when to set Log.Message.Timestamp.Type to CreateTime

submited by

Style Pass

2024-10-03 21:30:09

Apache Kafka is a very popular distributed messaging system, and it’s now used by one-third of the Fortune 500, including seven of the top 10 global banks.

As we have known, Apache Kafka is adopted as a scalable, event-driven backbone for service architectures. Therefore, any pitfall in Kafka will cause a serious issue.

From the system design perspective, any pitfall will indicate a system design issue. On this blog, I will share a Kafka pitfall when to choose CreateTime as Log.Message.Timestamp.Type. It includes what’s the issue and impacts, how to troubleshoot, where’s the design issue, and how to work around it.

After figuring out the root cause, I realized there are more impacts other than space shortage. I have outlined them in the section “Design review on Kafka timestamp-based retention policy”.

We did a quick check on the Operation System, Network Traffic, and Disk i/o. However, there is no clue what’s going wrong. So the immediate operation was imposed to increase space, then we can prevent Kafka from service outage.