Technology Radar | An opinionated guide to technology frontiers | Thoughtworks
We've come up with various ways to show blips evolving from one Radar volume to another. Blips can be new to the volume or move between rings.
We discussed several topics in this edition of the Radar (some of which eventually failed to make the final cut) where teams are employing tools to adapt to/from Kafka. Some of these tools allow more traditional interfaces to Kafka (such as ksqlDB, Confluent Kafka REST Proxy, and Nakadi), while others are designed to provide extra services such as GUI frontends and orchestration add-ons. We suspect that part of the underlying reason for this bounty of tools is the underlying sharp-edged complexity of some of Kafka’s parts combined with increased presence in organizations that need to bend it to existing architectures and processes. Some teams end up treating Kafka as a next-generation enterprise service bus — one example of The slippery slope of convenience theme — but other teams use Kafka to provide universal access to business events as they happen. These organizations recognize that it is sometimes easier to have a centralized infrastructure with adaptation at the edges and try to avoid sprawl with careful design and governance. In any case, it shows that Kafka continues toward status as a de facto standard for asynchronous publish/subscribe messaging at volume.
An antipattern as old as the Radar is the tendency for teams to place behavior within their ecosystem at convenient but improper nexus points that lead to long-term technical debt and worse. Examples abound, including using a database as an integration point, using Kafka as a global orchestrator, intermingling business logic with infrastructure code and so on. Modern software development offers many places for developers to stash behavior, and inexperienced or inconsiderate teams often entangle concerns by not carefully considering the long-term consequences of inappropriate coupling. Inappropriate team structures and other deviations from Conway’s Law don't help either. As software systems become more complex, development teams must show diligence to both create and maintain thoughtful architecture and design, not slap-dash decisions for expediency. Often, thinking about the testability of a particular approach leads teams away from some of these potentially problematic decisions. Software tends toward complexity when left to its own devices. Careful design and, perhaps more importantly, ongoing governance works to ensure that schedule pressure or one of the other numerous disruptive forces doesn't cause teams to make convenient but improper decisions.
