In recent years, there has been a huge growth and emergence of mainly Internet of Things (IoT) devices which has brought many new challenges for the I

Experimental Design and Setup

submited by

Style Pass

2021-06-10 09:30:05

In recent years, there has been a huge growth and emergence of mainly Internet of Things (IoT) devices which has brought many new challenges for the IT-industry such as processing data between devices in real-time. Therefore, the demand for a scalable, high throughput, and low latency system has become very high.

It remains quite unclear how stateful cloud native applications, such as message queue systems, perform in Kubernetes clusters. The Kafka, NATS Streaming (STAN), and RabbitMQ systems are evaluated in Kubernetes based on metrics such as performance, scalability, and overhead in this thesis.

The significant rise in internet-connected devices will consequently have a substantial influence on systems’ network traffic, and current point-to-point technologies using synchronous communication between end-points in IoT-systems are not any longer a sustainable solution. Message queue architectures using the publish-subscribe paradigm are widely implemented in event-based systems. This paradigm uses asynchronous communication between entities and conforms to scalable, high throughput, and low latency systems that are well adapted within the IoT-domain.

This thesis evaluates the adaptability of three popular message queue systems in Kubernetes. The systems are designed differently, where e.g. the Kafka system is using a peer-to-peer architecture while STAN and RabbitMQ use a master-slave architecture by applying the Raft consensus algorithm. A thorough analysis of the systems’ capabilities in terms of scalability, performance, and overhead are presented. The conducted tests give further knowledge on how the performance of the Kafka system is affected in multi-broker clusters using multiple number of partitions, enabling higher levels of parallelism for the system. Furthermore, it has proven to be a difficult task for systems to choose the correct message broker that fits their system requirements, however, this thesis outlines the main characteristics of the systems and eases this process.