The architecture of the internal implementation

Kanaloa is developed using Akka. This document record the architecture of the actors, mainly for the purpose getting new contributors on board.

The core of this system is Queue, WorkerPoolManager and Worker. The Queue buffers requests, WorkerPoolManager creates and manages Workers who dequeue requests from the Queue and send them to the service. The service is abstracted as Handler and HandlerProvider. A Handler represents one instance of the service, while the HandlerProvider provides a list of Handlers. These two traits are not actors. They are implemented to support different protocols and service locators.

On the queue side, the QueueSampler samples the queue’s length and dequeue speed, the Regulator then applies the PIE traffic regulation logic based on the metrics collected by QueueSampler. On the worker pool side, the WorkerPoolSampler samples the performance of the Handler, i.e. the service, at each pool size, then the AutoThrottler resizes the worker pool based on the optimization logic during traffic oversaturations. The Dispatcher is the top supervisor in the hierarchy.

When kanaloa load balances between multiple service instances, it creates for each service instance a dedicated WorkerPoolManager. This WorkerPoolManager comes with its own WorkerPoolSampler and Autothrottler, which means that the concurrency throttling is per service instance.