This is my first post after two years. Have been in an accident recently and had some spare time to think while taking rest for a month.
"Do we really consider OS load in the load-balancers?" I asked myself one-day while drowsy. There are myriad number of load-balancers which balance network traffic at various layers of TCP/IP. Some are TCP based, some IP, some can load-balance at application level like HTTP etc. But, most of them revolve around network latency etc. and almost none checks for OS load to schedule traffic to a particular node.
"Why?" probably because it is less cumbersome to keep track of the metrics that determine or influence load-balancer's scheduler. But, what's the harm in trying to make a tool which can really load-balance traffic based on the OS load metrics like memory usage, CPU usage, io-wait time, etc. "No harm at all" I replied to my own question (yes, I am talking to myself now-a-days after the head-eye injury during accident). So let's start building a tool which can efficiently provide a "score card" to every node in the cluster and schedule traffic based on who has got a better score. As simple as that!
As a result, I started a project at GitHub, named "Themis"