In this new era of Big Data, the term commodity hardware became very common, but it’s up to you to define how to use it. The term commodity hardware in general mean Computer hardware that is affordable and easy to obtain. Typically it is a low-performance system that is PC-compatible and is capable of running Microsoft Windows, Linux, or others operating systems without requiring any special devices or equipment.
Why do we need datacenters with special configurations to be able to run servers 24/7 : ACs, UPS, raised floors, networking, security…etc. While we used to have a desktop computer in every office. Imagine a company having 30 desktop computers having an i7 8cores 4GB ram 500GB hard drives each, that’s the equivalent of a 240 Core super computer with 120GB ram, and 15 TB or storage ! But obviously we don’t use them as servers, because these PCs fail frequently and they are not designed to run 24/7.
That’s exactly what commodity hardware are supposed to be, they fail ! Today we have very accessible technologies that allow us to run a large number of low-cost, low-performance commodity computers working in parallel instead of using fewer high-performance and high-cost computers, and we call it commodity cluster computing.
Startups should be the first to profit from these technologies, and for example in web applications development resources per computer could be easily shared in the network instead of having a local development environment per developer. The more your hardware is connected, the more powerful is your cluster.
Some experiments :
Having at home a development board which is the Banana Pi, an A7 dual Core arm based board with 1GB ram, I installed debian jessie and make it join a Docker Swarm that I run on my own laptop using a simple network cable :
That’s two different architectures, and very low-cost hardware, exactly what could represent commodity hardware. You can see it running Docker Swarm with portainer GUI giving a total of 4 CPUs and 9GB ram. Now I can easily run some code on the banana pi or in the cluster :
But if I change my mind and decided to unplug the banana pi, or the hardware failed for any reason ? The two nginx containers will keep running on the available hardware. As simple as that !
Now just imagine the size the of the supercomputer that you have in front of you, and start thinking about what can you do with a supercomputer ?! How you can distribute it between daily tasks, and super computing or cloud computing tasks.