Nix or Docker?
Two Different Approaches, Two Different Solutions, One Problem At Hand.
This Post Is About:
Bob is a developer working in a software company called Foo. Foo is spending a humongous amount of resources on Virtual Machines for development, testing, and other purposes. Although Foo is trying to adopt CI/CD, these virtual machines are making everything hard for Bob and his team and are making portability a very distant goal.
In a meeting, a supervisor in Foo company asks Bob to look for different possible approaches to overcome this old VM trend and skyrocket the efficiency while giving zero concessions from portability.
Bob is not well-versed in the new trends yet. So he picks up a magazine, opens a page, and voila! There is an article that is pointing this very issue. "Aha!" says Bob and starts reading:
What is:
Docker:
First things first: Docker is not a VM, I mean in theory. Docker is designed and built to prevent what is known as the snowflake problem that is when a local development tarball is not valid in another developers’ local environment, due to different OSs and so on. So, docker came to rescue developers from this pain and it admittedly fulfilled its promises to a degree. Docker is a containerization technology that owes some great degree to Linux’s idea of ‘Everything is a file’. I mean, everything. Even plugged hardware is considered as files. This opens lots of doors for a careful tinker. Essentially because the whole operating system is a carefully designed file tree and it allows additional users and sub-systems to be built on top of it while having access to the hardware. Think of chrooting, for example. In essence, chrooting in a Linux system is merely creating a sub-tree in the Linux system with its own bin/, var/ home/ and also additional users, while the chrooted user is also a user himself and the chroot environment is a confined environment with its own environment variables, its own permissions, etc. Smells like a recursion🤓
This very idea is the birthplace of Docker. But wait, if all these are true, and Docker is a containerization process over a shared Linux kernel, does not this make it, hmm, some sort of virtual machine when it is used on OSX and/or Windows environments?
It surely does. As a matter of a fact, this results in two steps — albeit implicit- deployment process where the host machine hosts the Docker, Docker mounts the kernel, and source codes are imported through this pipeline. This may create some bottlenecks, discrepancies due to the mounted OS kernel that the Docker container will be built upon, due to some caching or dependency issues, or due to the lagging when the project is big yet not very well handled.
Nix:
Talking about Nix requires a little bit more caution since the word itself may mean the operating system; NixOs, the language Nix, which some call a programming language while others call an expression language (yet I have no intention to delve into this topic), or the Nix the package manager. Well, these are not the same but they are somehow like follow-up steps in a ladder, in reference to each other. In this blogpost, Nix will refer to the package managing system, unless otherwise is stated. Unlike Docker, Nix approaches the issue at hand in a purely functional way. But what the heck is 'purely functional'? Well, a function is a procedure where given the same input, always results in the same output, regardless of how many times it is called, and when or in which sequence it is called. This is the point that makes Nix a very powerful tool actually!
Nix aims to achieve what Docker also wanted to achieve, however in a very different methodology. Nix sees the problem of incompatibility and resource hunger which render the efforts of portability futile at times. Why? First of all, not all dependencies are that innocent. there are tons and tons of unseen implicit dependencies for many applications and even programming languages and their libraries.
Check for the dependency tree for Pandas library of Python 3!
In order to solve possible issues and bugs that can stem from such a tree, Nix comes up with a genius solution. Once a pure environment is created, the user has no reach to any sort of utility in their environment. Even compiling a C program that greets the world with hello requires needed dependencies to be met like gcc, bash, Coreutils. This is because Nix is actually here to solve the problem of ‘implicit dependencies’. It wants to make sure that once a compile function is made, which is called derivation in Nix, it will definitely work every time and everywhere the same way. Just like a function!
The powerful side of Nix is, once a derivation is made, it gives a hash code to it before adding it into the local Nix Store in /nix/store. This hash function is such a sensitive one that, sometimes whitespace causes it to change, not to mention a dependency will trigger a chain reaction in the dependency tree and ultimately a little tweak in the used library version will produce a totally different hash. But this is how it is supposed to happen anyway, right? We change the function, the outcome changes; we change the derivation, the hash code is changed. Thus, Nix makes sure that this derivation will have an identical outcome every single time!
The Idea Behind:
Docker:
Docker has a very well established community, helpful, well- supported, and has a very nice application to manage the containers. It also is a really very straight-forward process to use if the scale of the project is not big or complex. For example, writing a basic Flask application with one or two routes, dockerizing it, and deploying are not taking more than a couple of minutes.
So in its essence, Docker takes a curated environment with its explicit dependencies, applies the dependencies, takes the source code, and runs the given command, in a sterilized and to a degree, a purified environment.
So instead of old school muscular, full-fledged virtual machines that consume the resources of the host machines, Docker takes these files, in its virtual environment, runs on a Linux kernel, and freezes. Thus, in an expected scenario, a source code behaves the same in OSX or in Windows system.
Nix:
I must admit, starting from its documentation, Nix is not half user friendly as Docker. It requires very tedious manual work, dependencies must be complete to work as expected since Nix and Nix store can be seen as a graph database where nodes and edges are completing an entire road and one missing can cause a dead-end and one modified can lead to another path. Thus the user must put a little bit more effort but will end up in rock-solid portability. Also breaking things is not a big deal since Nix holds multiple environments at once, has a garbage collector against bloating, and supports rollback to any desired environment version.
In this very small example, we make a derivation with the name of HELLOW, giving all necessary and sufficient build inputs to build the program, making the derivation and then either invoking or porting to any other medium for later use. The derivation lives in the local Nix Store with the URI and name similar to this:
`/nix/store/2p5hc92sjackxwj4rwzrlk0r7znn1vcw-HELLOW.drv`.
This piece of the file includes all the required steps for reproducibility and nothing else!
Final Round:
I do not like the writings that leave the reader in limbo by not at least offering a preference over another. Hence, I want to put emphasis on the pros and cons of both technologies and will also offer my choice.
As for Docker, it has an amazing integration starting from operating systems to IDEs and even native cloud integration thanks to Docker Cloud and services like AWS, a winner hands down. Also, deployment is relatively easy and intuitive. Procedural if you will. Its community is robust and scattered around the internet without losing its robustness and integrity. Besides, its documentation is thorough and very very user friendly, providing enough examples and human-readable content. However, it also has downsides. It is still managing its processes one way or another in an implicit fashion, which can cause some complications. Regardless to say, Docker is resource hungry, even more when running on OSX or Windows, and the Linux kernel can cause bottlenecks.
As for Nix, I can say quite the opposite of docker both in terms of its cons and pros. It has a functional programming approach, thus given input produces expected output. This is very important if a company is shipping its product versions or patches continuously to millions of users since they want to make sure that their product will work exactly as intended every time. Thus, Docker’s implicit dependency is replaced with strongly explicit dependency handling in Nix. This makes it a better choice if scalability is an important factor. However, it has a steeper learning curve for many people and documentation is not very helpful and the community is not as robust. However, considering there are currently around 70.000 packages ready for use, I can say that it is not only a very serious competitor against Docker but also homebrew for OSX users.
PS: you can use nix to deploy Docker images without using dockerfile :)
Regardless to say, the winner for me is Nix, with all its pros outweighing its cons and providing me a rock-solid environment!
Bob has been using Nix for 4 years now. He is happy and proud. Way to go Bob!