My DevOps experience is centered around the needs of a company that developed a platform for marketplace integration (Ozon, Wilberries, Tmall, Yandex.Market и Goods).

Kubernetes

Initially, the company’s infrastructure was based on Docker Compose / Swarm , but, at some point, switching to Kubernetes became a reasonable option. For K8s deployments I used Kubespray and K3s .

I faced a number of problems when switched to a cluster-based development, in particular, with local deployments. I tried to organize it using local miniclusters ( kind , minikube ) with tools like Tilt and Skaffold , but, in the end, for a number of reasons, returned to Compose.

Nevertheless, packaging applications in Helm charts somewhat simplified their management in the cloud. To address the issue of a cluster configuration reproducibility at the application level, I tried Helmfile and Helm Terraform provider , and settled on the latter.

CI/CD

In CI/CD my experience is limited to GitLab and Drone . For each project that was supposed to be used in production, a pipeline was set up, which included running the code through linters and running all tests, building a new Docker image and deploying this image on the staging and, later, on the production servers. A built-in GitLab registry was used for Docker images. Dockerfiles have been optimized for building a production image.

I became interested in Drone when I discovered that GitLab workers can easily consume 10GB of RAM per one worker. But, at some point, it became possible to cheaply move GitLab to a separate server with a bunch of RAM, so the option with Drone has lost its relevance.

FaaS

The ability to have a FaaS service in your own cluster opens up interesting architectural possibilities. OpenFaaS turned out to be a great option, which is quite easy to write functions for. There were problems with local deployment (without a cluster), but an internal development server resolved the issue.

Nix

Another interesting idea that I have been working on lately is declaratively configurable Linux distributions. They can be used as base systems that are not (so) scary to update. Today, NixOS is the only serious contender in this field (not Guix , unfortunately). I maintain my system configuration in the git repo , but it would be interesting to try this system for larger tasks.

(see also system administration card )

---

Illustration

• Title: AWS logo. I sometimes look at their web services solutions when working on infrastructure issues. It’s a pity that sometimes their prices are so high.

• Infrastructure visualization in the form of “floating boxes”: I probably decided to do something similar by looking at the ELK and Vault . The logos near the layers speak for themselves: with HCL scripts we provide machine configurations ( Terraform ), with playbooks—system configurations ( Ansible ), with resources—clusters ( Kubernetes ), with charts—applications ( Helm ). It should be noted that sometimes the boundaries between layers are not so strict: for example, as mentioned above, Terraform’s HCL scripts may as well describe the configuration of Kubernetes clusters at the Helm chart level.

• The logos on the Helm layer: using charts, it’s relatively painlessly to deploy a router with automatic TLS ( Traefik ), a secrets storage with keys autorotation ( Vault ), a feature-rich environment for source code ( GitLab ), and then collect all the needed metrics for Prometheus .