You may have already heard about the benefits, but it is important to be aware of risks.

Firewall effectiveness. In a traditional deployment, a virtual machine has its own IP address. It is trivial to limit network access to and from it. There is wide support in equipment and software, and easy tracing of network flows. When Kubernetes pods on the overlay network connect to your PostgreSQL database outside Kubernetes, you can’t filter clients by IP, as PostgreSQL sees the IP of a multi-tenant Kubernetes node, removing additional important authentication factor. You usually approve the entire Kubernetes subnet. Vendors will convince you that is a feature, as IP-based security is a “legacy” approach, creating the demand for premium add-ons, and hardware.

Encryption in transit. Common pattern is to do TLS termination on the ingress. By default the network traffic on the overlay, between ingress and pods, isn’t encrypted. Your application needs to either explicitly enable encryption, which impacts latency, and usually requires managing certificates (which may not be well protected in Kubernetes), or to purchase advanced NICs to do it transparently (standard in public cloud).

Security monitoring. Wide range of tools are available to monitor network flows in IP networks. But monitoring VPNs (including overlays) is considered an advanced paid-for feature, usually also requiring higher tier network equipment.

Secrets storage. A major security disadvantage of Kubernetes is sharing the same kernel. This means sharing the Linux virtual file system. It usually results in the ability of root to non-invasively read files of any container, and launch processes in its namespace. Much higher data confidentiality is achieved in VMs, especially with SAN. All actions are logged, and disconnecting storage from a VM to connect it to other VM for extracting information, is very visible.

There may be solutions to the issues mentioned above, but it typically involves additional long-term license cost, paying extra for hardware, or investing employee time. It is much cheaper to solve the highlighted issues, when the application is contained within a few servers (VMs) on a standard “legacy” network.

For applications where security is crucial, choose proven VM environment, unless you can compensate the risk by other means.