Touches a chord this post.Systems like Kubernetes, Kafka are inherently complicated. My previous company got baremetal from AWS and installed k8s cluster on them. No offense to who architected it, we had multi country infra and made sense to take care of cost advantages on lower cloud costs using alternate providers
We got a lot of critical infra running on them and then slowly there was tech-debt that would start accumulating. Clusters have to get updated , older DNS versions in k8s are slow, networking (Older Weave versions was bursting through the seams when the traffic exploded with many applications onboarded). SRE teams get overwhelmed, constant requests for adding PVC (Kafka & C* was on k8s) took a toll.
Sanity prevailed in the end, there was decision to move to hosted PaaS infra, though I no longer work there, I just reminisced what we were going through.
Though a "cloud-independent" solution will save pennies, it will definitely drown dollars in personnel costs and the uptime/SLA
History repeats itself, because we don't learn from our mistakes (us or others)
We got a lot of critical infra running on them and then slowly there was tech-debt that would start accumulating. Clusters have to get updated , older DNS versions in k8s are slow, networking (Older Weave versions was bursting through the seams when the traffic exploded with many applications onboarded). SRE teams get overwhelmed, constant requests for adding PVC (Kafka & C* was on k8s) took a toll. Sanity prevailed in the end, there was decision to move to hosted PaaS infra, though I no longer work there, I just reminisced what we were going through.
Though a "cloud-independent" solution will save pennies, it will definitely drown dollars in personnel costs and the uptime/SLA
History repeats itself, because we don't learn from our mistakes (us or others)