@antoviaque That’s great, thanks for checking it out!
Yup, that’s what I was going for here! This whole experiment is just a bunch of “glue” scripts to make other existing tools work well together.
Yes, Kubernetes can be quite complex, so we definitely want a managed service.
In my (limited) experience, DigitalOcean’s managed Kubernetes service is the nicest offering (and conveniently, it’s also one of the most affordable). DigitalOcean takes care of all the complexity of managing Kubernetes itself, as well as auto-scaling the cluster, running the Kubernetes Dashboard, etc. (They also maintain the official DigitalOcean Terraform provider, which I used here.) I think it’s a pretty nice separation of concerns, where we are responsible for our application + databases, and they are responsible for the Kubernetes cluster itself and everything it relies on.
Amazon EKS is an alternative, but is way more expensive, and still relies on you to manage a lot yourself. We’ve been using it in production for several years now for LabXchange, so we have a good understanding of it. Perhaps @toxinu might want to weigh in on how much maintenance has been required lately? In the case of EKS, Amazon manages a much more minimal Kubernetes service, and you have to implement things like auto-scaling, the Kubernetes dashboard, and other features yourself - so it’s more complicated. It also relies on “community supported” / semi-official terraform modules like this.
So I think that using DigitalOcean would give us less to manage ourselves than we do today. If we used their managed MySQL service (which is also an option), we could have even one less thing.
Things we (and I’m including “Ocim” in this “we”) still have to manage are:
- MySQL (or just use RDS/DigitalOcean MySQL)
- MongoDB (currently Tutor deploys a separate MongoDB instance for each Open edX instance, and I don’t think it has automatic backups)
- HTTPS certs (Tutor deploys a separate load balancer for each instance, which would get expensive; we can use a very similar approach but with one load balancer per cluster, which will be much cheaper)
- DNS records
That’s part of what I’d like the reviewers of this project to comment on. So I don’t have a precise answer to that yet.
In some sense, this project acts like a “mini Ocim”, and provides some overlapping functionality (i.e. deploying and managing lots of Open edX instances) just using Gitlab.
Here’s how I’m thinking of the major pieces:
- Customer portal: To me the most obvious thing that it lacks which Ocim provides is a secure customer portal that allows customers to deploy their own instances, manage things like theming / billing, etc. So I think that’s a clear role for Ocim going forward.
- Orchestration: As for orchestrating upgrades and running the various tools like Terraform or Tutor, we can choose to use GitLab CI to do that as I’ve shown here (with Ocim triggering it as needed using the GitLab API) or implement that in Ocim (which wouldn’t be too hard, as it mostly involves running external commands which Ocim already does). I think we need a bit more discovery/experience with GitLab CI to answer this.
- Managing Open edX: Some Ocim code is related to specific nuances of Open edX, like the upgrade process. But Tutor also includes similar code to deal with those things, and has more community usage. So I think we should see if Tutor will cover our use cases and if we have an effective relationship with Tutor as a key upstream project; I think that’s a good goal and based on my experiments so far it seems like it’s possible. However, if not, we’ll need to continue having Ocim handle that.
If we can say that Ocim will mostly focus on (1) and we’ll integrate it with GitLab CI or GoCD for (2), and Tutor for (3), I would say that’s a big win in terms of simplification and re-using existing tools. But I’m not yet clear on how much if any of (2) we can/should move out.
Yes! I’m just waiting for our internal reviewers first: