Devstack tips and tricks. General discussion

I whipped up a tentative backport to Ironwood:

https://github.com/edx/devstack/pull/538

The original PR is still not finalized, but I was able to fire up both an Ironwood and a Juniper devstack. You still can’t run them at the same time, but it’s nice to have both releases at the ready for when they’re needed.

3 Likes

Aaand the multi-devstack PR was merged upstream. I’m working to update my Blockstore and LX patches to match.

3 Likes

I now have both ironwood and master up and isolated. Here is my guide for how that can be done with the new multi-devstack code.

  1. Destroy your current devstack. Sorry about this. I don’t know a way around it.
  2. Create a new directory for your ironwood devstack project.
  3. Check out the ironwood branch of the devstack in it and set up a new devstack with the ironwood release set.
  4. This devstack will use launch services as edx.devstack.*
  5. In your master devstack create a new file called options.local.mk that has COMPOSE_PROJECT_NAME=devstack-master
  6. Set up this devstack from scratch. Now you master devstack is ready with services at edx.devstack-master.*

If you need a juniper devstack eventually, just create a local tag for juniper right now, it doesn’t matter if edX hasn’t tagged it, you can still use it with Juniper since it’s close enough :slight_smile:

5 Likes

That’s the way I’ve been doing Juniper. Works fine. :+1:

Webpack hot reload doesn’t work for Vagrant devstack, because when you edit file in shared directory, change event doesn’t be propagated to the guest system.

So if you want to make hot reload work:

CHOKIDAR_USEPOLLING=true npm start

Be aware, that this can drain your battery.

1 Like

Could I know where you’re using the Vagrant devstack? Your use case sounds like you’re talking about Ocim here. If that is the case, is there any particular reason you’re running the frontend inside the Vagrant devstack instead of outside it? I think that is the recommendation for Ocim. I don’t see any good reason to run the frontend dev server inside vagrant.

Yep, you right, it’s for new Ocim frontend.

You can run it inside Vagrant if you don’t want to bother yourself with making your environment close to what we have inside VM.

I know that it’s pretty rare case when something goes wrong just because of different node (or npm) version, but anyway that can happen to you.

VM footprint on performance is not so significant for me (at least for now :upside_down_face:).

I think in this case there is no risk of that happening. In production, the frontend isn’t served by the Ocim server or run using the webpack dev server, it’s compiled and served from S3.

So as long as you have a decent version of node and npm installed, feel free to run it outside the VM.

2 Likes

npm and its ecosystem is wild and unsafe and it makes sense to run on it a VM o docker container.

1 Like

Fair enough, though that is true of most such package distribution systems including pypi. There have been incidents of malware in Arch and other distro repositories as well. We all must take the security precautions we all find to be necessary or reasonable.

So I’ll clarify my main point. There is no requirement to run the frontend development tools inside the Ocim VM if your aim is to match the production environment. The production environment of the Ocim frontend is S3 serving “compiled” static assets. The assets are not even built in the Ocim environment so running there won’t help either. The assets are built in a docker container by using CircleCI’s node:lts image.

1 Like

The key point here is that npm in not guaranteed to behave consistently across the versions that the various developers currently have installed on their OS. This may lead to “works for me” situations.

That is a very useful insight @kshitij.

So if you are doing frontend development that requires any re-compilation of assets, then to be strictly “production equivalent” you would want to make sure that that recompilation happens in a node:lts container.

Sure, but here is the thing:

  • The frontend code is written in TypeScript, which is compiled to JavaScript.
  • The version of TypeScript and all related code is locked via the package-lock.json.
  • So this is only a problem if, for the same input TypeScript code and dependencies, you get different output depending on different npm version (which would mean it installed the wrong dependency) or a different node version (as bug in TypeScript or node).
  • The real production runtime for frontend code is the end-user’s browser. So, the production environment of the code is already variable across various versions of Firefox, Chrome, Safari etc.

The node:lts container is also a variable environment, since it updates regularly with the latest lts release of node, and npm. I guess my point is, use whatever setup you have, and only bother with all this if you face issues.

Given the length of this thread :) I thought I would transfer this question from Nimisha to us, on our shared Slack channel with edX:

OpenCraft friends, Question for you: Do you use Devstack for your local development? Since you also tend to follow master , I’m curious whether your devs run into devstack issues as much edXers do.

I don’t want to hijack this thread, but if someone is willing to answer Nimisha, please do – or better, compile feedback, maybe partially based on the contents from the current thread? Feel free to create & schedule a task on this - and if you don’t have time now, just let her know when you’ll give a more thorough answer :slight_smile:

@braden Thanks for replying to Nimisha!

FYI, for anyone interested, Nimisha replied in turn with a few links to proposals to improve devstack, if you want to comment, or add our own list of griefs as I was suggesting above: