I noticed when I was trying to upstream a change that there is a discrepancy between the tests that are running in the upstream CI and our forks’ CI, namely the JS Tests. They don’t seem to be running in our Nutmeg or Palm, however runs upstream (it failed initially, so I had to make changes).
I wanted to ask if this is intended behavior, i.e. not having all the tests that run upstream also run in our forks (more specifically JS tests). If so, what do we do in the case where tests fail upstream? Wouldn’t that be a cause for concern as code being merged into our forks might have potential issues but are not caught as the full tests suite in the CI are not run?
I could be wrong, but I think the tests are essentially part of the platform code, in the .github folder, and they have changed significantly over time. The tests that run on our Nutmeg branch are the same as the ones from the upstream version at the time of Nutmeg. And so on.
The JS tests are actually configured on both forks/versions. The issue is that the JS tests are (perhaps incorrectly) configured to only run on PRs against master. Since your PR on our Palm fork would have been targeting opencraft-release/palm.1, the tests didn’t run.
I see, that makes a lot of sense. Thanks for the clarification, seems like it might be worth creating a ticket to fix this upstream, I can go ahead and create one if we think it’s worthwhile?