Mattermost is moving to Kubernetes

Announcements - Public
#1

@team, on the 14th of September I’m going to move mattermost to Kubernetes. This will affect its uptime. We probably will have about a 120 minutes of downtime I’ll start the migration around 9AM UTC. In the meantime, you still can use Forum or OpenEdX’s slack. :crossed_fingers:

Mattermost is not exporting user preferences. Ergo, the profile pictures and other preferences set by the user for itself won’t be migrated. This is somewhat expected by the Mattermost team as I understand correctly. During the migration, I’ll give one more shot to migrate preferences as well, but please be prepared, that there is a high chance we need to set our profile pictures and name display settings again.

Ticket to log time on: SE-4988

4 Likes
#2

Good luck with the migration @gabor! :crossed_fingers:

1 Like
#3

@team A little update here: I had to reschedule moving MM from tomorrow to Monday Tuesday 1PM UTC.

Update: Matterlink is deprecated (or at least archived), so we are going to use Matter2Mail. Also, it seems that the Mattermost server is hosting something called “link shortener” (more about that).

#4

Question to the @team: Who is using link shortener? If it is still needed, we need to deploy that separately, otherwise, we can simply retire it.

  • I’m using the link shortener
  • I’m using the link shortener but let’s get rid of it
  • I’m not using the link shortener, let’s get rid of it

0 voters

Please make your vote until the end of this week.

#5

@gabor, I have been using it for a long time, but then it suddenly started getting caught by the HSTS rules, so I replaced it with a Redirector extension.

If anybody wants to try it, I have the following rules enabled in Firefox:

  1. Startpage
    • Include pattern: https:\/\/www\.startpage\.com\/do\/dsearch\?query=(\w+\-\d+)&.*$
    • Redirect to: https://tasks.opencraft.com/browse/$1
    • Example URL:: https://www.startpage.com/do/dsearch?query=bb-213&cat=web&pl=ext-ff&language=english&extVersion=1.3.0
    • Pattern type: Regular Expression
    • Apply to: Main window (address bar)
  2. DuckDuckGo (backup)
    • Include pattern: https:\/\/duckduckgo\.com\/\?t=ffab&q=(\w+\-\d+)$
    • Redirect to: https://tasks.opencraft.com/browse/$1
    • Example URL:: https://duckduckgo.com/?t=ffab&q=bb-213
    • Pattern type: Regular Expression
    • Apply to: Main window (address bar)
2 Likes
#6

Thank you @Agrendalath, this is a really great feedback! Also, seems to be a good alternative for those who want to have something link shortener.

#7

@gabor I had plans for setting up and using the link shortener at some point, but then I started using dynamic bookmarks instead :upside_down_face:

I found that to be the simplest solution for accessing JIRA tickets quickly via the address bar (= no third-party extensions needed).

In case anyone finds it useful, my current setup (Firefox) is:

Bookmark name Bookmark location Keywords
BB task https://tasks.opencraft.com/browse/BB-%s bb
FAL task https://tasks.opencraft.com/browse/FAL-%s fal
MNG https://tasks.opencraft.com/browse/MNG-%s mng
SE task https://tasks.opencraft.com/browse/SE-%s se
STAR task https://tasks.opencraft.com/browse/STAR-%s star

In terms of how to use it, this should be very similar to the solution that @Agrendalath chose:

To access a specific ticket I just type the appropriate keyword followed by the ticket ID into the address bar (e.g. bb 1234) and hit enter.

3 Likes
#8

I do a similar thing in Chrome, by setting up “OpenCraft JIRA” as a “Site search” Search Engine in the browser’s search engine settings:

Screen Shot 2022-09-16 at 9.42.55 AM

Then I can just type “oc”, space or tab, then enter any ticket number like MNG-2649 or enter any phrase, and it will either take me directly to that ticket or search as needed.

4 Likes
#9

@team The poll is closed. The result is clear, nobody put a vote on keeping the link shortener.

In the meantime, I found a mattermost issue with autolink. I need a day or two to figure out a resolution and provide a fix. Due to that, I’m in favor of no rush and pushing the migration to the 28th of September, 10AM UTC.

1 Like
#10

FTR/Update:

We have an open PR for Mattermost that fixes an Autolink configuration parsing issue. Due to that PR is not yet merged and Autolink is essential for us, I did not start the migration. As the migration requires up to ~2h downtime (since nobody shall send any messages during that time), and I had no ~2 in a row for the migration, I’ll try to do the migration during the weekend or next Tuesday. If the PR is not merged until that and no new releases cut, I’m going to build Mattermost and create a docker container for that and push it to a temporary repository we have: opencraft / Operations / Containers · GitLab.

Why not backporting the change, creating an image and doing the migration just right now? I don’t want to mess up anything with one of our primary communication tools.

Also, I’m making this thread private as it makes no sense to keep it public. It has no meaningful information to anyone except us.

#11

@gabor Thank you for taking care of Mattermost :slight_smile:

With the open first approach that we follow, it goes the other way – we put everything in the open by default, unless we have a good reason to keep it private (like client information for example, who often prefer to keep information private by default). The current thread might be irrelevant for everyone else, but that doesn’t hurt anyone either - people don’t have to follow every thread. But if someone is curious to see how we cook things in our open kitchen, they can :slight_smile: Can we keep this thread public?

1 Like
#12

but that’s doesn’t hurt anyone either

That’s true. Moving back to public. :crazy_face:

2 Likes
#13

@team Well, this was a long ride, but finally we have the patched Mattermost team server released by Mattermost including my fix for plugin config parsing. This means we can really-really-really migrate to Kubernetes. In fact, here is a screenshot of the staging mattermost instance I tested some things on:

Screenshot 2022-11-21 at 18.30.16
Screenshot 2022-11-21 at 18.30.163358×1868 430 KB

As mentioned earlier, user profile preferences will be lost, therefore everyone will experience the following (at least what I can tell):

  • usernames are shown instead of your current setup
  • all archived/previous conversations will appear on the left side (yes, this includes ex-peers as well)

Although we will move all attachments, it may happen that some of them won’t be available due to an upload error or similar. If that’s the case and you recognize it, please let me know and I’ll re-upload that asset.

Depending on the network we need 2-3 hours of downtime. This means that Mattermost will be completely unavailable. In the meantime, you can use Open Edx’s Slack.

Since this is a “no-impact” change (on customers), I’d not wait too much with the deployment and do it this Thursday around ~11 AM CET. Please let me know if you have any concerns.

7 Likes
#14

@team Mattermost migration is done. Please make sure to:

  • :warning: Reset your password, otherwise you cannot login
  • Set a profile picture (if you want)
  • Setup MFA (very much recommended)
  • Adjust user preferences (if you want)

During the migration there were some files that were not uploaded successfully. Mostly screenshots about error messages from 2017-2020. I did not investigate these failures as it could take too much time for ~10 screenshots.

:warning: To close channels/conversations (even if archived), first, you must open them. After that, you can close it. The Desktop client had a small issue on my end to close those conversations, but using the web app solved it.

Cheers! :beers:

6 Likes
#15

Is MFA enabled on our new instance? I cannot find it under Profile -> Security, as described here. I checked the web page and the app.

#16

@gabor, I’m also unable to enable mobile push notifications: “Push notifications have not been enabled by your System Administrator.”.

#17

@Agrendalath it seems that these kind of setting were not migrated. Very well.

Push & MFA enabled.

cc: @keithgg

3 Likes
#18

Great work @gabor ! The new shiny Mattermost is working great for me.

1 Like
#19

@grabor Could you help me with this? I couldn’t reset my password :slightly_smiling_face:

Captura de pantalla de 2022-12-02 15-23-06
Captura de pantalla de 2022-12-02 15-23-06729×444 33.1 KB

#20

I seem to be having the same issue.