Proposition: A More Scalable OpenCraft — Improving Self Management

For the past while, there have been many issues I have not been a fan of.

Members at OpenCraft have been in different burnout cycles, across different cells.
Moreover, there has been a lack of effective resolutions to combat these issues.

It’s clear, across different forum threads that different members and Xavier have not been on the same page regarding self management and management.

Accordingly, I’ve prepared a proposition, A More Scalable OpenCraft — Improving Self Management.
The proposition is my attempt at communicating what many members have been requesting for a while now, while maintaining Xavier’s vision of OpenCraft and self management.

The proposition is 8 pages, so please create tickets to read the proposition and leave your comments about it on this forum thread, not the google document.

[ Ticket to log time — SE-5120 ]

1 Like

Thanks for caring for your coworkers, sharing your thoughts and offering alternatives, @nizar. I did a quick first read of your document but will need to read it again and take some time to think about all this. In the meantime, I would like to point out that efforts have been made to address the situation. It’s probably not your intent, but I’m feeling like your post and your document don’t take into account the things we’ve put in place to mitigate the issues: increasing hiring, trial projects, minimum rate increase, project-based cells, cell supporter role, etc.

It’s difficult for me right now to figure out where these initiatives (a lot of which are in progress and their effects not yet measurable) fit within your proposal. Maybe other people’s comments will help with this. I’ll take more time to think about it.

Again, thanks for bringing difficult discussions to the table, and I’m looking forward to participating.

1 Like

Ah, I’m sorry it sounded that way. My intention wasn’t to disregard all the efforts put towards mitigating the issue. These different mitigations have definitely been effective in addressing different issues.

I do trust in our capabilities and efforts to put out fires, but my concern at the moment isn’t with putting out the fires. My concern is to showcase the need to reorganize our efforts in mitigating fires, in a manner which would keep the fires from spreading and extinguish them.

I hope that better communicates the goal behind this proposition.

1 Like

@nizar Thanks for putting this together. I’ll read it, think about it, and do a full review.

I probably won’t have time to do the review before the holidays, and I’m only back for 3 days at the beginning of January before going for a second week of vacation, so the review will likely only happen from mid-January. But I wanted to mention quickly that I appreciate the initiative, to name issues you see, and try to find improvements and refactoring steps for it, while at the same time considering the history and culture of OpenCraft and avoiding the temptation of a full rewrite. That’s a difficult equilibrium, and we’ll talk more about it after the holidays, but I’ll be looking forward to try to figure out what our next iteration steps should be.

1 Like

@nizar

Moving my comments here from the Google doc, where I initially posted them-- forgot you wanted them here.

Project-based cells notes

As of yet, no internal projects have been used as the basis of a project cell. Using them as the basis of a project cell means a massive change in how we handle sustainability.

This may be a good thing, but it’s important to understand the ripple effects. It would mean that the budgets and hours of this cell are set by the sustainability of other cells. And those other cells would share their sustainability between their own non-billable tasks and the work needed to run these projects.

DevOps Cell

I do like the idea of creating more dedicated allowance for these devops tasks. Especially concerning things like platform upgrades-- which are currently rotating in a way that creates context switching.

But it also creates sustainability questions and introduces one other problem-- in that the DevOps burden of a cell is not tied to the development burden in the same way. I worry that this may cause a client cell to treat DevOps as a commons, where it can become easy to ignore or not fully consider the DevOps implications of changes because that’s ‘someone else’s job’. How would we make sure that cells account for their impact on DevOps work?

Documentation cell

This section I’m the least convinced by.

What information is missing, specifically? Shouldn’t we be looking at adding this information rather than changing our structure to solve these problems?

Documentation is every developer’s duty. I feel like instead of creating specialized cells to handle it we may need to create better ways of having information disseminated-- something that can be done ‘in-line’ with handbook changes or similar.

The trouble I see with adding a new type of cell for documentation is that it may fall to the same issues-- it needs the overhead of sprint planning. Creating cell types may be more expensive than integrating these roles into the cells. It also creates a potential commons problem for documentation.

Role-based cells

How does this compare to the recently merged Support Cells? We now have one support cell, which you are migrating to. We may create another in time.

Right now, recruitment is done within cells by cells via their recruitment specialist. Would you suggest moving one or more people to the support cell and making recruitment their full time?

Pursuers

I really like the idea of a pursuer because it reflects what someone like you or I tend to find ourselves doing anyway but creating a process/role space in which to do it.

There are a number of problems that come up which there is no obvious point person to handle. The recent changes to recruiting-- including building a whole new section of the site, weren’t within my normal duties as a business development specialist, yet I did them anyway. They were just what needed to be done. They also weren’t directly under the responsibility of the recruitment manager, either.

I’m not sure it would have been as possible to do them if I were bound to normal development epics and responsibilities.

This leaves me with a few significant questions, though:

How much of what a persuer is intended to do is actually covered by your new Developer advocate role? It sounds like there’s a lot of overlap here. As of yet, the developer advocate role has no official definition in the handbook-- I created a stub article recently but it’s just that-- a stub. It’s unclear where it begins and ends because no concrete proposal on its scope has yet been tendered and accepted.

I do like how currently, anyone can start an initiative like this in the company. This feels like an attempt to create more support structure around someone taking charge of an initiative like this, but I’m unsure if it’s the best approach.

Electing a person seems like a bit of overhead that isn’t needed-- it may be better to just make it explicit that someone can decide they want to take on an initiative, and that this means ‘becoming a persuer’ and then defining how that would work-- handing their epics off to their backup, reducing or removing committed dev hours until the problem is resolved and then giving them the space to focus on it until it’s completed.

Overall thoughts

I’m skeptical of the idea of creating more cell types as we have only just added project cells and solidified support cells. Without having given these time to play out for a while it’s hard to know how good of a solution this specialization is providing and it seems premature to create new cell types to solve problems. Cell creation has significant overhead costs, and it creates additional border issues of cross-cell interaction that have to be weighed.

I’m also concerned, as @gabriel mentioned, that we haven’t given enough time for the current interventions to work. I’d like to get you actively engaged in your developer advocate role full time and see if in that space you come up with targeted interventions that don’t require huge structural changes or if these patterns still necessitate such a reorganization after a few months.

2 Likes

Thanks for putting this together @nizar.

A few quick thoughts:

  • A DevOps cell has been discussed in the past and I think it would be a good thing. To avoid the “it’s someone else’s job” problem, the cell should focus mostly on establishing best practices and setting up/documenting key parts of our own infrastructure rather than handling all devops tasks.
  • Documentation cell seems overkill. I remember we discussed a possibility of having a rotating “documentation duty” role where each sprint somone would reserve a chunk of their time to work exclusively on documentation to address the poor documentation problem, but we never tried doing that in practice.

Thanks for putting this together, @nizar!

I’m not convinced about this section either. I do believe that everyone should be responsible for documentation, no matter which cell we are talking about. Documenting stuff is a great opportunity for everyone to keep its knowledge updated periodically.

Agreed. It would be awesome to have a dedicated sub-team fixing issues in prior it hits back on us. That could reduce fires, increase customer satisfaction, result in better company image.

+1. I specifically included a documentation step in our PR review templates for this reason. It’s also the case that the developer who wrote the code probably has the most relevant expertise to share by writing documentation. I don’t want people thinking “that’s not my job” when it comes to documentation. Instead, documentation should always be part of our processes, whether that involves writing code, planning upgrades, refining how OpenCraft works, onboarding people, or anything else.

I like that idea.

@nizar Thanks for the proposal.

Your proposal was also clarified in the video call you had with Fox (https://forum.opencraft.com/t/re-assigning-my-responsibilities-before-leaving/1236/2). I think that’s a good addition, since these type of changes are easier to discuss by speaking than by writing. In that call you mentioned that it could be posted at the forum. So if others are interested, they can watch it.

About your proposal. I grouped my answers by topic, but I’m replying to different people and different sections of the document.

Cell responsibilities and metawork

  • I agree with the comments about cell size and roles. If there are more roles than people in the cell, it means everybody gets roles. Rotations like firefighting also become very frequent in a small cell. These topics (cell size, amount of roles) were discussed in the survey about work and metawork and some roles were merged.
  • That survey may have produced more changes so it could be a good time to follow up on it
  • In a very small cell, capacity changes are noticeable. The effect will be much stronger in project cells: e.g. if there are 3 people in a cell, and 1 person takes holiday to relax from a stressful sprint, then the remaining 2 have more work

Surviving/Stress/Burnout

Members, lately, are dedicating most of their effort and focus towards “surviving” or accomplishing their handbook-defined responsibilities to avoid spillovers.

„Surviving“ to avoid spill-overs has been a very common style of work, even before cells.
When we did 1-week sprints and sprint meetings, a lot of the work happened in the hours before the meetings. Many tasks were reviewed and closed minutes before the end of the sprint.
This has been improved thanks to doing 2-week sprints, and to removing the meetings.

But we also introduced new checklists and deadlines, so the concept of „surviving“ still applies. I think there were discussions to try to help people to do things earlier instead of at the last minute. We’re still seeing that we need to ping people to submit a form, create the social chat, post the ops log, plan the sprint, post a video update, etc.
Instead of blaming people for not doing everything, we could try to understand why or when it’s hard to survive the deadlines, and try to support people.

The „developer advocate“/„pursuer“ role could help.

Role Based Cells

I think the main difference that you are proposing (vs. the current situation with „roles“) is that, in your proposal, people could focus on only doing the role they’ve chosen, with additional tasks being optional.
For instance, someone with the role of „recruitment“ does mainly that, and then if they have time they can help in other tasks, take epics etc.

That would make choosing roles much more intentional, with people taking the roles they want, instead of being forced to take roles (and epics, and clients, and rotations) because the cell is small and someone needs to take the roles.
There’s still the question of „who takes the roles that nobody wants to take“, and I think that with a bigger team, and support cells, there will be more chances of finding someone to take them.

I think another difference you propose is that those roles would be company-wide, so that there wouldn’t need to be a recruitment role inside each cell, but just one group of people in the company to assist all cells.
This looks like „departments“ inside a company and I wonder if we’re slowly rediscovering that. Sales department, accounting department, hiring department, designers, …
There’s a struggle here, because the current structure involves having each cell be more self-sufficient (like a mini-OpenCraft). Though we don’t have a marketing role in each cell, or a prospects role, or a design department in each cell.
It will be interesting to see which roles are inside cells and which roles are in support cells (outside development cells); I’ll track how OpenCraft does it in the future.

Sustainability is complex even with 3 cells, and it will get much more complex with project cells. I’d recommend ignoring detailed calculations about sustainability, at least until the sustainability dashboard can do all the tedious work.
A simpler rule of thumb, like „the company should take much more billed work than unbilled work“, can suffice until then.

Devops cell

By having a project-based DevOps cell, things can be much more organized and members can work much more efficiently without unnecessary context switches.

By the way, there was already a „devops subteam“ when we had subteams.
It doesn’t mean that they worked just in devops tasks.

I don’t remember why subteams were removed.
I found https://forum.opencraft.com/t/new-plan-for-subteams-epic-management/33 but didn’t have time to relearn that part of history.
It looks like a a reason to remove subteams was that they caused „team fragmentation“. Well, cells are doing that too!

Documentation cell

As others mentioned, a „documentation cell“ may be too ambitious, and in addition everyone must do a bit of documentation. But if we expand the scope, and by „documentation“ we mean rules, processes and metawork, then there’s enough work for a person. Some duties in these areas are:

  • simplifying (reducing/removing) processes
  • meeting with newcomers that went through the onboarding course and handbook, and applying their feedback. Same, for people who are leaving the company
  • tracking the parts that are easily skipped or forgotten by newcomers or core (e.g. candidates that don’t learn about the right time-logging practices and are rejected because of that)
  • making sure all processes are at least mentioned in the documentation (e.g. right now, trial projects and accelerated epics are 2 important processes we follow, that are not documented in the handbook)
  • keeping documentation and processes up to date and pinging others
  • automating
  • etc.

Pursuers

They look very similar. I don’t mind the name, but let’s define the roles by the functions they do. If we don’t scope the functions, they can change and the spirit of the idea of lost. E.g. the „developer advocate“, someone who had to care about developers’ well-being, was dilluted into a role about collaborating with edX and the community.

I think (and I’m guessing a bit) that in your proposal you expect that a pursuer role would be very temporary, and focus on just one problem at a time and work until it’s fixed (hence the name, I suppose). This is in contrast to the permanent roles and duties of the current cell roles.
(It’s also in contrast with just reporting problems —e.g. in sprint retrospective or stress reports— and not pursuing it).

Combining it with your proposal of role-based cells, we could have a pursuer cell (or problem-fixing cell, or any other name), which supports all cells. I think we have a support cell experiment going on (I don’t know).
Whatever the way it’s implemented, I only wish for having someone actively fixing problems that affect the team.

Proposal, in general

Here’s a possible relationship: Someone (like the developer advocate) should be watching that all the attempts are being implemented and producing good results.
If there isn’t a person pushing them forward, sometimes they lose steam (see for instance how we stopped doing sprint retrospectives).

In order to better prioritize client work, internal projects, and internal issues, it’s important to introduce three new specialized cells: client based cells, project based cells and role based cells.

I’d prefer not to expand the already large handbook with new roles and knowledge.
If we have to change processes, I’d suggest changing some (maybe by opening a PR to change the affected sections), or making more processes optional

But we could start with practical steps first, like for instance having someone taking care of everything you mentioned, but without having to rewrite the handbook yet.
For instance, let’s try having the new developer advocate role spend some time fixing problems, and doing lots of small changes.
If it works, that itself may be enough. If at some point the developer advocate feels blocked because structural changes are required, then we can discuss them. Otherwise, so many PRs could create a lot of metawork for everyone else, and we also wanted to avoid this metawork.

Apparently the changes also need to go slowly, to give time to the recent changes (project cells etc.).
Though I feel that the „project cells“ was complex and not very well defined, and I wouldn’t mind replacing it by something simpler now instead of having to wait until it cracks.

Before any next steps are taken, opinions and comments need to be received about the proposition. Once the proposition has been discussed among members, the future steps can be decided.

@nizar Note that not everyone participates in discussions like this one. Some of them may have time just to read but not to answer. Others prefer to reduce the amount of metawork.

@nizar Thanks again for the discussions!
I hope some of them produce changes.

1 Like

After speaking in depth with Nizar about this, and having read through Reinventing Organizations, I’ve determined that I’ve fundamentally misunderstood what Nizar is trying to say, and with his agreement I’ll be writing a revised version of this proposal which integrates what he’s driving at with some ideas of my own next week.

4 Likes

Looking forward to it!

1 Like

@nizar Thank you for the additional effort towards improving interal processes and combating burnout.

While it may not be feasible to apply the proposal as is, it does contain a number of ideas that we could build on.

I agree :100: We’re a company that specializes not only in platform development but also in hosting, and yet, we haven’t found a way to consistently treat DevOps work as a first-class citizen.

We do spend time fixing issues as they occur (via the recurring Ocim Tech Debt & Bugs and DevOps epics); but more dedicated efforts addressing specific infrastructure and/or security-related aspects still tend to get pushed aside. The main factors that are contributing to this are:

  1. Issues with raw capacity.
  2. Lack of core team member availability. Even if a cell has enough raw capacity to work on internal projects, core team members are needed to lead and coordinate work on the corresponding epics: If each core member belonging to a cell is already at their limit of clients/epics/roles that they can simultaneously handle, this effectively leaves no room for the cell to take on additional projects, irrespective of the number of capable newcomers that the cell has available. (This is an issue that affects all types of projects that a cell could potentially take on, not just internal ones.)
  3. Permissions. We used to let newcomers work on internal infrastructure-related projects but as far as I’m aware this is no longer the case.
  4. Sustainability considerations. With the way that we currently handle sustainability, internal projects that are using non-billable cell accounts are first in line to be deprioritized by a cell if it needs to improve its sustainability ratio.

This is more of a side note, but we really need to complete BB-3265 Document upgrade process in the handbook to make the upgrade process more efficient – whether platform upgrades ultimately become part of the responsibility of a DevOps cell or not.

DevOps work that is directly related to development (such as deploying a new feature to a client’s instance once it’s complete), as well as fixing client-specific infrastructure issues would remain part of the responsibilities of regular cells.

Also, as @mtyaka mentioned, the DevOps cell could still delegate some “pure” DevOps tasks (involving changes to internal, non client-specific infrastructure) to other cells; it wouldn’t be expected to handle everything that comes up, but rather to lead and set a direction for projects concerning our internal infrastructure (including Ocim).