Proposition: A More Scalable OpenCraft — Improving Self Management

Thank you, @daniel and @jill – you’re right that we’re starting to get the conversation fragmented. I’ve scheduled a ticket for my next sprint (next week, which coincides with the dev sprint rollover, too) to create several follow-up tasks. I’ll use these either for creating relevant MRs or else starting dedicated follow-up discussions.

In the meantime I’m going to let team members continue to trickle in and read. I’d especially be happy to see @antoviaque 's take on things. I’m out Friday, and have some things to deliver Thursday, so I know there are some threads I haven’t responded to, but I will–if not directly on the document, in the follow-up tasks.

By the way — this is a long read, a huge chunk to process, and something that will have potentially big impacts on how we work, so I definitely advise that people schedule a task to read and comment. It took me ~3h to read, process, and comment.

1 Like

Sorry for replying to this thread only now, but last month was tricky, and I wanted to make sure to have the time and attention to do justice to the work done here. Thanks a lot for that @nizarmah, @Fox and everyone else who participates to the discussions – there are lots of inspiring proposals in there, and it’s great to be able to think about those topics from new perspectives. Hopefully good will come out of this!

Cell types & priorities

It looks really useful to define types of cells according to their prioritization scheme: client-based cells correspond to the current prioritization we use across all of OpenCraft, ie “client first”. But this also means that we don’t handle our own projects well, since they always come after the client priorities - so having project-based cells who prioritize their project above all else, including internal ones, removes the interference of conflicting client priorities.

Devops cell

We definitely need more specialization and better assignation of work for DevOps, and experience has proven that the “DevOps specialist” role didn’t really solve our issues there. The risk with a “DevOps” cell though is that it makes it look like DevOps is something that is the sole responsibility of that cell… Maybe it would be less ambiguous to have that cell formed more around specific projects? Like the Grove/Ocim cell, which would be in charge of maintaining the deployment/hosting architecture – which then would in turn be used by other cells, to deploy their own clients/projects instances. All cells would still do the DevOps from their projects, and even contribute to Grove/Ocim if that serves their purposes – but there would be more specialization on the individual projects, a better sense of ownership, and less context switching.

+1 – that could be made part of the responsibilities of the same cell. I think it’s still worth having a cell specifically responsible for moving Grove/Ocim forward (as a type of project cell), but this same cell could also be responsible for establishing the best practices accross all OpenCraft DevOps.

Documentation cell

I agree with the current consensus that a whole cell for this might be overkill. However, it’s also true that maintaining and improving the documentation of OpenCraft is something that has become large and important enough to warrant affecting specialized and dedicated attention. We rely on it so much for sharing information and building our processes that we’d all benefit daily from this.

If this isn’t a full cell, maybe be a support cell role? One person assigned to it, possibly full time if the workload justifies it - and we could recruit specifically for it. Also I agree that it shouldn’t be the person writing everything - we still all need to keep writing documentation, that’s how we share our knowledge and experience. But that could be more the reviewer on all documentation, as well as the editor / refactorer of the documentation, to keep it consistent and well structured?

Recruitment

I’ll be interested in what the recruitment managers & @gabriel think about this. Given how the recruitment works, it could make sense to assign recruitment to a more specialized support role or cell, to help with reducing context switching, and providing more focus/accountability.

That said, the volume of recruitment can vary over time - I don’t think it’s ever a full time currently, and it can sometimes go back to zero. Over the past year, we have been recruiting almost non-stop, but we have often had times where we didn’t in the past, and that might come back. We might be able to pass on more work than what the recruitment managers do, by adding some of @gabriel 's work for recruitment/onboarding, and maybe some of the additional improvement work you mentioned @nizarmah (analyzing recruitment stats, improving the process, finding the best sources of candidatures, etc.) – but we need to keep in mind that there could be periods where there would not be any recruitment to do.

Project-based cells & sustainability

That’s definitely a point that needs to be considered, the consequences on how to calculate sustainability can be tricky. Maybe the cells could “buy” services from each other, through billed hours budgets? For example, the Grove/Ocim cell could “sell” its maintenance services to the cells who use hosted Open edX instances, by passing on X unbilled hours to each of those cells? Though, to a later point made by @Fox , it would add complexity to an already complex system for sustainability, and it might be worth refactoring the whole thing, taking project-based cells into account. (cf below)

Role based cells

It does look like this is similar to the support cell / deathstar.

Pursuer

I also like the fact that this offers to define a type of initiative/person that already happen, and is key to solving issues and keeping the iterative approach alive. It allows to define and provide support to those who start these initiatives – like more dedicated time, moving out of the development flow and to the support cell. This is be a disruptive change though, which impacts the responsibilities within the cell, and require non-cell budget. So to start with, I’ll want to review each new pursuer designation, based on a specific scope/mission & time budget.

Redefined sales role/process

It does makes sense that the decision to accept new projects come from each cell, it’s similar to how each cell decides on its own recruitment, based on the epic & sustainability manager predictions. I also agree that the sales role itself probably make sense to keep separate – as a delegation of each cell to a support cell, to keep it specialized. Within each cell we still need someone responsible for figuring out when sales should resume/stop; the natural role for this is likely the epic & sustainability manager?

Btw, I liked the approach to make epic/capacity planning more reliable @braden in [STAR-2224] Revisiting Approaches to Self-Management and Scaling - Google Documenten – it could actually be worth investigating on its own. Maybe you, @Fox @DouglasDraper and @tikr could take a task specifically to discuss how that could work, and create a corresponding handbook PR to experiment with it?

Removing managers

Making an organization fully decentralized, without me or Braden as backup managers, would require a lot more processes and group decisions. Just look at what happened to Zappos, Gitlab, etc. when they tried it – they ended up having to revert to a traditional hierarchy, because it was too difficult to handle edge cases, the ones the processes were not taking into account, or effects like the tragedy of the commons. And required more processes to handle that. I think our intermediate version is already providing too many processes, and this would only add to it. I’m keeping the BDFL status, because it’s an important failsafe.

I do agree though that there are different parts of self-management, and I definitely want to keep going in the direction of giving more latitude, decision power and autonomy to the person who do the work. It always feels like a win when a topic has been fully delegated to the team as a whole, and I don’t need to intervene or take care of it. It’s probably true that we (I?) have erred a bit too much on the side of ensuring that things keep happening, by adding too many rules or checklist items, so relaxing rules to replace them by personal judgement seem like an important next step. I’ve also replied inline to some of the elements related to this in the document.

Handbook - Separating core & cell-specific rules

+1 to this, it would be good to allow for more differentiation/experimentation within local contexts. Actually the original version of the handbook contained sections for cell-specific rules, but those were never used. It’s likely right that it’s a result of the centralized way we have come up with processes yes, and the fact that until fairly recently I was often the one driving most of the large processes changes.

To keep things sane though, rather than having multiples branches of the handbook for each cell, I’d simply have sections in the handbook describing the special rules of each cell. I would also consider the “core” rules as a sort of higher priority legal text, like a constitution - the “local” rules can’t contradict the core rules, which would be the part kept consistent across OpenCraft; this would mean that we need to keep the core rules minimal, and that doesn’t over-specify things, to keep that flexibility at the cell level.

We should also use the opportunity to differentiate parts of the handbook which are mandatory rules, and the ones which are advice - that’s a point @daniel brought up in the past that would be helpful to make the “core” set lighter, especially.

Sustainability

It’s true that sustainability calculations are tricky, and currently have many issues, including many blocked projects because of it. It would be good to get an idea of what would replace it, exactly - because that’s something that is hard to delegate the responsibility for to the cells in the first place. That seem like a topic big enough for its own thread and project though. :)

Exposing more of the finances for this seem overkill and going to the opposite of our goals - it would create even more metawork, and won’t make it easier to take decisions for budgets - on the contrary. I would keep the approach of using hours as a proxy for $, and find a simpler way to make it apparent when the number of hours billed allow us to afford the internal projects that we want.

Compensation

We already had a pretty lengthy conversation about this recently, I don’t want to reopen this for some time. The solution might not be perfect for everyone, but no solution is ever going to be perfect, and we are always going to have some people who want or need more money - and we can’t always say yes, or fix the processes to keep them. That’s life, and trying to keep tuning this will just remain a large distraction.

Task force

That also sounds like a project cell to me. I think what you’re saying is that the project cell could be made lighter - and we can indeed look at ways to do that, some of the proposals above help with this (by pushing out things like recruitment). But it’s also not a coincidence that the project cells still have a certain amount of process and responsibilities – someone still need to plan for capacity, deal with the client if there is one, etc. Maybe the best on this would be to look concretely at the responsibilities/processes that we want to remove from project cells, and see if that doable & how.

Quick wins

  • I agree that resuming the retrospectives is important, +1
  • We do need to do something about the number of core developers in Serenity; I’m not a fan of moving people between cells, which is really disruptive, but it is an extreme case - unless we will soon have newcomers confirmed?
  • For sales, cf above, it makes sense to move forward in that direction, as long as we don’t add new roles/metawork to cells

Next steps

There are LOTS of things being discussed here, and if we try to pursue everything at once, it will be overwhelming. A good next step would indeed be to start discussing on concrete merge requests. I would pick 2-3 of these points max (maybe even one to start?), see it through, and then come back to this list.

Which ones would that be?

6 Likes

I’m really glad to see such positive replies on this thread. I just want to post a small request; please do your best to keep the split discussions public on the forum. I hope I’ll have the opportunity to see how things progress. :crossed_fingers:

Also, massive thanks to anyone who put time towards this thread or is planning to. :hugs:

4 Likes
My choices (hidden so you can think of your own priorities before reading mine)

I would personally suggest, in order:

  1. DevOps cell - establish devops best practices for the team and proactively solve problems (upgrade Ubuntu 16.04!)
  2. Make capacity planning more reliable
  3. Simplify/refactor sustainability calculations
3 Likes

First of all, Thank you, @antoviaque for your thoughtful responses here. After reading Reinventing Organizations, it put in front of me a model that was both reminiscent and also distinct from what we’re currently doing.

I wonder if long-term we can ever embody the ideals presented in the book fully. The ‘obvious’ solution after reading it was to dive in as deep as I could and see where that took me. So, I wrote the proposal around that.

I knew that even if we couldn’t fully implement the archetypal model it would set the stage for useful discussion and give us a chance to re-examine much. And boy, has it. I feel both that I better understand the boundaries of what we can do and what we are willing to do, and what is most important to the team. I don’t think I could have done that without an idealized target to start with.

Aside from the quick wins, the items that Braden mentioned would also be my priorities. During my sprint this week I’ll perform the prepwork here for moving these forward, especially on the front of creating follow-up tasks and getting them scheduled.

4 Likes

@Fox Yes, we took inspiration from Reinventing Organizations, but the goal isn’t to implement it literally – more to take the pieces that are useful, but still come up with our own sauce. The literal implementation seem to be working for some, but not for others, and it looks like there is a quite significant amount of cases where it didn’t. I suspect some of it is related to the type of industry, and maybe on the predictability of the work & processes.

Thanks btw for moving the overall discussion forward – and the next steps @braden and you have prioritized look good to me :+1:

I’m a bit late to the party, but I’ve read through the documents and thread and left a few comments.


My choices
  1. Simplify sustainability

See my comments on:
[STAR-2224] Revisiting Approaches to Self-Management and Scaling - Google Documenten

  1. DevOps Task Force

Have a dedicated group of people responsible for maintaining and pushing DevOps tasks forward, until we catch up with the tech debt and then maybe bring back the DevOps Assignee (to keep the responsibility in one place).

3 Likes

I have created and scheduled:

  • Log in - OpenCraft For revising the retrospective process
  • Log in - OpenCraft For revising the sales process
  • Log in - OpenCraft For pre-work/discussion on a suggested solution for the DevOps cell brought up by @farhaan (Merging Bebop and Serenity, and splitting off the DevOps cell in the process). Forum thread here. Currently private since we might need to discuss client details for handoffs or what have you, though maybe that won’t happen and we’ll be able to make it public.
  • Log in - OpenCraft has been created and scheduled for refactoring sustainability/capacity planning.

I think that’s enough to get us started. Yay for progress!

6 Likes

@Fox Thanks!

What about “2. Make capacity planning more reliable” – is that something that would be handled separately?

Also, including that change, that would get us to 5 changes handled simultaneously – even maybe 6 considering that we are discussing implementing the recruitment changes in https://forum.opencraft.com/t/hasta-la-vista-leaving-in-february/1234/17 . That might be a bit too much to chew at once, especially given the size and impact of some of these changes? Prioritizing those is hard, as they are all good changes, but like with code doing too much at once runs the risk of taking longer with everything. Could we put a hard limit to 3 at once? As soon as one is done, we can start another one, but I think it will quickly become overwhelming otherwise.

2 Likes

Log in - OpenCraft is required for doing that, I think, since it’s going to change what figures we use for planning. That alone will likely improve our forecasting some, but I figure follow-up should wait until we know what we’re dealing with.

I’ve already staggered the tickets a few sprints, but I could, say, kick revising the sales process back since we’re not doing much selling right now anyway. I do think the research for cell split/merge for DevOps needs to be top priority and the Sprint retrospective is an easy thing to change/add to the handbook. Those can be done next week-- which is why I’ve scheduled them then.

Tim wrote up a timeline of the tickets I created so you can see how they’re spaced. If you still think these are a bit too tight together, I’ll kick the sales process and sustainability changes forward a bit more.

@Fox Alright – sounds good then. Let’s just remain mindful of this too then, and reconsider if it starts being too many changes at once.

2 Likes