For the past while, there have been many issues I have not been a fan of.
Members at OpenCraft have been in different burnout cycles, across different cells.
Moreover, there has been a lack of effective resolutions to combat these issues.
It’s clear, across different forum threads that different members and Xavier have not been on the same page regarding self management and management.
Accordingly, I’ve prepared a proposition, A More Scalable OpenCraft — Improving Self Management.
The proposition is my attempt at communicating what many members have been requesting for a while now, while maintaining Xavier’s vision of OpenCraft and self management.
The proposition is 8 pages, so please create tickets to read the proposition and leave your comments about it on this forum thread, not the google document.
Thanks for caring for your coworkers, sharing your thoughts and offering alternatives, @nizar. I did a quick first read of your document but will need to read it again and take some time to think about all this. In the meantime, I would like to point out that efforts have been made to address the situation. It’s probably not your intent, but I’m feeling like your post and your document don’t take into account the things we’ve put in place to mitigate the issues: increasing hiring, trial projects, minimum rate increase, project-based cells, cell supporter role, etc.
It’s difficult for me right now to figure out where these initiatives (a lot of which are in progress and their effects not yet measurable) fit within your proposal. Maybe other people’s comments will help with this. I’ll take more time to think about it.
Again, thanks for bringing difficult discussions to the table, and I’m looking forward to participating.
Ah, I’m sorry it sounded that way. My intention wasn’t to disregard all the efforts put towards mitigating the issue. These different mitigations have definitely been effective in addressing different issues.
I do trust in our capabilities and efforts to put out fires, but my concern at the moment isn’t with putting out the fires. My concern is to showcase the need to reorganize our efforts in mitigating fires, in a manner which would keep the fires from spreading and extinguish them.
I hope that better communicates the goal behind this proposition.
@nizar Thanks for putting this together. I’ll read it, think about it, and do a full review.
I probably won’t have time to do the review before the holidays, and I’m only back for 3 days at the beginning of January before going for a second week of vacation, so the review will likely only happen from mid-January. But I wanted to mention quickly that I appreciate the initiative, to name issues you see, and try to find improvements and refactoring steps for it, while at the same time considering the history and culture of OpenCraft and avoiding the temptation of a full rewrite. That’s a difficult equilibrium, and we’ll talk more about it after the holidays, but I’ll be looking forward to try to figure out what our next iteration steps should be.
Moving my comments here from the Google doc, where I initially posted them-- forgot you wanted them here.
Project-based cells notes
As of yet, no internal projects have been used as the basis of a project cell. Using them as the basis of a project cell means a massive change in how we handle sustainability.
This may be a good thing, but it’s important to understand the ripple effects. It would mean that the budgets and hours of this cell are set by the sustainability of other cells. And those other cells would share their sustainability between their own non-billable tasks and the work needed to run these projects.
DevOps Cell
I do like the idea of creating more dedicated allowance for these devops tasks. Especially concerning things like platform upgrades-- which are currently rotating in a way that creates context switching.
But it also creates sustainability questions and introduces one other problem-- in that the DevOps burden of a cell is not tied to the development burden in the same way. I worry that this may cause a client cell to treat DevOps as a commons, where it can become easy to ignore or not fully consider the DevOps implications of changes because that’s ‘someone else’s job’. How would we make sure that cells account for their impact on DevOps work?
Documentation cell
This section I’m the least convinced by.
What information is missing, specifically? Shouldn’t we be looking at adding this information rather than changing our structure to solve these problems?
Documentation is every developer’s duty. I feel like instead of creating specialized cells to handle it we may need to create better ways of having information disseminated-- something that can be done ‘in-line’ with handbook changes or similar.
The trouble I see with adding a new type of cell for documentation is that it may fall to the same issues-- it needs the overhead of sprint planning. Creating cell types may be more expensive than integrating these roles into the cells. It also creates a potential commons problem for documentation.
Role-based cells
How does this compare to the recently merged Support Cells? We now have one support cell, which you are migrating to. We may create another in time.
Right now, recruitment is done within cells by cells via their recruitment specialist. Would you suggest moving one or more people to the support cell and making recruitment their full time?
Pursuers
I really like the idea of a pursuer because it reflects what someone like you or I tend to find ourselves doing anyway but creating a process/role space in which to do it.
There are a number of problems that come up which there is no obvious point person to handle. The recent changes to recruiting-- including building a whole new section of the site, weren’t within my normal duties as a business development specialist, yet I did them anyway. They were just what needed to be done. They also weren’t directly under the responsibility of the recruitment manager, either.
I’m not sure it would have been as possible to do them if I were bound to normal development epics and responsibilities.
This leaves me with a few significant questions, though:
How much of what a persuer is intended to do is actually covered by your new Developer advocate role? It sounds like there’s a lot of overlap here. As of yet, the developer advocate role has no official definition in the handbook-- I created a stub article recently but it’s just that-- a stub. It’s unclear where it begins and ends because no concrete proposal on its scope has yet been tendered and accepted.
I do like how currently, anyone can start an initiative like this in the company. This feels like an attempt to create more support structure around someone taking charge of an initiative like this, but I’m unsure if it’s the best approach.
Electing a person seems like a bit of overhead that isn’t needed-- it may be better to just make it explicit that someone can decide they want to take on an initiative, and that this means ‘becoming a persuer’ and then defining how that would work-- handing their epics off to their backup, reducing or removing committed dev hours until the problem is resolved and then giving them the space to focus on it until it’s completed.
Overall thoughts
I’m skeptical of the idea of creating more cell types as we have only just added project cells and solidified support cells. Without having given these time to play out for a while it’s hard to know how good of a solution this specialization is providing and it seems premature to create new cell types to solve problems. Cell creation has significant overhead costs, and it creates additional border issues of cross-cell interaction that have to be weighed.
I’m also concerned, as @gabriel mentioned, that we haven’t given enough time for the current interventions to work. I’d like to get you actively engaged in your developer advocate role full time and see if in that space you come up with targeted interventions that don’t require huge structural changes or if these patterns still necessitate such a reorganization after a few months.
A DevOps cell has been discussed in the past and I think it would be a good thing. To avoid the “it’s someone else’s job” problem, the cell should focus mostly on establishing best practices and setting up/documenting key parts of our own infrastructure rather than handling all devops tasks.
Documentation cell seems overkill. I remember we discussed a possibility of having a rotating “documentation duty” role where each sprint somone would reserve a chunk of their time to work exclusively on documentation to address the poor documentation problem, but we never tried doing that in practice.
I’m not convinced about this section either. I do believe that everyone should be responsible for documentation, no matter which cell we are talking about. Documenting stuff is a great opportunity for everyone to keep its knowledge updated periodically.
Agreed. It would be awesome to have a dedicated sub-team fixing issues in prior it hits back on us. That could reduce fires, increase customer satisfaction, result in better company image.
+1. I specifically included a documentation step in our PR review templates for this reason. It’s also the case that the developer who wrote the code probably has the most relevant expertise to share by writing documentation. I don’t want people thinking “that’s not my job” when it comes to documentation. Instead, documentation should always be part of our processes, whether that involves writing code, planning upgrades, refining how OpenCraft works, onboarding people, or anything else.
Your proposal was also clarified in the video call you had with Fox (https://forum.opencraft.com/t/re-assigning-my-responsibilities-before-leaving/1236/2). I think that’s a good addition, since these type of changes are easier to discuss by speaking than by writing. In that call you mentioned that it could be posted at the forum. So if others are interested, they can watch it.
About your proposal. I grouped my answers by topic, but I’m replying to different people and different sections of the document.
Cell responsibilities and metawork
I agree with the comments about cell size and roles. If there are more roles than people in the cell, it means everybody gets roles. Rotations like firefighting also become very frequent in a small cell. These topics (cell size, amount of roles) were discussed in the survey about work and metawork and some roles were merged.
That survey may have produced more changes so it could be a good time to follow up on it
In a very small cell, capacity changes are noticeable. The effect will be much stronger in project cells: e.g. if there are 3 people in a cell, and 1 person takes holiday to relax from a stressful sprint, then the remaining 2 have more work
Surviving/Stress/Burnout
Members, lately, are dedicating most of their effort and focus towards “surviving” or accomplishing their handbook-defined responsibilities to avoid spillovers.
„Surviving“ to avoid spill-overs has been a very common style of work, even before cells.
When we did 1-week sprints and sprint meetings, a lot of the work happened in the hours before the meetings. Many tasks were reviewed and closed minutes before the end of the sprint.
This has been improved thanks to doing 2-week sprints, and to removing the meetings.
But we also introduced new checklists and deadlines, so the concept of „surviving“ still applies. I think there were discussions to try to help people to do things earlier instead of at the last minute. We’re still seeing that we need to ping people to submit a form, create the social chat, post the ops log, plan the sprint, post a video update, etc.
Instead of blaming people for not doing everything, we could try to understand why or when it’s hard to survive the deadlines, and try to support people.
The „developer advocate“/„pursuer“ role could help.
Role Based Cells
I think the main difference that you are proposing (vs. the current situation with „roles“) is that, in your proposal, people could focus on only doing the role they’ve chosen, with additional tasks being optional.
For instance, someone with the role of „recruitment“ does mainly that, and then if they have time they can help in other tasks, take epics etc.
That would make choosing roles much more intentional, with people taking the roles they want, instead of being forced to take roles (and epics, and clients, and rotations) because the cell is small and someone needs to take the roles.
There’s still the question of „who takes the roles that nobody wants to take“, and I think that with a bigger team, and support cells, there will be more chances of finding someone to take them.
I think another difference you propose is that those roles would be company-wide, so that there wouldn’t need to be a recruitment role inside each cell, but just one group of people in the company to assist all cells.
This looks like „departments“ inside a company and I wonder if we’re slowly rediscovering that. Sales department, accounting department, hiring department, designers, …
There’s a struggle here, because the current structure involves having each cell be more self-sufficient (like a mini-OpenCraft). Though we don’t have a marketing role in each cell, or a prospects role, or a design department in each cell.
It will be interesting to see which roles are inside cells and which roles are in support cells (outside development cells); I’ll track how OpenCraft does it in the future.
Sustainability is complex even with 3 cells, and it will get much more complex with project cells. I’d recommend ignoring detailed calculations about sustainability, at least until the sustainability dashboard can do all the tedious work.
A simpler rule of thumb, like „the company should take much more billed work than unbilled work“, can suffice until then.
Devops cell
By having a project-based DevOps cell, things can be much more organized and members can work much more efficiently without unnecessary context switches.
By the way, there was already a „devops subteam“ when we had subteams.
It doesn’t mean that they worked just in devops tasks.
I don’t remember why subteams were removed.
I found https://forum.opencraft.com/t/new-plan-for-subteams-epic-management/33 but didn’t have time to relearn that part of history.
It looks like a a reason to remove subteams was that they caused „team fragmentation“. Well, cells are doing that too!
Documentation cell
As others mentioned, a „documentation cell“ may be too ambitious, and in addition everyone must do a bit of documentation. But if we expand the scope, and by „documentation“ we mean rules, processes and metawork, then there’s enough work for a person. Some duties in these areas are:
simplifying (reducing/removing) processes
meeting with newcomers that went through the onboarding course and handbook, and applying their feedback. Same, for people who are leaving the company
tracking the parts that are easily skipped or forgotten by newcomers or core (e.g. candidates that don’t learn about the right time-logging practices and are rejected because of that)
making sure all processes are at least mentioned in the documentation (e.g. right now, trial projects and accelerated epics are 2 important processes we follow, that are not documented in the handbook)
keeping documentation and processes up to date and pinging others
automating
etc.
Pursuers
They look very similar. I don’t mind the name, but let’s define the roles by the functions they do. If we don’t scope the functions, they can change and the spirit of the idea of lost. E.g. the „developer advocate“, someone who had to care about developers’ well-being, was dilluted into a role about collaborating with edX and the community.
I think (and I’m guessing a bit) that in your proposal you expect that a pursuer role would be very temporary, and focus on just one problem at a time and work until it’s fixed (hence the name, I suppose). This is in contrast to the permanent roles and duties of the current cell roles.
(It’s also in contrast with just reporting problems —e.g. in sprint retrospective or stress reports— and not pursuing it).
Combining it with your proposal of role-based cells, we could have a pursuer cell (or problem-fixing cell, or any other name), which supports all cells. I think we have a support cell experiment going on (I don’t know).
Whatever the way it’s implemented, I only wish for having someone actively fixing problems that affect the team.
Proposal, in general
Here’s a possible relationship: Someone (like the developer advocate) should be watching that all the attempts are being implemented and producing good results.
If there isn’t a person pushing them forward, sometimes they lose steam (see for instance how we stopped doing sprint retrospectives).
In order to better prioritize client work, internal projects, and internal issues, it’s important to introduce three new specialized cells: client based cells, project based cells and role based cells.
I’d prefer not to expand the already large handbook with new roles and knowledge.
If we have to change processes, I’d suggest changing some (maybe by opening a PR to change the affected sections), or making more processes optional
But we could start with practical steps first, like for instance having someone taking care of everything you mentioned, but without having to rewrite the handbook yet.
For instance, let’s try having the new developer advocate role spend some time fixing problems, and doing lots of small changes.
If it works, that itself may be enough. If at some point the developer advocate feels blocked because structural changes are required, then we can discuss them. Otherwise, so many PRs could create a lot of metawork for everyone else, and we also wanted to avoid this metawork.
Apparently the changes also need to go slowly, to give time to the recent changes (project cells etc.).
Though I feel that the „project cells“ was complex and not very well defined, and I wouldn’t mind replacing it by something simpler now instead of having to wait until it cracks.
Before any next steps are taken, opinions and comments need to be received about the proposition. Once the proposition has been discussed among members, the future steps can be decided.
@nizar Note that not everyone participates in discussions like this one. Some of them may have time just to read but not to answer. Others prefer to reduce the amount of metawork.
@nizar Thanks again for the discussions!
I hope some of them produce changes.
After speaking in depth with Nizar about this, and having read through Reinventing Organizations, I’ve determined that I’ve fundamentally misunderstood what Nizar is trying to say, and with his agreement I’ll be writing a revised version of this proposal which integrates what he’s driving at with some ideas of my own next week.
@nizar Thank you for the additional effort towards improving interal processes and combating burnout.
While it may not be feasible to apply the proposal as is, it does contain a number of ideas that we could build on.
I agree We’re a company that specializes not only in platform development but also in hosting, and yet, we haven’t found a way to consistently treat DevOps work as a first-class citizen.
We do spend time fixing issues as they occur (via the recurring Ocim Tech Debt & Bugs and DevOps epics); but more dedicated efforts addressing specific infrastructure and/or security-related aspects still tend to get pushed aside. The main factors that are contributing to this are:
Issues with raw capacity.
Lack of core team member availability. Even if a cell has enough raw capacity to work on internal projects, core team members are needed to lead and coordinate work on the corresponding epics: If each core member belonging to a cell is already at their limit of clients/epics/roles that they can simultaneously handle, this effectively leaves no room for the cell to take on additional projects, irrespective of the number of capable newcomers that the cell has available. (This is an issue that affects all types of projects that a cell could potentially take on, not just internal ones.)
Permissions. We used to let newcomers work on internal infrastructure-related projects but as far as I’m aware this is no longer the case.
Sustainability considerations. With the way that we currently handle sustainability, internal projects that are using non-billable cell accounts are first in line to be deprioritized by a cell if it needs to improve its sustainability ratio.
This is more of a side note, but we really need to complete BB-3265 Document upgrade process in the handbook to make the upgrade process more efficient – whether platform upgrades ultimately become part of the responsibility of a DevOps cell or not.
DevOps work that is directly related to development (such as deploying a new feature to a client’s instance once it’s complete), as well as fixing client-specific infrastructure issues would remain part of the responsibilities of regular cells.
Also, as @mtyaka mentioned, the DevOps cell could still delegate some “pure” DevOps tasks (involving changes to internal, non client-specific infrastructure) to other cells; it wouldn’t be expected to handle everything that comes up, but rather to lead and set a direction for projects concerning our internal infrastructure (including Ocim).
As promised, I have completed a revised proposal to address our scaling issues. This proposal’s suggestions are intense and far-reaching. It will take some time to read and chew through the ideas I’m proposing. It also contains a ‘Quick Wins’ section that we could begin almost immediately, irrespective of the larger discussion, which may take some time to complete. (Please let me know your thoughts on this so we can move forward on those, first of all.)
I expect there to be questions. Let’s dig into them!
@Fox thanks for bringing this forward. It was a pleasure reading.
First of all, I want to reply to this thread mentioning my proposition wasn’t to propose current actions right now. My initial proposition was to bring forth new thoughts in addressing the problem we’re mainly suffering from at the moment, the burn-out.
The burn-out’s effects are showcased wonderfully in Fox’s proposal.
Accordingly, things such as the documentation cell and the devops cell or the recruitment cell were just examples of the idea of departmentalization, to an extent.
My goal wasn’t to achieve change from this single proposal, but to introduce the new ideas, for them to be discussed and argued (with or against). The goal was to throw something and receive the necessary feedback to take those ideas from an unpolished perspective to a more polished one, with your help.
Luckily, although my proposal was misunderstood (my bad), the next step after that was achieved. The next step was to take those argued ideas and present them into something that’s more feasible and accepted, which Fox did wonderfully.
I think the concerns I had and discussed with you in calls, Fox, were addressed in your proposal. You also brought forth ideas which I’ve always wanted to talk about, but didn’t have a good idea on how to do so.
Sure, there’s a lot of feasible suggestions and other far-reaching ones, but just as much sound. The goal of these proposals isn’t to get all the ideas accepted, but at least a good share, in order to start progressing forward again. And even if some of the ideas you raised today seem far-reaching, I have no doubt that they will rise again in the future.
It’s nice to see the main problem receiving more attention.
I look forward to seeing how your ideas will improve the state of things at OpenCraft.
Great to read ideas from Fox Piacenti and review thoughts from the original work of Nizar Mahmoud. Such an interesting and complicated discussion that I’ll just offer a couple of thoughts below that are limited to business development and don’t address the capacity planning or the many other issues raised.
I have observed that burnout results as much from the quality of the work being performed (too many unfulfilling, repetitive tasks) as the quantity (being busy is great if we’re doing fun work). I think two of the important functions of business development that are essential for OpenCraft to scale sustainably and maybe even minimize burnout include:
Prospecting and pursuing clients that fit an ideal profile (organizations oriented to open source, committed to Open edX innovation, etc.) and whom we are likely to be able to help
Qualifying and selecting projects, through sales discovery, that are likely to be both rewarding for OpenCraft, fun to work on and have a high likelihood of closing
If performed properly, these business development tasks reduce burnout by ensuring we’re working with the right clients and on the most attractive projects.
Regarding distributing bizdev across the cells, I’m not sure developers should assume these tasks. First, is it really their core competency? Second, wouldn’t it seem like piling on another meta task with more context switching, etc.?
I believe a dedicated bizdev team is better able to broadly consider the portfolio of current clients and projects in the pipeline to assess which new prospects represent the best possible additions, both in terms of diversification of the client base as well as the availability of sufficient resources. Distributing business development responsibilities across the cells may result in the loss of this broader perspective needed to shape the future client base and avoid suboptimal projects.
As mentioned, I don’t know the answer to the capacity planning question, whether it’s some central clearing function or having it distributed among the cells. However, I don’t think decentralizing business development is the best solution to this challenge.
I’ve had a couple of people raise this question now, which tells me I’m not communicating my idea as effectively as I’d like. My goal is not necessarily to create new roles for cells to take on at this time. In fact, I think that would be a bad idea, especially for sales and administrative work. The example given was there to show how sales might have evolved if it had originated from the cell.
The primary thing I’m trying to illustrate is that roles designed by the cell for the cell’s needs will work better than those created by management independently. If we want to design a role we don’t ever want to hand to the cell, we’ll still want to imagine what it would look like if the role originated within the cell and use that to inform our design of the role.
To your point about qualifying leads-- I’ve placed a comment in the document I’d like your thoughts on. I’d like you to weigh in there and see what we can do re: Keeping the inflowing work stable and consistent.
My thoughts here are copied from @Fox document to keep them in one place. With respect to the ideas from @braden@opencraft.com on Jan 28, I like his notion of building in flexibility by dividing an individual’s efforts into “active” projects (with clients) and “ongoing” (alternative name could be “building the franchise” as it’s work on the work). However, I wouldn’t lose the formal processes of planning, forecasting and estimation. Yes, the purchase cycle is lengthy, but that doesn’t mean that, over time, we can become more accurate in predicting demand for development. It just takes focus, but over time it is possible to assign probabilities to close as well as deal close dates that are close to reflecting what will actually happen. I also think these predictions are a specialized skill that is based on the observations and data an organization collects over time. For this reason it is a function that should be centralized and one that ideally rests with bizdev/sales.
So I like @braden@opencraft.com 's idea of built-in flexibility but combined with the rigor of some basic predictive analytics. Interested in your thoughts on this Fox and those of @gabriel@opencraft.com and @tim@opencraft.com.
@Fox Thanks for adding your ideas and proposals. There are several proposals, from documentation, to cells, to salaries, to sales. Maybe they could be discussed in separate places?, or maybe we could try the approach of opening MRs to the handbook.
I feel that every person has an opinion about the issues and the Google Docs right side is too small to hold large discussions. We could use the forum and separate forum threads for each topic.
To advance towards conclusions, each forum thread can have a corresponding MR.
I left some comments. From my side, I’m more interested in the style of how we fix things. The problems described in the document should be fixed but I don’t mind how they’re fixed as long as they’re fixed. I guess that not everyone wants to be involved in all discussions about processes, but they all prefer processes that work well.
My main OpenCraft wish is to have someone fixing the problems that people find. That person (or a group of people) should have power to try changes, and time, and budget, and will probably have to deal with burnout without burning out themselves.
The developer advocate role is a good way of doing of helping people if it’s implemented as originally intended , and if it’s kept that way.
In the document you also mention that we should go back to doing sprint retrospectives, and commenting about the problems experienced each sprint (like we did in the biweekly checkup form / stress report). These processes must work, but in the past they stopped working for several reasons, so a next step is to research why and to fix them (maybe by trying new ways of doing them).
This is all metawork because we’re trying to fix problems in the problem-fixing process in OpenCraft.
to this suggestion. The background/context information in @Fox 's doc provides a great perspective, but I think we’re getting caught up in discussing that, without addressing the concrete suggestions he’s made. And they’re really good! Some MRs to enact them would help move this along.
We’re a company that specializes not only in platform development but also in hosting, and yet, we haven’t found a way to consistently treat DevOps work as a first-class citizen.
, and if it’s kept that way.
It ain’t easy, that’s for sure!
to this suggestion. The background/context information in