Feedback: following processes

Discussions - Public
1

Hi everyone, I just wanted to post some observations and thoughts on the processes after rejoining OpenCraft. We’re a very unique company, and our processes are important for ensuring everything keeps ticking over efficiently. Perhaps it’s been a slow change over time here, but after being away for three years, I’ve noticed a big difference in how our processes are being followed - often things are only partially followed, late or missed altogether.

I don’t want to point fingers or call anyone out, although I do need to share some examples. I simply would like to draw attention to some things and start a discussion.

I’ll start with some observations:

Participation in estimation sessions

I can only view recent estimation sessions for Bebop, but here the average session participation over the last 5 sprints (357 - 361 inclusive): is only 2.8 participants. This is less than 30% of the team on average.

Worklogs

We’ve been having a recurring issue with worklogs not being uploaded in a timely manner. See https://forum.opencraft.com/t/sprint-updates-bebop/691/2555 where it was reported by Tim, and there’s been some ongoing discussion.

Jira ticket descriptions

We have a very clear standard for ticket descriptions, but very often I’ve encountered tickets without a description, or with a partially written description. Some examples:

  • BB-10064 (partially finished template)
  • BB-9951 (no description)
  • BB-9946 (partially finished template)
  • BB-9916 (no description; description only added later upon request)
  • BB-9902 (unclear description, does not follow the template)
  • BB-9990 (has a description now, but didn’t in the thursday before the sprint)

Pull request descriptions

Likewise here we have a template and high standards for descriptions on PRs, but I’ve encountered several with partial or missing descriptions. This makes it hard, especially when there’s not even a link to the relevant ticket. Some examples:

End of sprint updates

End of sprint updates are due on the last Monday of the sprint, and there’s a checklist item on the first Tuesday of the next sprint to catch up on sprint updates. However, these updates are regularly days late (so I’m catching up on sprint updates all week) or not at all.

Some brief examples:

  • BB.360: 7/10 updates, with 3 definitely late (40% members with timely update)
  • BB.359: 8/10 updates, with 3 definitely late (50% members with timely update)
  • BB.358, 10/10 updates, with 5 definitely late (50% members with timely update)

Similar story for mid sprint updates.

Sprint checklists

In Bebop at the end of last sprint, 4 members were pinged a reminder in mattermost about late filling in the checklist, and the previous sprint 5 were pinged.

Responding to pings

I’m not going to share any examples, but I’ve noticed several times where pings have been ignored. :(

Retrospective process

I found this really confusing, because the handbook said one thing, the sprint checklist said another, and the calendar indicated yet something else, and in reality each cell did something else altogether. See BB-9943 for more details.

General thoughts

This surprises me for things like the estimation session participation and filling in the checklist, because we now have a dedicated tool for tracking checklists - everyone has a personalised checklist every sprint with reminders.

I know it may be easy to neglect filling in a full ticket description, posting an update, etc., especially if it’s probably only going to be you working on a ticket, or it’s a small task, or there isn’t much to update, etc. But in our async remote work environment, coupled with our self management structure, clear written communication is essential. Otherwise information is lost, context is forgotten, and we start gathering many minor inconveniences - eg. needing to spend time finding a ticket that wasn’t linked, or asking why a PR wasn’t merged, or checking the status of a ticket.

We have a lot of flexibility to enjoy at OpenCraft; let’s not push that flexibility too far and get lax with processes that are important.

Thoughts on addressing issues

Just some quick thoughts:

  • If some things are habitually not happening and there doesn’t seem to be any consequences, perhaps those processes need to be removed or revised? Maybe there are opportunities to be leaner.
  • Is there fatigue among the Sprint Planning Manager and Sprint Manager roles for chasing up things like ticket and PR descriptions, late updates, etc.? (Side note: I’m happy to take on one of these roles if anyone wants to hand them off)
  • Are there other things impeding us - overwork, feeling too busy, feeling like some things aren’t worth it, not enough time? Things to discuss.
  • Do we just need a reminder here, to take responsibility of these things and return from the trend of becoming lax?

Closing note

I don’t want this to be a negative thing. I believe in OpenCraft and the structure we have, and I hope these observations can be helpful - observations from a re-new-joiner, observations of things that may not have been noticed otherwise or have been accepted as normal.

Ticket: BB-10101

7 Likes
2

Thank you for penning this down. I am sorry if you had a bad experience with the processes. I think as a team we can take this feedback and improve the processes, especially for me, I will make sure to have a better ticket description and do my sprint update on time, which I have been struggling for past few sprints.

1 Like
3

Thank you for taking the time to highlight this, Samuel. I share your sentiments on several points from my personal perspective. Your thoughts on addressing the issues are spot on as well! If anyone is experiencing similar observations or frustrations regarding certain aspects of our processes, it’s healthy to discuss them and take follow-up actions.

1 Like
4

@samuel Thank you for your thoughtful post about this :+1: It’s probably a boiling frog type of issue, where over time we sometimes missed a specific thing, but then didn’t correct it - even if it’s small things, over time and years it can accumulate. That’s where it’s useful to get your perspective, you are able to point out the difference compared the last time you were here, a few years ago.

Imho one thing that I have observed is that to ensure something is consistently done, it’s important to have someone actively and consistently checking that the work is done. And also sometimes the availability or interest for this type of role can evolve over time. The fact that you care about this and are able to call it out makes me think you would be good in one of those roles, especially the ones around sprint management - @pooja @paulo what do you think of your sprint manager and sprint planning manager roles, and how is your workload looking? If either (or both) of you is interested in passing it on, it could be a good thing to consider taking on @samuel

Also, because as you have pointed out, some of it might be due to processes or their documentation/checklists not being updated to reflect the current reality. It could make sense to do a small epic to iterate there - with time scheduled in everyone’s sprint to review your findings above and make suggestions, and tasks to update the handbook accordingly. @samuel Would you be interested in taking this on?

1 Like
5

The sprint planning role has no specific process related to the ones listed by @samuel, but given that estimates are essential for planning, I have been reminding the cell members in Mattermost. Ref

On missing pings, not sure I’m included here since there were no examples, but I just found this one from last night, and it was outside my working hours.

Lastly, I’m good with Sprint Planning and the workload, but keep in mind the role is always available; anyone in the cell is welcome to take it on at any time. BB-3149

1 Like
6

One thing that comes to mind with this is the section in our ticket description template:

h3. Review timeline

* PR to be sent for review by <date>
* First PR review to be completed by <date>
* _[Optional]_ Draft/WIP PR sent for review by <date>

I don’t think I’ve ever seen this used. Even if it was used, it’s not in a format likely to be useful. How would someone remember? If we were expecting team members to manually create calendar entries to remind themselves of upcoming PRs to review… we were expecting this to be ignored :slight_smile: No one’s going to do that. I’m not even confident we can reliably set these dates as assignees.

In fact, I usually see team members do what they SHOULD do here-- which is ping their reviewer ahead of time when they have a very large thing coming up for review and want to make them aware that it’s getting close to done and they can take an initial look if they want to avoid having to do a full review from scratch so close to the sprint boundary. That’s really what this is intended to avoid, near as I can tell-- people suddenly being overwhelmed with reviews at the last minute, leading to spillover.

I would also like this clarified. I’m not sure we’re even doing something that I would call a retrospective process at current. We used to have a recurring meeting that you were expected to join if you were working during that time, but I hear that’s been abandoned. In Deathstar we have never really had a retrospective process. However, many of these issues with process that Sam is bringing up now are the kind of thing a retrospective is supposed to flag much earlier.

I think we really need to figure out a way to make retrospectives work. They aren’t merely a chechklist item-- they’re process maintenance.

I’m not sure what the best option is. It’s possible we could start collecting more retrospective info in our checklists. Right now we just collect general sentiment. We don’t collect things like ‘I am irritated with having to complete x process when no one seems to use it’ or ‘I feel like my pings aren’t getting answered on time’ or ‘Our tool for x is error prone and I keep losing hours trying to figure out how to tweak its myriad configuration options.’

And even if we did we don’t have a process for reviewing these items and scheduling follow-up tasks to fix them. So maybe that’s what we should do-- start collecting issues in our sprint checklists and delegate someone who will create tickets to either start forum discussions or resolve issues or whatever. They don’t have to be the assignee of said tickets-- in fact they may even preferentially assign them to the person who is noticing the issue. But the nice thing about having a ticket is that now there’s budgeted time to figure out the issue and so the problem isn’t so overwhelming.

I’m pretty sure there is intended budget for continuing to evolve processes, or there should be. If there’s not an epic for tracking/consuming that budget, there should be and someone can create tickets consuming that budget to fix issues identified in retrospectives.

7

Thanks for bringing this up @samuel . A few of these points have come up in the past as well, so it’s helpful to revisit them. I think there are a few factors at play here:

  1. Many of the concerns mentioned fall outside the sprint managers’ responsibilities. For instance, ticket descriptions are owned by the epic owners or whoever creates them, so there isn’t a mechanism enforcing template compliance. The same applies to PR descriptions - the person opening the PR is expected to follow the template, and while we could check during code reviews, it’s not systematically enforced otherwise. The same goes for time logs.

  2. While we do have a clear checklist, there’s currently no reminder or notification system for certain tasks. For example, it’s easy to miss when sprint estimation is due unless you check manually. We used to get email notifications when estimation sessions were created, but now we only get notified when they close.

  3. Sprint retrospectives have been a recurring challenge. When we moved to a fully async setup, participation dropped - partly due to timezone mismatches, and over time, others lost interest too. Falcon’s approach of posting retrospectives in sprint updates is a good alternative, but it hasn’t been formalized across teams yet.

I do try to post reminders for sprint updates, but forum pings often get missed or ignored.

If we start enforcing every small process step, we risk turning this into a more traditional, top-down management structure - which isn’t what we want for OpenCraft. While I definitely think we can improve our processes around some of these issues, I’d like to think most of them just need a reminder from time to time to keep us on track.

8

I don’t think this is about any specific person, more a general shift in the team. I have certainly noticed more pings that get unanswered within the team in recent years. I often bring it up directly in those cases, but it looks like some of the drift is beyond the individual corrections, and a tendency at the team level.

That could be a good candidate for a rule update there :+1: Same thing for the retrospective process - including it with the sprint updates seems to be a better approach than the meeting, and makes more sense given that it’s async. It will need some processes and handbook updates, which can be part of the process improvements epic.

I think it would be helpful if @samuel took on one or several of the sprint & epic management related roles, if he is still willing to. At least for some time while attempting to fix the processes and the handbook, it would allow to have to deal with the issues and people involved first hand, and experiment more easily with testing changes in how we deal with lapses?

Actually, since you have been dealing with these topics directly too, maybe it would make sense to work on the process improvements epic the three of you @paulo @pooja @samuel ? Would you have the interest and the availability?

The cell management budget is also there for this type of things. And if there is a larger update needed we can always allocate a dedicated budget - like here for the issues pointed out by @samuel

So maybe the epic planning manager also needs to be involved? @tikr what do you think?

Sounds like another good improvement we could make as part of the above-mentioned epic. :+1:

We don’t want to mindlessly enforce stupid rules for sure, which is why part of the work here would be to update the rules to make them fit reality. But part of what gives us our freedom (async, remote, etc.) is to have a few basic rules that we follow consistently. If this erodes, it becomes much harder to keep the same level of reliability in our work.

4 Likes
9

Thanks for your responses everyone! This feels like a really productive discussion.

I’m quite happy to take on any of these roles, and would definitely be interested to take on an epic to address these things. I’d also be very happy to work with @paulo and @pooja on the epic if they’re interested. :slight_smile:

No no that case was fine, and you responded quickly! I’m talking about cases where there’s been a request directly to someone, but several working days go past without response.

Agreed, this. ^

Aha agreed, this was a blind spot for me actually - I remember that being in the template when I was at OpenCraft last time, and it was very rarely used then too. So I’ve just tuned it out…

I believe here with our flat structure, we all have both the opportunity and responsibility to remind each other when we see things like ticket descriptions not clear, etc. For example, one of the useful aspects of the ticket estimation sessions are that everyone on the team has an opportunity to read through the ticket and request clarification or improved descriptions, regardless of their role or whether they will work on the ticket.

This all feels like a great opportunity to update our processes and improve how we approach the processes - so that reality and the handbook converge on something we’re all happy with. :slight_smile:

5 Likes
10

I’m not sure that the issues Pooja referenced would be best solved in the context of epic planning (cf. below). But thinking about it more, a couple aspects that could be worth addressing/revisiting in that area would be posting epic updates on time and, to a lesser degree, making sure they contain the right level of detail.

Moving the deadline for posting epics to Wed W1 and updating the epic update template already helped with these aspects, but maybe there’s a way to improve further (without introducing more manual checks – @farhaan and I already do a couple of those each sprint and post sync reminders on Mattermost)?

So yes, for a portion of the work it would make sense to involve the epic planning manager :+1:

–

Regarding the issues that Pooja referenced:

  • Template compliance: This seems most closely related to sprint planning. If tickets don’t have proper descriptions, they’re hard or even impossible to estimate. This affects people’s ability to properly plan for the sprint following the current one. So it would make sense for the sprint planning manager to check the tickets in the upcoming sprint and flag the ones that don’t have a proper description. (IIRC, something like this used to be part of the set of responsibilities of the sprint planning manager role, but it sounds like it got dropped?)
  • PR descriptions: Aside from upstream reviewers, the people most directly affected by poor PR descriptions are internal reviewers; and they’re also usually first in line to notice when a PR description doesn’t follow the template. If we could find a way to catch this issue more reliably at that level, I think that would be preferable. If both ticket assignee and reviewer take ownership of their responsibilities when it comes to creating high-quality PRs, that’s two levels of checking already, and it shouldn’t be necessary to introduce yet another process/mechanism for managing that aspect.
  • Time logs: While missing work logs mainly affect epic owners trying to review budget status and adjust sprint scope based on it, I’d be curious to see if we could find an efficient way to address this at team level. Starting with a conversation about what’s making it so difficult for people to submit their work logs right away/once per day would probably be helpful here.
3 Likes
11

Sure thing, feel free to take BB-3149 @samuel

I wasn’t aware of an epic to improve processes; that seems like a great idea. I share the ideas Pooja mentioned on how adding processes could lead to a rigid structure. Having an epic to actually look at all the processes and evaluate how they interact and how to improve that is a good thing.

As we add more and more processes, metawork starts to compete for time and budget with the actual work on tickets. And with more processes and more people to follow them, even if the ratio of missed processes remains constant, the amount of missed processes increases and starts showing up everywhere.

Now, on some specific processes, the estimation session is never fully attended, but it never seemed necessary to be. It might require time that people on hectic sprints don’t have, and even a sample is enough to get good enough estimates. Sometimes it is more important to have a few people with context estimating than everyone. I believe there were two occasions when I had to manually set the story points and ping people where I couldn’t estimate because there weren’t enough votes on all tickets, but given how long I’ve been doing this, it doesn’t worry me.

Ticket and PR descriptions seem mainly a case of a forgotten “let me just create this and update it later”. That isn’t common, and when it blocks me, I just ping the ticket reporter/assignee to update it. Also, there are times when the template does not seem to fit the ticket or PR, especially for very simple PRs.

1 Like
12

Great! So as a next step, @samuel can you create the new epic and schedule a discovery task for it for the upcoming sprint? To establish a list of tasks, assignees and a budget for it. And put as reviewers on the discovery & epic: @paulo @pooja @tikr – and then assign the tasks between the 4 of you?

@samuel for the sprint/epic management roles, which one(s) do you want to take? Since each of these roles comes with its own workload, it’s better if you pick I think @samuel , depending on what you think will help the most for the epic and the goal to correct the issues discussed in this thread. Btw, hanks for being open to passing those on at least for some time @paulo @pooja @tikr

1 Like
13

Thanks @antoviaque :slight_smile:

I’ve created the epic BB-10142, and initial discovery BB-10143. @pooja @paulo @tikr please take a look and assign yourselves as first/second reviewer as you would like. :slight_smile:

I don’t want to step on any toes here, so I don’t mind. I also want to avoid biting off more than I can chew. :sweat_smile: I’m already reviewer for the sprint manager now, and since @paulo is happy, I’m quite happy to take sprint planning manager. Maybe that will work for now?

@paulo the question of reviewer for BB-3149: @maxim is currently reviewer - not sure if he would like to stay reviewer, or if you would like to take over reviewer? What do you think @maxim ?

:+1: I feel like this epic is more about refining current processes and even removing processes that we’re finding less useful, rather than adding new ones.

3 Likes
14

The epic and discovery ticket look good @samuel, thanks!

I took one of the reviewer positions on both of them.

:100:

If we can adjust or get rid of processes that aren’t helpful and create the conditions for everyone to follow the remaining/important practices consistently, the epic will have fulfilled its purpose! :slightly_smiling_face:

3 Likes
15

@samuel Thank you for the epic and the discovery task! :+1: Definitely include me in the reviews, I’ll schedule them in my sprints.

Sounds good to me :+1:

1 Like
16

@samuel I’m ok with giving up this role, I assigned you as a reviewer.

1 Like
17

As an assignee of tickets, I don’t feel I get much value from other people estimating story points on the ticket. The only exception is when they leave a comment with useful context on an issue I don’t have much experience/context on. As a participant of such sessions, I don’t feel like my vote on the story points is has any value for the assignee (at the end of the day they can ignore it and set the value to whatever they want), and my comments are usually ignored. On top of that the process of participating in the session is just annoying:

  • you have a small window to participate
  • many tickets are small tickets that don’t need estimation
  • if a ticket doesn’t have a description and you ask the reporter to add it, most of the time they add it next day, and by that time the session is closed
  • because there are many tickets, you have to change what ticket you’re logging time on every couple minutes

I don’t know how to solve these issues and make this process more useful.

18

As someone who’s often struggled with processes, I have a few thoughts about this.

  1. Regarding Epic updates: I think there was discussion earlier about moving these to Listaflow. We can have epic updates entirely in Listaflow with a checklist that can change in Listaflow and be automatically updated. Each sprint you can simply link (or Crafty can do this) the latest epic update checklist on the epic. One advantage of this is that you you can be assigned this early and can fill it in progressively till the due date.
  2. Estimation Sessions: I agree that there is limited value here. For a project epic we’ve already estimated each task, so the ones who’ve done the discovery should assign the points. The advantage of this is that if a task is sized as 2 points but you’re not confident you can do it in that timeframe, you can leave it to someone who is.
  3. PR Descriptions: I think I often miss linking them because I generally don’t use them personally. I use a bookmarklet that finds the JIRA ticket from the PR, so I’ve never encountered this issue. I will be more careful since most people don’t use this approach.
  4. End of Sprint updates: The mistake I make is to leave these for the end of the day. However, on most most days I have personal stuff I need to handle by evening so they get missed. I think I can definitely do the mid-sprint update at the start of the day, but for the end of the sprint update I just need to get better at managing that time.
19

I feel like the value of estimation sessions is more than setting story point estimates. I think the real value lies in being a formal process of exposing tickets to everyone in the cell. This means more chance for:

  • catching issues with ticket descriptions (ambiguities, undefined scope, etc.)
  • flagging potential blockers, linking to similar issues from the past
  • helpful drive-by comments, tips, ideas, suggestions
  • flagging things that may be much more/less complex than originally estimated
  • Agreed on the small participation window, that does make it harder.
  • Re small tickets: why don’t they need estimation? Isn’t it just easier for small tickets - you can quickly read through, hit 0.5 or 1, and go to the next. And there’s always the chance that a small ticket has something hidden that makes it really a large ticket.
  • Agreed on the ticket description missing problem - that’s indicative of a wider problem in our processes though: tickets should have a clear description before being put in the next sprint and the estimation session. The deadline for putting tickets in the next sprint is day 9 (Wednesday). If we don’t have a clear description then, how can we commit to working on it in the next sprint? (And we can’t use the excuse that the assignee already knows the scope; the point of a clear description is also to have a written record, and so the work can continue if the assignee is unexpectedly unavailable.)
  • On logging time to tickets every couple of minutes: this is explicitly not required according to the handbook.

Maybe we could rethink the estimation session then? If the story points estimate isn’t valuable, perhaps we could replace it with a semi-formal ticket review session. It could be just a self-directed expectation of going through the backlog of all tickets in the upcoming sprint, looking at each ticket briefly with the aim to:

  • identify any tickets you want to work on or review
  • get an overview of what the next sprint looks like
  • flag any potential blockers, over/under estimation, conflicts, etc.
  • give any tips, suggestions, ideas, etc.

This could be over the Thursday to Friday before a new sprint.

20

Thanks @maxim . :slight_smile:

@paulo if you’re happy, I’ll swap the reviewer/assignee on BB-3149, and open a PR to the handbook with the change. :slight_smile:

1 Like