We’ve been doing a lot of recruitment lately and we can only expect that to continue. Clients give us big projects and part of our onboarding process involves handing tickets to newcomers who are trying to get their bearings.
Hiring is one of the hardest problems in software, as we all know. The only way to find out if a developer is a good fit for our needs is to have them work on a task. While some work can be done in planning to avoid giving a task to a newcomer that is critical to complete on a deadline, it cannot always be avoided and a procedure must exist for handling the possibility of failure.
Tickets that are assigned to newcomers and which may cause problems if unfinished by the end of the sprint should be closely monitored. I’d like to suggest adding a filter in Jira to highlight newcomer tickets so epic owners can see them at a glance, but how the epic owner tracks which tickets to pay attention to is less important than that they be closely monitored.
As part of this, encourage the developer to push early, and create their WIP PR early. That way you can monitor if reasonable progress is being made. While this does mean you’ll be able to see progress early, do keep in mind that we’d be looking at unfinished code, so try to weigh the quality verses the completeness accordingly.
If it is clear that the newcomer will not be able to complete the assignment in time or will not be able to produce the quality of work needed, post your assessment and reasoning for it to the newcomer’s dev@opencraft.com review thread. While most replies to the thread must wait until the end of a newcomer’s trial, this may be done as soon as the need arises.
@antoviaque will then meet with the newcomer and announce their trial is ending.
Epic owners should then ask for either another newcomer (if available and practical) or find an assignee for the work to get the task back on track.
Move the hours spent on the ticket from the client epic to the newcomer’s onboarding budget. If this is enough to drain their onboarding budget, notify @antoviaque and their contract will be terminated immediately. Otherwise they may be given lower stakes tasks until the trial period is completed. If the number of hours expended exceeds the newcomer’s onboarding budget, put the remaining hours into Learning Failures (perhaps by cloning the ticket, assigning it to the new developer, and changing the old one’s account).
If the Learning Failures account was needed, we didn’t catch the problem early enough. Ping your cell’s sustainability manager for review of the hours added. If the number is too high, we may be forced to take it from the epic budget, which is what we’re trying to avoid.
The most important feature of this procedure is to allow for budgets to be kept safe from these failures. Reviewing code early and often allows us to know when a problem will arise and when to scramble and find alternative arrangements. Moving the hours onto the newcomer’s onboarding budget makes sure that the client is not impacted from a budget standpoint, and that the epic budget isn’t jeopardized.
The last parts about learning failures and taking from the epic budget are absolute last resorts. If we do that, we probably weren’t paying close enough attention. A litmus test for whether this procedure works is how often we get to that point.
There’s one point I’m still a bit fuzzy on, since I’ve not yet been part of reviewing a newcomer: If this class of failure happens, would my suggestion of ‘the newcomer may be given smaller tasks to finish out their onboarding budget’ actually be true or would they be let go then? I’d like some clarification there.
Please let me know what your thoughts are on this procedure. After discussion I will be creating a pull request for the handbook.
I think immediately ending the trial could be too harsh in some cases. It is possible for a newcomer to mess up a ticket in a way that needs such recovery but not in a way that justifies ending their trial.
I think it makes sense to have a review thread as soon as a newcomer joins. So as not to bias everyone perhaps this can go to the mentor instead of to dev, and the mentor can escalate if needed.
I’ve faced this issue a lot in the discussions epic, and I definitely feel something like this would have smoothed some of the early issues.
I will probably have more to say on this but I’m away for 2 weeks, but I agree with this post and any PR for such a change to the handbook.
I also agree that immediately ending the trial is too harsh.
But I go even further: I actually don’t think this policy should be specific to newcomers at all. Even tickets assigned to a core developer can sometimes get into a problematic state where someone else needs to take it over. You can probably think of some ticket that continually spilled over from sprint to sprint to sprint which someone else might have been able to complete sooner, and not just ones assigned to a newcomer.
Sometimes a ticket is way more complex than expected, or is just not a good fit for someone’s skillset, or requires access that a person doesn’t have. Sometimes the assignee just develops some kind of mental block about a particular ticket that prevents them from working on it or making progress or thinking outside of the box as needed. This can happen even with someone who’s generally a very strong developer and team member. So in my opinion that situation doesn’t automatically mean the person isn’t a good fit for the team, just that they’re not a good fit for the ticket at that time. (Of course, if this happens multiple times, then it probably does mean they’re not a good fit for our team.)
We do need to start by keeping a growth mindset in mind (is it possible for the reviewer to apply some guidance / mentoring / stern warnings that will get this on track?) but when that’s not going to help, it’s very important to get someone else to take it over so that it doesn’t jeopardize the schedule / epic / budget / software quality. And then see if there are any lessons to be learner to avoid this in the future.
I was going to write this long explanation but I think I can summarize:
If it is clear that the newcomer will not be able to complete the assignment in time…
Xavier will then meet with the newcomer and announce their trial is ending
Between 2 and 3, there could be a if metric_of_failing_to_deliver >= n then action(), 3 being one of the actions. The action can be discussed as you suggest, on a thread, or maybe voting with Monday (the UI is lovely, right?)
I also want to say that metric_of_failing_to_deliver will be clear much earlier the better the ticket is specified/estimated, unless it’s the assignee’s first task. In that case, why do they have a risky ticket as a first task? (Mine was golden, thank you @jill)
Assuming the ticket is sufficiently specified, then metric_of_failing_to_deliver falls almost entirely upon the assignee’s hands. There would be less uncertainty and that implies smarter actions, as you can put a number on it and see where you stand.
(me being a newcomer is now thinking: “Anything you say may be used against you” )
@kshitij@braden@fox I think there’s some missing context on this specific point. I remember that we discussed that we’d only end the trial if the newcomer exhausts his onboarding budget, which seems reasonable to me.
For other cases, I agree that it’s too harsh to end the trial (specially if it’s because of a single ticket).
I wish we had a process for saying quickly and strongly, “you’re not doing a good enough job on this ticket, so as the epic owner I’m going to stop you there, move your ticket to the ‘Learning Failures’ account and create a new ticket and re-do this from scratch.” Someone needs to be the bad guy.
^ This is the main topic of SE-3409.
I’ve had to do this once or twice, and it doesn’t really feel good to yank the ticket out of someone’s sprint and saying that they were not doing a good enough job there. I’ve asked for @antoviaque’s help when I had to do and was careful to give proper feedback in the write-up.
It ended up being a good thing to do: it unblocked the ticket, recovered the epic budget and removed a stress-generating ticket from the assignee’s sprint.
Having this process in place is really important, specially in cases where the epic/project budget is small, where 5-point tickets can deal huge blows in the budget.
This makes sense.
Mental blocks can happen and sometimes it’s difficult to acknowledge them (specially if the assignee is the one blocked).
This doesn’t only applies to tickets: epics that grow too much, are long running or yield frustrating results can also cause mental blocks and avoidance by the owner. In those cases is also best to reassign - it will lower stress for a core team member and provide a “fresh” start to the epic to be unblocked.
Hmm. I think I was conflating two things here-- the ticket was mainly oriented around the case that a newcomer wasn’t going to work out and might jeopardize an epic. The suggestions from @antoviaque 's quotes within the ticket suggest that the trial would end. However I think that was a statement on one narrow particular case-- it just happened to be the most consequential case-- where a newcomer, out of their depth, sinks the budget of a project and it appears to be based on their aptitude rather than the size/complexity of the requirements being too much for a newcomer (or, in the case of an existing core dev-- much larger/requiring more learning than anticipated).
That said, I would expect core members to communicate when they’re in over their head. I’d expect them to recognize when a tickets is getting out of hand, and to reach out to the epic owner to change plans.
Newcomers might be nervous and not know that they can speak out if they think that the task they’re handed is too large/complex for them to get done in the right amount of time. They might think that they’re showing weakness during their candidacy-- and I think an epic owner monitoring the ticket to verify that it’s on track is more important there. Expecting an epic owner to monitor other core devs as closely as they would a newcomer doesn’t make sense, and would be exhausting.
Perhaps what we should have is a general process for rescuing problem tickets for an epic, and then an addendum in this process for newcomer considerations-- both for the case when the task is just too much, and the case where it doesn’t appear they’re cut out for the role of core dev.
I do think adding the filter for newcomer tickets would be helpful still, since these are the cases where there’s the most uncertainty, and we should be able to rely on core team members to speak up more readily. Not to say Epic owners shouldn’t also pay attention to core dev tickets, but they shouldn’t need to pay as much as newcomer tickets, where the outcome can be more variable.
Some of this responsibility can go to the reviewer on the ticket - reviewers should be core members who know the processes well anyway. They can (and do; I’ve been on both sides of this already) ping the newcomer assignee to ask for status updates, remind them about opening PRs early with wip code, etc.
What is Learning Failures? This is the first I’ve heard it referenced.
Agreed, but the numbered points above don’t really emphasise this. Maybe there needs to be a point between 2 and 3 as @daniel.francis noted that explains the metrics used here. It currently goes from “clear that there will be spillover” to “Xavier will announce their trial is ending”, which seems a pretty harse jump.
I have one suggestion as part of this: A template checklist (milestone based) can be added to the high risk newcomer tickets which could encourage more communication by newcomer on the ticket and would act as a timeline. It would also allow reviewer/epic owner to follow up on specific updates during the sprint and identify risk at early stage.
An example checklist could be:
[X] I have understood the ticket requirements and updated my approach on ticket (by Wednesday Week 1)
[X] I have created a WIP PR and notified the reviewer (by Thrusday Week 1)
[X] I have addressed the review comments and updated my PR for final review (by Wednesday Week 2)
[ ] I have missed a milestone/ I'm facing difficulty and have notify the reviewer/mentor/epic owner.
Yup, I had the same question. I assume this would be a cell non-billed budget, akin to the overflow budget? Imho we should be careful to monitor budgets and catch issues way before they exceed the newcomer onboarding budget, so this should rarely ever be needed.
I’m hesitant to add a new checklist, as we already have a lot of them. It may be necessary to add one, but I’d like to prove the need first by trying other techniques first, such as flagging the tickets and seeing if that’s enough for epic owners and reviewers to be more mindful about tickets that need intervention early.
The Learning Failures account was referred to in Braden’s original statement:
I wish we had a process for saying quickly and strongly, “you’re not doing a good enough job on this ticket, so as the epic owner I’m going to stop you there, move your ticket to the ‘Learning Failures’ account and create a new ticket and re-do this from scratch.” Someone needs to be the bad guy.
I assumed that this account already existed for other purposes, but I’ve just checked, and it does not. I’m good with removing the Learning Failures budget from the process, especially since, as you mentioned, we should be acting early enough to prevent the need for such an account.
Thank you all for your input on this ticket. I’ve created an MR here which outlines a procedure that I feel has more nuance and handles cases where the issue is outside of the assignee’s control more gracefully.
When a ticket gets out of hand due to a miscalculation of complexity (not enough points assigned), unforeseen issues, or scope creep, isn’t there a procedure to split the ticket into smaller chunks that can be estimated and reassigned that will minimize the impact on budgets and commitments to clients?
)