18F Engineering 2016 End of Year Assessment Guide
18F, as a part of GSA, has a mature performance management and recognition system. This includes an end-of-year performance assessment. 18F Engineering’s goals for the end-of-year assessment are to have engineers talk with their facilitator and supervisor about the past year and create actionable plans for career growth in the coming year. These reviews ideally build on a robust, granular, continuous feedback loop between an engineer, their facilitator, and other coworkers.
Assessments are based on an individual’s performance plan. GSA’s Office of Human Resources Management (OHRM) has a set of appraisal materials applicable to all of GSA. These include a good guide on Appraising Objectively and Fairly, and the requirement to evaluate performance on a 5 point scale. Performance plans have what 18F calls “objectives” and OHRM calls “critical elements.” Each “critical element” in the performance plan is rated, and there is also a summary rating. Some ratings can earn performance awards. You can find more detail, including what each rating number means, in the Appraisal Rating System & Performance Awards document.
End-of-year assessments are given to everyone hired into GSA on or prior to May 15, 2016, and are due by November 14, 2016.
In late September, a group of Engineering leadership and engineers met to determine what outcomes we wanted from the end-of-year review process, and what we’d do to achieve those outcomes.
The number one desired outcome was actionable feedback for individuals. In order for a review to be valuable, the person being reviewed needs to finish the process with knowledge of — and, ideally, a plan for — how to grow over the following year. At the end of the review period, every engineer should have a solid understanding of what to do next to continue to improve and deliver better value to their teams.
Individuals deserve fair ratings. These ratings should be calibrated both within Engineering and across 18F. The ratings and feedback should be aligned with the individual’s performance profile (as is required of course), and not biased towards individuals that the raters have interacted with more than others or those whose strengths align the most with their raters’ interests. Positive and constructive feedback should be balanced, and all feedback should be backed by evidence. Ideally the individual agrees with the rating they’re given.
Individuals should be rated not only on their technical abilities and contributions, but on their total contribution to their teams and 18F. This means the reviewers must seek to understand the individual’s role on their teams: for example, were they doing any project management or design work in addition to committing code. The reviewer should seek feedback from the individual’s project leads and more experienced technical staff they’ve worked with. Additionally, if an individual has contributed to other areas of 18F through guild and working group contributions, the reviewer should gather feedback from leaders and stakeholders in that group.
The person conducting the review, usually an Engineering Supervisor, completes these steps.
- Have the engineer complete a self-review.
- Gather feedback from reviewers (key stakeholders including project leads, technical leads, peers identified in step 1, etc)
- Synthesize that feedback into a review.
- Review reviews with the Director.
- Meet with the engineer and their facilitator to deliver and discuss the review.
More details on each step:
1. Conduct self-reviews
Have each individual complete a self-review. This review should ask for:
- Which projects and accomplishments they’ve worked on over the last year, including both major staffed projects and also side-projects, working groups, guilds, etc.
- A self-assessment against each objective in their performance plan.
- Peers (in engineering, or elsewhere) that they’d like us to solicit feedback from (in addition to project leads, tech leads, guild leads, etc., which we’ll talk to by default).
- Any specific areas they’d like feedback on.
- Answers to these six “engineering climate” questions (the same as we asked in the mid-year review plus one new one):
- Do you feel you’re growing as an engineer?
- Do you ever feel like you’re not treated fairly? Please explain.
- Do you feel like you can raise work issues? Do you feel like your issues are actually heard?
- On a scale from 1 to 5, how satisfied are you with your work (1 is “not satisfied”, 5 is “very satisfied”)?
- On a scale from 1 to 5, how motivated are you to do your work (1 is “not motivated”, 5 is “very motivated”)?
- On a scale from 1 to 5, how satisfied are you with the feedback you’ve received throughout this year?
The answers are sensitive, and should only be shared with other supervisors, and director if necessary.
2. Gather feedback from reviewers
Now that you have the self-review, you can collect feedback from reviewers.
First, identify reviewers. The engineer’s facilitator can help with this. They should be the key stakeholders in the work the person has done: project leads, technical leads, working group and guild leads, and peers the person nominated above. You should identify at least 3-5 people, but if there are too many it’s OK to prioritize and pare the list down. Be sure to include people who can speak to recent project work.
Send a request for feedback to each of the reviewers you identified above. This should ask for:
- An assessment against each objective in their performance plan (as for #2, above).
- A request for feedback on the specific areas the individual identified in #4 above.
- A free-form general feedback area.
Please remind reviewers to aim to be as specific as possible — ask for specific examples to back up any feedback they give. Assure reviewers that their responses will be synthesized and anonymized before they’re presented to the person under review (so we’re not going to say “well, so-and-so thinks you stink at Go!”)
If necessary, you can arrange short (15-30 minute) meetings with a reviewer to go over their feedback and gather specific examples. Whether you need to do this will depend on how good and specific written feedback is. Arranging a meeting can be a good method of chasing a response: it takes less time for the reviewer to tell you feedback than for them to write it up.
3. Synthesize feedback and draft a written review
Now you’ve got a bunch of feedback, so the next step is to synthesize it down into a coherent written review.
First, you’ll want to review all that feedback along with the person’s facilitator (if applicable). You’re looking for patterns, commonalities, or feedback that aligns with what you or the facilitator have personally observed. You’re looking to distill down all that feedback into a review for each objective in the person’s performance profile. Include as many specific examples of behavior as possible — good feedback is specific and actionable (for a deeper treatment if this idea, see this article on The Situation-Impact-Feedback Tool).
It’s important to be especially cognizant of unconscious bias here. Remember that people are especially susceptible to assume that underrepresented people in tech — women, people of color, etc. — are less qualified than their white male counterparts. One way to correct for this bias is to make sure that reviews are concrete and specific (as above). Another way is to be cognizant of the kinds of reviews that are especially applied to underrepresented minorities. For example, the word “abrasive” is far more common in reviews of women than of men. Other similar kinds of phrases to watch out for are “tone”, “aggressive”, “judgemental”, “too nice”, and so on. In general, if you’re writing a review that seems like it speaks to the person’s character, you may be headed in the wrong direction.
GSA has guidance on writing objective and fair reviews that’s worth reviewing as you start to write your review.
You’ll need to provide a score (1-5) for each objective area in the person’s performance profile. The performance plan has criteria for each score; these plans are available in CHRIS. You should also review the overall guidance on the rating system scores. You’ll also need to give the engineer a single overall score (1-5), which is calculated from the objective scores.
4. Review your drafts with the Director
Before finalizing the reviews, meet with the Director of Engineering and review your reviews. They’ll check that your reviews are fair, and also compare scores across the whole chapter to check for bias, patterns, and consistency across Engineering.
5. “Ship” the review, and go over it with the engineer
Now that the review is done, you need to file the “paperwork” and deliver the review.
First, create the review in CHRIS. Then export a PDF version and send it to the engineer at least a day in advance of meeting with them to deliver the review.
Then, meet with the engineer (and their facilitator, if applicable) to discuss the review. You’ll discuss the rating and possible performance award if applicable. This is also a good opportunity to deliver feedback that didn’t make it into the written review — especially feedback on areas that the engineer requested way back in step 1. Talk about ways the engineer can address specific feedback (and how we can support them). This can feed into a plan for next year, so take notes to help with follow up and evolution of next year’s plan.
Finally, discuss their answers to the “engineering climate” questions. There may have been details they didn’t want to provide in the survey version.
Lastly, the engineer will need to digitally sign the PDF. You’ll store those PDFs in a Google Drive folder (Talent will send us the link), Talent’s tracking spreadsheet, and CHRIS.
Our intention for this process is that it will be the start of a more regular, granular feedback system. Engineers deserve frequent feedback on their performance, and there have been many organization and leadership changes in Engineering this year that have disrupted regular feedback loops. Our new structure, where engineers will have a more established relationship with their facilitator and supervisor, will allow us to ensure that feedback happens. Providing regular feedback will make the annual review process a simple extension of regular feedback, as it will be more a culmination of the year’s progress.
In November 2016 we conducted a retro of the 2016 Engineering end of year assessment. We asked ourselves how the process we used worked, and what we’d improve. Those in 18F can review that document for details.