How to Evaluate Project Managers
Taylor Campbell — Mar 1, 2019 (updated March 3)
In which Taylor argues that project success is an incomplete standard for evaluating PMs, and we need to better understand the job so we can define better standards.
Audience: Software project managers and their bosses.
Photo by rawpixel on Unsplash
Software projects are hard
It feels like managing software projects should be easier. We’re rarely solving new problems, but the failure rate — missed deadlines, buggy software, unhappy clients — is close to half.
The problem, as with most things, is the humans.
Each client has a different vision for their project, each team has a different set of skills. There’s a million combinations of expectations and skillsets and personalities and misfortunes that can align to push your project off-track. Developers are bad at estimates. Clients are bad at knowing what they want. Simple tasks are bad at being simple.
Given this, knowing how to evaluate project managers is hard. If you’re a PM, how should you evaluate yourself? If you manage PMs, how do you know who’s doing a good job, and who’s just scheduling meetings and avoiding responsibility?
The obvious answer is to look at outcomes: If the project succeeds, the PM did a good job. If it fails, they did not.
Project success is a flawed metric
The problem is, project success is a bad metric. Remember the humans I mentioned? Building software is complex — no one can guarantee project success, and no one can mitigate every risk.
So judging PMs solely on results is unfair and unproductive. Results are obviously relevant, but you can’t judge competency off of one project outcome. At the same time, waiting for a trend of failures will cost you dearly.
We need better metrics. If you only evaluate PM effectiveness at the end of the project, you’re doing it wrong.
So how can we evaluate project managers?
Instead of waiting for the end and assigning a pass-fail, we need to define objective standards of measurement. But to do this, we need to know what we’re measuring — we need to know the job of a PM.
Note to bosses:
- This is the hard part: If you think your PM’s job is to “make your projects succeed,” you can’t properly evaluate them, and you’re incentivizing them to hide problems and avoid responsibility. This guarantees your projects will fail.
What should project managers do?
As PM, your job is information synthesis and communication.
You can wrangle a dozen spreadsheets, break down requirements into 15-minute increments, hold hours of meetings every day, but if you aren’t spending deliberate time synthesizing those messy sources of information, and communicating them to all parties, your project will fail.
Project management is hard. It’s a lot of work to track and organize information. It’s not all fast decisions and scheduling meetings – you have to spend time thinking about the project.
- If you feel you can “manage” multiple projects, step back and make sure you’re not just a meeting-scheduler and note-taker. Those are necessary, but aren’t the valuable part of the job. The max number may be more than one, but be aware and get help if you don’t have time to think about each project.
Information synthesis – how do I evaluate that?
Concretely, as PM, you should be able to answer these questions at any point:
- What are the project deliverables?
- When are they due?
- How long will they take (roughly)?
- How much time has been spent so far?
- Are they on track?
- Do you know what needs to be done, when, and by whom?
- What could go wrong? How likely is it?
- Which team members are critical?
- Which are risks to the project?
You should not need to schedule a meeting to get these answers.
If the answers above live only in your head, your project will fail. Your job is communication, remember? Your team, boss, and client should have honest, regular updates on progress and risks.
Send written updates at least once per sprint, if not every week. This forces you to think about the answers, which is critical. If it takes too long to gather the information, fix that problem, don’t stop sending updates.
Your goal is to avoid surprises. If something goes wrong, and it wasn’t included as a risk in the previous update, go back and think about how it was missed. What can you do so it’s not missed again?
How can the PM know all of this?
There’s no magic spell that will give you schedule and progress. It takes good planning and hard work – good requirements, disciplined estimation, regular organization and prioritization.
Tools aren’t a silver bullet – you can do this with sticky notes on a wall – but if you use one, spend time learning how to use it effectively so it doesn’t cause friction for your team. Jira doesn’t automatically solve your organization problems.
Don’t rely on your team for point-in-time updates on project status
If you ask, “How are things going? Any blockers?” at every standup, you’ll get individual answers (“my tasks are going well”). These are useful and necessary, but individual statuses do not sum up to the project status.
You’re the funnel for all project information. Blindly passing on what’s said is not useful – you need to also consider what’s left unsaid, and synthesize it all into a useful snapshot of status.
What about risks?
It’s impossible to foresee every issue. Some risks are universal to every project – do I have the right team? Is this timeline possible? Others are project-specific – “this client sure changes their mind a lot…”
You should certainly try to mitigate all the obvious risks, and regularly ask your team for risks they see. They’re closest to the work, so they’ll often sense underlying issues before they become apparent.
- Even if some risks are just remote possibilities, don’t let them fall out of mind — include them in every status report.
In the worst case, the project will seem to be progressing well, then everything will blow up at the end. Those last backlog items hid a large chunk of uncaptured requirements. The client wasn’t paying attention at demos, and just assumed you’d built his favorite feature differently.
Your team often won’t be surprised at these sorts of issues. But it requires a lot of team trust to get honest communication about vague forebodings. You need to make it clear that you welcome these fears, and then not punish the person who raises their hand by demanding solutions (“Well, Bob, I know you don’t ‘feel’ like we’re going to hit our deadline, but everyone else is on track, so maybe it’s just you.”).
Even if their misgivings turn out to be wrong, they still point to communication lapses. Remember those weekly updates you’re sending? Everyone should know how much work is remaining, and how much progress is being made across the whole project, not just their sphere.
Vague, individual misgivings probably aren’t actionable. To get useful data, survey your team regularly on their confidence in the project, and track the response trends over time.
Tracking the whole team over time smooths out any ‘bad week’ dips, and gives you a trendline with actionable data.
Sales pitch: Anonymous team polling as-a-service
Save time managing team surveys! We built ProjectPoll to help you find and fix these problems early, so your project ships on time, on budget, with happy clients.
Project management is hard. But if you want to be a better PM, you can do it! It’s is a skill, like estimation, or requirements analysis, or development. Learn from your mistakes. Make each project better than the last. When in doubt, over-communicate. Be completely, transparently honest with yourself and your team. If you’re too over-extended to think deliberately about your projects, find help.
It’s easy to think that a PM’s job is to “deliver projects successfully,” but that’s an incomplete metric. Use the questions above to better evaluate your PMs (or yourself). Your goal is no surprises – sometimes projects fail, but they shouldn’t fail without warning.
Thanks for reading!
What questions would you add to the list? I’d love to hear from you! Send me an email – firstname.lastname@example.org.
And if you want to improve your team communication, check out ProjectPoll. We’d love to help your projects succeed (with no surprises).