Skip to content
← Field notes
Rubric deep-dive · Pillar 01

Rubric deep-dive: craft and judgment (Pillar 01).

25% of the composite. Two senior practitioners read your real shipped work against a calibrated reference set. Not tool familiarity. Judgment. What we look for and how to prepare.

By · Guest expertAgencies6 min read

Craft and judgment is the pillar most agencies expect to win and most agencies underperform on. Twenty-five percent of the composite, judged by two senior practitioners from the discipline, reading the team's real shipped work against a set of reference examples the panel has scored before.

02

What we are not testing

We are not testing tool familiarity. Not whether the team uses the current trend framework, the current stack, the current design system methodology, or the current PM methodology. Trend fluency is a mile wide and an inch deep. A team can pass a trend check while producing brittle work, and fail one while producing durable work.

We are testing judgment. Judgment is what you did when you had to choose between two decent options, one easier and one correct. Judgment shows up in the parts of the work that are not obvious. The seams between components, the acceptance criteria that got rejected, the request that was declined, the assumption that was surfaced and disputed. That is what we read for.

03

How the review actually happens

Two senior practitioners read two shipped exemplars in the team's own medium. Not case studies, not slides, not screenshots. The actual artifact. Production repositories for software. A campaign with brief, creative, and 90-day reporting for marketing. Deliverables with source files for design. Decision memos or strategy artifacts for product. Access is read-only. Time-boxed to two hours per reviewer.

Each reviewer scores five criteria. Output quality and idiom (30%). Approach and tradeoffs (25%). Verification and QA (15%). Peer review culture (15%). A live exercise for borderline cases (15%). If both reviewers agree, the score is entered. If they disagree by more than one level, a third reviewer joins. All three read the same artifacts and the final score is a considered average.

04

What pushes an agency from meets-bar to strong

The gap between level-2 (meets bar) and level-3 (strong) is usually not visible in the work itself. It is visible in the walk-through. A level-2 team can show you what they did. A level-3 team can show you what they didn't do, and why. They have opinions about the alternates they considered and rejected. They can name the failure modes they were designing against, and point to the parts of the work that address them.

That kind of walk-through only comes from a team that has argued about the work internally, in writing, before showing it externally. If we ask a lead engineer or lead designer or lead PM to defend a choice and their first instinct is to explain how it is "industry best practice," the score drops. If their first instinct is to name the counter-option and describe why it lost, the score rises.

05

The red flags that block issuance

Any criterion scored 0 blocks the badge. Level 0 in craft and judgment usually means one of three things. The same scaffold copy-pasted across engagements with no adaptation, visible within thirty minutes of reading two projects side by side. Assumptions and inputs left implicit where a peer would document them, showing up as "I remember why we did X" but only in the head of the person who made the decision. Style and conventions changing within a single deliverable, with naming schemes that shift mid-project and different formatting for the same kind of file. None of these are catastrophic alone. All three together, in a single project, are.

06

How to prepare

The best preparation is to pick your two exemplars honestly. Not your prettiest work. Your most defensible work. Projects where you have written artifacts of the decisions you made, where the peer reviews are on file, where the QA record shows a real check that caught a real problem. If you pick your prettiest work and cannot defend the decisions, you will score lower than if you had picked a less flashy project you actually remember arguing about.

The second preparation is to walk through the exemplars with someone from outside your team before the review. Not to rehearse. To find out what parts of the work you cannot defend cleanly. Those are the parts to prepare for.

Takeaways
  • 01The pillar tests judgment, not tool fluency. Teams that argue about their choices score higher than teams that follow current best practice.
  • 02The read is on real artifacts, not slides. Read-only access to production work, two hours per reviewer.
  • 03The gap between meets-bar and strong is visible in the walk-through. Teams that can name their rejected alternates score higher.
  • 04Pick your most defensible exemplar, not your prettiest.
  • 05Copy-paste scaffolding, implicit assumptions, and in-project style drift each block issuance if consistent.
Related
Score yourself

Where does your shop sit on this pillar?

The self-assessment mirrors the pillar weights and gives you a private draft score before you commit to the full intake.