Introduction:
A product engineering evaluation checklist operationalises the selection criteria that consistently predict delivery success — creating a documented basis for the decision that can be reviewed and learned from. This checklist is designed for significant engagements: MVP builds, platform modernizations, or ongoing product development programmes. Adapt it to your specific context.
Pre-Evaluation Preparation
Preparation Item | Status | Notes |
Product or system scope defined — what is being built or modernized | ☐ | Vague scopes produce incomparable proposals |
Technology stack requirements identified | ☐ | Match specific stack expertise to requirement |
Compliance requirements documented (HIPAA, PCI-DSS, SOC 2, etc.) | ☐ | Compliance expertise is not universal; verify specifically |
Timeline and milestone requirements defined | ☐ | Firms with different delivery models have different profiles |
Budget range established | ☐ | Helps shortlist by model; prevents wasted evaluation cycles |
Internal management bandwidth assessed | ☐ | Determines which pricing models are appropriate |
Success metrics defined | ☐ | Share with candidates; their response reveals outcome orientation |
Capability Assessment
Checklist Item | Status | Evidence Required |
Portfolio includes comparable engagements (same domain, technology, scale) | ☐ | Case studies with named clients and measurable outcomes |
Direct reference contacts available from comparable engagements | ☐ | Named contacts who will take calls, not written testimonials |
Full-spectrum capability — discovery, design, engineering, QA, DevOps | ☐ | Which disciplines are employed internally vs. subcontracted? |
Domain-specific compliance expertise verified | ☐ | Specific implementations, not general claims |
AI engineering capability assessed — architecture, governance, experience | ☐ | Describes AI-native products built; has written AI governance policy |
Glassdoor/LinkedIn reviewed — employee tenure, engineering leadership depth | ☐ | High attrition is a delivery risk signal |
Discovery and Planning
Checklist Item | Status | Evidence Required |
Assessment conducted before proposal submitted | ☐ | Any proposal without assessment is a guess |
Discovery methodology described with specific outputs | ☐ | Validated problem statement, prototype, scoped requirements |
Can describe a discovery producing "do not build" recommendation | ☐ | Reveals genuine discovery practice vs. confirmation bias |
Technical assessment includes codebase review (for modernization) | ☐ | Architecture forensics required before modernization proposals |
Requirement specification standard documented | ☐ | "Stories must meet this standard before sprint planning" |
Estimation methodology explained | ☐ | Specific approach beats "we use story points" |
Engineering Standards
Checklist Item | Status | Evidence Required |
Automated test coverage requirement stated (target %) | ☐ | 70%+ enforced by CI pipeline, not a guideline |
Code review process documented — who reviews, what criteria | ☐ | Every PR reviewed by senior engineer; specific criteria |
CI/CD pipeline described — what runs on every commit | ☐ | Unit tests, integration tests, lint, security scan, accessibility check |
Definition of Done documented and viewable | ☐ | Written document; consistent across team |
Technical debt management practice described | ☐ | Formal register; sprint allocation; escalation threshold |
AI coding tool governance policy provided | ☐ | Which tools permitted; client code data protection |
Security embedded in development — not final-stage audit only | ☐ | OWASP checks in CI; dependency scanning; SAST in pipeline |
Delivery and Governance
Checklist Item | Status | Evidence Required |
Sprint review format described — client sees working software | ☐ | Not presentations; not screenshots; live demo |
Business outcome metrics agreed before development begins | ☐ | Specific, measurable; not just delivery outputs |
Escalation path for delivery concerns defined | ☐ | Named contacts; specific process; response timeline |
Change management process described | ☐ | Written change request; client approval; transparent pricing |
Communication rhythm documented | ☐ | Specific schedule; not "we communicate proactively" |
Project transparency — real-time backlog and sprint status access | ☐ | Client access to tools; not mediated through PM |
IP, Security, and AI Governance
Checklist Item | Status | Evidence Required |
IP assignment to client from day one — in contract | ☐ | Unambiguous language; no work-for-hire ambiguity |
Repository access from first sprint — in contract | ☐ | Client owns repo; partner works in client-owned repository |
Data protection agreement covers all subcontractors and AI tools | ☐ | Explicit; not assumed from general NDA |
AI coding tool policy — tools named; data protection confirmed | ☐ | Written policy; not verbal assurance |
Security certifications (ISO 27001, SOC 2 Type II, or equivalent) | ☐ | Current certificates; not expired |
Post-Delivery and Support
Checklist Item | Status | Evidence Required |
Post-delivery support model defined in contract | ☐ | Duration; SLA; what included; what charged extra |
SLA for critical production issues stated | ☐ | Specific response and resolution times; not "best efforts" |
Knowledge transfer plan described | ☐ | ADRs, runbooks, handover sessions; not just final document |
Post-programme iteration roadmap discussed | ☐ | Reveals relationship vs. project-close orientation |
Reference Verification
Checklist Item | Status | Notes |
Three direct reference contacts from comparable engagements | ☐ | Same domain, technology, and scale as your engagement |
Reference calls completed — not email Q&A | ☐ | Live calls produce information written Q&A does not |
"How did they handle scope changes or unexpected complexity?" asked | ☐ | Most revealing; look for specific stories not generalities |
"Did they surface problems early or late?" asked | ☐ | Reveals communication culture under pressure |
"How smooth was knowledge transfer?" asked | ☐ | Reveals post-delivery quality |
"Would you engage them again?" asked | ☐ | Binary and revealing |
Scoring Framework
Section | Suggested Weight | Rationale |
Engineering Standards | 25% | Most predictive of delivery quality |
Delivery and Governance | 20% | Determines whether problems surface early or late |
Discovery and Planning | 15% | Determines whether the right thing gets built |
IP, Security, and AI Governance | 15% | Risk management; non-negotiable minimums in regulated industries |
Reference Verification | 15% | Most reliable signal of actual delivery behaviour |
Capability Assessment | 5% | Necessary but least predictive; can be presented attractively |
Post-Delivery Support | 5% | Underweighted in most evaluations; significant long-term impact |
Codesis Technologies welcomes evaluation against this checklist and can provide documented evidence for each item:
