How do I evaluate a software agency's technical capability before hiring?

Why the Standard Vetting Process Falls Short

Most buyers compare agencies on price, team size, and a polished portfolio deck. None of those signals reliably predict technical execution. A deck can be assembled by a sales team; what matters is what the engineers actually ship — and how they handle the messy parts of a build.

A Practical Evaluation Framework

1. Verify live products, not just logos

Ask for two or three products the agency built that are currently live and serving real users. Pull them up yourself. Check performance (Google PageSpeed, Lighthouse), review App Store or Play Store ratings if applicable, and look at uptime monitors if accessible. Agencies with genuine track records welcome this; those without them deflect.

2. Run a paid discovery or scoping engagement

A one- to two-week paid discovery session — where the agency scopes your problem, proposes an architecture, and delivers a written technical plan — is the single best signal of capability. You see how they ask questions, where they spot risks, and whether their engineers can communicate with non-technical stakeholders. If an agency refuses a paid discovery, that's a red flag.

3. Review actual code or architecture artifacts

Request a sanitized code sample, an architecture diagram, or a technical design document from a past project. Look for clean separation of concerns, documentation habits, and evidence of automated testing. For US engagements, also verify the team's familiarity with SOC 2, HIPAA, or relevant compliance frameworks if your product touches sensitive data.

4. Call references — and ask the right questions

Reference calls are only useful if you ask specific questions: What broke during the project, and how did they fix it? Did they hit their milestones? Would you give them a harder problem? Generic questions get generic answers. Aim for at least two US-based client references if you're a US buyer — timezone alignment and communication norms matter.

5. Assess their deployment and security posture

Ask how they handle CI/CD pipelines, environment separation (dev/staging/prod), secrets management, and incident response. Agencies that can't answer these questions fluently are likely to create technical debt you'll pay for later.

Honest Tradeoffs to Keep in Mind

• Large agency vs. boutique: Larger agencies often have more process overhead; boutiques can be faster but depend heavily on a few key people. Know which you're actually getting.
• Offshore vs. onshore: Cost differences are real, but so are communication overhead and timezone friction. The best offshore teams compensate with strong async documentation and overlap hours.
• IP and contract terms: Confirm who owns the code, data models, and third-party integrations at contract signing — not after launch. In the US, this is especially important if you plan to raise funding or sell the company.

Where CodeNicely Fits

CodeNicely is an AI-first product studio that publishes its track record openly — including products like GimBooks (Y Combinator-backed, 5M+ downloads) and Vahak (800K+ trucks onboarded). They offer NDA-first engagements, milestone-based pricing, and full IP ownership transferred to the client — terms worth benchmarking against any agency you evaluate.