How to Check a Dev Partner's References Before You Sign

Published by CodeNicely, the product studio founded by Meghal Agrawal and Ujjwal Agrawal in 2017

For: A CIO or VP-Engineering at a mid-to-large Indian enterprise — manufacturing, BFSI, or logistics — who has shortlisted two or three dev partners for a ₹50–200 lakh engagement and has been offered a reference list by each vendor, but suspects the list was curated to include only clients who agreed in advance to say yes

Every reference on the list a vendor just emailed you has already agreed to say yes. The call will not tell you whether to sign — unless you drop the generic quality-and-deadlines script and ask instead about what happened to the MSA's IP-assignment clause and the source-code escrow twelve months after go-live. Vendors who cut corners there cut them everywhere, and references will admit it if you ask directly. This is the interview prep for that call.

Context first. Gartner's 2024 analysis found 21% of IT outsourcing engagements fail inside 24 months, and 44% of terminated contracts were driven by price-led selection rather than technical fit. For a ₹50–200 lakh engagement, the reference call is the cheapest piece of due diligence you will do and the one most buyers execute worst — because they ask the vendor's questions, not their own.

Before the call: set it up to fail the vendor, not the reference

Three ground rules.

• Ask for five references, not three. Specifically request: one engagement that went to production and is still live, one that was descoped mid-way, one that ended more than two years ago, one in your industry, and one where the vendor replaced the delivery lead mid-project. If the vendor can only supply the first category, that is itself the finding.
• Insist on the client's CTO or engineering head, not the business sponsor. Business sponsors remember whether the vendor was pleasant. Engineering heads remember whether the code was maintainable.
• Schedule 45 minutes, not 20. The useful material surfaces after minute 25, once the reference has exhausted the prepared talking points.

The 16 questions, grouped by what they actually expose

Group 1: IP, escrow, and the clauses that reveal everything else (ask these first)

1. "What does your current MSA say about IP assignment, and did you have to renegotiate it after go-live?"

Why it matters: IP protection is the single most common due diligence gap in IT outsourcing, per vendor due diligence checklists. A vendor who drafts loose assignment language — "work product" without defining pre-existing IP, background IP, or derivative models — is telling you they intend to reuse your code.

Good answer: "Clean assignment on delivery, pre-existing libraries listed in an exhibit, no renegotiation needed."

Red flag: "We had to go back and get specific language added for the ML models they built on our data" — or worse, hesitation.

2. "Did you require source code escrow, and have you tested a release scenario?"

Why it matters: ISACA auditors have documented cases where escrow was contractually required but the deposited code was undocumented and unusable. Escrow on paper is worthless. Escrow with a tested release scenario is a real control.

Good answer: "Yes, deposited quarterly with a third party, we ran a verification exercise in year two and the build reproduced."

Red flag: "It's in the contract but we haven't checked" or "The vendor pushed back on escrow."

3. "If AI or ML components were built, who owns the trained model weights, and can the vendor use them for other clients?"

Why it matters: Deloitte's 2024 Global Outsourcing Survey found 83% of executives now use AI inside outsourced services, and most MSA templates predate this. Model weights trained on your data are the new IP battleground.

Good answer: Specific contractual language naming weights, embeddings, and fine-tunes as client property, with a non-reuse covenant.

Red flag: "We assume the standard IP clause covers it." It does not.

4. "What happened to the vendor's margin on your account after year one?"

Why it matters: This is a trick question. The reference will not know the margin, but the reaction will tell you whether the vendor repriced aggressively, cut senior staff, or started pushing change requests to recover margin.

Good answer: "Pricing held, team composition held."

Red flag: "They swapped out our senior engineers for juniors in month eight" or "Every clarification became a CR."

Group 2: Attrition and team substitution

5. "How many of the original named team members were still on your account at month 12?"

Why it matters: NASSCOM workforce data puts Indian IT-services attrition near 13% in 2024, down from a 2022 peak above 20%. A ten-person team can realistically rotate its entire roster in a year. The question is whether the vendor absorbs that churn or passes it to you as ramp-up tax.

Good answer: "Six of eight, with 60-day overlap for the two replacements."

Red flag: "We didn't track it — people kept changing."

6. "When a key engineer left, how did you find out?"

Why it matters: Mature vendors notify the client before resignation is public, with a succession plan. Immature vendors notify you on the Monday the person doesn't show up.

Good answer: "Account manager called us two weeks before the exit date with a named replacement already shadowing."

Red flag: "We noticed in standup when they stopped joining."

7. "Did the vendor charge you for ramp-up time on replacement engineers?"

Why it matters: Attrition is the vendor's problem commercially. If they billed you for the first four weeks of a replacement's learning curve, that is a contract-drafting failure you can avoid.

Group 3: What actually broke in production

8. "Walk me through the worst production incident in the first six months post-launch. Who paid for the fix?"

Why it matters: Every project has a first-six-months incident. The question is whether the vendor treated it as warranty work or billed it as a change request.

Good answer: A specific incident, root cause named, fixed under warranty inside the defect window.

Red flag: "There weren't really any issues" — this is a reference reading from a script — or "They charged us for the hotfix."

9. "What percentage of your post-launch spend went to planned enhancements versus fixing shortcuts taken before go-live?"

Why it matters: This exposes the technical debt the vendor shipped. Price-led selection, cited in 44% of terminated contracts, almost always manifests as post-launch rework budget.

Good answer: "80/20 enhancements to fixes."

Red flag: "Honestly, most of year two was stabilization."

10. "Did the vendor hand over runbooks, architecture decision records, and a working CI/CD pipeline — or did your team reconstruct them?"

Why it matters: Documentation is where corner-cutting is invisible until the vendor leaves. ISACA has flagged this specifically as the escrow-equivalent risk even for in-contract vendors.

Group 4: Commercial behaviour under stress

11. "What was the first change request worth more than ten percent of the original contract value, and what triggered it?"

Why it matters: Scope-creep is inevitable; weaponised scope-creep is not. You want to know whether the vendor absorbed small clarifications or treated every ambiguity as revenue.

Good answer: A clearly new requirement — a regulatory change, a new integration — not something a competent BA should have caught in discovery.

Red flag: "They CR'd us for things we assumed were in scope."

12. "If you had a dispute, did it go to the steering committee or straight to legal?"

Why it matters: Vendors with mature governance resolve at the steering committee. Vendors who escalate to legal early are telling you about their default posture.

13. "What is in the exit clause, and have you ever had to invoke it?"

Why it matters: Gartner's 2024 data shows 24% of outsourcing clients report vendor lock-in as a challenge. The exit clause is where lock-in gets built or broken: knowledge transfer duration, data-return format, named-personnel transition.

Good answer: "90-day KT, data returned in open formats, source code released on termination regardless of cause."

Red flag: "We never read it carefully" or "The vendor said it was standard."

Group 5: The questions that only work on a reference call

14. "If you were running this procurement again tomorrow, would this vendor make your shortlist — and would they win?"

Why it matters: Separates goodwill from endorsement. Many references will shortlist the vendor again out of loyalty but admit, when asked the second half, that they would pick differently now.

15. "Who on the vendor's side would you specifically ask for by name, and who would you refuse?"

Why it matters: Every references knows who the real engineers are. Named requests tell you which delivery lead to insist on in your SOW. Named refusals tell you which bench the vendor rotates onto struggling accounts.

16. "What did the vendor's sales team promise during pursuit that delivery could not honour?"

Why it matters: This is the question references wait for. It almost always surfaces specific overpromises — a timeline, a technology certification, a named architect who turned out to be fractional. Use the answers to re-read the proposal you were just given.

Reading the pattern across five calls

No single answer disqualifies a vendor. Patterns do.

• Three or more references say team composition shifted sharply after month six — assume your team will too, and price the ramp-up tax into the SOW or push it onto the vendor commercially.
• Any reference mentions MSA renegotiation around IP after delivery — have your counsel redline the IP exhibit before signing, specifically covering model weights, embeddings, and derivative works. (This is not legal advice; your lawyer should review the specifics.)
• No reference has tested escrow release — escrow is theatre at this vendor. Either make verification a contractual milestone or treat the clause as absent.
• References consistently praise the vendor's responsiveness but struggle to name a hard engineering decision the vendor made well — you are buying a staffing company, not an engineering partner. Price accordingly.

What the reference call cannot tell you

Be honest about the limits. References will not reveal the vendor's financial health, their concentration risk (what percent of revenue is one client), or whether the delivery leader you met is about to resign. Those come from a separate stream — audited financials, a D&B pull, and backchannel checks via your own network on LinkedIn. Reference calls are necessary and insufficient.

They also will not tell you about the engagements that ended badly enough that the client refused to be a reference at all. Ask the vendor directly: "Name a client who would not take this call, and tell me why." The answer, or the refusal to answer, is data.

One note on cost and timeline, because you will be asked

Reference calls take roughly 45 minutes each plus 30 minutes of prep and 15 minutes of notes — call it 90 minutes per reference, five references per vendor, two or three vendors. That is 15–22 hours of senior engineering time for a ₹50–200 lakh decision. The ratio is absurdly in your favour. The thing that stretches this out is scheduling, not conversation. Block the calls inside a two-week window or they will slip to six.

For the engagement itself, the cost and timeline you were quoted in the proposal are shaped by three decisions you still control: how many external integrations are in scope at signing versus deferred, whether the vendor is responsible for data migration from legacy systems, and whether the pilot is production-grade or throwaway. A conversation with a partner like CodeNicely's enterprise team — or any serious shortlisted vendor — should rescope against those three variables before any number is treated as a quote. For context on how engagement shape drives numbers, our digital transformation and AI studio pages describe the delivery models in more detail.

Frequently Asked Questions

How many references should I ask for from a shortlisted software development vendor in India?

Ask for five, not the standard three, and specify the categories: one live production engagement, one descoped project, one older than two years, one in your industry, and one where the delivery lead was replaced. If the vendor cannot supply across those categories, that itself is a signal — not necessarily disqualifying, but worth probing in the technical deep-dive.

Can I trust vendor-supplied references at all, or should I only use backchannel references?

Use both. Vendor-supplied references are curated but still useful if you ask operational questions they cannot deflect — IP clause renegotiation, escrow testing, attrition in the named team, who paid for the first production incident. Backchannel references via your own network on LinkedIn fill the gap for the engagements the vendor would not list.

What is the single most important question to ask on a dev partner reference call?

"What happened to your MSA's IP assignment clause and source code escrow twelve months after go-live?" Vendors who cut corners on these clauses tend to cut corners everywhere else — on documentation, on team substitution, on post-launch warranty work. The answer predicts the operational pattern better than any question about quality or deadlines.

How long does a proper vendor reference check take for an enterprise engagement?

Roughly 15–22 hours of senior engineering time across two or three shortlisted vendors — about 90 minutes per reference call including prep and notes, with five references per vendor. The duration end-to-end stretches because scheduling slips, not because conversations run long. Block the calls inside a two-week window or expect six.

What should I do if a vendor refuses to allow reference calls with a specific client?

Ask directly why that client is off-limits — NDA, confidentiality, ongoing dispute, or simply "they're busy." Then ask the vendor to name the engagement they are least proud of and explain what they would do differently. A vendor who can answer that honestly is more trustworthy than one whose entire reference list is spotless. Spotless lists are curated lists.