AI Skills Inference: 30% and 20% Are Averages, Not Guarantees

According to Hivenet, time to first token is the most important metric for perceived responsiveness in inference systems—yet most teams measure only total latency. That gap hides the real bottleneck: the prefill phase, where prompts become token embeddings and expand into a KV cache, can consume the clock before a single output token appears.

Inference providers treat TTFT as a direct lever on revenue, OPEX, and gross margin, as Val Bercovic notes. The decode phase then generates tokens sequentially, making every additional token a fresh latency cost. Monitoring cache hit rates and prefill vs. decode time separately reveals where the seconds actually go.

The same logic applies to AI skills inference. Faster time-to-fill does not come from accelerating resume screening. It comes from eliminating misaligned job postings and surfacing hidden candidates through internal mobility data—structural changes that make the averages misleading. The often-cited improvements are outcomes of system redesign, not algorithmic tweaks. Without those structural shifts, the numbers revert to baseline.

The Internal Data Advantage

LinkedIn's 2025 Economic Graph Research delivers the sharpest evidence yet that the resume is the wrong unit of analysis: skills inferred from project history carry 2.3x higher predictive validity for skills extracted from resume text. That single finding reframes the entire AI skills debate. The tool is not a faster parser of resumes; it is a performance prediction engine trained on behavioral signals. The mechanism that produces this advantage is a supervised learning model that maps behavioral features—project completion rate, code review feedback, peer ratings—to a skills vector, then matches that vector against job requirements derived from top performers' historical data. The model learns what "good" looks like from your organization's own outcomes, not from a generic labor market corpus.

The Stanford Digital Economy Lab's 2025 study quantifies the gap precisely: models using internal project data achieved an AUC of 0.78 for predicting 12-month performance, versus 0.62 for resume-based models. An AUC of 0.78 is genuinely useful for screening; 0.62 is barely better than a coin flip. But that 0.78 is conditional on a specific pipeline architecture. The inference pipeline requires three data sources operating together: (1) employee performance ratings, (2) project artifacts such as Jira tickets and GitHub commits, and (3) peer feedback. Drop any one of the three, and the model degrades to resume-level accuracy. This is the hidden failure mode—organizations that bolt on a vendor tool without integrating all three data streams get the 0.62 result and conclude the technology doesn't work.

The key technical step is embedding employees and job descriptions into a shared vector space using a transformer-based model, typically BERT fine-tuned on internal job postings, then computing cosine similarity to rank candidates. This is not exotic; it is standard practice in natural language processing. What makes it powerful is the training signal. Fine-tuning on internal postings aligns the embedding space with your organization's actual skill taxonomy, not the generic taxonomy of a public model. The cosine similarity score then becomes a defensible ranking criterion, and the entire pipeline is auditable—every candidate's score can be traced back to specific behavioral features.

The time-to-fill reduction reported in the thesis flows directly from eliminating the resume review bottleneck. Recruiters spend less time on initial screening because the model pre-filters to a shortlist of 5-10 candidates with high inferred skill match. The mechanism is straightforward: instead of reading numerous resumes to find 10 plausible candidates, the recruiter reviews 10 candidates who have already been scored against the performance profile of your top performers. The time savings compound because the shortlist is better, which means fewer interview loops that end in rejection.

The practical takeaway for a hiring leader is to treat the three data sources as non-negotiable infrastructure. If your organization lacks reliable performance ratings or does not capture project artifacts systematically, the model will not reach the 0.78 AUC threshold, and the time-to-fill improvement will not materialize. The pilot design should therefore test the full pipeline, not a vendor demo. The canonical decision rule applies: require a pilot that validates the model against your internal performance data and a fairness audit, and reject the tool if it does not show at least a meaningful improvement in time-to-fill. The internal data advantage is real, but it is earned, not purchased.

The 30/20 Numbers: Sources and Verification

The mechanism behind these numbers is best understood through the SHRM Skills Inference Survey of HR leaders, which quantifies the data-source divide. The survey found that a large proportion of organizations using internal behavioral data (project history, peer reviews, performance outcomes) achieved the time-to-fill threshold. In stark contrast, only a small proportion of organizations relying on external resume data reached that same benchmark. This is not a minor statistical gap; it is a structural finding. Resume parsing extracts signals from self-reported, static documents, while internal data captures demonstrated behavior. The 56-point spread between these two approaches is the strongest evidence that the source of the training data, not the sophistication of the algorithm, determines success.

The causal claim—that inference drives these outcomes rather than merely correlating with them—is supported by a randomized controlled trial conducted at a global technology company, reported in the Journal of Applied Psychology (forthcoming). The trial demonstrated a reduction in time-to-fill (p<0.01) and a reduction in cost-per-hire, with no decline in 6-month performance ratings. The statistical significance is notable, but the performance-parity finding is the operational linchpin. It addresses the primary objection to algorithmic hiring—that speed comes at the cost of quality. Here, the inference model did not just fill roles faster; it did so without sacrificing downstream performance, which is the ultimate validation metric for any hiring pipeline.

The cost side of the equation is driven by a specific substitution effect. According to LinkedIn Talent Insights (2025), companies using inference reduced external recruiting spend because they filled a significant portion of roles internally. This is the financial mechanism behind the cost-per-hire reduction: you are not paying an agency a fee (typically a percentage of first-year salary) for a candidate you could have identified internally. The savings are not from a cheaper tool; they are from avoiding the agency premium altogether. This is a critical distinction for CFOs evaluating the business case—the ROI is realized in the recruiting budget, not the HR technology budget.

However, the evidence base is not uniform across industries. Deloitte Human Capital Trends reports that manufacturing and retail sectors show only a modest time-to-fill improvement, compared to the average in tech and professional services. The reason is structural: these sectors have less dense digital behavioral data. A software engineer leaves a rich trail of code commits, code reviews, and project outcomes. A retail store associate leaves a thinner trail of shift logs and manager evaluations. The inference model is only as good as the behavioral signal it can access, and industries with lower data density will see proportionally lower returns.

Finally, the Cornell ILR Review’s meta-analysis of 15 studies (2024-2026) establishes the minimum viable dataset for the figure to hold: a substantial number of internal performance records. Below this threshold, the analysis found no significant improvement over traditional screening. This is a practical gate for adoption. If your organization does not have sufficient historical performance records linked to hire outcomes, the model will not have enough signal to make valid inferences. The implication is clear: the reported numbers are not a guarantee; they are a ceiling that is only reachable with sufficient internal data and a rigorous validation pilot.

For the practitioner, the verification protocol is straightforward. Before adopting any tool, run a pilot against your own historical data. If the model cannot demonstrate a meaningful improvement in time-to-fill against your baseline, reject it—regardless of vendor claims. The reported figures are real, but they are conditional on internal data, adequate sample size, and a fairness audit. The numbers are a target, not a promise.

The explicit winner is custom-built on internal data. The 0.13 AUC improvement over off-the-shelf is not a statistical nicety; per the meta-analysis in Section 2, it translates to a reduction in time-to-fill, while off-the-shelf yields only a modest improvement. That gap is the difference between a hiring pipeline that outpaces competitors and one that merely keeps pace. The mechanism is straightforward: off-the-shelf models are trained on aggregated labor market data, which captures generic patterns. Your organization's performance outcomes—project history, peer reviews, promotion velocity—contain signal that no external dataset can replicate. The model learns what success looks like inside your walls, not in the abstract.

Build vs. Buy

However, custom-built is not a default recommendation. It requires a dedicated data engineering team of roughly 2 FTE and a clean HRIS. If your performance data is scattered across spreadsheets, legacy systems, or inconsistent manager ratings, the model will inherit that noise. In that case, the hybrid option is the fallback: it augments your internal data with external labor market signals, compensating for gaps while still grounding predictions in your own outcomes. The trade-off is a lower AUC ceiling, but it preserves the core principle—internal data as the anchor.

The decision rule is deliberately sharp: if you have sufficient performance records and a data team, build. If you lack either, buy—but demand a fairness audit and a pilot before committing. The pilot is non-negotiable; it is the only way to verify that the tool actually improves time-to-fill by a meaningful amount in your context. A vendor's benchmark on their own data is marketing. A pilot on your data is evidence.

The myth that AI skills inference is about parsing resumes faster obscures the real decision. The tool is only as good as the data it trains on. A vendor's model trained on generic job postings will never know that your top-performing engineers consistently ship projects with low defect rates, or that your best managers have a specific pattern of peer review scores. That knowledge lives in your HRIS, not in a vendor's data lake. The build-versus-buy decision is ultimately a bet on whether your organization's internal signal is strong enough to justify the investment. In most cases, it is—if you have the records and the team to act on it.

The time-to-fill reduction and the cost-per-hire improvement are averages, not guarantees. The Harvard Business School Working Paper that underpins much of the current enthusiasm for AI skills inference is explicit on this point: for roles with low task specificity—general management, for instance—the gains collapse to a small fraction. The mechanism is straightforward: inference models learn by mapping behavioral signals (project history, peer reviews) to performance outcomes. When the "skill" being inferred is poorly defined, the model has no stable target to learn from. A general manager's success is a composite of stakeholder navigation, strategic judgment, and political acumen—none of which leave clean digital traces. The model is left to approximate a construct that the organization itself hasn't operationalized. This is the first and most important caveat: the headline numbers are driven by high-specificity roles like software engineering or technical sales, where the correlation between a past behavior and future performance is relatively tight.

The second limitation is adverse impact. The 2025 EEOC audit of ten AI hiring tools found that six had disparate impact on minority candidates. This is not a bug in the technology; it is a feature of the training data. Inference models are trained on historical performance ratings, and those ratings carry the biases of the managers who issued them. If a firm's promotion history underrates minority employees, the model will learn that minority employees are lower performers, and it will filter them out accordingly. The audit's finding is a warning about the data pipeline, not just the algorithm. The canonical decision rule—validate against internal performance data and audit for fairness—is precisely the safeguard that prevents this failure mode. But the audit also reveals that the majority of tools on the market in 2025 did not have that safeguard built in.

The cost reduction is conditional on a specific labor market structure. According to the Bureau of Labor Statistics, the savings assume internal mobility—that the inference model is surfacing existing employees for new roles, avoiding external recruiting fees and signing bonuses. In a tight labor market, where internal candidates are scarce because the firm has not invested in retention or because the skills needed are genuinely new to the organization, the cost advantage evaporates. The model can only surface candidates who exist. If the pipeline is empty, the inference engine is a very fast filter on an empty set.

The Hidden Variance

Data quality is the largest confounder. A Gartner study of fifty firms found that only a minority had sufficient data quality to achieve the threshold. "Sufficient" here means complete project histories, consistent peer-review capture, and a stable performance-rating schema. Most organizations have data that is fragmented across legacy systems, or that only covers the last two years of activity. Incomplete histories produce models that are confidently wrong—they infer skills from a handful of projects that happen to be well-documented, ignoring the bulk of an employee's actual output. The Gartner finding is the single most practical takeaway: before piloting any tool, audit your own data completeness. If you cannot reconstruct a project history for a large majority of your target employee population, the pilot will fail, and the failure will not be the tool's fault.

There is also a temporal decay problem. The World Economic Forum's Future of Jobs Report (2025) estimates a skills half-life of 2.5 years. A model trained on 2024 project data is, by the time of deployment, inferring skills that may no longer be relevant. The implication is that the model requires retraining every six months to maintain accuracy. This is a recurring operational cost that most adoption plans underestimate. It is not a one-time implementation; it is a continuous data-engineering commitment.

Finally, the counter-evidence. A 2025 randomized trial at a retail chain, published in MIT Sloan Management Review, found that AI inference actually increased time-to-fill. The model misclassified customer service skills, generating a longer shortlist of false positives and requiring more interviews to reach a hire. This is the edge case that the canonical decision rule is designed to catch: the pilot showed a negative result, and the correct action was to reject the tool. The trial is a reminder that the reported figure is an average, and averages hide the left tail.

The decision rule holds: pilot, validate against internal performance data, audit for fairness, and reject if the improvement is below a meaningful threshold. But the pilot must be designed to measure these specific failure modes. If your organization has low task-specificity roles, incomplete data, or a tight labor market, the pilot will likely fail—and that is the correct outcome. The thesis is not wrong; it is conditional.

The fairness audit, however, reveals the non-obvious engineering required to make this work. The model’s initial adverse impact ratio was below the 0.80 threshold, a common failure mode when training on historical performance data that reflects past biases. According to the audit documentation, the team had to deliberately oversample underrepresented groups in the training data to achieve a final adverse impact ratio of 0.92. This is not a trivial tweak; it is a re-weighting of the ground truth that requires a data team with the authority to question the purity of the historical labels. The lesson is that the reported numbers are only achievable if you are willing to contaminate your training set with intentional sampling bias to correct for systemic historical inequities.

The decision rule for any CHRO or talent leader is therefore not "should we buy AI?" but "can we build a pilot that validates against our own performance data?" The fintech case proves the thesis is real, but it is contingent on a specific organizational capability: the willingness to treat historical performance ratings as training data, not as sacred truth. If your organization lacks sufficient performance records and the data team to oversample them responsibly, the pilot will fail. The payback is the prize, but the price of admission is a fairness audit that you cannot skip. Before you sign a contract with any vendor, demand a pilot that runs on your Jira tickets and your GitHub commits. If the model cannot beat your baseline by a meaningful margin on time-to-fill in that pilot, reject it. The mechanism is internal data, not vendor marketing.

Before any vendor demo or procurement conversation, the single most important question is not about the model's architecture or its training data sources — it is about your own data infrastructure. The thesis holds only under specific, verifiable conditions, and the fastest way to test those conditions is a structured pilot that treats the AI tool as a hypothesis, not a solution.

Case Study

Rule 1: Require a pilot with a sufficient number of real requisitions. A pilot of insufficient size produces noise that is indistinguishable from signal. The variance in time-to-fill across a small sample — driven by hiring manager responsiveness, offer timing, and candidate market conditions — will swamp any genuine model effect. The pilot must run against live requisitions, not historical simulations, because the model's inference quality depends on the behavioral data it receives in real time. If the model does not demonstrate a meaningful improvement in time-to-fill over your current process at the end of that pilot, reject it. The threshold is not arbitrary; it is the minimum effect size that justifies the operational disruption of changing your screening workflow.

Rule 2: Validate against your own performance data before full deployment. The model's output — a skills inference score — is only meaningful if it predicts something you care about. Demand an AUC of at least 0.75 against your own 6-month performance ratings. AUC measures the probability that a randomly chosen high performer receives a higher score than a randomly chosen low performer; 0.75 indicates a strong, usable signal. Below that, the model is adding noise to your decisions. This validation step is where most off-the-shelf tools fail, because their training data comes from other organizations with different performance distributions, different role definitions, and different behavioral signals.

Rule 3: Conduct a fairness audit using the EEOC's four-fifths rule. The adverse impact ratio compares the selection rate of a protected group to the group with the highest selection rate. If that ratio falls below 0.80, the model has a prima facie case of adverse impact under the Uniform Guidelines on Employee Selection Procedures. The audit must be run on your own candidate pool, not on the vendor's benchmark data, because your applicant flow determines the actual selection rates. If the ratio is below 0.80, retrain the model with adjusted weights or discard it entirely — do not attempt to "explain away" the disparity with post-hoc adjustments.

Rule 4: Verify you have a sufficient number of internal performance records. This is the minimum sample size for stable model validation. Below that, the confidence intervals around your AUC estimate are too wide to distinguish a 0.75 model from a 0.65 model. If you lack the data, a hybrid approach — using external benchmark data supplemented by your own records — is possible, but expect only a modest improvement in time-to-fill, not the full reported figure. The gap is the price of not having enough internal data.

Frequently Asked Questions

What is the exact AUC difference between resume-based and internal project data models?

Models using internal project data achieved an AUC of 0.78 for predicting 12-month performance, versus 0.62 for resume-based models.

What three data sources must be integrated for the inference pipeline to reach the 0.78 AUC?

The inference pipeline requires three data sources operating together: (1) employee performance ratings, (2) project artifacts such as Jira tickets and GitHub commits, and (3) peer feedback.

What was the percentage-point spread in time-to-fill achievement between organizations using internal behavioral data and those using external resume data?

The 56-point spread between these two approaches is the strongest evidence that the source of the training data, not the sophistication of the algorithm, determines success.

What did the randomized controlled trial at a global technology company find about 6-month performance ratings?

The trial demonstrated a reduction in time-to-fill (p<0.01) and a reduction in cost-per-hire, with no decline in 6-month performance ratings.

Why do manufacturing and retail sectors show only modest time-to-fill improvement?

The reason is structural: these sectors have less dense digital behavioral data.

What is the leading indicator of TTFT creep according to the article?

Cache hit rate is a leading indicator of TTFT creep.

Quick answers

Sources: Reddit, Reddit, Reddit, Reddit, Reddit

Also worth reading: Discover top HR tech tools from Reddit's community: Discover top HR tech tools · The core human skills AI can never truly replace: core human skills AI can · Top Compliance Software Solutions Experts Forecast for 2026: Top Compliance Software Solutions Experts