Structuring the AI Platform Team: Roles, Ratios, and the First Three Hires
AI platform team structure for mid-market regulated firms: platform vs embedded models, the first three hires in order, and when a fractional architect wins.
Your AI platform team structure will decide more about your AI program's success than your model choice will. The question is not whether your firm does AI work; it already does, in pockets, usually unsanctioned. The question is who owns the shared parts: the gateway, the eval gates, the audit trail, the cost telemetry. This article is for CTOs and VPs of Engineering at mid-market regulated firms, roughly 100 to 2,000 employees, who need to staff that ownership without building an ivory tower. It covers the three org models, the first three hires in order, and the case for buying architecture fractionally instead of hiring it.
AI platform team structure: three models, chosen by coordination cost
There are three working models, and each fails in a predictable way.
| Model | Shape | Where it wins | Where it breaks |
|---|---|---|---|
| Centralized platform | One team owns all AI infrastructure and often the use cases too | Early days, one or two use cases, scarce talent | Becomes a ticket queue; product teams wait weeks for prompt changes |
| Embedded | AI engineers sit inside product teams; no central function | Speed; domain context lives with the builder | Duplicated gateways, inconsistent controls, audit trail per team or none |
| Hub-and-spoke | Small central platform owns shared controls; use-case engineers embed in product teams | Regulated firms past two or three production use cases | Needs a real platform lead; fails as a part-time side job |
Team Topologies, the closest thing the industry has to a shared vocabulary here, frames this as cognitive load: a platform team exists to absorb complexity so stream-aligned teams can ship (IT Revolution). The framework's guidance for small companies is blunt: a single team can run a thin platform and rely on advisors instead of formal enabling teams (Umbrex Team Topologies FAQ). Structure follows scale.
The market context matters. Gartner projected that 80% of large software engineering organizations would run dedicated platform teams by 2026, up from 45% in 2022, but reported adoption sat closer to 55% in 2025, and nearly 30% of platform teams measure no success metrics at all (CloudMagazin, March 2026). That stat comes from large enterprises. A 400-person firm copying their org chart is copying the costume, not the cause.
In regulated industries there is one argument that cuts across all three models: NIST's AI Risk Management Framework puts accountability structures first. Govern 2 requires that specific teams and individuals are empowered, responsible, and trained for managing AI risk (NIST AI RMF Core). "Everyone owns it" is not an accountability structure. Whoever your model assigns the controls to, name them.
What the platform owns, and how big it should be
The AI platform is not a research lab. It is a control plane. The owned surface:
- The gateway. All model traffic, internal and external, routes through one chokepoint with authentication, logging, and policy enforcement. One gateway, not one per team (see ai-gateway-architecture-enterprise).
- The eval gate. Golden datasets and CI thresholds that block deployment when quality regresses.
- The audit trail. Immutable records of prompts, retrieved context, model versions, and outputs, retention-matched to your regulator.
- Access control at the retrieval layer. Permissions enforced in the pipeline, never in the prompt.
- Cost telemetry. Per-request, per-team, per-use-case spend attribution.
Controls as architecture: these are properties of the platform, not of any single use case. That is the entire argument for centralizing them.
On size, Gartner-scale ratios do not transfer to the mid-market. Our field rule from assessment work: one platform engineer per two to four consuming teams, or per 15 to 25 engineers building on AI features, whichever binds first. Below two consuming teams you do not have a platform; you have a person with good defaults. Above the ratio, the platform team becomes the bottleneck its critics warn about.
The first three hires, in order
Sequence matters more than the roster. Each hire de-risks the next.
| # | Role | What they own first | Why this order |
|---|---|---|---|
| 1 | Platform architect (senior, hands-on) | Gateway, identity, audit trail, model and vendor selection | Architecture decisions made before use case three are cheap. Made after, they are migrations. |
| 2 | Applied AI engineer, embedded | The single highest-value use case, end to end | Proof of value funds the program. A platform with no shipped use case is overhead. |
| 3 | Platform engineer / SRE | Eval CI, observability, cost telemetry, on-call for the platform | Turns hire 1's design into a running service and frees the architect from pager duty. |
The mistake we see most is inverting hires 1 and 2: an applied engineer ships a demo fast, the demo becomes production by default, and the controls get retrofitted around decisions nobody remembers making. The second mistake is hiring 3 before 2, which produces a beautifully instrumented platform serving zero users.
Expect to pay for seniority. In Q1 2025 job-posting data, platform engineer roles averaged $172,038 with a range of $143,001 to $201,074, and 85% of posted roles were senior-level (Kube Careers via Platform Engineering). Built In puts average total ML engineer compensation at $212,022 (KORE1 ML Engineer Salary Guide, March 2026). Three senior hires land at $600K or more in loaded annual cost. Time to fill runs months, not weeks, in this market (see ai-talent-gap-hiring-trends-2026).
When a fractional architect beats hire number two
Hire 1's job is front-loaded. Designing the gateway, choosing the control architecture, writing the policy-as-code baseline: this is six to twelve weeks of senior judgment. Operating it afterward is closer to one steady-state FTE, and that operator is hire 3's profile, not hire 1's.
That mismatch creates the fractional case. Buy the architecture fractionally at two to three days a week for a quarter, then hire operators into a finished design. The arithmetic: instead of $172K base plus 25-40% loading for a senior architect you may not keep busy after month four, you pay for the design work only, and your first full-time hire is a platform engineer at a lower band who inherits documented decisions instead of inventing them.
The triggers where fractional wins:
- Fewer than three consuming teams today. The platform work is real but part-time.
- You need controls in place this quarter for a client, auditor, or regulator, and a senior search takes longer than the deadline.
- Your use-case roadmap is validated but your architecture is not, and you want the design reviewed by someone who has built it before.
- Key-person exposure already worries you. A fractional engagement that ends with documentation and trained operators reduces bus factor instead of adding to it (see key-person-risk-ai-platform).
The triggers where it loses: you have four or more teams consuming AI services, the platform is on the critical path daily, or politics requires a named internal owner with hiring authority. Fractional architects advise; they do not win turf wars.
The case against a platform team at all
The strongest version of the opposing view deserves airtime, because it is right about many firms.
Platform teams fail quietly and often. Nearly 30% never measure adoption or outcomes, and practitioner forums are full of the same autopsy: a portal gets installed, golden paths get announced, and developers route around both (TianPan community discussion, March 2026). Centralization also creates information asymmetry: the platform team holds the context every product team needs, so a queue forms, and the team that was hired to remove friction becomes the friction (TianPan, May 2026).
For a mid-market firm, the counter-argument writes itself. Three senior hires at $600K loaded is a bet the roadmap has to repay. Embedded engineers with strong defaults, a shared eval harness, and an advisor on retainer can carry a firm to three or four production use cases without a platform org at all. Team Topologies says as much for small companies: thin platform, advisors, explicit interfaces, revisit at scale.
Concede the point where it holds. Under two consuming teams, do not build the team. The evidence also says, however, that embedded-without-controls is where regulated firms get hurt: duplicated gateways, unaudited prompts, permissions bolted onto prompts. The skeptic is right about the org chart and wrong about the controls. You can run thin on people. You cannot run thin on the audit trail.
Our read
In regulated industries the hub-and-spoke model arrives earlier than the headcount math suggests, because the spokes cannot each own compliance. Our position: hire or contract the architect first, centralize the five control surfaces from day one, embed everyone else, and keep the central function at one or two people until four consuming teams force the ratio. Treat the platform team as a control plane with an SLA, not a center of excellence with a mission statement. And write the exit criteria now: the day the platform team approves use cases instead of enabling them, it has failed.
What this means for you
- Finance: Your regulator's recordkeeping expectations map directly onto the audit trail surface. Assign it to a named owner before your second use case ships, not after an exam finds the gap.
- Healthcare: PHI in prompts and retrieved context makes the gateway and retrieval-layer access control your two non-negotiables. Hire 1 owns both; a fractional architect who has done HIPAA-adjacent deployments beats a generalist search you cannot close in time.
- Legal: Matter-level permissions are the control that fails first in embedded models. Centralize permission filtering even if everything else stays in product teams.
- CPA/advisory: Seasonal demand argues for fractional. Staff the architecture between busy seasons, hire operators when the client-facing assistants go live.
- All firms: Run the ratio check this week. Count teams building on AI features. If the answer is two or fewer, buy advice and defaults, not headcount.
Book a free assessment: systemics.ai.
References
- Here is Why Platform Engineering May Be a More Lucrative Career Than You Think (Kube Careers Q1 2025 job market data), Platform Engineering, June 2025. https://platformengineering.com/features/here-is-why-platform-engineering-may-be-a-more-lucrative-career-than-you-think/
- Platform Engineering 2026: Why 80% of Companies Are Adopting Internal Developer Platforms, CloudMagazin, March 2026. https://www.cloudmagazin.com/en/2026/03/01/platform-engineering-2026-why-80-of-companies-are-adopting-internal-developer-pl/
- AI RMF Core (Govern 2: accountability structures), NIST AI Resource Center, accessed August 2026. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/
- The Four Team Types from Team Topologies, IT Revolution, February 2023. https://itrevolution.com/articles/four-team-types/
- Team Topologies (Organization Frameworks FAQ), Umbrex, July 2026. https://umbrex.com/resources/frameworks/organization-frameworks/team-topologies/
- Machine Learning Engineer Salary Guide, KORE1, March 2026. https://www.kore1.com/ml-engineer-salary-guide/
- Golden Paths for AI Agents: How Platform Teams Can Enable Adoption Without Becoming a Bottleneck, TianPan, May 2026. https://tianpan.co/blog/2026-05-06-golden-path-internal-platform-ai-agent-self-service
- Gartner Predicted 80% Platform Engineering Adoption by 2026, But We're at 55%, TianPan community forum, March 2026. https://tianpan.co/forum/t/gartner-predicted-80-platform-engineering-adoption-by-2026-but-were-at-55-are-we-overhyping-portals-while-underinvesting-in-platform-fundamentals/3471
Start with a free architecture assessment.
Thirty minutes, no slides. Tell us where your AI posture stands and we will tell you honestly whether we can help.