Principal Engineer, LLM
Upstart ยท United States | Remote
๐บ๐ธ Remote (US)Principal$238,400 โ $330,200/yr4h ago
At a glance
- Where
- Remote โ open to applicants in the United States
- Pay
- $238,400 โ $330,200/yr โ as listed by Upstart
- Level
- Principal
- Experience
- 8+ years
- Education
- Bachelor's degree, or equivalent experience
- Hours
- Team operates on East/West coast time zones
- Travel
- Digital first role; most teams meet onsite once or twice per quarter for 2-4 consecutive days
- Posted
- November 19, 2026
- Last confirmed open
- 53m ago
- At Upstart
- 26 open remote roles โ mostly Other (6), AI & Machine Learning (4), Finance & Legal (4). Pay is listed on 26 of them ($79,000 โ $330,200/yr).
- How to apply
- Directly on Upstart's own site โ no NodJobs account needed.
About the role
This role leads technical pathfinding for Upstart's Applied AI team, building 0-to-1 LLM and agent prototypes across the agent platform, borrower assistant orchestration, and internal workflows. It suits a principal-level engineer with 8+ years of software experience and hands-on background shipping non-deterministic LLM products to real users.
What you'll do
- Rapidly prototype ambiguous bets across the agent platform, cross-agent context orchestration, borrower-facing assistant experiences, and internal workflows
- Partner with Product and Design to validate non-deterministic UX and establish clean architectural patterns
- Hand off golden datasets, automated evaluation harnesses, failure-mode maps, latency/cost benchmarks, and architectural blueprints to engineering squads
- Align early with receiving squad tech leads, support joint go/no-go decisions including killing unviable bets, and stay engaged through initial productionization
- Review early PRs and debug edge-case model behavior during productionization
- Treat latency spikes, context loss between agents, hallucinations, and fallback paths as systems design problems
- Make pragmatic applied AI trade-offs on model routing, fine-tuning, or distillation when latency, unit economics, or reliability require it
- Lead by example with agentic coding workflows and verification harnesses, act as early adopter of internal tooling, and help product squads adopt higher-leverage practices
- Build and support a reliable platform that lets non-technical users build, launch, and operate AI agents (voice/chat/text)
- Build the orchestration layer behind a unified personal assistant that shares context across specialized agents
- Improve internal operational workflows that directly connect to applicant and borrower experiences
- Act as a technical pathfinder and force multiplier for the team and partner product squads, raising the applied AI bar across squads
- Share practical requirements with internal tooling teams and elevate engineering judgment across teams through active collaboration
What you'll need
- 8+ years of software engineering experience
- Hands-on experience shipping non-deterministic, user-facing LLM or AI products to real users
- Deep practical experience designing production LLM systems with structured outputs, tool calling, context engineering, retrieval/memory patterns, and multi-step or multi-agent orchestration
- Strong distributed-systems and backend fundamentals: async orchestration, streaming UX, caching, timeouts and fallbacks, rate-limit handling, and end-to-end observability/tracing
- Proven experience building golden datasets, automated evaluation harnesses (deterministic checks, heuristics, LLM-as-a-judge), regression gates, and failure-mode analyses
- Instinct to use the simplest reliable tool first (deterministic logic, prompting, context engineering, guardrails) with judgment on when to route, fine-tune, or distill
- Comfortable moving fast across backend and frontend stacks such as Python, TypeScript/React, and REST or streaming APIs
- Track record of aligning with partner squads, handing off ownership cleanly, and partnering with Product, Design, and business leaders on latency budgets and fallback experiences
- Bachelor's degree in Computer Science, Engineering, Mathematics, or equivalent practical experience
- Partner directly with operations or business leaders to turn ambiguous goals into concrete latency budgets and resilient fallback experiences
- Design non-deterministic UX and orchestration together, navigating trade-offs across latency, unit economics, reliability, and user trust
Nice to have
- Startup or 0-to-1 incubation experience owning ambiguous problems end to end under real constraints
- Experience designing reusable agent primitives, SDKs, or orchestration platforms that help peer teams build, evaluate, and launch AI workflows
- Hands-on experience fine-tuning or distilling smaller models in production to improve latency, unit economics, or reliability
- Deep daily fluency with agentic coding workflows and automated verification harnesses, and experience helping peers adopt them
- Experience in regulated or high-trust consumer domains such as fintech, lending, or healthcare
Benefits listed by Upstart
- Competitive base pay, bonus opportunities, and annual equity grants vesting quarterly
- 401(k) or Group Retirement Savings Plan with company match
- Employee Stock Purchase Plan (US only)
- Medical, dental, vision, and wellness coverage; supplemental health coverage for Canada
- Health Savings Account contributions for eligible plans (US only)
- Life insurance and disability coverage
- Paid time off, sick leave, and company holidays
- Paid family and parental leave
- Family-centered benefits for fertility, parenthood, and caregiving
- Employee Assistance Program with mental health support
- Financial wellness resources including financial planning tools and a financial concierge (US only)
- Annual wellness allowance
Tools & skills mentioned
PythonTypeScriptReactstreaming APIsLLMtool callingretrievalLLM-as-a-judgeobservabilitytracingfine-tuningdistillation
Summarized by NodJobs from Upstart's posting. Read the full posting on Upstart's site โ
NodJobs is free for job seekers โ we never charge to apply, and this role takes you straight to the company's own application.
Similar live jobs
Applied Scientist
Upstart
๐บ๐ธ Remote (US)AI & Machine Learning$141,500 โ $196,000/yr
8d ago
Staff Applied Scientist
Upstart
๐บ๐ธ Remote (US)AI & Machine Learning$199,500 โ $276,000/yr
8d ago
Senior Applied Scientist
Upstart
๐บ๐ธ Remote (US)AI & Machine Learning$167,700 โ $231,800/yr
8d ago
Senior AI Data & Analytics Engineer II
Samsara
๐บ๐ธ Remote (US)AI & Machine Learning$137,445 โ $231,000/yr
1mo ago
Senior Fullstack Engineer, FanX AI
SeatGeek
๐บ๐ธ Remote (US)AI & Machine Learning$144,000 โ $209,000/yr
14d ago
Manager, Business Risk and AI Automation
๐บ๐ธ Remote (US)AI & Machine Learning$154,700 โ $216,600/yr
2d ago