AI Software Engineer, Legal Prompting & LLM Dev.

Job Description:

The AI Software Engineer, Legal Prompting \& LLM Development is responsible for building production-grade applications that interact with state-of-the-art Large Language Models (LLMs) to deliver cutting-edge, time-saving solutions to attorneys throughout our firm and in support of our clients. This role sits at the intersection of software engineering, prompt engineering, and emerging agentic AI infrastructure — building not just LLM-powered applications, but the surrounding ecosystem of tools, integrations, and protocols that allow AI to interact safely and effectively with legal workflows and firm data.

The AI Software Engineer will develop application code as well as English prose, persona-based prompts to LLMs, while also designing and operating the infrastructure that connects those models to the firm's systems, including Model Context Protocol (MCP) servers, retrieval-augmented generation (RAG) pipelines, vector stores, APIs, and agent orchestration frameworks. Works primarily within the Microsoft technology stack (Azure, .NET, C#, SQL Server) and with languages such as TypeScript/JavaScript, Python, and other modern languages as needed, integrating with internal and third-party APIs to deliver production-grade solutions. Works across programming languages and can produce reliable backend code while also writing creative, lucid, and effective prose instructions. A remote work schedule is available for this position.

A career at Nixon Peabody is the opportunity to do work that matters. It’s the chance to use your knowledge to shape what’s ahead. To share, to innovate, to learn at a firm that taps the power of collective thinking.

We’ve created a dynamic, energizing environment that promotes success for our clients and each other. We offer fast growth, connectedness and training in business as well as law. And our rigorous standards assure you are part of a diverse team of top talent at every turn.

If you’re someone who’s looking toward the future, we’d love to hear from you.

Location:

Boston, MA; Chicago, IL; Los Angeles, CA; New York City, NY; Rochester, NY; San Francisco, CA; Washington, DC

LLM Application Development

• Design, develop, and deploy LLM-integrated applications that enhance legal workflows across transactional, litigation, regulatory, and advisory practice areas.
• Develop backend services across the Microsoft stack and in languages such as TypeScript/JavaScript, Python, C#, and others as needed, that interact with LLM providers (OpenAI, Anthropic, etc.), external APIs, SQL and NoSQL databases, and document management systems.
• Build and maintain RESTful and event-driven APIs that expose AI capabilities to internal applications and downstream consumers.

Prompt Engineering \& Evaluation

• Write and refine persona-based prompts, system instructions, and few-shot examples to guide LLMs in delivering accurate, defensible, and legally appropriate responses.
• Build prompt evaluation harnesses, regression test suites, and offline/online evaluation pipelines (e.g., LLM-as-judge, golden datasets) to measure quality, hallucination rates, and latency.
• Continuously test and iterate on prompts and code to optimize model performance, cost, and user experience.

MCP Servers \& Tool Integration

• Design, build, and operate Model Context Protocol (MCP) servers that expose firm systems — document management (e.g., iManage, NetDocuments), time and billing, CRM, research platforms, and internal knowledge bases — as secure, governed tools for AI agents.
• Define tool schemas, authentication flows, rate limiting, and audit logging for MCP endpoints, ensuring outputs are scoped to user permissions and ethical walls.
• Maintain a catalog of reusable MCP tools and resources that can be composed across multiple AI products at the firm.

Retrieval, RAG \& Knowledge Infrastructure

• Build and tune retrieval-augmented generation pipelines, including chunking strategies, embedding model selection, hybrid search (lexical \+ semantic), and reranking.
• Work with vector databases (e.g., Pinecone, Weaviate, pgvector, Azure AI Search) and orchestration frameworks (e.g., LangChain, LlamaIndex, Semantic Kernel) to ground LLM outputs in firm and client data.

Agentic Workflows \& Orchestration

• Develop multi-step and multi-agent workflows that combine planning, tool use, and human-in-the-loop checkpoints for sensitive legal tasks.
• Implement guardrails, content filters, PII redaction, and citation/verification layers to ensure responsible use.

MLOps, Observability \& Security

• Containerize services (Docker) and deploy via CI/CD pipelines to cloud environments (Azure preferred; AWS/GCP a plus), using infrastructure-as-code (Terraform, Bicep) where appropriate.
• Instrument applications with logging, tracing, and LLM-specific observability tools (e.g., LangSmith, Arize, Weights \& Biases, OpenTelemetry) to monitor quality, cost, and drift in production.
• Partner with Information Security and the Office of the General Counsel to ensure solutions meet client outside counsel guidelines, data residency requirements, and confidentiality obligations.

Collaboration \& Translation

• Collaborate with attorneys, legal professionals, and product teams to understand domain-specific needs and translate them into technical solutions.
• Assess the integration of LLMs into existing legal workflow systems and recommend improvements.
• Perform other duties as assigned.

To perform this job successfully, you must be able to perform each essential job responsibility listed above, satisfactorily, with or without reasonable accommodation. Nixon Peabody retains the right to change or assign other duties to this position. The requirements listed below are representative of the skills and abilities required.

• 4-6 years of production-level software engineering experience on a commercial or internal product team. Prior experience developing software solutions in the legal industry strongly preferred.
• Bachelor’s degree in Computer Science, Engineering, or a related technical field.
• Legal background highly preferred (e.g., J.D., paralegal, legal tech industry experience, or work with legal software vendors).
• Demonstrated experience in legal practice or support roles is a plus.
• Strong programming skills in modern object-oriented languages such as TypeScript/JavaScript, C#, Python, or Java (typing, async, packaging, testing), with the ability to work fluently across the Microsoft technology stack.
• Experience designing and consuming RESTful APIs and working with SQL databases; familiarity with NoSQL and vector stores a plus.
• Hands-on experience with LLM APIs (OpenAI, Anthropic, Cohere, Azure OpenAI) and/or open-source models (e.g., LLaMA, Mistral).
• Proficiency with prompt engineering techniques (chain-of-thought, structured outputs/JSON mode, function/tool calling, few-shot design).
• Experience building or integrating with Model Context Protocol (MCP) servers, custom tools, or function-calling endpoints for agentic systems.
• Familiarity with orchestration frameworks such as LangChain, LlamaIndex, LangGraph, Semantic Kernel, or Pydantic AI.
• Experience implementing RAG pipelines with embeddings, vector databases, and reranking models.
• Experience with evaluation frameworks (Ragas, DeepEval, promptfoo) and LLM observability platforms.
• Familiarity with containerization (Docker), CI/CD, and cloud deployment (Azure preferred).
• Excellent written communication skills — especially in crafting clear and effective LLM prompts and technical documentation.
• Ability to translate legal context and goals into prompt instructions, tool definitions, and system requirements.
• Strong analytical and problem-solving capabilities, with sound judgment about when to use deterministic code versus probabilistic models.
• Ability to thrive both independently and as part of a collaborative team.

In accordance with applicable Federal and State laws, the anticipated annual salary range for this position, depending on location, is as follows:

• Rochester, NY: $107,000 to $158,950
• Boston, MA: $118,000 to $174,900
• Chicago, IL: $118,000 to $174,900
• Los Angeles, CA: $118,000 to $174,900
• Washington, DC: $118,000 to $174,900
• New York, NY: $129,500 to $192,500
• San Francisco, CA: $129,500 to $192,500

In addition to a standard benefits package, this role may be eligible for additional contingent compensation based on an array of factors, including but not limited to: work performance, geographic location, work experience, education, and qualifications. Because such contingent compensation is not yet calculable or may be zero ($0) in some circumstances, the above-listed salary range(s) are Nixon Peabody’s good faith estimate of the annual salary(ies) it reasonably expects to pay for the position at the time of this posting.

Nixon Peabody LLP is an Equal Opportunity / Affirmative Action Employer: Disability / Veteran. Nixon Peabody is committed to creating a fair workplace and does not discriminate against any employee or applicant for employment because of race, creed, color, national origin, sex, age, disability, marital status, sexual orientation, citizenship status, or any other characteristic protected by applicable law. Pursuant to the San Francisco Fair Chance Ordinance and the Los Angeles Fair Chance Initiative, we will consider for employment qualified applicants with arrest and conviction records.

To comply with Federal law, Nixon Peabody participates in E-Verify. All newly hired employees are verified through this electronic system established by the Department of Homeland Security (DHS) and the Social Security Administration (SSA) to confirm their identity and employment eligibility. Please refer to the Notice of E-Verify Participation and the Right to Work posters on the Nixon Peabody Careers page for more information.