Fabric Data Engineer — Workplace Engineering

About The Role

Vanguard is standing up Microsoft Fabric as the enterprise data and analytics foundation that powers our Workplace AI, Power BI, and cross-cloud analytics estate. We are partnering with Microsoft on a CDAO-led Fabric Enablement engagement and are building this capability on an F256 Reserved capacity, integrated with the broader Vanguard data, identity, and security stack — including OneLake Direct Lake against AWS S3, Entra ID and Okta federation, and Microsoft Purview.

Role Summary

We are hiring a hands-on Fabric Data Engineer to own the data layer of that capability. This is a builder's role, not an architect-only role. The engineer designs and implements scalable data products in OneLake — lakehouses, warehouses, pipelines, notebooks, semantic-model-ready Delta tables — and is accountable for the lifecycle, governance, and operational health of the Fabric platform. The complementary AI Engineer role consumes that foundation to build agents, copilots, and Foundry orchestrations; this engineer makes sure the data underneath is governed, monitored, and ready.

You will partner closely with the AI Engineer on AI-ready data products and semantic-layer handoffs; with our Technical Project Manager on program delivery, enablement, and change management; and with our Cloud Domain Architect on platform alignment. You will work alongside the Microsoft CDAO Fabric Enablement team and Vanguard partners across CDAO and Workplace Engineering. You will be a core member of the emerging Workplace AI Fusion Team. This is a strategic engineering and implementation role, not a support position.

Key Responsibilities

(

Fabric Build \& Data Engineering)

• Design and implement scalable data storage in OneLake using Lakehouses (Delta) and Warehouses (T-SQL); choose the right item for each workload and configure SQL analytics endpoints, shortcuts, and OneLake security.
• Build and maintain Spark notebooks (PySpark), Data Factory pipelines, Dataflows Gen2, Copy Jobs, and mirroring for batch and incremental ingestion at enterprise scale.
• Build Real-Time Intelligence solutions: Eventstreams, Eventhouses / KQL databases, Activator reflexes, and Spark structured streaming for low-latency workloads.
• Optimize Lakehouse tables (OPTIMIZE, V-Order, Z-Order, partitioning) and Direct Lake semantic-model-ready datasets so downstream Power BI and AI agents perform predictably.

ALM \& Lifecycle Engineering

• Implement source control, branching, and CI/CD using native Fabric Git integration (Azure DevOps and GitHub), Fabric Deployment Pipelines, and the Microsoft fabric-cicd Python library.
• Automate Dev / Test / Prod promotion against the Fabric REST API using service principals and Workload Identity Federation; codify environment-aware bindings via Variable Libraries and parameter.yml.
• Operate a Feature → Dev → UAT → Prod branching pattern — native Git on Feature and Dev workspaces, pipeline-pushed promotion to UAT and Prod — with mandatory PR review, cherry-pick promotion, and one repo per team to scope blast radius.
• Own the lifecycle of Fabric data components from creation through retirement, ensuring every environment is reproducible from the GitHub pipeline rather than from the Fabric UI.

Platform Operations \& Monitoring

• Operate the Fabric F256 capacity: monitor CU consumption with the Capacity Metrics App, manage smoothing windows, diagnose interactive and background throttling, and right-size workloads.
• Build telemetry using the Monitoring Hub, per-workspace Workspace Monitoring (Eventhouse-based KQL logs), Eventhouse monitoring, and the Admin Monitoring Workspace to surface refresh failures, pipeline errors, and semantic-model health.
• Define dashboards and alerts for ingestion, transformation, refresh, and capacity health; drive root-cause analysis on production incidents and feed lessons back into platform standards.
• Define and operate the on-call model for production data pipelines and Fabric items in partnership with Tier 3 Engineering.

Standards, Governance \& Security

• Define and enforce Fabric platform standards through Terraform-based IaC using the official microsoft/fabric provider (workspaces, capacities, domains, items), workspace templates, naming and tagging conventions, and automated CI policy checks against the Fabric REST API.
• Manage tenant settings, domains, and capacity allocation in partnership with the Fabric Center of Excellence; align identity with Entra ID and Okta federation; rotate service principals and use PIM for elevated admin roles.
• Implement RBAC patterns that separate workspace control-plane roles (Admin / Member / Contributor / Viewer) from OneLake data-plane roles (folder and table level); operate RLS, CLS, OLS, dynamic data masking, and item-level sharing.
• Integrate Microsoft Purview for sensitivity labels, DLP, metadata scanning, lineage, and impact analysis; manage endorsement (Promoted / Certified) so AI agents and BI consumers only ground on trusted datasets.

Integration \& Interoperability

• Build cross-cloud integration patterns: OneLake Direct Lake against AWS S3, Mirrored Databases for Snowflake, SQL Server, and Cosmos, and shortcuts that avoid Athena and ODBC where Direct Lake delivers better performance.
• Publish governed, AI-ready data products with Prep for AI configured on semantic models so Fabric Data Agents, Copilot Studio, and Azure AI Foundry can ground on certified Vanguard data.
• Coordinate with Data, Cloud, Identity, and Security domain teams on data-sharing patterns, private link configuration, and on-prem data gateway operations across the current 6–8 gateway footprint.

Tier 3 Escalation \& Expert Support

• Serve as Tier 3 escalation for complex Fabric, OneLake, pipeline, capacity, and Direct Lake issues across the enterprise.
• Provide deep technical consultation to Workplace Engineering, CDAO, and partner teams onboarding workloads to Fabric.
• Build reusable patterns, reference implementations, and internal playbooks for ingestion, modeling, deployment, and capacity operations that scale beyond a single engineer.

Innovation \& Strategic Oversight

• Lead proof-of-concept work for new Fabric capabilities (Mirrored Databases, GraphQL APIs, the SQL Database item, Real-Time Intelligence enhancements, Fabric MCP integration, evolving Direct Lake and Prep-for-AI features).
• Partner with the Microsoft CDAO Fabric Enablement engagement to bring product roadmap insights back into Vanguard's implementation.
• Contribute to the Workplace AI and enterprise Data roadmap and operating model, and partner with champions and train-the-trainer initiatives to translate engineering work into adoption outcomes.

Required Qualifications And Skills

• 8\+ years of professional software / data / platform engineering experience, with 5\+ years building production data solutions on the Microsoft and / or Azure data stack.
• Hands-on production experience with at least three of: Microsoft Fabric (Lakehouse, Warehouse, Pipelines, Notebooks, Real-Time Intelligence), Azure Synapse, Azure Data Factory, Databricks, Power BI semantic models, Azure SQL / SQL Server.
• Strong skills in SQL, PySpark, and KQL — the core Fabric language trio — and comfort moving between batch, streaming, and interactive analytics workloads.
• Demonstrable experience designing and shipping CI/CD for data platforms: Git workflows, automated deployment, environment promotion, secret-less authentication, and infrastructure-as-code.
• Working knowledge of Terraform (preferred) or Bicep for cloud platform automation, including provider versioning, state management, and policy-as-code patterns.
• Experience implementing security and compliance controls in a regulated environment: Purview, Sentinel, Defender, Conditional Access, MIP, DLP, RBAC, RLS / CLS / OLS, dynamic data masking.
• Identity fluency with Entra ID (Azure AD) and federated IdPs (Okta preferred); experience with service principals, managed identities, and Workload Identity Federation.
• Experience working in financial services, healthcare, or another heavily regulated environment, or a credible plan to come up to speed quickly.
• Bachelor's degree in Computer Science, Engineering, or equivalent practical experience.

Preferred Attributes

• DP-700 (Microsoft Certified: Fabric Data Engineer Associate) required or in-progress within 6 months of hire; DP-600 (Fabric Analytics Engineer Associate) and AZ-305 (Azure Solutions Architect Expert) preferred.
• Hands-on experience with the Microsoft fabric-cicd Python library and the microsoft/fabric Terraform provider.
• Experience operating a Fabric Center of Excellence, Power BI CoE, or comparable data-platform CoE.
• Experience with cross-cloud data integration patterns (OneLake ↔ AWS S3, mirroring, shortcuts) and BCDR for analytics platforms at enterprise scale.
• Experience configuring Prep for AI on semantic models and partnering with AI / agent engineers on certified data-product handoffs.
• Background contributing to internal communities of practice, champions networks, or developer enablement programs.
• Prior experience as a hands-on engineer in a Fusion Team (engineers \+ product \+ data \+ analysts) or Data / AI Center of Excellence model.
• Additional vendor certifications welcomed but not required: AZ-204, SC-100, DP-203 (legacy, retired March 2025 but still relevant context).

Special Factors

Sponsorship

Vanguard is not offering visa sponsorship for this position.

About Vanguard

At Vanguard, we don't just have a mission—we're on a mission.

To work for the long-term financial wellbeing of our clients. To lead through product and services that transform our clients' lives. To learn and develop our skills as individuals and as a team. From Malvern to Melbourne, our mission drives us forward and inspires us to be our best.

How We Work

Vanguard has implemented a hybrid working model for the majority of our crew members, designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.