Data Engineer

$25 Billion Manufacturing Business.

Databricks Engineer position - must have 3\+ years Databricks.

Here's the rewritten job description focused on Databricks engineering rather than architecture:

Platform \& Engineering

• Develop, optimize, and maintain scalable data pipelines and workloads on Primoris' Databricks Lakehouse platform, following established Medallion layer patterns across ingestion, transformation, and serving layers
• Build and deploy production-grade Databricks solutions (including serverless where appropriate) supporting batch, streaming, ML, and GenAI workloads, with a focus on performance, cost efficiency, security, and reliability
• Implement and maintain solutions leveraging Unity Catalog, Delta Lake, and Medallion architecture patterns to support both analytical and operational use cases
• Collaborate with enterprise architecture, security, infrastructure, and application teams to ensure Databricks implementations align with broader platform and integration standards

Delivery \& Operations

• Develop, deploy, and migrate priority use cases and workloads into production on the Databricks Lakehouse
• Serve as a primary point of contact for technical challenges related to development and production workloads, including triage, root cause analysis, and escalation coordination
• Work effectively within hybrid delivery teams — internal employees, consulting partners, and offshore resources — ensuring clear accountability and consistent engineering outcomes
• Uphold and contribute to engineering standards covering CI/CD, automated testing, observability, monitoring and alerting, and incident runbooks
• Build, review, and maintain workflows, jobs, and streaming pipelines (e.g., Auto Loader, DLT, Structured Streaming) with a focus on robustness, recoverability, and SLA adherence
• Support the adoption of Databricks product innovations, private previews, and platform upgrades following change management and release best practices

Data Engineering

• Write and review high-quality, performant multi-language notebooks and modules (SQL, Python/PySpark) across data engineering and data science/AI/ML use cases
• Design and implement streaming and batch data pipelines using Auto Loader, Delta Live Tables (DLT), and Structured Streaming, handling structured, semi-structured, and unstructured data at scale
• Apply performance tuning techniques across Spark jobs, Delta tables, and cluster configurations to optimise throughput, cost, and reliability

Governance, Security \& Compliance

• Implement Row-Level Security (RLS), column masking, data encryption/decryption, and other data protection controls using Unity Catalog and cloud-native Azure capabilities
• Work with security and compliance stakeholders to implement data governance models, access patterns, data classification, auditing, and regulatory controls as required
• Ensure all implementations align with Primoris' security, networking, and compliance standards for Azure and Databricks

Stakeholder Collaboration \& Communication

• Participate in discovery workshops and requirements sessions with business and technical stakeholders to understand data flows, pain points, and use case priorities
• Translate requirements into clear implementation plans with defined tasks, dependencies, and delivery milestones
• Provide regular updates on progress, risks, and blockers to leads and stakeholders, with practical options and recommendations

Engineering Standards, Mentoring \& Reuse

• Contribute to and follow pragmatic engineering standards covering coding patterns, data modelling, observability, cost optimisation, and workspace conventions
• Support junior and mid-level engineers through code reviews, knowledge-sharing sessions, and pairing on complex tasks
• Build and maintain reusable engineering assets — starter notebooks, pipeline templates, utility libraries, and runbooks — to accelerate delivery and improve consistency
• Take proactive ownership of your workstreams: surface risks early, drive decisions within your scope, and remove blockers to keep delivery on track

Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, Data Science, or equivalent practical experience
• 5\+ years of hands-on experience in data engineering or analytics engineering roles, with 2\+ years working on Databricks or a comparable distributed data platform
• Proven hands-on experience building and maintaining Databricks pipelines using the Medallion architecture across ingestion, transformation, and serving layers
• Hands-on expertise with Databricks on Microsoft Azure (required), including familiarity with Azure security, networking, identity, and storage services
• Strong proficiency in Python (including PySpark) and SQL, with demonstrated ability to write performant, production-ready code for large-scale data processing
• Hands-on experience implementing Auto Loader, Delta Live Tables (DLT), and Structured Streaming solutions for structured, semi-structured, and unstructured data
• Working experience with Unity Catalog capabilities including data governance, RBAC, lineage, auditing, and secure data sharing
• Familiarity with implementing data security controls in Databricks — RLS, column masking, encryption — in alignment with enterprise security policies
• Experience working in CI/CD-driven delivery environments with automated testing, version control, and observability tooling
• Evidence of continuous learning and staying current with the Databricks Lakehouse platform, Azure services, and modern data engineering patterns

Nice to have:

• One or more Databricks certifications (e.g., Databricks Certified Data Engineer Associate/Professional)
• Experience with cloud data warehouse technologies such as Synapse, Snowflake, BigQuery, or Redshift
• Familiarity with infrastructure-as-code tools (e.g., Terraform) for managing Databricks workspace configuration