Lead Data Engineer

Purpose of Position

The Lead Data Engineer is a hands-on technical leader responsible for architecting, developing, and guiding the implementation of modern data solutions in a cloud-based environment. This role provides technical leadership to Senior Data Engineers and Senior Analytics Engineers, ensuring best practices in data engineering, analytics, and reporting. The Lead Data Engineer plays a key role in strategic initiatives, driving the design and delivery of scalable data platforms, and enabling actionable insights through advanced Power BI dashboards and reports.

Tasks and Responsibilities

• Serve as the technical lead on data engineering and analytics initiatives, ensuring alignment with architectural standards and business goals
• Provide technical mentorship and guidance to Senior Data Engineers and Senior Analytics Engineers
• Design and implement scalable ETL/ELT pipelines using Azure Data Factory (ADF), Databricks, and DBT
• Lead the development of Data Vault 2.0 models and ensure consistency across projects
• Architect and orchestrate data workflows using Databricks LakeFlow, Apache Airflow, and ADF
• Oversee data ingestion and replication strategies using tools such as Fivetran, SQDR, Rivery, or custom Python scripts
• Develop and optimize Power BI dashboards, reports, and semantic models to support business intelligence and self-service analytics
• Collaborate with business stakeholders, architects, and analysts to translate requirements into technical solutions
• Ensure data quality, governance, and security standards are met across all data products
• Lead code reviews, enforce development standards, and promote best practices in data engineering and analytics
• Support CI/CD automation, DevOps practices, and version control for data pipelines and reporting assets
• Troubleshoot and resolve complex data issues in production and development environments
• Contribute to enterprise data strategy and participate in architectural review boards
• Maintain documentation for data architecture, pipelines, and reporting solutions
• Infrequent travel as needed

Minimum Education

• Bachelor's degree from an accredited institution or equivalent industry experience

Minimum Experience

• 12+ years of experience in data engineering or software development roles
• 3+ years in a technical leadership or lead engineer capacity
• 5+ years of experience with Azure Data Factory and Azure Data Lake Storage
• 3+ years of experience with Databricks, Delta Live Tables, and Unity Catalog
• 3+ years of experience with Power BI dashboard and report development
• 2+ years of experience with DBT and Apache Airflow
• Experience with Data Vault 2.0 modeling (certification preferred)
• Strong experience with Python, PySpark, and SQL

Knowledge, Skills, Abilities

• Deep expertise in cloud data platforms, especially Azure and Databricks
• Strong understanding of ELT/ETL, data modeling, and data warehousing principles
• Proven ability to lead and mentor technical teams in a collaborative environment
• Advanced skills in Power BI, including DAX, Power Query, and data modeling
• Experience with data ingestion and replication tools (e.g., Fivetran, SQDR, Rivery)
• Proficiency in Python, PySpark, and SQL for data transformation and automation
• Experience orchestrating workflows using Apache Airflow and Databricks LakeFlow
• Familiarity with DevOps, CI/CD pipelines, and version control systems (Git, TFS)
• Strong communication and stakeholder engagement skills
• Knowledge of data governance, metadata management, and compliance frameworks
• Experience with Agile/Scrum methodologies
• Exposure to AI/ML tools and concepts within Databricks is a plus
• Experience in Education or Aviation industries is a plus

Physical Demands and Work Environment

The physical demands and work environment described here are representative of those that must be met and/or encountered by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. While performing the duties of this job, the employee is regularly required to use hands to finger, handle, or feel; reach with hands and arms; and communicate. The employee may be required to stand, walk, and sit. Specific vision abilities required by this job include the ability to view monitors, technical documents, and reference material. The employee must occasionally lift or move up to 25 pounds. The noise level in the work environment is usually low to moderate.