About Quabyt
Quabyt delivers GenAI, cloud, and data engineering solutions for scaling businesses. Our Data Engineering practice builds robust ETL/ELT pipelines, data platforms, and analytics foundations that power AI and product teams.
Role Summary
We’re looking for a hands-on Data Engineer with Azure experience to design, build, and operate modern data platforms using Microsoft Fabric, Azure Data Factory, Synapse Analytics, and lakehouse architectures. You’ll leverage zero ETL integrations, real-time analytics, and unified data platforms to support analytics and GenAI use cases. You’ll work across the stack (ingestion, transformation, semantic modeling, querying, monitoring, and cost optimization) and collaborate with product, ML, and cloud teams.
Key Responsibilities
- Design, implement, and maintain modern data platforms using Microsoft Fabric (OneLake, Data Factory, Synapse Data Engineering, Data Warehouse).
- Build lakehouse architectures with Delta Lake format on OneLake; implement medallion architecture (bronze, silver, gold layers).
- Leverage zero ETL integrations (Fabric Mirroring, Azure Synapse Link, Power BI Direct Lake) to minimize data movement and latency.
- Ingest data from relational databases, event streams, APIs, and files using Azure Data Factory, Fabric Data Pipelines, and Event Hubs.
- Develop and maintain data pipelines using Fabric Notebooks (PySpark, Spark SQL), Data Factory dataflows, and orchestration.
- Build semantic models and implement data mesh principles with Fabric domains and workspaces.
- Optimize query performance using Delta Lake optimization (Z-ordering, compaction, liquid clustering), partitioning strategies, and Fabric compute optimization.
- Implement data quality checks, monitoring, and alerting using Fabric monitoring, Azure Monitor, and data validation frameworks.
- Define and enforce data governance, security (Microsoft Purview, RBAC, row-level security), encryption, and compliance policies.
- Collaborate with ML engineers, analysts, and Power BI developers to deliver end-to-end data solutions.
- Contribute to architecture decisions, best practices, and documentation; mentor junior engineers.
Required Skills
- 2+ years of professional data engineering experience; demonstrable experience building data platforms on Azure.
- Hands-on expertise with Microsoft Fabric (OneLake, Lakehouses, Data Pipelines, Notebooks) or Azure Synapse Analytics in production.
- Strong SQL and PySpark/Spark SQL skills; experience optimizing analytical queries and distributed processing.
- Experience with Delta Lake format, lakehouse architecture, and medallion design patterns (bronze/silver/gold).
- Hands-on experience with Azure Data Factory (copy activities, dataflows, pipelines, triggers) and orchestration.
- Knowledge of zero ETL patterns (Fabric Mirroring, Synapse Link, Direct Lake mode) and real-time analytics.
- Familiarity with data modeling for analytics and ML (star schema, dimensional modeling, semantic models).
- Solid understanding of Azure security (Entra ID, RBAC, managed identities, encryption, Key Vault) and governance (Microsoft Purview).
- Experience with monitoring/logging tools (Azure Monitor, Log Analytics, Application Insights, or similar).
- Excellent communication, problem solving, and collaboration skills.
Preferred Skills
- Experience with real-time streaming (Azure Event Hubs, Kafka on Azure, Fabric Eventstreams, KQL Database).
- Familiarity with Fabric Real-Time Intelligence, Power BI integration, and Direct Lake mode.
- Experience with Azure Databricks, Azure Synapse Spark pools, or Fabric Spark compute.
- Knowledge of data mesh principles, domain-oriented data ownership, and Fabric workspace architecture.
- Experience supporting ML pipelines (Azure ML, MLflow, feature stores) and AI workloads.
- Python programming for data engineering, transformations, and automation (Azure SDK, Fabric APIs).
- Experience with Infrastructure as Code (Bicep, Terraform) and CI/CD for data platforms (Azure DevOps, GitHub Actions).
- Familiarity with Microsoft Purview for data cataloging, lineage, and compliance.
- Experience working at a consulting or product engineering firm.
Personal Qualities
- Strong problem-solving and analytical skills
- Excellent communication and teamwork abilities
- Self-motivated and able to work independently when required
- Passionate about learning new technologies and keeping up with industry trends
- Detail-oriented with a focus on writing clean, efficient, and maintainable code
What We Offer
- Opportunity to work on GenAI, cloud-first projects for diverse clients.
- Collaborative engineering culture with mentoring and career growth.
- Competitive salary and benefits (location-adjusted).
- Flexible work arrangements.