Job Type: Contract

Job Category: IT

Job Description

Role: Data Engineer (MSFT Project)

Location: Remote (must work in PST hours)

Client Environment:

Microsoft ecosystem (Redmond, WA). The role is aligned with Azure data engineering, automation, and distributed data processing workloads.

Core Responsibilities & Skill Insights

1. Azure Technologies (Synapse, IaaS VM, SQL Server, ADLS Gen2)
• You’ll manage and integrate Azure Synapse Analytics for data warehousing, SQL Server (on both Azure and On-prem), and Azure Data Lake Storage Gen2 for large-scale storage.
• IaaS VM refers to setting up or maintaining virtual machines hosted in Azure.
2. Azure Automation with PowerShell
• Building and maintaining runbooks — automated scripts used for system and data tasks (like database backups, maintenance, or pipeline restarts).
• Heavy scripting using PowerShell for Azure administration.
3. SQL Backup/Restore Operations (Azure & On-Prem)
• Hands-on with SQL database management, especially performing backups and restores across Azure Storage Accounts and on-prem environments.
4. Distributed Computing / Spark Processing
• Must know how to process large datasets using Apache Spark, ensuring scalability and performance optimization in data pipelines.
5. Fabric Reporting (Preferred)
• Microsoft Fabric is the new end-to-end analytics platform; prior exposure is a plus but not mandatory.
6. Microsoft Experience (Preferred)
• If you’ve worked at Microsoft or on Microsoft projects before, it’s a strong advantage (preferred for vendor alignment and process familiarity).
7. Scala Programming & OOPs Concepts
• Development using Scala for Spark-based ETL pipelines; solid grasp of object-oriented principles.
8. Data Storage Formats (Parquet, Delta Lake)
• Knowledge of modern data storage formats used in big data processing for performance and versioning.

Core Tech Stack Summary

Azure Synapse | Azure SQL | ADLS Gen2 | PowerShell | Spark | Scala | Parquet | Delta Lake | Azure Automation

Required Skills