Job Type: Contract

Job Category: IT

Job Description

Role:  Databricks Platform Admin
Location: St. Louis, Missouri (Hybrid – 3 days onsite/week)
Contract

Databricks Platform Admin – Azure Native Data Platforms, Technology

What you'll do

·  Design, implement, and manage scalable and secure Databricks platforms to ensure optimal performance and reliability.

·  Create and manage Databricks workspaces and clusters, including right-sizing and lifecycle management of resources.

·  Automate infrastructure provisioning and deployment using Infrastructure-as-Code tools like Terraform, GitLab CI/CD, and scripting languages (Python, Bash, PowerShell).

·  Monitor platform health, performance, and cost by building dashboards and integrating with monitoring tools.

·  Troubleshoot and resolve issues related to cluster configuration, job failures, and performance bottlenecks within Databricksenvironments.

·  Collaborate with cross-functional teams including data scientists, ML engineers, and product teams to deploy and maintain data pipelines and services on Databricks.

·  Implement security and governance policies, including user management, access controls, and compliance adherence using Unity Catalog and other Databricks security features.

·  Optimize compute and storage resources for cost efficiency without compromising performance, leveraging features like Delta Lake and Photon.

·  Stay updated with new Databricks features and coordinate with product teams to evaluate and adopt them.

·  Document platform architecture, best practices, troubleshooting guides, and provide mentorship to junior engineers on platform usage and optimization

What you'll need

·  Required hands-on experience in architecting, deploying, managing, and supporting Azure Databricks Platform, Azure Datalake and related solutions.

·  Required hands-on experience working with Cloud provider tools and architecture (Azure) .

·  Required hands-on experience using scripting languages and command-line (CLI) tools such as Ansible, SQL, Terraform, Azure CLI, Databricks CLI, PowerShell and/or Bash.

·  Preferred experience in management and tuning of data platforms like Azure ADLS storage and Databricks.

·  Knowledge working with and maintaining CI/CD pipelines and versioning tools such as Git preferred.

·  Knowledge with a broad set of data ingestion and data pipeline patterns, and associated tools like (Qlik Replicate, Qlik Compose, Databricks, Kafka, Matillion ELT and Snowflake Services)

·  Experience with working in an agile product development team

·  Excellent communication skills and ability to articulate system designs and patterns to varying levels of leadership.

Required Skills