Job Type: Full Time

Job Category: IT

Job Description

Role: Data Ops Engineer

Location: Jersey City, NJ

FTE only

Job Description

Must Have Technical/Functional Skills

Technical Skills

• Strong hands-on experience with HDFS, Spark, Hive/Impala, and Python-based data processing.
• Expertise in debugging ingestion failures, pipeline issues, and performance bottlenecks.
• Knowledge of workflow orchestrators such as Airflow, Oozie, ADF.
• Experience with Git, artifact management, and CI/CD tools (Azure DevOps/Jenkins).
• Incident management: triage, RCA, remediation; ability to handle on-call rotation.
• Monitoring using Cloudera Manager, logs, and metrics with focus on SLA/MTTR.
• Strong Shell scripting skills and automation of operational tasks.
• Basic understanding of Azure/Databricks jobs, clusters, storage.

Functional Skills

• Strong analytical thinking and structured problem-solving.
• Ability to document runbooks and operational playbooks clearly.
• Effective communication with cross-functional teams including release management and offshore teams.
• Good understanding of production governance and stability requirements.

Roles & Responsibilities

• Monitor and troubleshoot data ingestion/transformation workflows across hybrid environments.
• Manage job schedules, track failures, and implement fixes to ensure smooth pipeline execution.
• Automate repetitive operational tasks and contribute to CI/CD improvements.
• Collaborate with platform engineers, release managers, and global teams for stable releases.
• Perform RCA, maintain SLAs, and continuously optimize performance.
• Maintain operational documentation, runbooks, and health-check procedures.

Required Skills