Job Type: Contract

Job Category: IT

Job Description

Role- AWS Cloud Lead

Location: Wilmington, DE / Plano, TX - Onsite
Contract

Key Responsibilities

• Data Pipeline Development Design| develop| and implement efficient and scalable ETLELT processes using Apache Spark with Java| integrating data from diverse sources into data lakes (e.g.| S3) and data warehouses (e.g.| Redshift).
• AWS Service Utilization Leverage a range of AWS services for data storage| processing| and analytics| including but not limited to S3| Redshift| Glue| EMR| Lambda| Kinesis| and DynamoDB.
• Performance Optimization Optimize Spark applications and data pipelines for performance| cost-efficiency| and reliability| including tuning Spark configurations and utilizing appropriate AWS resources.
• Data Modeling and Architecture Design and implement data models for structured and unstructured data| and contribute to the overall data architecture strategy within an AWS environment.
• Collaboration and Communication Work closely with data scientists| analysts| and other stakeholders to understand data requirements and deliver solutions that meet business needs.
• Monitoring and Troubleshooting Implement monitoring solutions for data pipelines and infrastructure| and troubleshoot issues to ensure data quality and system stability.
• Security and Governance Implement data security measures and adhere to data governance policies within the AWS ecosystem.
• Documentation Create and maintain comprehensive documentation for data engineering processes| designs| and deployments.

Required Skills and Qualifications:

• Programming Expertise Strong proficiency in Java| with experience in developing Spark applications.
• Big Data Technologies In-depth knowledge and hands-on experience with Apache Spark| including Spark SQL| Data Frames| and RDDs.
• Cloud Computing Extensive experience with AWS services| particularly those related to data engineering (S3| Redshift| Glue| EMR| Kinesis| Lambda).
• Data Warehousing and Lakes Experience with data warehousing concepts and technologies (e.g.| Redshift| Snowflake)| and building data lakes on S3.
• Databases Proficiency in SQL and experience with both relational and NoSQL databases.
• Strong understanding and practical experience in designing and implementing ETL/ELT processes.
• Problem-Solving Excellent analytical and problem-solving skills| with the ability to troubleshoot complex data issues.

Communication Strong communication

Required Skills