Job Type: Full Time

Job Category: IT

Job Description

Data & Analytics Solutions Architect

Location: Toronto, ON
Work Model: Hybrid
Job Type: Full Time

Primary Skills

Enterprise Data Architecture, Open-Source Data Platforms, Apache Spark/Kafka

Job Description

We are seeking an experienced Data & Analytics Solutions Architect with 14+ years of experience in enterprise data architecture, open-source data platforms, distributed data processing, and data engineering. The ideal candidate will lead the design and modernization of scalable, secure, and highly available data platforms supporting batch processing, real-time streaming, analytics, and AI/ML workloads.

This role requires deep technical expertise across open-source data technologies, modern lakehouse architectures, data integration, DataOps, and enterprise governance. The successful candidate will collaborate with business and technology stakeholders to define end-to-end architecture, establish technical standards, and deliver production-ready data and AI solutions aligned with business objectives.

Key Responsibilities

• Engage with business and technology stakeholders to understand data strategy, platform requirements, architecture goals, and transformation priorities.
• Design and define end-to-end architectures for enterprise open-source data platforms supporting batch, streaming, analytics, and AI workloads.
• Lead modernization initiatives involving data lakes, lakehouse platforms, distributed processing frameworks, and enterprise data ecosystems.
• Architect scalable, secure, highly available, and resilient data platforms using open-source technologies and industry best practices.
• Design data ingestion, integration, streaming, storage, and consumption patterns using Apache Spark, Kafka, Flink, NiFi, and Trino.
• Develop modern data lakehouse architectures using Apache Iceberg, Hudi, Delta Lake, object storage, and open table formats.
• Establish standards for data governance, security, metadata management, lineage, privacy, and observability.
• Evaluate and recommend open-source technologies based on business requirements, scalability, interoperability, supportability, and total cost of ownership.
• Define architecture standards, reference architectures, reusable frameworks, CI/CD practices, and DataOps operating models.
• Provide technical leadership across data modeling, engineering, platform implementation, migration, testing, deployment, and production readiness.
• Drive platform performance optimization, capacity planning, reliability engineering, failure recovery, and operational excellence.
• Conduct architecture reviews, create High-Level Design (HLD) and Low-Level Design (LLD) documents, assess non-functional requirements (NFRs), and guide risk mitigation.
• Support solution design, RFP responses, effort estimation, technical presentations, and client workshops.
• Assess business use cases and recommend appropriate approaches across BI, rules-based systems, statistical analysis, forecasting, machine learning, and GenAI.
• Define measurable business KPIs, baselines, acceptance criteria, and success metrics for analytics and AI initiatives.
• Integrate data, ML, and GenAI reference architectures covering data sources, feature pipelines, models, LLMs, vector stores, serving layers, and enterprise applications.
• Support AI/ML and LLM workload deployment on Kubernetes in collaboration with platform and operations teams.
• Establish MLOps and LLMOps practices, including experiment tracking, model registries, feature stores, automated quality gates, and data/model/prompt versioning.
• Support AI observability and monitoring for data drift, model drift, LLM quality, tracing, latency, availability, and alerting.
• Mentor architects and engineers, support technical assessments and hiring, and develop reusable accelerators, playbooks, and engineering best practices.

Required Skills & Experience

Enterprise Data Architecture

• 14+ years of experience in enterprise data architecture, open-source data platforms, distributed data processing, and data engineering.
• Proven experience designing enterprise solutions using open-source technologies across the end-to-end data-to-insights value chain.
• Strong experience in architecture definition, HLD/LLD creation, NFR assessment, migration planning, technical governance, and production readiness.
• Experience leading large-scale data platform modernization and cloud migration initiatives.

Open-Source Data Technologies

• Hands-on expertise in Apache Spark, PySpark, Kafka, Flink, Airflow, NiFi, and Trino.
• Strong programming and query skills in Python, Scala, Java, and SQL.
• Experience with Apache Iceberg, Hudi, Delta Lake, object storage, and open table formats.
• Knowledge of event-driven architectures, Change Data Capture (CDC), APIs, schema management, and enterprise integration patterns.
• Experience with PostgreSQL, Cassandra, MongoDB, and OpenSearch, including data modeling, indexing, workload management, and high availability.

Data Engineering & DataOps

• Strong understanding of batch, streaming, and real-time data processing architectures.
• Experience with Airflow orchestration, CI/CD, automated testing, observability, metadata management, data lineage, and platform operations.
• Expertise in performance tuning, workload optimization, failure recovery, and distributed processing.
• Experience defining reusable engineering frameworks and production deployment standards.

Security, Governance & Compliance

• Strong knowledge of RBAC, identity management, encryption, privacy controls, auditability, metadata governance, and vulnerability management.
• Understanding of enterprise security, resilience, and operational requirements, particularly within Banking, Financial Services, and Insurance (BFSI).

AI/ML & GenAI Architecture

• Understanding of AI/ML platform architecture, including feature engineering, model training, batch and real-time inference, and enterprise integration.
• Knowledge of MLOps and LLMOps practices using tools such as MLflow and Kubeflow.
• Exposure to model registries, feature stores, experiment tracking, prompt versioning, and automated quality gates.
• Understanding of AI observability, data/model drift monitoring, LLM evaluation, tracing, latency, and operational runbooks.
• Ability to assess and recommend BI, analytics, ML, or GenAI solutions based on feasibility, cost, risk, and measurable business outcomes.

Leadership & Communication

• Strong stakeholder management, technical leadership, architecture governance, and problem-solving skills.
• Experience supporting solutioning, RFP responses, effort estimation, and client-facing architecture discussions.
• Ability to mentor technical teams and establish architecture standards and best practices.

Preferred Skills

• Experience with Kubernetes-based deployment of AI/ML and LLM workloads.
• Relevant certifications in cloud data engineering, Kubernetes, enterprise architecture, data management, security, DevOps, or open-source data platforms.
• Experience collaborating with platform vendors, open-source communities, and enterprise engineering teams.
• Bachelor's degree in Computer Science, Engineering, Information Systems, Data Management, or a related discipline.

Required Skills