2 Candidate Submittal Slots, New High Level Policy
MSP Owner: Rob Finton
Location: Atlanta, GA - Onsite, local candidates preferred
Duration: 6 months
skill id: 10728208
Competencies: 10+ years experience
Digital : Google Data Engineering
integration/data engineering - 10+ years overall
Google Cloud Platform (GCP) - 5+ years
GC/USC preferred - Atlanta local candidates
Job Title Integration Lead Developer (GCP Data Engineering)
Relevant Experience: 10+ years overall integration/data engineering experience with 5+ years leading cloud-based integrations and data platforms on Google Cloud Platform (GCP).
Technical/Functional Skills Data Engineering & Integration
• Data ingestion, transformation, orchestration, and integration patterns (batch + real-time + event-driven)
• Strong understanding of ETL/ELT, data quality checks, schema evolution, and metadata management
• API-led connectivity and microservices-based integrations
Google Cloud Platform (GCP)
• BigQuery (BQ): data modeling, partitioning/clustering, performance tuning, cost optimization, materialized views, BQ scripting
• Cloud Storage (GCS): landing zones, lifecycle policies, encryption, retention
• Cloud Run: containerized integration services, scaling, secure service-to-service auth
• Pub/Sub: event-driven integrations, ordering, retries, DLQ patterns, idempotency
• Cloud Composer / Airflow: DAG design, scheduling, sensors, backfills, SLAs, alerting
• Cloud Logging/Monitoring: operational observability, dashboards, alerts
Orchestration & Pipeline Engineering
• Airflow DAG development standards, reusable operators/hooks, secrets management, environment-based configurations
• Dependency management, failure handling, retries, idempotent design, replay strategies
Programming / Frameworks
• Python (preferred) for DAGs, data processing, and integration services
• SQL (advanced) for BQ transformations and optimization
• Optional/plus: Java / Spring Boot for microservices; dbt for transformation workflows
DevOps / Platform
• CI/CD for data pipelines and services (Cloud Build / Jenkins / GitHub Actions)
• Containerization (Docker), artifact management
• IaC (Terraform preferred) for GCP resource provisioning
• Secrets and access management (IAM, service accounts, workload identity)
Security & Governance
• Data security (PII/PHI masking), encryption, least privilege IAM
• Auditability, lineage awareness, and compliance-friendly design
Experience Required
• Demonstrated ability to design and implement scalable integration and data platforms on GCP, supporting high-volume batch and real-time workloads.
• Strong hands-on expertise with BigQuery, including performance tuning, cost controls, and dimensional/data vault-style modeling as applicable.
• Deep experience building and operating Airflow / Cloud Composer DAGs with enterprise scheduling, SLAs, alerting, and recovery procedures.
• Strong experience implementing event-driven architectures using Pub/Sub, including replay, deduplication, and DLQ patterns.
• Solid production experience with Cloud Run services for integration/microservice workloads and secure connectivity to internal/external systems.
• Proven ability to integrate across enterprise systems using REST APIs, file-based integrations, and messaging patterns.
• Ability to produce solution designs, technical specifications, runbooks, and operational SOPs.
Roles & Responsibilities
- Lead end-to-end integration delivery for data and application integration initiatives on GCP (batch + streaming + event-driven).
- Architect and develop Airflow DAGs (Cloud Composer) for ingestion and transformation workflows with standardized templates, reusable components, and strong operational controls.
- Build scalable BigQuery data pipelines, optimize query performance, and implement cost governance (partitioning, clustering, query optimization, workload segregation).
- Develop and maintain Cloud Run services for integration APIs and workflow-trigger services; enforce secure IAM-based access controls.
- Implement Pub/Sub topics/subscriptions for event-based integration; design for resiliency (retries, DLQ, idempotency, ordering if required).
- Establish and enforce best practices for logging, monitoring, alerting, and incident triage; participate in RCA and preventive actions.
- Collaborate with architects, product owners, and downstream consumers to define interfaces, SLAs, data contracts, and operational support models.
- Support release planning, deployment, post-release monitoring, and ongoing production support for integrations and pipelines.
- Mentor engineers, conduct code/design reviews, and ensure engineering excellence and documentation completeness.
- Drive continuous improvement through automation, standardization, and reduction of manual operational steps.
Generic Managerial Skills
• Ability to lead and coordinate onsite/offshore teams, manage deliverables, and ensure quality and timelines.
• Strong stakeholder management, communication, and cross-team collaboration skills.
• Self-starter with ownership mindset, strong analytical/problem-solving capability, and ability to work with minimal supervision.
Education BS/MS in Computer Science / Engineering (or equivalent)., Project Code :
MSP Owner: Rob Finton
Location: Atlanta, GA - Onsite, local candidates preferred
Duration: 6 months
skill id: 10728208
Competencies: 10+ years experience
Digital : Google Data Engineering
integration/data engineering - 10+ years overall
Google Cloud Platform (GCP) - 5+ years
GC/USC preferred - Atlanta local candidates
Job Title Integration Lead Developer (GCP Data Engineering)
Relevant Experience: 10+ years overall integration/data engineering experience with 5+ years leading cloud-based integrations and data platforms on Google Cloud Platform (GCP).
Technical/Functional Skills Data Engineering & Integration
• Data ingestion, transformation, orchestration, and integration patterns (batch + real-time + event-driven)
• Strong understanding of ETL/ELT, data quality checks, schema evolution, and metadata management
• API-led connectivity and microservices-based integrations
Google Cloud Platform (GCP)
• BigQuery (BQ): data modeling, partitioning/clustering, performance tuning, cost optimization, materialized views, BQ scripting
• Cloud Storage (GCS): landing zones, lifecycle policies, encryption, retention
• Cloud Run: containerized integration services, scaling, secure service-to-service auth
• Pub/Sub: event-driven integrations, ordering, retries, DLQ patterns, idempotency
• Cloud Composer / Airflow: DAG design, scheduling, sensors, backfills, SLAs, alerting
• Cloud Logging/Monitoring: operational observability, dashboards, alerts
Orchestration & Pipeline Engineering
• Airflow DAG development standards, reusable operators/hooks, secrets management, environment-based configurations
• Dependency management, failure handling, retries, idempotent design, replay strategies
Programming / Frameworks
• Python (preferred) for DAGs, data processing, and integration services
• SQL (advanced) for BQ transformations and optimization
• Optional/plus: Java / Spring Boot for microservices; dbt for transformation workflows
DevOps / Platform
• CI/CD for data pipelines and services (Cloud Build / Jenkins / GitHub Actions)
• Containerization (Docker), artifact management
• IaC (Terraform preferred) for GCP resource provisioning
• Secrets and access management (IAM, service accounts, workload identity)
Security & Governance
• Data security (PII/PHI masking), encryption, least privilege IAM
• Auditability, lineage awareness, and compliance-friendly design
Experience Required
• Demonstrated ability to design and implement scalable integration and data platforms on GCP, supporting high-volume batch and real-time workloads.
• Strong hands-on expertise with BigQuery, including performance tuning, cost controls, and dimensional/data vault-style modeling as applicable.
• Deep experience building and operating Airflow / Cloud Composer DAGs with enterprise scheduling, SLAs, alerting, and recovery procedures.
• Strong experience implementing event-driven architectures using Pub/Sub, including replay, deduplication, and DLQ patterns.
• Solid production experience with Cloud Run services for integration/microservice workloads and secure connectivity to internal/external systems.
• Proven ability to integrate across enterprise systems using REST APIs, file-based integrations, and messaging patterns.
• Ability to produce solution designs, technical specifications, runbooks, and operational SOPs.
Roles & Responsibilities
- Lead end-to-end integration delivery for data and application integration initiatives on GCP (batch + streaming + event-driven).
- Architect and develop Airflow DAGs (Cloud Composer) for ingestion and transformation workflows with standardized templates, reusable components, and strong operational controls.
- Build scalable BigQuery data pipelines, optimize query performance, and implement cost governance (partitioning, clustering, query optimization, workload segregation).
- Develop and maintain Cloud Run services for integration APIs and workflow-trigger services; enforce secure IAM-based access controls.
- Implement Pub/Sub topics/subscriptions for event-based integration; design for resiliency (retries, DLQ, idempotency, ordering if required).
- Establish and enforce best practices for logging, monitoring, alerting, and incident triage; participate in RCA and preventive actions.
- Collaborate with architects, product owners, and downstream consumers to define interfaces, SLAs, data contracts, and operational support models.
- Support release planning, deployment, post-release monitoring, and ongoing production support for integrations and pipelines.
- Mentor engineers, conduct code/design reviews, and ensure engineering excellence and documentation completeness.
- Drive continuous improvement through automation, standardization, and reduction of manual operational steps.
Generic Managerial Skills
• Ability to lead and coordinate onsite/offshore teams, manage deliverables, and ensure quality and timelines.
• Strong stakeholder management, communication, and cross-team collaboration skills.
• Self-starter with ownership mindset, strong analytical/problem-solving capability, and ability to work with minimal supervision.
Education BS/MS in Computer Science / Engineering (or equivalent)., Project Code :