Cloud Administrator

Project overview

This contract Cloud Administrator role supports and optimizes AWS cloud and data platforms, partnering with data engineering teams to keep production workloads reliable, scalable, secure, and cost-efficient. Core requirements include hands-on AWS and Amazon EMR administration, Spark and Hadoop ecosystem support, Redshift and Airflow operations, and EKS/Kubernetes administration. The role also requires Terraform, Docker, Helm, CI/CD, Python and Bash automation, plus production troubleshooting, monitoring, capacity planning, and cost optimization.

The stated experience band is 4–6 years, and the location is Pune; the work mode is not specified. The description names Jenkins, GitLab CI/CD, GitHub Actions, AWS-native CI/CD, CloudWatch, Prometheus, and Grafana as tool options rather than identifying one exclusive implementation.

Responsibilities

• Administer, manage, and optimize Amazon EMR clusters, including provisioning, scaling, upgrades, patching, performance tuning, troubleshooting, and capacity planning.
• Manage Big Data technologies including Apache Spark, Hive, Hadoop, HDFS, YARN, Tez, and Hive Metastore.
• Monitor and optimize Spark workloads covering executor sizing, memory management, dynamic allocation, shuffle optimization, and job performance.
• Troubleshoot Spark jobs, Hive queries, HDFS storage, YARN resource allocation, and EMR cluster issues.
• Configure EMR Instance Fleets/Groups, Managed Scaling, auto-termination, and cost optimization strategies.
• Plan and execute EMR version upgrades and migrations while maintaining workload compatibility and platform stability.
• Administer and optimize Amazon Redshift, including WLM, query tuning, schema design, storage optimization, capacity planning, and data loading.
• Manage Apache Airflow DAG scheduling, monitoring, troubleshooting, and workflow optimization.
• Collaborate with data engineering teams to design, automate, and optimize ETL/ELT pipelines using Airflow, Spark, Redshift, and AWS services.
• Design, implement, and manage AWS infrastructure, including EC2, S3, RDS, EKS, Lambda, VPC, Route 53, ALB/NLB, CloudFront, EFS, and IAM.
• Design, deploy, upgrade, scale, and troubleshoot Amazon EKS clusters and node groups.
• Manage Kubernetes workloads including Deployments, StatefulSets, Services, ConfigMaps, Secrets, Ingress, and RBAC.

Requirements

• AWS cloud infrastructure administration
• Amazon EMR cluster administration
• Apache Spark workload operations and tuning
• Hadoop ecosystem administration, including HDFS, YARN, Hive, Tez, and Hive Metastore
• Amazon Redshift administration and query optimization
• Apache Airflow DAG and workflow operations
• Amazon EKS and Kubernetes administration
• Terraform infrastructure as code
• Docker and Helm application deployment
• CI/CD pipeline design and operations
• Python and Bash operational automation
• Production operations, monitoring, troubleshooting, capacity planning, and cost optimization

Project details

• Start: flexible
• Remote: hybrid
• Location: India, IN