Senior Data Infrastructure & Streaming Engineer

Project overview

Want to make an impact? We offer that

Due to continued growth, we are currently hiring a Senior Data Infrastructure & Streaming Engineer to join our team. emagine is a dynamic, fast-growing international consulting and technology company, founded on real-world experience, focused on custom-fit solutions. We have the proven people and expertise to complete complex projects, while delivering lasting knowledge-transfer benefits to empower our partners for the next challenge.

Are you intrigued? Do you want to learn more?

A snapshot of your key responsibilities as a Senior Data Infrastructure & Streaming Engineer would be:

• Design, build and support highly available Apache Kafka and Apache Spark platforms across multi-data centre environments.
• Develop and maintain real-time data streaming and large-scale data processing solutions.
• Automate infrastructure deployment and configuration using Terraform, Ansible and Kubernetes.
• Integrate streaming platforms with enterprise systems, databases, data lakes and microservices.
• Implement security controls including authentication, authorisation and encryption.
• Monitor, maintain and optimise Kafka and Spark clusters to ensure performance, scalability and reliability.
• Implement observability solutions using monitoring, logging and alerting tools.
• Troubleshoot platform issues and support incident resolution and root cause analysis.
• Collaborate with infrastructure, security and engineering teams to deliver enterprise-grade data platforms.

Requirements

Desirable Experience

So are you ready to join our team?

It’s important to remember, emagine is an Equal Employment Opportunity Employer; we value bringing together individuals from diverse backgrounds to develop innovative solutions for our customers.

Opportunity type: Contract (12 months initially), Dublin (Hybrid)

• Strong hands-on experience administering and supporting Apache Kafka in enterprise environments.
• Proven experience designing, deploying and optimising Apache Spark clusters and workloads.
• Experience with Kubernetes, Docker and containerised platforms.
• Strong experience with Infrastructure as Code and automation tools including Terraform and Ansible.
• Strong Linux administration experience (RHEL and/or Ubuntu).
• Experience integrating data platforms with enterprise applications, databases and data lakes.
• Knowledge of security, authentication and access management technologies including LDAP, Kerberos, OAuth2 and TLS.
• Experience implementing monitoring and observability solutions using tools such as Prometheus, Grafana and ELK.
• Strong understanding of high-availability, disaster recovery and performance tuning principles.
• Experience supporting multi-data centre or geographically distributed environments.
• Experience with Proxmox, VMware or other virtualised infrastructure platforms.
• Knowledge of enterprise storage technologies including SAN and NVMe storage.
• Experience with Keycloak, FreeIPA or Active Directory integrations.
• Experience working within large-scale data, analytics or regulated environments.

Project details

• Start: flexible
• Duration: 10-12 months
• Remote: hybrid
• Location: Dublin, Ireland, IE
• Languages: English