Senior Data Engineer

The opportunity

We are partnering with a global technology organisation supporting them in hiring a Senior Data Engineer to join a high-impact data platform team. This team sits at the core of a data-driven product, processing extremely high-volume, near-real-time data to power reporting, optimisation and decision-making systems. The environment is complex, fast-moving and built around large-scale distributed systems with strict performance and reliability expectations. This is a senior role suited to an engineer who enjoys owning systems, working close to production and contributing to the evolution of data infrastructure at scale.

The opportunity (continues): You’ll design, build and maintain reliable, scalable, enterprise-level data processing systems that support both existing platforms and new initiatives. The work involves operating under tight SLAs, ensuring data accuracy, consistency and availability while enabling near-real-time insight.

You’ll focus on optimising data pipelines and workloads across a modern stack, making efficient use of compute and storage resources. Transparency around data quality and system behaviour is central to the role, as is improving accessibility of data for analysts, data scientists and downstream consumers.

The team operates in a collaborative environment, combining deep technical work with mentorship and shared ownership. You’ll contribute to technical decision-making, tooling evaluation and long-term platform improvements while supporting more junior engineers.

What you’ll be working with

The data platform is built around a modern distributed ecosystem, including Python as a primary language, Spark and streaming technologies for near-real-time aggregation, and orchestration tools such as Airflow. Containerisation and Kubernetes are used to manage workloads, alongside strong monitoring and observability practices.

You’ll work with technologies spanning Kafka, distributed storage and query layers, SQL-based warehousing, and cloud-based analytics tooling. The environment includes both on-prem and cloud components, with exposure to ongoing platform evolution and migration initiatives.

What we’re looking for

This role suits someone with extensive experience in data engineering, typically eight years or more, who is comfortable working with large-scale distributed production systems.

• Strong, current experience with SQL and Python is essential, alongside several years of hands-on Spark experience.

• A solid understanding of Hadoop-based ecosystems, data warehousing concepts and query optimisation is expected.

• Experience with Scala or Java is beneficial, particularly for engineers who enjoy working across multiple languages.

• Comfort operating in Linux environments with a strong grounding in data infrastructure fundamentals such as availability, retention, backup strategies and capacity planning.

• Exposure to cloud platforms and migration work is advantageous.

• The role is fully remote, operating on East Coast US working hours.