Data Engineer Job at Intercontinental Exchange Holdings, Inc., Atlanta, GA

SUlWUXhKL09WS1M3dU1IRUZIK3BZelM3R0E9PQ==
  • Intercontinental Exchange Holdings, Inc.
  • Atlanta, GA

Job Description

Overview:

Job Purpose

We're seeking a talented Data Engineer to join ICE Data Services in a cross-cutting role that will help define and implement our next-generation data platform. In this pivotal position, you'll lead the design and implementation of scalable, self-service data pipelines with a strong emphasis on data quality and governance. This is an opportunity to shape our data engineering practice from the ground up, working directly with key stakeholders to build mission-critical ML and AI data workflows.

 

You'll be working with a modern, on-premises data stack that includes:

  • Apache Airflow for workflow orchestration (self-hosted on Kubernetes)
  • dbt for data transformation and testing
  • Apache Flink for stream processing and real-time data workflows
  • Kubernetes for containerized deployment and scaling
  • Git-based version control and CI/CD for data pipelines
  • Oracle Exadata for data warehousing
  • Kafka for messaging and event streaming

We emphasize building systems that are maintainable, scalable, and focused on enabling self-service data access while maintaining high standards for data quality and governance.

 

The ideal candidate is a problem-solver who enjoys working on complex data systems and is passionate about data quality. You thrive in collaborative environments but can also work independently to deliver solutions. You're comfortable working directly with technical and non-technical stakeholders and can communicate complex technical concepts clearly. Most importantly, you're excited about creating systems that empower others to work with data efficiently and confidently.

 

Responsibilities

  • Design, build, and maintain our on-premises data orchestration platform using Apache Airflow, dbt, and Apache Flink
  • Create self-service capabilities that empower teams across the organization to build and deploy data pipelines without extensive engineering support
  • Implement robust data quality testing frameworks that ensure data integrity throughout the entire data lifecycle
  • Establish data engineering best practices, including version control, CI/CD for data pipelines, and automated testing
  • Collaborate with ML/AI teams to build scalable feature engineering pipelines that support both batch and real-time data processing
  • Develop reusable patterns for common data integration scenarios that can be leveraged across the organization
  • Work closely with infrastructure teams to optimize our Kubernetes-based data platform for performance and reliability

Knowledge and Experience

  • 2+ years of professional experience in data engineering
  • Deep expertise with Apache Airflow, including DAG design, performance optimization, and operational management
  • Strong understanding of dbt for data transformation, including experience with testing frameworks and deployment strategies
  • Proficiency with SQL and Python for data transformation and pipeline development
  • Familiarity with Kubernetes for containerized application deployment
  • Experience implementing data quality frameworks and automated testing for data pipelines
  • Knowledge of Git-based workflows and CI/CD pipelines for data applications
  • Ability to work cross-functionally with data scientists, ML engineers, and business stakeholders

Preferred Knowledge and Experience 

  • Experience with self-hosted data orchestration platforms (rather than managed services)
  • Experience with stream processing frameworks like Apache Flink or similar technologies
  • Background in implementing data contracts or schema governance
  • Knowledge of ML/AI data pipeline requirements and feature engineering
  • Experience with real-time data processing and streaming architectures
  • Familiarity with data modeling and warehouse design principles
  • Prior experience in a technical leadership role

 

#LI-HR1 #LI-ONSITE

----------: Intercontinental Exchange, Inc. is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to legally protected characteristics.

Job Tags

Full time,

Similar Jobs

Cisco

Protective Intelligence Analyst (Remote8) 2002813 Job at Cisco

 ...sufficient number of applications are received. Protective Intelligence Analyst Meet the Team Ciscos Security and Trust Organization...  ...relations, political science, intelligence studies, criminal justice, journalism, or a related field. ~2-4 years of government... 

The Jernigan Agency

WORK FROM HOME/HOME BASED INSURANCE AGENT Job at The Jernigan Agency

 ...This Jernigan Agency is looking for individuals interested in working remotely as sales representatives. We are hiring coachable individuals...  ...a computer and phone to service the clients. This is all online so and internet connection is a must. We provide all of the... 

Rolfson Oil

CDL Driver-Fuel Transport Job at Rolfson Oil

 ...IMMEDIATE OPENINGS IN ODESSA, TX!!! With Rolfson Oil, youre in the drivers seat. We started in 1985 in Watford City, North Dakota...  ..., Wyoming, Colorado, New Mexico, and Texas. We are 100% oil field, 100% of the time and looking for drivers to fuel our team. Benefits... 

Quail Park

Night Janitor/Housekeeper Job at Quail Park

 ...Night Janitor | Housekeeper Quail Park of Lynnwood Have you always wanted a fun and rewarding job? Do you have a passion for working...  ...a resident requires two-person transfer assistance during night-time hours Answer the resident emergency call system Provides... 

Guardian Angel Senior Services

Helping Hand for Elderly - Help & support in ADLs Job at Guardian Angel Senior Services

 ...someone's life,no prior experience required we provide FREE training ! GUARDIAN ANGEL SENIOR SERVICES is looking for a...  ...companionship duties. (Not applicable for non-Drivers) Helping with personal hygiene care like dressing/undressing; showering, feeding etc....