Senior Data Engineer, Data Infrastructure - Shift4

Shift4 is looking for a Senior Data Engineer to join our Data Platform team. We are seeking a hands-on individual with experience in building and scaling modern data infrastructure, passionate about exploring AI technologies and applying them to enhance and evolve our data systems. This role is ideal for someone focused on data systems, performance, automation, and reliability, who will design, build, and maintain core components of our data pipelines, storage, monitoring, and observability stack, with a strong focus on improving workflows and system capabilities, while embracing the Shift4 way.

Responsibilities:

• Build and maintain scalable, reliable, and high-performance data infrastructure across cloud and on-prem environments
• Partner with engineering, analytics, and observability teams to support data flow from ingestion to consumption
• Develop infrastructure as code and automation tools to improve deployment, monitoring, and recovery
• Help troubleshoot performance bottlenecks and reliability issues in large-scale data pipelines
• Implement data quality processes aligned with security standards to ensure accuracy, consistency, and compliance with financial regulations and PCI requirements.
• Continuously improve the performance and efficiency of the data platform with proactive tuning and benchmarking
• Proactively explore and apply AI tools to enhance efficiency across the data engineering lifecycle, while improving accessibility and usability of data solutions for teams across the organization

Qualifications:

• 7+ years of experience in data engineering, infrastructure, or DevOps roles
• Strong hands-on experience with message queue systems such as
  Apache Kafka, Confluent and RabbitMQ
• Practical experience with the Elastic Stack (Elasticsearch, Logstash, Kibana), including designing data pipelines, index management, performance and costs optimization.
• Hands-on experience with time series databases such as Prometheus and InfluxDB
• Experienced with visualization and observability platforms including Grafana, Prometheus, Elastic stack. Designed and maintained interactive dashboards for real-time monitoring of system metrics, logs, and traces, enabling proactive issue detection and improved operational efficiency
• Strong knowledge of cloud platforms (AWS, GCP, or Azure) and operating production data infrastructure, ensuring scalability, reliability, and performance of data systems.
• Infrastructure as code experience using Terraform or Ansible
  Proven proficiency in CI/CD methodologies with the ability to design and implement automated pipelines from scratch
• Responsible for organization-wide monitoring systems, with deep familiarity and understanding of the various infrastructure components, services, and devices across the organization
• Background in observability and performance tuning at scale
• Passion for automation, performance, and clean system design