Lead Site Reliability Engineer

Our Purpose

Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.

Title and Summary

Lead Site Reliability Engineer

Overview:

The Identity Solutions program within the Services group works to prevent fraud by securing a user’s identity through their devices, identity information, and usage patterns such as passive Biometrics. Comprised of best-in-class solutions from Ekata (a Mastercard company), we use complex machine learning, combined with device, identity, and transaction information from billions of transactions to reduce user friction to enable highly secure real time transactions.

The Identity Solutions – ‘Cloud Infrastructure - Edge’ team is looking for a Lead Site Reliability Engineer to help support, design, and build our highly available and scalable AWS Cloud infrastructure which our products are built on. The team builds infrastructure for Engineering projects, extends our platform for the future, and improves the availability and performance of our applications. This position also designs, develops systems, automation, and tools to help make it easier for Engineering teams to deploy services in a fast, automated and reliable fashion.


In this role You will:

• Be a team and highly accountable project leader in a geo-diverse team. Mentoring the more junior members and being an SME.
• Plan, design, build, and scale our infrastructure.
• Work with tools such as Jenkins, Ansible, Argo CD, Terraform, CloudFormation, Resource Manager and many more to ensure that our stack is well represented as Infrastructure as Code.
• Manage, design, and improve security and availability monitoring and processes for all services, ensure defined security policies are consistently implemented across all environments.
• Deploy and supervise workloads to cloud environments, proven experience with all of the core services within AWS including instance management, IAM configuration, Database, Caching and general support/troubleshooting.
• Have a deep understanding of the core components required to run Kubernetes (EKS) and be able to build a cluster from scratch or scale out if needed.
• Have perfected load balancing, service mesh and always looking for ways to improve availability, response time, and uptime.
• Maintain, and manage quality documentation for systems owned by the Infrastructure team.
• Design, build, and improve monitoring tools to identify and resolve issues before they happen. Mainly with Prometheus.
• Help and advise other teams, troubleshoot and solve failures and performance problems, participate in on-call rotations.
• Have a basic passion for working with Go, Python, Rust or even Bash to build custom tools and improve system integration. Take code ownership to the next level and act as an advocate for writing code that aligns with industry best practice.

All About You:
• Excellent spoken and written English skills. Is a team player and values collaboration.
• BS degree in Computer Science or equivalent experience.
• Led / been a leader within a team and projects.
• AWS Networking Services
• Proficiency with:
o Amazon Virtual Private Cloud (VPC)
o AWS Direct Connect
o Transit Gateway
o Elastic Load Balancing (ELB)
o Route 53, Resolver endpoints, DNS firewall
o Security Groups and NACLs
o AWS Firewall
o VPC flow log analysis
• Understanding of hybrid connectivity (VPNs, Direct Connect) and multi-region architectures
• Understand implementation of latency reduction techniques and tools - iperf, sockperf
• Proven deep skills with Linux or UNIX systems and related protocols/software with 3+ years’ experience. Robust DevOps and/or BizOps experiecne.
• A command of Linux systems including troubleshooting, memory management, tuning, I/O subsystem, RAID, and security.
• Experience with provisioning tools such as Ansible/Chef/Terraform.
• Experience with Jenkins or other CI/CD tools.
• Programming aptitude in Go, Python, and Bash.
• Knowledge of database systems such as MySQL or PostgreSQL.
• Experience building and deploying Containers, including orchestration tools such as Kubernetes, Mesos, or Docker Swarm.
• Experience with AWS cloud providers (AWS, Azure, GCP)

Work Schedule & On-Call Requirements:
• Standard working hours are ~10 AM – ~6/7 PM UTC, Monday through Friday.
• Participate in a follow-the-sun on-call rotation approximately every 1.5 months once the team is fully staffed.
• During the initial staffing period, team members should expect to participate in the on-call rotation approximately every 3-4 weeks until the team reaches full capacity.
• Ability to respond to and troubleshoot production infrastructure and networking issues during assigned on-call periods.

We cultivate a culture of inclusion (https://www.mastercard.us/en-us/vision/who-we-are/diversity-inclusion.html) for all employees that respects their individual strengths, views, and experiences.

Corporate Security Responsibility

All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:

• Abide by Mastercard’s security policies and practices;

• Ensure the confidentiality and integrity of the information being accessed;

• Report any suspected information security violation or breach, and

• Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.