This website uses cookies to ensure you get the best experience. Learn more

Hadoop Software/Platform Architect

Who We Are

We are the Analytics Data Infrastructure team in the Infrastructure & Data Engineering organization. Our team owns the Hadoop platform in full, including its availability, reliability, security & compliance, performance, and engineer usability. We are at the center of unlocking the power of data to reveal intelligence that moves eBay forward. Critical elements of our team DNA are quality and being customer-focused.

The Role

We are looking for a hands-on Hadoop Architect who will drive operational and engineering excellence within the team.

Primary Job Responsibilities

  • Be the go-to domain expert for all things Hadoop (Spark, HBase, HDFS, Yarn, Hive, etc.)

  • Maintain cluster availability and reliability as part of a DevOps on-call rotation (ex: recovery from and RCA for namenode failovers, hands-on implementation of long-term fixes; ex: namenode garbage collection tuning)

  • Mentor junior DevOps Engineers

  • Provide hands-on technical leadership and implementations for software development projects that automate and improve the usability of our Hadoop platform (ex: full-stack self-service queue management features)

  • Lead operations projects to improve the architecture and implementations of our Hadoop platform (ex: tiered storage implementations, namenode federation)

  • Develop and maintain upgrade roadmaps and implementation plans for various Hadoop components

  • Ideate constantly for new ideas to drive improvements in our focus areas (availability, reliability, security & compliance, performance, and engineer usability)

  • Drive cost savings in areas of compute and storage through efficiency efforts (ex: tiered storage, erasure coding)

  • Assist internal customers on the creation and optimization of SparkSQL

  • Participate in one or more scrum teams that develop eBay-specific distributions of open source software (Hadoop, Spark, etc)


  • Must have 7+ years of experience as a staff level engineer

  • Significant practical experience deploying and operating Hadoop clusters at-scale

  • Significant experience in Hadoop architecture and experience various innovative solutions like Tiered storage, federated namenode. 

  • Significant practical experience in being customer-centric and empathetic in understanding deeply how the work you do enables others

  • Significant practical infrastructure experience (data center design awareness, fault domain requirements, HA and DR strategies)

  • Significant practical OS experience (Linux: RHEL/CentOS, Ubuntu, shell programming [bash, perl, etc.])

  • Significant practical Compute experience (hyper-scale)

  • Significant practical Storage experience (local, ISCSI, object)

  • Significant practical Network experience (practical knowledge of the OSI model from top to bottom)

  • Significant practical Security experience (Kerberos, host-based firewall implementations [iptables/ipset], zoning strategies]

  • Prior experience with DevOps support will be good to have.

  • Prior experience working on geo-diverse teams in an Agile environment

  • Prior Open Source development experience and contributor (nice to have)

  • Strong interpersonal networking skills with the ability to relate to executives and other team members through all organizational levels

  • Excellent communication skills in oral and written English


eBay Inc. is an equal opportunity employer.  All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status.  If you are unable to submit an application because of incompatible assistive technology or a disability, please contact us at [email protected]  We will make every effort to respond to your request for disability assistance as soon as possible.

For more information see:  EEO is the Law Poster and EEO is the Law Poster Supplement

This website uses cookies to enhance your experience. By continuing to browse the site, you agree to our use of cookies

View our privacy policy and accessibility info

Apply Now

Share this