Who We Are
We are the Analytics Data Infrastructure team in the Infrastructure & Data Engineering organization. Our team owns the Hadoop platform in full, including its availability, reliability, security & compliance, performance, and engineer usability. We are at the center of unlocking the power of data to reveal intelligence that moves eBay forward. Critical elements of our team DNA are quality and being customer-focused.
We are looking for a hands-on Hadoop Architect who will drive operational and engineering excellence within the team.
Primary Job Responsibilities
Be the go-to domain expert for all things Hadoop (Spark, HBase, HDFS, Yarn, Hive, etc.)
Maintain cluster availability and reliability as part of a DevOps on-call rotation (ex: recovery from and RCA for namenode failovers, hands-on implementation of long-term fixes; ex: namenode garbage collection tuning)
Mentor junior DevOps Engineers
Provide hands-on technical leadership and implementations for software development projects that automate and improve the usability of our Hadoop platform (ex: full-stack self-service queue management features)
Lead operations projects to improve the architecture and implementations of our Hadoop platform (ex: tiered storage implementations, namenode federation)
Develop and maintain upgrade roadmaps and implementation plans for various Hadoop components
Ideate constantly for new ideas to drive improvements in our focus areas (availability, reliability, security & compliance, performance, and engineer usability)
Drive cost savings in areas of compute and storage through efficiency efforts (ex: tiered storage, erasure coding)
Assist internal customers on the creation and optimization of SparkSQL
Participate in one or more scrum teams that develop eBay-specific distributions of open source software (Hadoop, Spark, etc)
Must have 7+ years of experience as a staff level engineer
Significant practical experience deploying and operating Hadoop clusters at-scale
Significant experience in Hadoop architecture and experience various innovative solutions like Tiered storage, federated namenode.
Significant practical experience in being customer-centric and empathetic in understanding deeply how the work you do enables others
Significant practical infrastructure experience (data center design awareness, fault domain requirements, HA and DR strategies)
Significant practical OS experience (Linux: RHEL/CentOS, Ubuntu, shell programming [bash, perl, etc.])
Significant practical Compute experience (hyper-scale)
Significant practical Storage experience (local, ISCSI, object)
Significant practical Network experience (practical knowledge of the OSI model from top to bottom)
Significant practical Security experience (Kerberos, host-based firewall implementations [iptables/ipset], zoning strategies]
Prior experience with DevOps support will be good to have.
Prior experience working on geo-diverse teams in an Agile environment
Prior Open Source development experience and contributor (nice to have)
Strong interpersonal networking skills with the ability to relate to executives and other team members through all organizational levels
Excellent communication skills in oral and written English
eBay Inc. is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status. If you are unable to submit an application because of incompatible assistive technology or a disability, please contact us at [email protected] We will make every effort to respond to your request for disability assistance as soon as possible.
For more information see: EEO is the Law Poster and EEO is the Law Poster Supplement