Company Description: The Best Engineered e-commerce Platform in the World
Salary: Highly Competitive, Depending on Experience
Position Type: Permanent
Job Number: 9857
The Data Platforms team at our client is looking for an exceptionally talented Data engineer with an outstanding track record reflecting achievements and skills in modernizing and improving the data infrastructure from the ground up. So, if you are passionate about working with very Large Data sets (structured/unstructured), building large scale Data processing platforms, implementing world class data governance and operational controls, solving complex performance challenges and building robust ETL pipelines then we would love to talk to you.
Our Data Environment
Our Event driven Ecommerce Microservices generate massive volume of data every second. Our Data Platform ingests and organizes this and other third party high speed, high volume data and enables strategic and operational decisions and reporting by Analysts and Data Scientist in every domain of our Ecommerce business.
Our Data Platform supports Strategic and Operational Analytics and leverages Azure hosted Big Data clusters as well as a constellation of Big Microsoft SQL Server instances. Our platform leverages Spark, Hive, Kafka, Redis, Elastic Search, Cosmos DB.
What you will bring
· Degree in Computer Science, Information Technology, Math or related technical field.
· Natural inclination for designing well thought out data solutions as well as solid hands-on implementation capability
· 5+ years’ experience in engineering data solutions using technologies including Spark (batch/streaming), Scala/Java (building data pipelines vs frameworks), Hadoop, Hive, HBase, Kafka, Spark, Oozie, Yarn.
· Solid hands-on experience in building data pipelines, deploying and managing Big Data infrastructure, establishing deployment and operational excellence of Big Data clusters.
· Experience in Relational databases, Data Warehousing, SQL, ETL and/or NOSQL databases will be sweet.
· Proven experience in Building or improving large scale data infrastructure from the ground up. This could include building Data warehousing stores, Data Architecture, Data Integration, building highly performant data movement pipelines, building tools and automation to facilitate Data and operational governance, Data Lineage, Automation and Monitoring.
· Proven Performance Tuning Accomplishments and Advanced tuning and trouble shooting skills in Large data environments on the Hadoop or RDBMS stack.
· Advanced knowledge of internals of at least one RDBMS (MSSQL preferred) and best practices.
· Data Ops experience including hands on skills in scripting language such as Python, Perl or Bash
· Proven experience in trouble shooting, problem determination and rapid problem resolution
· Experience and ability to work under high pressure in a complex technical environment.
What you will Do
· Play a pivotal design and hands on implementation role in improving the Data infrastructure in a project-oriented work environment.
· Solve complex SQL and Big Data Performance challenges.
· Mitigate Risks in our data infrastructure by developing the best in class tools and processes.
· Implement controls, policies, processes and best practices in the Data Engineering space.
· Help us improve our database deployment and change management process.
· Provide reliable and efficient Data services as part of the database team.
· Work closely with the devs on development best practices and standards.
· Be a mentor.
Andiamo provides equal employment opportunities to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability or genetics. In addition to federal law requirements, Andiamo complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.
Please Note: All inquiries will be treated with the utmost confidentiality. Your resume will not be submitted to any client company without your prior knowledge and consent.