Python+ Hadoop Developer

Client of Forstaffing Published: September 21, 2016
Mumbai, India
Job Type


Designation: Python + Hadoop Developer

Experience required- 5-7 years

Reporting To: Architect
Vacancy: 1

Cab- No
Working Days: 5/6 (alternate Saturday off)
Shift: Day shift job

Job Location – Mumbai

Position type: permanent

Salary- upto 13L/annum (international travel @client site will be there)

Skills : Python, Java, Hadoop, Git, Amazon Web Services, Celery, ETL
We are looking for a capable DevOps Engineer with a strong background in Big Data Technologies and Python.

Technology Stack for Application

Data Storage + Analytics: AWS/ Cloudera (on-Premise) Hadoop Ecosystem with MongoDB & Elastic Search on S3 or on Premise
Queueing System: RabbitMQ
Programming: Python
Front-End: HTML5 for WebApp and Objective C for ios. Potentially hybrid framework for iOS and Android
CDN: AWS or On Premise Routing

- Responsible for the daily operation of data system such as hadoop / spark / olap, and ensure that the data platform is stable and running efficiently.
- Responsible for application deployment, maintenance, administration and monitoring systems
- Responsible for Automation of build, deployment and operational tasks throughout the Software Development Life Cycle (SDLC).
- Bring a DevOps mind-set to working with teams including Release Management, Operations, Production Engineering, and Engineering to ensure end-to-end solutions are designed and implemented to make build, delivery and monitoring mechanisms reliable, transparent, measurable, scalable and transportable.
- Responsible for data platform capacity planning, performance optimization, architecture audit and other works
- Responsible for development work of automation, maintenance and monitoring
- Develop and demonstrate detailed, independent ownership for supported systems, including configurations, monitoring and documentation.
- Troubleshoot and resolve functionality and performance issues the application stack, from hardware, operating system, network and security to address issues that impact release and service delivery.
- Follow best engineering practices such as architectural design, unit and regression testing, test-driven development, and continuous integration frameworks
- Work with the infrastructure team to ensure that all the required monitoring, exception handling and fault tolerance is in place for a production quality platform

- Degree in computer background or related field; at least 5+ years of intensive Python development in a production environment
- Experience with at least one other development or scripting language (Java, Node, Go, etc.) and good understanding of at least one programming language (C/C++)
- Knowledge and comprehension in the development of ETL processes and frameworks for large-scale, complex datasets
- Experience with open source asynchronous task queues like Celery
- Industry experience working on distributed, scalable, service-oriented platforms
- Experience with multi-tiered system operations in high-volume, PCI-compliant transactional environments, including web, application, message queuing and database operational support and services.
- Fundamental understanding of Network and Load Balancing technologies!
- Experience building, implementing and supporting monitoring tools!
- Have deep understanding of Cloudera/AWS ecosystem
- Proven ability to deliver high quality, production ready code
- Strong understanding of the agile software development life cycle
- Experience in the management, configuration, operation and maintenance of hadoop, spark, hive and data warehouse.
- Provide expertise on reliability and performance challenges that you've conquered in the past
- Thrive in a rapid development environment with an intense focus on quality
- Have an allergic reaction to the words "defer" and "works on my machine"
- Strong customer service and communication skills (verbal and written) to effectively interact with a diverse team of people across business and engineering

Drop files here browse files ...

Related Jobs

February 14, 2019
Quantitative(Python + C#/ C++) Developer   Mumbai(Andheri), India
September 22, 2016

Comments Closed