Quest Hire Solutions
Location: Bangalore & Chennai
Category: Information & Technology
Job Type: Full Time
Salary: Best in the industry
Job Summary:
We are looking for a skilled Data Engineer/Senior Data
Engineer with 2 to 10 years of experience to join our dynamic team. In this
role, you will be responsible for designing, building, and maintaining scalable
data pipelines and infrastructure for optimal data extraction, transformation,
and loading. You will collaborate closely with cross functional teams to
understand data requirements, architect solutions, and ensure data quality and
reliability.
Skills:
Set 1: Python, PySpark, SQL
Set 2: Hadoop, Spark, Scala, Hive
Responsibilities:
1. Design and develop robust data pipelines using
Python, PySpark, Scala, and SQL to support various data processing needs.
2. Implement efficient ETL processes for both
batch and real time data processing.
3. Work with largescale distributed systems such
as Hadoop and Spark to process and analyse data.
4. Build and optimize data storage solutions
using technologies like HDFS, S3, and Hive.
5. Collaborate with data scientists and analysts
to understand data requirements and provide scalable solutions.
6. Implement data quality checks, monitoring, and
troubleshooting to ensure data accuracy and availability.
7. Continuously optimize and improve data
infrastructure and processes for performance and scalability.
8. Stay updated with industry trends and emerging
technologies in data engineering and big data.
Requirements:
·
Proven
experience as a Data Engineer, Data Analyst, or similar role, with 2 to 10
years of experience.
·
Strong
programming skills in Python and Scala, with proficiency in SQL for querying
and data manipulation.
·
Handson
experience with big data processing frameworks such as Apache Spark, Hadoop, or
similar technologies.
·
Experience
designing and implementing ETL processes and data pipelines.
·
Solid
understanding of database design, data warehousing, and data modelling
concepts.
·
Excellent
problem solving and analytical skills, with a strong attention to detail.
·
Strong
communication and collaboration skills to work effectively in a team
environment.
Best regards,
Quest Hire Solutions