Data Engineer (SQL, Python, Linux, AWS and PySpark)
Search Jobs
Search by job, company or skills
Search by job, company or skills
Data Engineer
Immediate starters required
Location: Hyderabad, TG & Pune, MH
Responsibilities
·Develop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation.
·Collaborate with product and technology teams to design and validate the capabilities of the data platform
·Identify, design, and implement process improvements: automating manual processes, optimizing for usability, re-designing for greater scalability
·Provide technical support and usage guidance to the users of our platform's services.
·Drive the creation and refinement of metrics, monitoring, and alerting mechanisms to give us the visibility we need into our production services.
Qualifications
·Experience building and optimizing data pipelines in a distributed environment
·Experience supporting and working with cross-functional teams
·Proficiency working in Linux environment
·5+ years of advanced working knowledge of SQL, Python, and PySpark
·Knowledge on Palantir
·Experience using tools such as: Git/Bitbucket, Jenkins/CodeBuild, CodePipeline
·Experience with platform monitoring and alerts tools
Job ID: 126998679
Skills:
RDS, Pyspark, AWS Glue, Dynamodb, Emr, Sql, Lambda, Cloudwatch, Iam, Sqs, Sns, Python, AWS, Lake Formation, CloudTrail
Skills:
Pyspark, S3, Aws Services, Lambda, Iam, ECS, Spark, Python
Skills:
Machine Learning, Microservices, Nosql, Nlp, Docker, Flask, Python, Sql, Django, Git, FastAPI, Rest Apis, Kubernetes, vector databases, Pinecone, Claude, Azure OpenAI, RAG Retrieval-Augmented Generation, ChromaDB, Generative AI, LLM-based solutions, FAISS, CI CD pipelines, OpenAI, Weaviate
Skills:
Api Development, Unit Testing, Django, Numpy, Git, Pandas, Docker, Flask, Azure, Python, AWS, async programming
Skills:
Python, Reactjs, AI tools
