I
AI Engineer (Gen AI, Databricks)
I
AI Engineer (Gen AI, Databricks)
Infosys- Posted 6 days ago
- Over 50 applicants have applied
Job Description
Gen Ai + Databricks (with Data Engineering exp is preferred)
Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks. Build and optimize ETL/ELT workflows for structured and unstructured data. Develop and deploy Generative AI solutions using LLMs. Implement RAG (Retrieval-Augmented Generation) architectures and vector databases. Work with large-scale datasets using Spark and PySpark. Integrate AI solutions with cloud services and enterprise applications. Collaborate with Data Scientists, Data Engineers, and business stakeholders. Monitor, troubleshoot, and optimize data and AI workloads. Ensure data quality, governance, and security standards. Mandatory Skills 2-5 years of experience in Data Engineering and/or GenAI Databricks PySpark / Apache Spark Python SQL Data Pipeline Development ETL/ELT Development Delta Lake Data Warehousing Concepts Generative AI LLMs (OpenAI, Azure OpenAI, Llama, Claude, etc.) RAG Frameworks Vector Databases Git
Preferred Skills Azure Databricks or AWS Databricks LangChain / LangGraph Azure OpenAI MLflow Kafka Airflow Snowflake DBT Knowledge of MLOps concepts Good to Have Prompt Engineering Agentic AI Semantic Search REST APIs Docker CI/CD Exposure
Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks. Build and optimize ETL/ELT workflows for structured and unstructured data. Develop and deploy Generative AI solutions using LLMs. Implement RAG (Retrieval-Augmented Generation) architectures and vector databases. Work with large-scale datasets using Spark and PySpark. Integrate AI solutions with cloud services and enterprise applications. Collaborate with Data Scientists, Data Engineers, and business stakeholders. Monitor, troubleshoot, and optimize data and AI workloads. Ensure data quality, governance, and security standards. Mandatory Skills 2-5 years of experience in Data Engineering and/or GenAI Databricks PySpark / Apache Spark Python SQL Data Pipeline Development ETL/ELT Development Delta Lake Data Warehousing Concepts Generative AI LLMs (OpenAI, Azure OpenAI, Llama, Claude, etc.) RAG Frameworks Vector Databases Git
Preferred Skills Azure Databricks or AWS Databricks LangChain / LangGraph Azure OpenAI MLflow Kafka Airflow Snowflake DBT Knowledge of MLOps concepts Good to Have Prompt Engineering Agentic AI Semantic Search REST APIs Docker CI/CD Exposure
More Info
Key Skills
Generative AI
LLMs
Claude
Azure OpenAI
Data Pipeline Development
Vector Databases
RAG Frameworks
Llama
Delta Lake
OpenAI
About Company
Skills:
Workflows, Apis, Cloudformation, Pyspark, Sql, Devops, MLops, Terraform, Spark, Databricks, Azure, Python, AWS, Databricks Lakehouse, dbt
Skills:
Pyspark, Spark SQL, RAG implementation, vector databases, MLflow, Claude, Pinecone, Azure OpenAI, LLM APIs, Google Vertex AI, ChromaDB, Azure AI Search, fine-tuning techniques, LangChain, PEFT, Databricks platform, Databricks Mosaic AI, Databricks Vector Search, Anthropic, prompt engineering, Semantic Kernel, Unity Catalog, Langgraph, FAISS, LoRA, Delta Lake, OpenAI, Weaviate, QLoRA
Skills:
Macros, Data Modeling, Pyspark, Sql, Databricks, Data Ingestion, Performance Optimization, dbt Data Build Tool, Materializations, Medallion Architecture, Testing Documentation, CI CD Integration, Unity Catalog, Incremental Models, Delta Lake, Databricks Workflows
Skills:
Microservices, Typescript, Databricks, Azure cloud services, Rest Apis, Python, Azure DevOps, LLMOps, Vector DBs, AKS, Azure OpenAI, Azure AI Speech, DAX Models, GitHub Copilot, Agentic AI, Image Video Audio Generation APIs, RAG
Skills:
Spark SQL, Pyspark, Sql, Devops, Terraform, Databricks, Python, Azure DevOps, Infrastructure as Code, MLflow, CI CD, Unity Catalog, GitHub Actions, Delta Lake, Structured Streaming, Databricks Workflows