Job Overview:
Master Works is looking for a highly skilled LLM Engineer to join our AI Core Delivery team within the AI & Analytics CoE. In this role, you will develop, deploy, and optimize advanced language model capabilities, including Retrieval-Augmented Generation (RAG) pipelines and agentic AI systems. You will play a key role in scaling AI Core as the central enterprise AI platform by ensuring production-ready performance, robust observability, and seamless integration of LLMs with internal and external systems.
Key Responsibilities
- Model Development & Optimization
- Build and optimize RAG pipelines for efficient document processing, indexing, and retrieval
- Implement agentic AI logic for tool integration, task automation, and external data access
- Enhance query understanding, response ranking, and AI-generated output quality
- Develop evaluation benchmarks to measure LLM performance, accuracy, and relevance
- LLMOps & Deployment
- Collaborate with API developers and backend engineers to deliver secure, observable LLM endpoints
- Deploy and manage LLMs in cloud-native or hybrid environments with support for scalability and multi-cloud readiness
- Implement CI/CD workflows for model updates, rollback mechanisms, and performance tuning
- Backend & Data Integration
- Integrate LLM capabilities with enterprise data sources and knowledge bases
- Support multi-source retrieval and embedding strategies to improve AI responses
- Design pipelines that ensure low latency, high throughput, and fault tolerance
- Security & Compliance
- Apply enterprise-grade authentication, access management, and encryption standards
- Ensure compliance with internal governance policies, data privacy, and audit requirements
- Continuous Improvement
- Monitor system performance, debug issues, and fine-tune models based on real-world feedback
- Stay updated on advancements in LLM architectures, open-source tools, and evaluation frameworks
Requirements
- Bachelor's or Master's degree in Computer Science, AI, Data Science, or related field
- Hands-on experience with LLM development and deployment (OpenAI, Anthropic, Hugging Face, LLaMA, etc.)
- Strong understanding of Retrieval-Augmented Generation (RAG), embeddings, and vector databases
- Experience with cloud environments (AWS, Azure, GCP) and infrastructure-as-code (IaC)
- Proficiency with Python or similar languages for AI/ML pipeline development
- Familiarity with container orchestration (Docker, Kubernetes) and CI/CD practices
- Knowledge of observability tools for monitoring AI performance in production environments
Preferred Skills
- Experience integrating agentic AI capabilities with external APIs and automation tools
- Strong grasp of prompt engineering, contextual memory systems, and LLM evaluation
- Exposure to enterprise AI security and compliance standards
