Lead Engineer – Data Platform & Acquisition (LLM Corpora)

• Location: Delhi NCR

• Core Mandate: Pipeline engineering, dedup/quality/decontamination, licensing negotiations, vendor management.

• Responsibilities:
• Architect high-throughput data pipelines capable of processing petabytes of text and multimodal data.
• Implement rigorous data deduplication, quality filtering, and benchmark decontamination protocols.
• Lead licensing negotiations for proprietary datasets and manage external vendors for large-scale digitization and crowdsourcing tasks.

Education

B.E./B.Tech/M.Tech/M.S./Ph.D. in CS, EE, Maths or related