Senior Data Engineer
Senior Data Engineer
NewVision Software & ConsultancyEarly Applicant
- Posted 2 months ago
- Be among the first 10 applicants
Job Description
Sr. Data Engineer (6-9 yrs)
Responsibilities:
- Data masking: implement PII masking pipeline
- Builds & maintain synthetic data generator
- CI/CD pipeline integration with auto-seed data
- Validates generated data against the ground truth – referential integrity, volume accuracy, etc. data validation characteristics.
Technical skills:
- Python (Faker, factory-boy, custom generators) or Node.js
- Database management: SQL Server, PostgreSQL — insert at scale
- CI/CD pipeline integration (GitHub Actions, Azure DevOps)
- Data masking techniques: tokenization, hashing, format-preserving encryption
- API-based data seeding (REST / GraphQL endpoints)
- Performance: generating & loading millions of records efficiently
More Info
Key Skills
GitHub Actions
factory-boy
data masking techniques
performance generating loading millions of records efficiently
API-based data seeding
custom generators
Faker
format-preserving encryption
About Company
Skills:
Data Quality, Azure Data Factory, Data Modeling, Power Bi, Etl Development, SQL Server, Azure Databricks, Data Warehousing, SSIS, Dax
Skills:
snowflake , Aws Lambda, S3, Pyspark, AWS Glue, Python, Sql, Etl, ELT
Skills:
Google Cloud Platform (GCP), BigQuery, Dataproc, Sql, Cloud Storage, DataFlow, Python, Medallion Architecture, Cloud Composer, DataOps, CI/CD, Pub/Sub
Skills:
snowflake , Java, Data Factory, Performance Tuning, Scala, Data Governance, Sql, Metadata Management, Spark, Python, OneLake, Access Control, Snowflake Snowpipe Streaming, Fabric Warehouse, Fabric Lakehouse, Microsoft Purview, Snowpark, Snowflake Tasks, Azure Event Hubs, Lakehouse architecture, Data quality frameworks, Microsoft Fabric, CI/CD practices for data pipelines
Skills:
snowflake , Sql, Azure Data Factory, Python, Data Warehousing Concepts, Git, Hadoop, Kafka, Microsoft Azure, Spark, Big Data tools, NoSQL databases