Big Data Engineer

Experience 7–8 Years Employment Type Full-Time Work Mode Remote

Job Summary

We are looking for an experienced Big Data Engineer with 7–8 years of experience in designing developing and maintaining scalable big data pipelines and data processing solutions.

The ideal candidate

should have strong hands-on experience with Scala PySpark AWS Glue big data pipeline development and data validation/profiling along with good knowledge of APIs databases testing and modern web technologies.

Key Responsibilities

Design develop and maintain scalable big data pipelines. Develop data processing solutions using Scala and PySpark. Build and manage ETL/ELT pipelines using AWS Glue. Perform data validation profiling quality checks and transformation. Work with large and complex datasets to ensure data accuracy and consistency. Develop and integrate REST APIs where required. Work with SQL and NoSQL databases including PostgreSQL MongoDB and Elasticsearch. Implement testing and validation using Scala/Java. Troubleshoot pipeline and data-related issues and optimize performance. Collaborate with data engineers developers QA and business teams. Follow best practices for data engineering testing documentation and deployment. Primary

Skills

Scala PySpark AWS Glue Big Data Pipeline Development Data Validation & Profiling Secondary

Skills

REST APIs Async JavaScript SQL / NoSQL PostgreSQL MongoDB Elasticsearch Web Design Testing with Scala / Java

Required Qualifications

7–8 years of experience in Big Data/Data Engineering. Strong hands-on experience with Scala and PySpark. Experience developing and maintaining big data pipelines. Strong experience with AWS Glue. Experience in data validation profiling and data quality. Good knowledge of SQL/NoSQL databases. Strong debugging and problem-solving skills. Good communication and collaboration skills.

Work Location

Remote