Top 3 Required:
Azure Cloud, Azure Data Bricks with Spark Scala, and Azure DevOps
Job Description:
We are seeking a skilled and motivated Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering, with expertise in Azure Cloud, Azure Data Bricks with Spark Scala, and Azure DevOps. The candidate should also possess a solid understanding of ETL processes, data engineering principles, and warehouse systems.
Key Responsibilities:
Design, develop, and maintain scalable data pipelines and systems on Azure Cloud.
Utilize Azure Data Bricks with Spark Scala to process and analyze large datasets.
Implement and manage CI/CD pipelines using Azure DevOps.
Develop and optimize ETL processes to ensure efficient data flow and transformation.
Collaborate with cross-functional teams to understand data requirements and deliver solutions.
Ensure data quality, integrity, and security across all data engineering processes.
Monitor and troubleshoot data pipelines to ensure smooth operation and minimal downtime.
Stay updated with the latest industry trends and best practices in data engineering and cloud technologies.
Qualifications:
Bachelor's degree in Computer Science, Information Technology, or a related field.
Proven experience as a Data Engineer or in a similar role.
Strong proficiency in Azure Cloud services and Azure Data Bricks with Spark Scala.
Hands-on experience with Azure DevOps for CI/CD pipeline implementation.
In-depth knowledge of ETL processes and data engineering principles.
Understanding of warehouse systems and data modeling.
Excellent problem-solving skills and attention to detail.
Strong communication and collaboration skills.Years of Experience:11.00 Years of Experience