Requirement and Details

  • Place Work : Mumbai
  • Time – 12-9 Shift Only

Key Functional Responsibilities

  • Develop, maintain, and optimize data pipelines using Hadoop, Kafka, Hive, and other Data Engineering technologies.
  • Implement data transformation and ETL processes to support data analytics and reporting.
  • Work with large datasets, both structured and unstructured, to extract valuable insights.
  • Collaborate with cross-functional teams to understand data requirements and design robust data solutions.
  • Ensure the scalability, performance, and security of data platforms.
  • Troubleshoot and resolve data-related issues and ensure data quality and integrity.
  • Monitor and fine-tune data workflows to improve efficiency and performance.
  • Stay up-to-date with emerging technologies in the data engineering space and recommend their integration when appropriate.
  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver on their data needs.

Key Technical Responsibilities and Skillsets

  • Utilize Azure and Spark components such as Azure Data factory, Azure Data Bricks to manage and process big data.
  • Implement data ingestion and integration solutions using Azure Services(preferred).
  • Experience in building Azure Data Bricks(ADB) Pyspark-python, Spark-SQL Notebooks to perform data transformations.
  • Develop and maintain data processing, extraction flow and querying solutions using Azure Data Factory, Azure Data Bricks and other Azure services.
  • Knowledge of spark optimizations to manage resource allocation and cluster operations through for efficient data processing.
  • Design, schedule, and monitor data workflows with Azure Data factory for seamless data orchestration.
  • Having hands on maintaining Delta Lake and maintains data in different zone layer.
  • Good hands-on experience in writing notebooks using Pyspark.
  • Apply programming skills in Python to develop and optimize data processing scripts and applications.
  • Involved in Pyspark development to implement Batch Validation, Delta Identification Tables implementation.
  • Demonstrate a strong understanding of data warehousing platforms and data storage concepts.
  • Execute SQL queries for data querying and manipulation with proficiency.
  • Contribute to data modeling and database design efforts to ensure effective data structures.
  • Employ data serialization formats like JSON, Avro, or Parquet for data interchange and storage.
  • Tackle complex data-related challenges with robust problem-solving and analytical skills.
  • Communicate effectively within the team and collaborate to achieve data objectives.
  • Utilize version control systems like Git/Azure DevOps for code management and collaboration.

To apply for this job email your details to career@translab.io

Start Your Journey with

Embark on a transformative journey into the future of innovation as you kickstart your exploration of translab.io, where limitless possibilities await at every click and command.