Databricks Engineer

Key skills

• Python
• SQL
• Spark
• Databricks
• AWS
• Azure
• GCP
• Kafka

Job description

We are looking for an experienced Databricks Engineer with 5-8 years of experience in designing, developing, and optimizing large-scale data solutions. The ideal candidate should have strong expertise in Databricks, PySpark, Spark, SQL, cloud platforms (AWS/Azure/GCP), and modern data engineering practices to build scalable and high-performance data pipelines.

Technical Requirements:

• 5-8 years of experience in Data Engineering and Big Data technologies.
• Strong hands-on experience with Databricks, PySpark, and Spark SQL.
• Experience with Delta Lake, Databricks Workflows, and Unity Catalog.
• Proficiency in Python and SQL.
• Experience with Azure/AWS/GCP cloud platforms.
• Knowledge of Azure Data Factory (ADF), Apache Airflow, or similar orchestration tools.
• Understanding of data warehousing concepts and ETL/ELT methodologies.
• Experience with Kafka/Event Hub for streaming data processing is preferred.
• Familiarity with Git, CI/CD pipelines, and DevOps practices.
• Strong problem-solving, analytical, and communication skills.

Responsibilities:

Roles & Responsibilities

Design, develop, and maintain data pipelines using Databricks.

Build ETL/ELT workflows for large-scale data processing.

Develop and optimize data transformation logic using PySpark and Spark SQL.

Implement and manage Delta Lake for reliable and efficient data storage.

Work with batch and real-time data ingestion frameworks.

Monitor and optimize data pipelines for performance and scalability.

Collaborate with data analysts, data scientists, and business stakeholders.

Ensure data quality, security, and governance standards are followed.

Troubleshoot production issues and provide ongoing support.

Preferred Skills:

Technology->Data Engineering->Databricks

Apply on the official Infosys careers page

Check your resume match score for this job free on Job Scanner