Databricks Admin

We are looking for a skilled Databricks Administrator to manage, maintain, and optimize the Databricks platform within our data ecosystem. The ideal candidate will have hands-on experience with Databricks workspace administration, cluster management, security, and performance optimization, ensuring efficient big data processing and analytics. Technical Requirements: 2–5 years of experience in Databricks Administration / Big Data Platforms. Strong hands-on experience with Databricks (Workspace, Clusters, Jobs, Notebooks). Experience with Apache Spark (PySpark/Scala) and distributed data processing. Knowledge of cloud platforms (Azure preferred, AWS/GCP also acceptable). Experience with data storage systems like ADLS, S3, Blob Storage. Good understanding of data engineering concepts, ETL/ELT pipelines. Familiarity with security best practices (IAM roles, data encryption, private endpoints). Proficiency in SQL and scripting languages (Python, Shell, etc.). Experience in performance tuning and cost optimization for clusters and workloads. Responsibilities: Administer and manage Azure/AWS Databricks workspaces. Configure and maintain clusters, including creation, scaling, and optimization. Manage user access, roles, and permissions using RBAC and integration with identity providers (Azure AD, etc.). Monitor cluster performance and optimize cost, resource utilization, and job execution. Implement and manage job scheduling, workflows, and pipelines. Ensure data security and governance, including data encryption and access policies. Integrate Databricks with data sources such as ADLS, S3, Snowflake, or SQL databases. Troubleshoot issues related to jobs, clusters, notebooks, and integrations. Support CI/CD pipelines and DevOps processes for Databricks deployments. Monitor usage through logs, metrics, and audit trails. Maintain documentation for architecture, configurations, and processes. Collaborate with data engineers, data scientists, and DevOps teams. Preferred Skills: Technology->Data Engineering->Databricks