Cloud Engineer

Greetings from LanceSoft !!

Role: Databricks

Location: Sri Lanka

Duration: Permanent

We’re looking for candidates with strong technology and data understanding in big data engineering space, having proven delivery capability. This is a fantastic opportunity to be part of a leading firm as well as a part of a growing Data and Analytics team.

Your key responsibilities

• Develop & deploy Azure Databricks in a cloud environment using Azure Cloud services
• Configure and replicate the Azure Databricks platform, including workspaces, clusters, and storage based on existing Hadoop workloads.
• Migration of Hadoop jobs (YARN, SPARK), ETL workflows (Sqoop scripts, Hive), Kafka streams, Impala queries and database objects to Azure Databricks, ensuring efficient data cleaning, transformation, and loading.
• Set up and manage workflow orchestration using Databricks Jobs Scheduler, handling dependencies and scheduling of data processing tasks.
• Implement monitoring solutions to track the health and performance of the Databricks environment, configuring alerting mechanisms for proactive issue resolution.
• Ensure all code is maintained in the existing Git repository, undergoing peer reviews to maintain quality and readability.
• Conduct System Integration Testing (SIT) and QA testing parallel to development, ensuring all critical workflows and reports function correctly.
• Document and address any issues identified during testing, compiling detailed reports for stakeholders.
• Implement the cutover strategy and production migration according to the deployment plan, communicating with stakeholders and ensuring non-restricted hours migration.
• Implement data validation checks to confirm the integrity and completeness of the migrated data.
• Document the deployment process, including encountered issues and their resolutions, and provide stakeholders with information on how to report any issues.
• Provide hyper care support post-deployment, addressing any issues promptly and transitioning to standard operational support as the environment stabilizes.

Skills and attributes for success

• 3 to 7 years of Experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases, NoSQL, and data warehouse solutions.
• Extensive hands-on experience implementing data migration and data processing using Azure services: Databricks (notebooks, workspaces, delta lake, clusters), ADLS, Azure Data Factory, Azure Functions, Synapse/DW, Azure SQL DB, Azure Event Hubs etc.
• Hands on experience in programming like python/pyspark
• Need to have good knowledge on DWH concepts and implementation knowledge in ACID transactions
• Well versed in DevOps and CI/CD deployments
• Must have hands on experience in SQL and procedural SQL languages
• Strong analytical skills and enjoys solving complex technical problems

To qualify for the role, you must have

• Be a computer science graduate or equivalent with 3 to 7years of industry experience
• Have working experience in an Agile base delivery methodology (Preferable)
• Flexible and proactive/self-motivated working style with strong personal ownership of problem resolution.
• Strong analytical skills and enjoys solving complex technical problems
• Proficiency in Software Development Best Practices
• Excellent debugging and optimization skills
• Experience in Enterprise grade solution implementations & in converting business problems/challenges to technical solutions
• considering security, performance, scalability etc
• Excellent communicator (written and verbal formal and informal).
• Participate in all aspects of solution delivery life cycle including analysis, design, development, testing, production deployment, and support.
• Client management skills

Ideally, you’ll also have

• Experience on HealthCare domains

Skills and attributes for success

• Use an issue-based approach to deliver growth, market and portfolio strategy engagements for corporates
• Strong communication, presentation and team building skills and experience in producing high quality reports, papers, and presentations.
• Experience in executing and managing research and analysis of companies and markets, preferably from a commercial due diligence standpoint.

You have to wait 20 seconds

Generating Apply Link...