HDFC Life Embraces the Cloud for Effective Financial Reporting – ET CIO

Quick access to data is paramount. This is the doctrine that HDFC Life has adopted without reservation. And they should know because they are India’s leading private life insurer with the largest number of distribution partners.

But the organization was crippled by a less than adequate on-premises enterprise data warehouse. Sunil Jain, Chief Architect and Executive Vice President, HDFC Life soon realized that the EDW was struggling to keep pace with business demands. He knew full well that he needed a cloud-enabled and fast modernization.

Sunil Jain, Chief Architect and Executive Vice President, HDFC Life explains how he did it in an interview with ETCIO

When we talk about large-scale digital operations, data becomes the mind and the cloud becomes the backbone. HDFC Life created a large on-premises data warehouse in 2013. What were the limitations you faced with the model? Why did you choose to modernize it with a cloud-based approach to achieve agility, reliability, and flexibility?

At HDFC Life, we have a large corporate data warehouse that handles sales, operations, and financial reporting. It is necessary to extract, transform and load (ETL) processes which are essential for us to prepare data models and generate daily reports in a timely manner. These processes are essential to the functioning of specific teams and play an important role in the satisfaction of internal users.

In the on-premises model, we were experiencing performance issues that hampered financial reporting. There were delays in the Extract, Transform, Load (ETL) processes that impacted our preparation of the data models on time.

ETL jobs started at midnight were taking more than 12 hours to complete. This had a negative impact on day-to-day decision-making. Employees did not have the reports handy in the morning. They had to wait until noon to work effectively with data from the day before. With issues of speed, agility, and suboptimal performance, user satisfaction was declining.

The watershed moment came when the 2021 financial close nearly failed. It was then that we decided to migrate from our on-premises infrastructure to a high-performance cloud solution to allow the process to run smoothly with increased scale and reliability and to meet deadlines and expectations.

Can you tell me more specifically how HDFC Life migrated its corporate data warehouse to the cloud?

As lead architect, I was looking for a cloud solution that offered security, scalability and reliability. I needed an advanced product that would not only support my organization’s growth goals – both organic and inorganic, but also allow us to undertake an easy transition to the cloud, without binding us. We preferred a managed service, with tied SLAs so we could provide the reliability, system performance, rich functionality, and user experience to run our complex models. Given the complexity of the existing data transfer layers, we were looking for a smooth migration.

Some of the challenges were managing the large data migration from site to cloud, rewiring data integrations to multiple data sources, VAPT and information security authorization, and managing the entire transition from site to cloud within 48 hours so there is no business impact.

The biggest challenge during the POC was the migration of huge data. The size of the data we currently hold, in terms of reports, analytics, etc., is around 30TB. So migrating this huge data from the on-premises solution to the cloud was a huge task for us. .

This was a four month project – very rich in detail and risk mitigation with daily updates and weekly governance.

What are some of the benefits you realized after the migration?

Driven by the growing demand for innovative insurance and annuity products and growth opportunities for life insurance companies in India, it was imperative for us to be able to expand our platform and provide services based on data analysis, risk analysis and profitability across various dimensions and demographics.

This meant more resource-intensive ETL tasks on our existing large databases, forcing us to look at better EDW product architecture to meet reporting and analytics needs.

With our current database service, we were able to quickly meet our requirements without rewriting code and meet workload demands without disrupting operations and staying on budget. We achieved a 48% improvement across all ETL workloads, enabling business users to perform faster queries against the enterprise data warehouse.

The speed of overnight processing tasks increased by 40%, giving data analysts more agility to make decisions. Our cloud infrastructure has improved processing speed and maximum availability has reduced ETL job runs by up to 6 hours, giving analysts access to critical data at the very start of the workday. This has significantly boosted satisfaction and improved business efficiency. Since the migration, the maintenance ticket on the infra is almost negligible thanks to the performance and the automated patches and updates of our cloud database service on the cloud infrastructure.

From an ETL point of view, with this type of architecture and infrastructure, the power was such that we improved the work by 40%, which indirectly contributed to providing 57% of the data. Since we have been in service, there has not even been a single failure, as far as the infrastructure is concerned.