AWS Database Blog

Australia Finance Group’s Journey to Cloud Databases: Migrating from Oracle Exadata to HAQM RDS

Australia Finance Group (AFG) is one of the largest mortgage aggregators in Australia, according to the Mortgage and Finance Association of Australia. AFG migrated their Oracle Siebel Customer Relationship Management (CRM) application to a managed database with HAQM RDS for Oracle. In this blog post, we describe their approach to migrating the database from Oracle Database 11.2 on Oracle Exadata on-premises, to Oracle Database 12.1 on HAQM RDS for Oracle. We also provide tips for successful migrations of large database applications to HAQM RDS.

Background
AFG’s applications were hosted out of two data centers, and more than 80 percent of IT expenditure at AFG was consumed by IT operational costs. AFG wanted to reduce operational costs and free resources for innovation by migrating these applications to the cloud and decommissioning the data centers.

At AFG, the application most critical to the business is the Oracle Siebel CRM. This application helps AFG coordinate communications with customers across multiple channels and lodge home-loan applications electronically. The Oracle platform is used by all 2,800 of AFG’s brokers and their support staff.

For the CRM database, AFG was looking for a solution to reduce operational costs and result in low management, maintenance, and administrative overheads. At the same time, AFG needed a solution that was highly available and scalable.

On-premises setup
In the on-premises setup, the Siebel CRM database (Oracle 11.2) was deployed on an Oracle Exadata X2-2 Quarter rack system. The Exadata Quarter rack had two database servers, each with two six-core Intel Xeon processors and 96 GB RAM, for a total of 24 CPU cores and 192 GB RAM. The Exadata quarter rack also had three Exadata Storage servers, with a total of 36 CPU cores for SQL processing and 1.1 TB of Exadata smart flash cache. The Siebel CRM database size was 1.2 TB.

For disaster recovery, the Siebel CRM database was replicated to another Oracle Exadata quarter rack in a secondary data center using Oracle Data Guard replication.

The following diagram shows this setup.

Oracle Data Gaurd Replication

Choosing between Oracle on HAQM EC2 and HAQM RDS for Oracle
Before migrating the Siebel CRM database to AWS, AFG had to decide between Oracle on HAQM EC2 and HAQM RDS for Oracle. They decided to deploy the Siebel CRM database on HAQM RDS for Oracle instead of Oracle on HAQM EC2 due to the following considerations:

  • HAQM RDS has a lower management and maintenance overhead because it is a managed service.
  • HAQM RDS has a lower administration overhead due to automated patching and simple backup and recovery process.
  • It is relatively easier to implement a Multi-AZ deployment using RDS, compared to a Multi-AZ deployment with HAQM EC2 and Oracle Data Guard.
  • You can easily snapshot an RDS instance to create a copy. This approach is useful if you want to clone your production database to replicate and troubleshoot application issues.
  • With HAQM RDS, you can easily scale up or scale down the compute and memory resource of your instance in a few minutes. This scaling comes in handy if you need to temporarily scale up the database instance while you identify and resolve application performance issues.

Multi-AZ deployment on AWS
The following diagram shows the final Siebel CRM deployment on AWS after migration.

The Siebel application servers and the Siebel database on HAQM RDS were deployed across multiple Availability Zones (AZs). Multi-AZ deployments provide enhanced availability and durability for HAQM RDS DB instances, making them a natural fit for production database workloads.

When you provision a Multi-AZ DB instance, HAQM RDS automatically creates a primary DB instance and synchronously replicates the data to a standby instance in a different AZ. In case of an infrastructure failure (for example, instance hardware failure, storage failure, or network disruption), HAQM RDS performs an automatic failover to the standby instance, which becomes the new primary. Because the endpoint for your DB instance remains the same after a failover, the Siebel servers can connect to the new primary instance and resume operations without any data loss.

To learn how to set up HAQM RDS for Oracle as the database backend of your Siebel CRM application, see Installing a Siebel Database on Oracle on HAQM RDS in the HAQM RDS documentation.

Migrating the database using Oracle Streams
The Siebel CRM database was 1.2 TB in size, and AFG wanted to minimize the downtime during migration. To do so, they took an Oracle Data Pump export of the on-premises database, copied it to HAQM S3 and imported it into the HAQM RDS instance. Then they configured Oracle Streams replication between the on-premises database and the HAQM RDS instance. They used Oracle Streams replication because HAQM RDS for Oracle does not currently support Oracle Data Guard replication, which may have been a more obvious choice.

Before cutover, AFG stopped the production application servers, waited for the remaining changes to be replicated to HAQM RDS, and shut down the on-premises database. Then the Siebel application was installed on HAQM EC2 instances and Siebel Enterprise was configured to use the database on HAQM RDS. Finally they cut over by repointed the DNS to the Siebel application on AWS.

The following diagram shows replication of database changes from the on-premises setup to HAQM RDS using Oracle streams.

Oracle Streams replicates information from one Oracle database to another and can capture both data definition language (DDL) and data manipulation language (DML) changes. The capture process in the source database captures information from the database redo log, converts it into logical change records (LCRs), and stores these in a source queue. The LCRs are propagated from the source queue to a destination queue in the destination database. The apply process takes the LCRs from the destination queue and applies them to the destination database. You can configure Oracle Streams to replicate changes for tables, schemas, tablespaces, or databases. The source database must be in ARCHIVELOG mode and supplemental logging must be enabled.

Performance tuning and sizing the HAQM RDS instance
The simplest way of sizing your HAQM RDS instance might be to find the CPU cores, RAM, disk size, and IOPS used on-premises and select an appropriate HAQM RDS instance based on that. Although this method is easy, it’s not optimal. Tuning and right-sizing your database before migration can help reduce AWS infrastructure costs and also database licensing costs.

In this case, AFG was migrating from an Oracle Exadata machine on-premises to Oracle Enterprise Edition on HAQM RDS. AFG had to identify some of the queries benefiting from Exadata capabilities such as Smart Flash Cache and Smart Scan and tune or optimize them on HAQM RDS. The DBA team and the Siebel team at AFG worked together to identify and tune the database before migration to AWS. The teams took the following steps:

  • They set up a performance test environment on HAQM RDS and imported the full production data.
  • They used a SQL record and playback tool from Ponder Proserve to identify differences in query execution time. They recorded a day’s worth of transactions from the production database on-premises and replayed these transactions on the HAQM RDS instance. To find which queries were running slower, the run time of each of these queries on HAQM RDS was recorded and compared to the run time for each on the production database.
  • To solve issues of slowness, the teams did the following:
    • Individual queries performing poorly were identified and tuned by adding or adjusting the indexes, or by making changes to the Siebel application.
    • The teams migrated from Oracle 11.2 on Exadata to Oracle 12.1 on HAQM RDS for Oracle. They had to adjust the optimizer_features_enable and optimizer_index_cost_adj database parameters to optimize the execution plans and to favor Index scans instead of full table scans.
    • To identify the optimal HAQM RDS instance size, the teams experimented with different HAQM RDS instance sizes and disk options while playing back the recorded queries.

AWS Infrastructure Event Management
AFG took advantage of AWS Infrastructure Event Management (IEM) to help ensure a smooth migration and mitigate risks. AWS IEM assisted AFG in reviewing and fine-tuning the migration plan before cutover and support was available during the migration to help them in resolving issues that occurred post migration.

Summary and lessons learned

This blog post outlines the steps AFG took to migrate their mission-critical Siebel CRM database running on Oracle Exadata to HAQM RDS for Oracle with minimal downtime. While migrating large databases to HAQM RDS, we recommend keeping the following points in mind:

  • Assess your current environment and identify the infrastructure resources available to your database (CPU cores, RAM, disk size, disk IOPS, and so on). While estimating the compute requirements on AWS, factor in the difference in processor performance levels and clock speeds on-premises compared to on For example, the R4 instances on AWS use the Intel Xeon E5-2686 v4 (Broadwell) processors, which are faster than, say, a 5-year-old Intel Xeon processor model on-premises. Also factor in the utilization levels of the hardware on-premises while estimating your compute, storage, and memory requirements on RDS.We recommend using comparative sizing based on existing Oracle Automatic Workload Repository (AWR) reports to size your database. When you are migrating from Exadata, you should also considering the savings due to smart scans and storage indexes while calculating the throughput requirements. For more details on sizing on AWS, see the whitepaper Demystifying the Number of vCPUs for Optimal Workload Performance on the AWS website.
  • Set up a performance test environment on HAQM RDS, and import the full data from on- You can use Oracle Real Application Testing or similar tools to capture transactions from the production database and replay them in the performance test environment. This approach helps identify poorly performing queries that require tuning.
  • HAQM RDS should be your default choice while migrating to AWS. You can consider deploying Oracle on HAQM EC2 when you cannot use HAQM RDS for Oracle due to certain restrictions like the following:
    • The database size is close to or greater than the storage limit for HAQM RDS for Oracle.
    • The CPU, memory, or IOPS required is greater than what is available on the largest RDS instance.
    • You are using an application that requires Sys or System user access.
  • Migrating large databases to HAQM RDS within a small downtime window requires careful planning:
    • We recommend that you take a point-in-time export of your database, transfer it to AWS, import it into HAQM RDS, and then apply the delta changes from on-premises.
    • You can use AWS Direct Connect or AWS Snowball to transfer the export dump to AWS.
    • You can use AWS Database Migration Service (AWS DMS) to apply the delta changes and sync the on-premises database with the HAQM RDS instance. You can also use Oracle Streams replication as described in this blog post, but setting it up is more complicated compared to AWS DMS. When AFG migrated to AWS, AWS DMS was not available, so they chose to use Oracle Streams.

About the Authors

Ashok Shanmuga Sundaram is a partner solutions architect with the Global System Integrator (GSI) team at HAQM Web Services. He works with the GSIs to provide guidance on enterprise cloud adoption, migration and strategy.