Migrating Open-Source Databases to Azure

Open-source databases like MySQL, MariaDB, and PostgreSQL are more popular than ever. However, running these databases on bare metal machines can be time consuming, and it can be difficult to scale up and down and ensure high availability for critical workloads. An attractive option is to run these databases in the cloud.

The Azure cloud is known as an excellent choice for enterprise workloads and has possibly the best security features of the big three public clouds. It also has advanced automated migration capabilities that make it easy to move your workloads to Azure. But is it suitable for the simple task of running an open-source database? Let’s find out.

Running Open-Source Databases as a Managed Service in Azure

The Microsoft Azure cloud provides several options for migrating open-source databases from your local environment to the cloud.

The following services can host these databases and take care of administrative tasks like maintenance and upgrades, scalability, and high availability. These managed services can also enhance open source security by automating patches and security updates and hardening the database host according to database security best practices.

• Azure Database for MySQL—a managed service that can be deployed as a single server or Flexible Server that can scale across multiple Azure virtual machines (VMs). It runs the original open-source MySQL engine and can support mission-critical workloads with high performance and automated scalability.
• Azure Database for MariaDB—a managed service that lets you run MariaDB, a popular fork of MySQL which is not controlled by Oracle.
• Azure Database for PostgreSQL—a managed service that lets you run PostgreSQL, with the same availability, performance, scaling, security, and administrative benefits as the MySQL service.

You can use several techniques to migrate your database to the relevant Azure managed service—see the migration techniques section below.

Moving Open-Source Databases to Azure in an IaaS Model

If you need more control over your cloud database, you have the option of running open-source databases in Azure in a self-managed model. You can create a VM in Azure and directly install a database like MySQL, or use tested, pre-configured VMs from the Azure Marketplace, which have popular database engines installed. Note that some Azure Marketplace VMs come at an additional cost.

Depending on your requirements and your system’s complexity, here are several approaches to implement an IaaS model for your database.

Lift and Shift

Begin by mirroring the on-premises architecture in the cloud by creating a virtual network infrastructure matching your data center. You then create a VM for each server install and configure your database management system on the VMs. Finally, with minor client code or configuration changes, lift and shift the system into the cloud.

This strategy is a step-by-step process for migration. Use Azure network gateway to move some parts of your system to the cloud incrementally, while some elements remain on-premises. This hybrid approach helps stabilize the system at each step and roll back if needed.

Use Marketplace Virtual Machines

Virtual machines pre-configured for MySQL, MariaDB, and PostgreSQL servers are available in the Azure Marketplace. You don’t need to install the database server software with these images, which can save considerable time and improve reliability. See if there is an image in the Marketplace with the correct version of the database software while you are setting up database VMs to use.

Use MySQL, MariaDB, or PostgreSQL Containers

Containerization is another virtualization technology you can consider. Containers are like VMs, but they share the operating system with the host computer. You can run more containers on a host VM, improving utilization. You’ll need container runtime software like Docker to run containers on a physical server. Prepare container images with your open source database, and run them within the VM.

Another option is to run containers directly in Azure without using VMs. You can do this via the Azure Container Instance (ACI) service. Or, if you have Kubernetes expertise, use the Azure Kubernetes Service (AKS) to manage large numbers of containers that need to communicate with each other.

5 Techniques for Migrating Open-Source Databases to Azure

Here are a few techniques you can use to migrate your database to Azure. These are relevant both for managed service migration and for IaaS migration.

1. Export and Import

Export and import lets you move a database while retaining tight control over the schema and data that is moved during migration. You can use your current database's export and import tool to select data to migrate to a new database and clean or modify data during the migration process.

Consider using export and import if you need to:

• Select a subset of tables in your local database to migrate to a cloud database.
• Migrate database objects such as constraints, views, functions, procedures, and triggers with full control of how these objects are organized in the destination database.
• Import data from external sources other than MySQL, MariaDB, or PostgreSQL into an Azure managed service for one of these database engines.

2. Backup and Restore

Backup and restore operations are typically used to protect databases from disasters, but you can also use them for migration. A backup operation creates an exact copy of the database, from which you can resume operations in case of damage to the original database.

You can move your entire database to another location, such as a database in the cloud, by taking a backup and restoring it to the new location.

Consider using backup and restore if you need to:

• Migrate an entire database or group of databases in one operation.
• Migrate a database with no changes to data, schema, or other database objects.

3. Azure Database Migration Service (DMS)

Azure provides DMS, an automated service that can help you migrate databases to the cloud. You can use it to perform seamless online migrations from multiple data sources to the Azure Data Platform. DMS supports migration to Azure Database for MySQL, Azure Database for MariaDB, and Azure Database for PostgreSQL.

Consider using Azure DMS if you need to:

• Perform online migration without disrupting existing workloads.
• Migrate databases that are compliant with the requirements of Azure’s managed database services (you can perform an automated assessment to identify compliance).

4. SQL Server Migration Assistant for MySQL

Azure provides a dedicated tool for migrating from MySQL to Azure SQL Database, the cloud version of Microsoft SQL Server. This is a GUI tool called SQL Server Migration Assistant for MySQL. It connects to the source MySQL database and target SQL Server database, which can either be a standalone SQL Server deployment or an Azure SQL Database instance.

Once connected, the tool copies the entire schema to SQL Server or Azure SQL Database and converts all data types to their SQL Server equivalents. It also migrates views, procedures, triggers, and other objects. Then the tool allows you to start migrating the actual data.

Consider using SQL Server Migration Assistant for MySQL if you need to:

• Migrate an existing MySQL server to Azure SQL Database.

5. Custom Application Code

If you have complex data transformation requirements, or want to perform a migration with special requirements, consider writing your own custom code to move data from an on-premises MySQL, PostgreSQL, or MariaDB database to the cloud.

The custom code approach provides great flexibility. You can customize how data is filtered, aggregated and transformed, migrated to multiple destinations, and combined from multiple sources. Use this method if you have needs that are not met by out-of-the-box backup or export tools.

The downside of this approach is that it requires more development time, testing, and management. To successfully migrate any data via custom code, you must thoroughly test it before running it on real data. This often increases project budgets as it requires a team of experienced data engineers. If you're considering writing custom migration code, don't underestimate the time and effort it requires.

Conclusion

In this article, I explained the basics of Azure managed services for popular open-source database engines like MySQL, MariaDB, and PostgreSQL. I also showcased five technical methods to migrate your open source database to the Azure cloud:

• Export and import—using the export/import tool in your database engine to create an export file representing your database or part of it and re-importing in the cloud.
• Backup and restore—creating a full backup of your database and restoring it in a cloud location.
• Azure Database Migration Service (DMS)—an automated tool that performs online migration of existing databases to Azure managed services.
• SQL Server Migration Assistant for MySQL—an automated tool that can help convert a MySQL database into a SQL Server database in Azure.
• Custom application code—performing migration via custom development, which provides the most flexibility.

I hope this will be useful as you evaluate the options for moving your open-source databases to the Azure cloud.