Deploy

This quickstart assumes that you have performed an assessment to understand upgrade breaking changes and limitations before beginning.

This quickstart outlines how to deploy Migration Assistant for OpenSearch and execute an existing data migration using Reindex-from-Snapshot (RFS). It uses AWS for illustrative purposes. However, you can modify the steps for use with other cloud providers.

Before using this quickstart, make sure you review Is Migration Assistant right for you?.

Because this guide uses AWS Cloud Development Kit (AWS CDK), make sure that the CDKToolkit stack exists and is in the CREATE_COMPLETE state. For setup instructions, see the CDK Toolkit documentation.

Step 1: Install Bootstrap on an Amazon EC2 instance (~10 minutes)

To begin your migration, use the following steps to install a bootstrap box on an Amazon Elastic Compute Cloud (Amazon EC2) instance. The instance uses AWS CloudFormation to create and manage the stack.

Log in to the target AWS account in which you want to deploy Migration Assistant.
From the browser where you are logged in to your target AWS account, right-click here to load the CloudFormation template from a new browser tab.
Follow the CloudFormation stack wizard:
- Stack Name: MigrationBootstrap
- Stage Name: dev
- Choose Next after each step > Acknowledge > Submit.
Verify that the Bootstrap stack exists and is set to CREATE_COMPLETE. This process takes around 10 minutes to complete.

Step 2: Set up Bootstrap instance access (~5 minutes)

Use the following steps to set up Bootstrap instance access:

After deployment, find the EC2 instance ID for the bootstrap-dev-instance.

Create an AWS Identity and Access Management (IAM) policy using the following snippet, replacing <aws-region>, <aws-account>, <stage>, and <ec2-instance-id> with your information:

 {
     "Version": "2012-10-17",
     "Statement": [
         {
             "Effect": "Allow",
             "Action": "ssm:StartSession",
             "Resource": [
                 "arn:aws:ec2:<aws-region>:<aws-account>:instance/<ec2-instance-id>",
                 "arn:aws:ssm:<aws-region>:<aws-account>:document/BootstrapShellDoc-<stage>-<aws-region>"
             ]
         }
     ]
 }

Name the policy, for example, SSM-OSMigrationBootstrapAccess, and then create the policy by selecting Create policy.
Attach the newly created policy to your EC2 instance’s IAM role.

Step 3: Log in to Bootstrap and build Migration Assistant (~15 minutes)

Next, log in to Bootstrap and build Migration Assistant using the following steps.

Prerequisites

To use these steps, make sure you fulfill the following prerequisites:

The AWS Command Line Interface (AWS CLI) and AWS Session Manager plugin are installed on your instance.
The AWS credentials are configured (aws configure) for your instance.

Steps

Load AWS credentials into your terminal.

Log in to the instance using the following command, replacing <instance-id> and <aws-region> with your instance ID and Region:

 aws ssm start-session --document-name BootstrapShellDoc-<stage>-<aws-region> --target <instance-id> --region <aws-region> [--profile <profile-name>]

Once logged in, run the following command from the shell of the Bootstrap instance in the /opensearch-migrations directory:
```
 ./initBootstrap.sh && cd deployment/cdk/opensearch-service-migration
```
After a successful build, note the path for infrastructure deployment, which will be used in the next step.

Step 4: Configure and deploy RFS (~20 minutes)

To deploy Migration Assistant with RFS, the following stacks must be deployed:

These commands deploy the following stacks:

Migration Assistant network stack
RFS stack
Migration console stack

Use the following steps to configure and deploy RFS, deploy Migration Assistant, and verify installation of the required stacks:

Add the basic authentication information (username and password) for both the source and target clusters as separate secrets in AWS Secrets Manager. Each secret must include two key-value pairs: one for the username and one for the password. The plaintext of each secret should resemble the following example:
```
{"username":"admin","password":"myStrongPassword123!"}
```
Be sure to copy the secret Amazon Resource Name (ARN) for use during deployment.

From the same shell as the Bootstrap instance, modify the cdk.context.json file located in the /opensearch-migrations/deployment/cdk/opensearch-service-migration directory and configure the following settings:

 {
 "default": {
     "stage": "dev",
     "vpcId": "<TARGET CLUSTER VPC ID>",
     "targetCluster": {
         "endpoint": "<TARGET CLUSTER ENDPOINT>",
         "auth": {
             "type": "basic",
             "userSecretArn": "<SECRET_WITH_USERNAME_AND_PASSWORD_KEYS>"
         }
     },
     "sourceCluster": {
         "endpoint": "<SOURCE CLUSTER ENDPOINT>",
         "version": "<SOURCE ENGINE VERSION>",
         "auth": {
             "type": "basic",
             "userSecretArn": "<SECRET_WITH_USERNAME_AND_PASSWORD_KEYS>"
         }
     },
     "reindexFromSnapshotExtraArgs": "<RFS PARAMETERS (see below)>",
     "reindexFromSnapshotMaxShardSizeGiB": 80,
     "otelCollectorEnabled": true,
     "migrationConsoleServiceEnabled": true
 }
 }

The source and target cluster authorization can be configured to have no authorization, basic with a username and password, or sigv4.

### Environment configuration examples

To avoid conflicts with existing deployments, consider using different context IDs and stage names:

 {
 "test-deploy": {
     "stage": "test",
     "migrationAssistanceEnabled": true,
     "migrationConsoleServiceEnabled": true,
     "reindexFromSnapshotServiceEnabled": true,
     "sourceCluster": {
         "endpoint": "https://migration-source-es710.us-west-2.es.amazonaws.com",
         "version": "ES_7.10",
         "auth": {
             "type": "basic",
             "username": "admin",
             "passwordFromSecretArn": "arn:aws:secretsmanager:us-west-2:123456789012:secret:migration-source-password"
         }
     },
     "targetCluster": {
         "endpoint": "https://migration-target-os219.us-west-2.es.amazonaws.com",
         "auth": {
             "type": "basic",
             "username": "admin",
             "passwordFromSecretArn": "arn:aws:secretsmanager:us-west-2:123456789012:secret:migration-target-password"
         }
     }
 },
 "prod-deploy": {
     "stage": "prod",
     "migrationAssistanceEnabled": true,
     "migrationConsoleServiceEnabled": true,
     "reindexFromSnapshotServiceEnabled": true,
     "// ... additional production-specific configuration"
 }
 }

Important Notes:

Use unique stage values to prevent resource naming conflicts.
Ensure secret ARNs are complete and accessible in your deployment Region.
Domain endpoints can be simplified names or full AWS URLs.
Deploy using ./deploy.sh <contextId> (for example, ./deploy.sh test-deploy).

After the cdk.context.json file is fully configured, bootstrap the account and deploy the required stacks using the following command:
```
 cdk bootstrap --c contextId=default --require-approval never 
```

Deploy Migration Assistant using the following command:

 cdk deploy "*" --c contextId=default --require-approval never --concurrency 5

From the same Bootstrap instance shell, verify that all CloudFormation stacks were installed successfully:

 aws cloudformation list-stacks --query "StackSummaries[?StackStatus!='DELETE_COMPLETE'].[StackName,StackStatus]" --output table

You should receive a similar output for your Region:

------------------------------------------------------------------------
|                              ListStacks                              |
+--------------------------------------------------+-------------------+
|  OSMigrations-dev-us-east-1-MigrationConsole     |  CREATE_COMPLETE  |
|  OSMigrations-dev-us-east-1-ReindexFromSnapshot  |  CREATE_COMPLETE  |
|  OSMigrations-dev-us-east-1-MigrationInfra       |  CREATE_COMPLETE  |
|  OSMigrations-dev-us-east-1-default-NetworkInfra |  CREATE_COMPLETE  |
|  MigrationBootstrap                              |  CREATE_COMPLETE  |
|  CDKToolkit                                      |  CREATE_COMPLETE  |
+--------------------------------------------------+-------------------+

RFS parameters

If you’re creating a snapshot using migration tooling, these parameters are automatically configured. If you’re using an existing snapshot, modify the reindexFromSnapshotExtraArgs setting with the following values:

    "reindexFromSnapshotExtraArgs": "--s3-repo-uri s3://<bucket-name>/<repo> --s3-region <region> --snapshot-name <name>"

You will also need to give the migrationconsole and reindexFromSnapshot TaskRoles permissions to the S3 bucket.

Step 5: Access the migration console

Run the following command to access the migration console:

./accessContainer.sh migration-console dev <region>

accessContainer.sh is located in /opensearch-migrations/deployment/cdk/opensearch-service-migration/ on the Bootstrap instance. To learn more, see Accessing the migration console.

Step 6: Verify the connection to the source and target clusters

To verify the connection to the clusters, run the following command:

console clusters connection-check

You should receive the following output:

SOURCE CLUSTER
ConnectionResult(connection_message='Successfully connected!', connection_established=True, cluster_version='')
TARGET CLUSTER
ConnectionResult(connection_message='Successfully connected!', connection_established=True, cluster_version='')

To learn more about migration console commands, see Migration console command reference.

Troubleshooting

The following section covers common deployment issues and resolutions.

Common deployment issues

Problem: AWS credentials not configured

Unable to locate credentials. You can configure credentials by running "aws configure".

Resolution:

Run aws configure and provide your access key, secret key, and Region.
Alternatively, set environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION.
Verify credentials with aws sts get-caller-identity.

Problem: Stack naming conflicts

Stack with id OSMigrations-dev-us-west-2-MigrationConsole already exists

Resolution:

Use a different stage value in your context configuration (for example, “test”, “staging”).
Or destroy existing stacks: cdk destroy "*" --c contextId=<existing-context>.
Ensure unique context IDs for parallel deployments.

Problem: Docker build failures

ERROR: failed to solve: public.ecr.aws/sam/build-nodejs18.x: pulling from host public.ecr.aws failed

Resolution:

Run docker logout public.ecr.aws to clear the authentication cache.
Retry the build process: ./buildDockerImages.sh.

Problem: CDK bootstrap required

This stack uses assets, so the toolkit stack must be deployed to the environment

Resolution:

Bootstrap the CDK in your Region: cdk bootstrap --c contextId=<your-context>.
Ensure you have configured the correct AWS credentials and Region.

Rollback procedures

If you need to remove a deployment:

Stop all running services:

console backfill stop  # If backfill is running

Destroy CDK stacks:

cdk destroy "*" --c contextId=<your-context> --force

Clean up manually if needed:
- Remove any remaining CloudFormation stacks from the AWS Management Console.
- Delete any orphaned resources, for example, Amazon Elastic Container Service (Amazon ECS) tasks and load balancers.

Next steps

After completing the deployment, proceed with the migration phases:

Create a snapshot: Create a snapshot of your source cluster.
Migrate metadata: Migrate cluster metadata to the target.
Backfill: Migrate documents and monitor the process.

For more information about the complete migration process, see Migration phases.

Step 1: Install Bootstrap on an Amazon EC2 instance (~10 minutes)
Step 2: Set up Bootstrap instance access (~5 minutes)
Step 3: Log in to Bootstrap and build Migration Assistant (~15 minutes)
- Prerequisites
- Steps
Step 4: Configure and deploy RFS (~20 minutes)
- RFS parameters
Step 5: Access the migration console
Step 6: Verify the connection to the source and target clusters
Troubleshooting
- Common deployment issues
- Rollback procedures
Next steps

WAS THIS PAGE HELPFUL?

✔ Yes ✖ No

Tell us why

350 characters left

Have a question? Ask us on the OpenSearch forum.

Want to contribute? Edit this page or create an issue.

Deploy

Step 1: Install Bootstrap on an Amazon EC2 instance (~10 minutes)

Step 2: Set up Bootstrap instance access (~5 minutes)

Step 3: Log in to Bootstrap and build Migration Assistant (~15 minutes)

Prerequisites

Steps

Step 4: Configure and deploy RFS (~20 minutes)

RFS parameters

Step 5: Access the migration console

Step 6: Verify the connection to the source and target clusters

Troubleshooting

Common deployment issues

Rollback procedures

Next steps

Related articles

OpenSearch Links

Get Involved

Resources

Contact Us