Migration Guide (Archive)

As we previously shared, the Parse service was shut down on January 30, 2017.

You will not be able to export or access your data in any way after your app is disabled on January 30, 2017. This document will guide you through the steps needed to set up your own open-source Parse Server. At this point, it is no longer possible to migrate a Parse.com database.

This migration guide assumes the use of Heroku and either mLab or ObjectRocket. These three services are easy to use, especially if you are new to deploying and managing your own backend stack. But, you can elect to use any infrastructure provider that supports Node.js environments.

After completion, you will have the following:

Parse Server running on your computer, allowing you to develop locally.
Parse Server running on Heroku or another infrastructure provider.
Your app's data stored in MongoDB hosted on mLab or ObjectRocket.
Your app's static files stored in S3 or another hosting provider.
No dependency on *.parseapp.com.
Your app's client-side code updated to point to your Parse Server instance, ready to be released.
No dependency on api.parse.com for the new app client.

We highly recommend that you first run through this guide with a development or test version of your app before working with a production app.

Migrating your Parse database

The first step is to migrate the data from your Parse hosted app to a self-hosted MongoDB. After this is done, your clients will continue talking to api.parse.com, and you may manage your app using the hosted Parse Dashboard.

Once your data is migrated to your self-hosted MongoDB, you will be able to adjust your app's request limit all the way up to 600 requests/second, at no extra charge. You will not be charged by Parse for any database storage fees as the data is no longer hosted on Parse's servers.

Once your data is migrated, your app's traffic to api.parse.com will be given a higher priority over apps that have not yet migrated their data. If your app has not migrated its data to a self-hosted MongoDB by April 28, 2016, a small number of database requests may fail with a 428 error code. If you run into this, migrate your data to a self-hosted MongoDB without delay.

1. Migrate Parse DB to Self-Hosted MongoDB

Set up a MongoDB instance that conforms to our database specifications. If this is your first time setting up a production MongoDB instance, we recommend using either mLab or ObjectRocket. These are database-as-a-service companies which provide fully managed MongoDB instances, and can help you scale up as needed.

Once you have Mongo set up, take note of the Mongo connection URL. Use the database migration tool to transfer your data. The database migration tool can be found in the Dashboard under App Settings → General → Migrate to external database.

Make sure to size your MongoDB at least 3x. Due to data being compressed in the hosted Parse database, make sure to size your Mongo at least 3X the current amount of data storage you are using (you can find this information in your hosted Parse app's Analytics overview page).
Ensure the user in the connection string has admin privileges. If the user in the connection string does not have admin privileges, you will need to set the failIndexKeyTooLong parameter to false yourself prior to the migration. You can configure it back to true after the migration is completed. (Why?)
Latency between Parse and your self-hosted MongoDB should not exceed 20ms. We recommend choosing either mLab or ObjectRocket for your hosted MongoDB as they both have datacenters in the US East geographic region.

If you plan on hosting your production database in a different geographic region, you can do so after first migrating your data out of Parse and into the self-hosted MongoDB in US East. You should also plan to host your Parse Server in the same geographic region to minimize latency.
Over-provision your database during the migration. Make sure you're running on a good sized instance or plan. This usually means working with someone like mLab or Object Rocket and paying more for a high performance line. If your migration job takes to long to achieve sync, you're probably using insufficient hardware. Once you finalize the migration, you can trim down your data, drop duplicated indexes, and finally downsize your instance.

Finding the right long term balance between price and performance will be left to you, but for migrating aim on the higher priced side for best performance. (This advice also applies to folks hosting on AWS, Digital Ocean, Heroku, or our other partners.)

Connections from Parse's servers will use IP addresses in the following range: 54.85.224.0/20

What happens next?

Copy Snapshot The database migration tool first takes a snapshot of your existing data and transfers it to your Mongo database.
Sync Next, it will pause to allow manual verification, while continuing to keep things in sync with writes that are coming in from your live app. While you are in this state, your app continues to read and write from your Parse hosted database.
Verify Connect to your Mongo instance and browse through the collections in the newly created database. Check the collection counts and do a few spot checks to ensure that your data was migrated successfully.

You may stop the migration and try again as many times as you need (until you click on Finalize). The tool will keep things in sync for up to 24 hours after the migration started. You can go to App Settings → General → Migrate to external database and click on "Migrate" to start over if needed. You will first need to clean up your target database as the tool will only migrate data to an empty database.

Verify and back up your data

Once the migration job completes the Sync phase, take some time to Verify your new database. You are responsible for the backup and recovery of your data, so go ahead and back up your new database before proceeding. Both mLab and ObjectRocket have step-by-step guides to perform your first backup.

Finalize the database migration

Once you're satisfied with the database migration, you can finalize the transfer in the migration UI.

It is important to note that the Finalize action is irreversible. You will not be able to switch back to the Parse managed database. You will be responsible for any backups and recovery of your data. Only proceed once you have backed up your new database.

I have finalized the migration. Now what?

At this point, your app is still hitting api.parse.com, but is using your MongoDB instance. You will need to administrate your database instance yourself, including maintaining indexes and scaling up. Parse cannot assist with the recovery of data that is no longer hosted on its servers, so it is very important that you back up your self-hosted database regularly.

Troubleshooting common errors

The destination database was not empty. The database you are migrating to is not empty. Please clean up and retry.
The destination database was too small. The database you are migrating to is too small to hold all of your data. Add more space to your host or buy more space if you are using a service such as mLab or ObjectRocket.
This migration was cancelled. You can try again from the app settings page. The job was cancelled manually by the user.
This migration was not finalized in time. When a migration job is ready to be finalized, Parse will keep the database in sync for 24 hours. If you do not take any action to finalize the job within 24 hours, the job will be cancelled automatically.
The mongo user provided is not authorized to do migration. The user in the connection string doesn't have the necessary access to complete the migration.
You must set failIndexKeyTooLong option. You need to set the failIndexKeyTooLong setting of your mongo server to false.(Why?)
The job failed too many times and reached the max retry limit. The job failed several times and we gave up trying. One possible reason can be that your database hardware cannot handle the migration load. If you have large collections (containing more than 1 million objects), please consider upgrading your database hardware. During the migration the system throughput is much higher than normal, so it requires better hardware. You can resize your host after the migration is done.
The job failed and we cannot clean up the destination and retry.The job failed, likely due to a timeout, and it could not try again as it was unable to drop the collections in the destination database. One possible reason can be that your database hardware is not sufficient. Please consider upgrading your database hardware (mainly, increase the RAM) and try again. If the problem remains, open a support ticket.

If your database migration job does not reach the final verification step after several attempts, take note of your migration job id and file a bug report with our team. Make sure to include as much detail upfront as possible:

Migration job id
MongoDB version
Storage engine
Hosting provider
Hardware details (CPU, RAM, available disk space)

Setting up Parse Server

There are a few areas where Parse Server does not provide compatibility with the Parse hosted backend. Carefully read through the list of compatibility issues with hosted Parse before proceeding.

2. Set Up Local Parse Server

Follow the instructions in the Parse Server Sample App and use the Mongo connection string from Step 1.

Go to the Security & Keys section of App Settings in your Dashboard and take note of the File Key and Master Key values. Pass that into the ParseServer constructor in index.js. You no longer need to use a client key with Parse Server.

Verification

Make sure saving an object and retrieving it via a query works:


curl -X POST \
  -H "X-Parse-Application-Id: YOUR_APP_ID" \
  -H "Content-Type: application/json" \
  -d '{"score":1337,"playerName":"Sean Plott","cheatMode":false}' \
  http://localhost:1337/parse/classes/GameScore

curl -X GET \
  -H "X-Parse-Application-Id: YOUR_APP_ID" \
  -H "X-Parse-Master-Key: YOUR_APP_MASTER_KEY" \
  http://localhost:1337/parse/classes/GameScore

You now have a Parse Server running locally that is connected to the data in MongoDB from step 1.

3. Cloud Code

We will now migrate your existing Cloud Code to run in Parse Server. Copy your app’s Cloud Code to the parse-server/cloud directory, replacing the example main.js. You will need to replace any relative paths like 'cloud/…' to './cloud'. The Parse.Cloud.useMasterKey() method, Parse.User.current() method, and native Parse.com Cloud Code modules are not available in Parse Server. See our compatibility guide to see a list of workarounds.

Verification

Run Parse Server and make some calls to it to verify that your Cloud Code is running correctly. If you had tests setup for the Cloud Code in your Parse hosted backend, point them to your local Parse Server and run the test suite.

Because the Parse hosted Cloud Code isn’t running a full node environment, there may be subtle differences in how your Cloud Code runs in Parse Server. We recommend exercising all your critical code paths to ensure full functionality.

4. Hosting

If you are using Parse Hosting, you can migrate all these web endpoints to the same Express app that is serving Parse Server. For example, you could mount Parse Server under /parse and your website at the root, like so:


var api = new ParseServer({ ... });
app.use('/parse', api);

// Web endpoints
app.get('/', homeController.index);
app.get('/about', aboutController.index);

// ...

5. Files

When using Parse Server, any new file you create will be saved on the data store you select through the files adapter (MongoDB, S3, ...). All of your existing files will remain in Parse's hosted service. As long as you specify the correct fileKey, Parse Server knows exactly how to access them, so they will keep working just fine.

Please be aware that new files that are uploaded to Parse Server will not be accessible to older clients that use api.parse.com as the hosted Parse server only supports serving static files from its own S3 bucket.

Migrating Parse Files

Parse's S3 bucket will no longer be accessible after the hosted Parse service is shut down on January 30, 2017. You will need to migrate your old files before that date. You may use the files utility to perform this migration.

6. App Settings

Go through your app settings panel and make sure to understand how these settings will be impacted by moving to Parse Server.

User Sessions

Require revocable sessions - This is required by Parse Server. If you are using legacy user sessions, you will need to migrate your user sessions prior to using Parse Server.
Revoke session on password change - This is required by Parse Server.

User Authentication

Enable new methods by default - This is hardcoded as true in Parse Server.
Allow username and password-based authentication - This is currently not optional in Parse Server, username and password based accounts are always enabled.
Allow anonymous users - This is currently not optional in Parse Server, anonymous users are always enabled.

Social Login

Allow FB auth - This is available in Parse Server if a Facebook App ID is configured.
Add a Facebook app (list of apps) - The ability to restrict to one Facebook App is not available in Parse Server.

7. Point Client to Local Parse Server

Update your app with the latest version of the Parse SDK (at least version 1.12 for iOS, 1.13.0 for Android, 1.6.14 for JS, 1.7.0 for .NET), which have the ability to change the server URL.

8. Checkpoint: Test Your App

Now, test your app locally. We recommend running a staging database using a snapshot of your production database, as there may be legacy data in your database that exercises code paths that wouldn't otherwise be exercised. Be very careful if your Parse Server is pointing to the same Mongo instance as your live app, as you could be mutating production data.

At this point, your app may be totally functional. Objects, queries, and users will work right out of the box.

Deploying Parse Server

9. Set Up Parse Server on Heroku or another infrastructure provider.

Follow the instructions for deploying the server to Heroku, NodeChef, AWS, Digital Ocean, Google Cloud Platform, Azure, or any other infrastucture provider.

10. Point Client to Parse Server

Now, update your client to point to the location of the API that you just deployed.

11. Checkpoint: Test Your App

Test your app now that it uses the Parse Server backend.

12. Publish Your App

You can now publish the new app, which will utilize your new backend. You should encourage users to update to the new version of your app. Once your app has been disabled on January 30, 2017, any calls to the hosted Parse backend service (api.parse.com) will cease to function.