Hello again! 👋🏻
Most of the time, in a project, we are working with different environments and different data between them. Sometimes this lack of synchronicity brings very tricky bugs.
Also, another circumstance where knowing this may be useful is when we are working with dummy data and want to bring back an earlier version of the data.
Pre-requisites
- You will only be able to import or export data if you are in the Blaze plan.
- The account that will make the migration will need the following roles:
- For Firestore: Owner, Cloud Datastore Owner, or Cloud Datastore Import Export Admin
- For Storage: Owner or Storage Admin
- Have a storage bucket in the same region as the Firestore Database.
- Have Google Cloud SDK CLI installed and configured. (Can be omitted, more on that later)
Once we take care of all the pre-requisites, we can move on to make, what I call, a Flight Plan. That means writing down the steps we need to do to accomplish the task.
The Flight Plan
Make a backup before the task:
It's a good practice to make a backup before getting our hands dirty, just in case.
- Export origin collection/database as is.
- Export destination collection/database as is.
- Make the modifications we want in the origin database. We may skip this step if we want to migrate the database as it is at the moment.
- Export the origin database with the changes.
- Import the changed export into the destination database.
Export a collection
Go to your Google Cloud Console: https://console.cloud.google.com and select the project where you want to do the export, in this case, app-dev.
Then, on the sidenav look for "Firestore" and then select "Import/Export"
Then, on the side nav look for "Firestore" and then select "Import/Export" where you will see this screen. Click on "Export".
Here we can choose to export the entire database or just a collection. We are going to export the whole database for the sake of this tutorial, so let's just select "Export entire database". As the destination, here we set the bucket where we are saving the export.
As the destination, here we set the bucket where we are saving the export. I rather use the one with the {{projectId}}.appspot.com. Feel free to create a new one if you are going to make backups or multiple migrations. Then click on the "Export" button.
Once it starts exporting data, Google will takes us to this screen where we can see the progress of the export.
Let's follow the same process for the destination database and continue with the changes (when it applies)
Making changes in the DB
Feel free to make the changes you want. In this example, we are going to just add a new document.
Now the origin DB has the changes we want to migrate to the destination DB so we can proceed with the next part.
Export
Now we are going to export again, but this time let's write down the name of the export. We can find it when we hover the "bucket" column in the row that belongs to the export we just triggered.
Move the export
Now we have the changes made to the database exported but in the DEV bucket. The thing is that to import the data, we must move the export to the UAT bucket. To do that, we are going to use the GSUTIL CLI.
The command is:
gsutil cp -r gs://app-dev-99948.appspot.com/2021-11-09T11:34:22_164 gs://app-uat-5d2bc.appspot.com/2021-11-09T11:34:22_164
Pro tip
You can use Cloud Shell if you want to avoid installing Google Cloud SDK
Import the data
Importing the changes is quite similar to exporting them. Let's go to Google Cloud Console > Firestore and then click on "Import/Export"
Click on "Import"
Then, inside the export we've just moved, we'll find an "export_metadata" file. Select it. Then click on the "Select" button.
It will bring us back to the last screen with the file selected and just click on "Import"
Once it finishes you will see the green tick on the left side of the table
And finally, we will find the migrated documents on Firestore.
And that's how we migrate Firestore's data between projects.
I'd like to thank Alvaro Lopez Vaquerizo who helped me shape the documentation I used for this post.
As always, I hope you can take anything useful from this and I'd like to know your thoughts.