Connect Google Cloud Firestore to Wix (External DB Collection)

Hello! I wanted to create a long and easy to follow tutorial about how to connect Google Firestore to your Wix websites/web apps. You can follow this tutorial with any editor you use. (Wix Editor, Editor X, Wix Studio or Wix Blocks)

Keep in mind if you are experienced developer you can connect any external db source to your Wix website. Some adapters already made by Wix to use with GCP, AWS or Azure.

Reason of this tutorial to be with Firestore is because Wix didn’t provide a tutorial about that (in fact there was a tutorial but they removed it I don’t know why) and Firestore is very cheap most of you probably will pay 0$/m because you will stay in free tier of Firestore and also Firestore is document based database which is same with Wix collections. But every good thing has it’s own bad sides.

If you want to connect different DB service like CloudSQL or even from a different provider like AWS checkout this docs by Wix.

Read Before You Start:
As I said earlier every good thing has it’s own bad sides. You need to know some details when you want to connect your db/collection to Firestore or any external source. When you connect your collections to external sources:

  • You will not be able to encrypt fields in Wix.
  • You will be able to use only few types. (Number, Text, Boolean, URL, Datetime, Image, Object)
  • Default field value feature of CMS will be disabled.
  • Validations for fields feature of CMS will be disabled.
  • You will still be limited with data request limits as long as you use wix-data APIs.
  • You will also use Google Cloud Run service and it has it’s own pricing. You can use Google Cloud Pricing Calculator to estimate your cloud costs.
  • Some wix-data APIs may not work/available with external sources. (I didn’t test all wix-data APIs)
  • You can’t set custom permissions for your collections inside Wix. It will be auto to private which is Admin All. (But you can use PERMISSIONS JSON in Cloud)
  • You can’t use Indexes in Wix. (Indexes also available in Firestore so don’t worry about that)

Let me know if I’m missing something in that list.

Now we are ready to go. :airplane:


Understand What We Will Do (How it Works)

What we do here is basically creating a collection/s in Firestore and connecting them to Wix with an adapter. We can think adapter as a translator between Wix and Firestore.

Let’s say we have a query that checks if we have a phone number of a person in our contacts collection and provide a boolean value to use in frontend, in data.jsw file in the backend of our website/web app:

import wixData from 'wix-data';

const options = {
	suppressAuth: true
}

export async function getContacts() {
	const { items } = await wixData.query("Firestore/contacts").find(options);
	return items.map((item) => {
		if (item.phone) {
			return {
				...item,
				hasPhone: true
			}
		} else {
			return {
				...item,
				hasPhone: false
			}
		}
	})
}

The adaptor will translate this into something that can Firestore understand and it will get the results from Firestore and return it to the website/web app.

This is what happens between Wix and Firestore. Or when you connect any other external source. As I said Wix has premade adapters for some sources like Firestore but you can build your own with external-database-collections SPIs.

Step 1 - Enable Firestore in Google Cloud

I assume that you have a Google Cloud account with a billing enabled project. Go to your project and search Firestore in the search bar and select Firestore:

You will see a page that shows you two option “Native Mode” or “Datastore Mode”, select Native Mode (this will be permanent for project-level).

After you select Native Mode Google Cloud will ask you for Firestore location. As we are in a Cloud provider there are tons of servers we can use. You will have two options:

  • Regional (Single Server)
  • Multi Region (Multi Servers in a Region) - Recommended for Better Experience

I will select Multi Region in Europe (EU) since my users will be coming from Europe most of the times. You should choose that location based on your users location. If you are targeting US market you should go with US.

Pricing may vary!

And then click “Create Database”. (It may take up to few minutes to create your Firestore Database in selected Region)

Step 2 - Setup Firestore Database and Collections

If you never used Firestore or Cloud services before UI might be not understandable at first view but it’s easy to use don’t worry.

Now you should see a page similar to this one:

Let me explain what we will do before we do. Since Firestore is document based database like Wix (Wix is using MongoDB which is document based database) we will create collections and we will create documents. I think you already know what is collections. And the documents inside our collections are the data we store in our collections.

You can also connect MongoDB with your own adapter.

Now let’s build an example collection to store members data. I want to have a collection like this:

Members (Collection)

  • _id - text (Auto Created)
  • _createdDate - datetime (Auto Created)
  • _updatedDate - datetime (Auto Created)
  • _owner - text (Auto Created)
  • name - text
  • phoneNumber - text
  • email - text
  • password - text

I want to keep it short for tutorial you can add extra fields if you want.

So here is the important part of Firestore. We need to set these fields and types inside a collection called “_descriptor” Wix using this collection to understand collections we use and fields + types we set inside these collections.

If we do not set _descriptor collection correctly Wix/Adapter will not be able to read and understand collections we have in Firestore.

Now let’s start building “_descriptor” collection for our custom made collections.

Click Start Collection, set Collection ID as _descriptor. After that it will ask us to create it’s first document/data in our collection. In this document we will store our “Members” collection fields and types associated with these fields.

Set Document ID as “Members” and add a field name as “id” select field type as string and write Collection ID to the value field which is Members in our case And we are good to go.

Check your setup and mine:

You must do this when you create another collection in Firestore. If you don’t adapter will not be able to fetch that collection and add it inside Wix.

Let’s keep going by adding collection fields and types of our Members collection fields in _descriptor. Click _descriptor collection and then select Members document.

Create a new field in that document. Select field type as array and name it as fields.

Let me explain what’s going on here. For each field we will have in Members collection we will create a data in this array and in each data we will have will hold two key-value a name for field and a type for the same field.

Let’s create our first field which is name field to store members name.

Add new field with map type.
Set first field name as “name” and add the value as name.
Set the second field name as “type” and add the value as text.

This is what it looks like:

"fields": [
	{
		"name": "name",
		"type": "text"
	}
]

Create other fields of Members collection and then your page should look like this:

Now we are done with _descriptor.

Do not forget that you need to do these steps when you create new collection or add new fields into existing collection. You will adjust settings from _descriptor.

Let’s create Members collection. Just create a new collection name it as Members and add values for the fields we set before. And click save.

For now we are ready to for the Cloud Run part. But of course we can come back and add new collections or fields in our collections.

Step 3 - Setup Adapter in Cloud Run

We have completed our job in Firestore it’s time to setup that connection to Wix using Cloud Run. Search for “secret” in search bar of Google Cloud and select Secret Manager.

It may ask you to enable Secret Manager APIs just enable it.

Click “Create Secret” name it as “SECRET_KEY” and add your connection password. This password will be used between adapter and your Wix website to access your Firestore database. And don’t share your secret/password.

And click Create Secret.

And we are ready to go Cloud Run. Search for “run” in Cloud Console and open Cloud Run.


Before we setup Cloud Run let me explain you why we use Cloud Run and what it does. Think Cloud Run as a server of our database that we can connect. We will use a premade “codes” (container) by Wix to setup that adapter I explained earlier. This is why we use Cloud Run and this is basically what it does.


In Cloud Run click Create Service to create our server between Firestore and Wix.

Paste that Container image URL: “gcr.io/wix-velo-api/velo-external-db” This will automatically set the name to velo-external-db.

And again select a region like in Firestore for your service to run in. Select the closest server to your users. In my case I will use europe-central2 which is located in Poland. You can search in regions and select best for your case.

And please read here carefully:
Settings in CPU allocation and pricing are important for cost and performance. If you are testing do it like me but learn more about these things (using Google :upside_down_face:) if you want to use it in your website to understand how these works.

Example: You can see more than 100€/m cost when you set min number of instances to 1 or higher.

Select CPU is only allocated during request processing.
Set min number of instances to 0 and max to 100.

Scroll down.

Select “Allow unauthenticated invocations” in Authentication settings. This will allow Wix adapter to make calls between service and your website/web app.

And click Create. Process may take up to 1-2min to create the service and deploy it to servers.

After you created service click Edit & Deploy New Revision. (Top menu next to Service Details) And select Security tab.

Here we need to create a Service Account for that service with specific permissions to keep it secure and give access to Firestore. Click Service account and create new Service Account.

After you create new Service Account you need to add following roles/permissions to that service account:

  • Cloud Datastore User
  • Firestore Service Agent
  • Secret Manager Secret Accessor

And click Done.

After that go back to Container tab and scroll down until you see “Environment variables”. In here you will need to enter three env variable.

  • PROJECT_ID
  • TYPE
  • CLOUD_VENDOR

Project ID is your Google Cloud Project ID. If you don’t know your project id you can find it by clicking project name (next to Google Cloud logo).

For type we will use: “firestore”
For vendor we will use: “gcp”

It should look like this:

Cloud Run service env variables for Firestore Wix adapter connection

Next we will add our secret key to the service. To do that you will see “Secrets” just under env variables. Click “Add a Secret Reference”.

1. Select the secret we’ve created before. (SECRET_KEY)
2. Select exposed as env variable method.
3. Name it as “SECRET_KEY”.
4. Select “latest” version.
5. Click Done.

When you ready scroll down more and click Deploy.

After your service deployed successfully copy your service URL and save it somewhere because we will need it when we connect it to Wix. And we will also need the password we set earlier.

Step 4 - Connect to Wix

Now we can go to our Wix Editor to connect our external DB to our Wix website/web app. Open your Editor and enable developer tools if you haven’t yet. And open collections tab then click “+ Connect an external database”.

Select Google Cloud and add the details:

  • Name (Whatever You Want)
  • URL Endpoint (Cloud Run Service URL)
  • Secret Key (Secret Key/Password We Set in Cloud Secret Manager)

And click connect.

If there is no problem you will see a success popup.

Success popup after connecting external DB to Wix

And after that you can see your collection and it’s fields.


Testing Cloud Run URL

If there is a problem you can check your schema request response with your Cloud Run service URL just add “/schemas/list/” to the end of the URL and test it with secret key. And you will need a body like that:

Example URL: https://velo-external-db-id-lm.a.run.app/schemas/list/
Method: POST
Body in JSON:

{
    "requestContext": {
        "settings": {
            "secretKey": "secretKey/password"
        },
        "role": "OWNER"
    }
}

You can use Postman to test it.

Lean More here.

Example Test Website

I’ve created a test website that you can use to test external DB connection I made.

  • You can create new data.
  • You can update existing data.
  • You can remove data.
  • You can query collection.
  • You can filter members with password and email together.

Here is the website I’ve created. https://exweiv.editorx.io/external-db-test


Let me know if you have any questions. And kolay gelsin :purple_heart:.

12 Likes

This is an amazing tutorial!

Great job!

3 Likes

@Quentin_Plomteux Thanks!

1 Like

@loeiks Could you share your experience in term of cost (all feature combined)

What about performance do you notice any difference between external database and local one?

@Quentin_Plomteux

Firestore is quite cheap when we compare it to others because you will be in free tier up to:

  • Up to 50K read/day
  • Up to 20K write/day
  • Up to 20K delete/day

When you compare this to Wix’s limits it’s crazy. And let’s say we have 10GiB data in collections which means millions of documents. Maybe even billion depends on the document size.

You will pay 1-2$/m. Check here.

BUT BUT BUT if you use wix-data APIs these limits are not important because you will be also limited to Wix’s data request limits as I know. And Wix has very low limits with new update and overall monthly limit which you don’t have in Firestore. Check.

About the performance I think there is no big difference I didn’t test it enough to compare with local collections. But I can say it’s fast enough. Since we can change Cloud Run performance settings like CPU and RAM we can even have higher performance when we run heavy queries.

And here is the Cloud Run pricing which you will probably again stay in the free tier (all depends on your Cloud Run settings and of course usage).

When you change CPU to 1 and ram to 512mb which is default values. It will be even free. I selected 4CPU and 8GB RAM probably you will never need that much for a external DB connection service. But even with that let’s say we made 1M requests to the DB and each request run 1.5s you will pay 4.87$/m for the Cloud Run. If you lower the CPU and RAM it will be cheaper or even free. (Play with it in calculator).

5 Likes

Hi @loeiks! Hope all is well.

I’m new to GCP and I really appreciate how you meticulously captured every step of the setup process.

Yet, I’m getting a WDE0116: rs.fields is not iterable error at Step 4 and I was wondering if you would happen to know what could be the issue.
2023-09-17 23_58_40-Wix Website Editor _ Bondoloi

I checked my GCP log and it’s returning POST 500 error, so I double checked my Step 2 setup. Could I be missing anything below?


I’m also attaching my Cloud Run setup, in case it may be relevant. I’m not sure if the missing permissions warning in the Security tab, could have anything to do with the error (although, I did create a New Service Account according to the guide)


I would be deeply thankful if someone could kindly shed some light on the issue.

Thanks and regards

1 Like

You have a very small mistake that you will say WHAAAT.

You are missing “s” in fields key in your _descriptor collection.

It should be fields not field. That’s why Wix can’t read your fields :slight_smile:

Let me know if you still get the error.

1 Like

Yikes, such a basic mistake too. It’s working great now.

Thanks very much for responding super fast! That was awesome :innocent:

No problem :slight_smile:


Anyone have problem you can repost this post guys and tag me into your posts. How? Here is how to repost:

1 Like

Hi @loeiks, hope all is well.

A quick question, have you ever successfully run any wixData operations WITHOUT suppressing the permission, like the example at the bottom of their instruction page?

let options = {
  "suppressAuth": true
}

Without the options parameter, I’m getting a GCP error (in both Wix preview and live modes), despite having already allowed unauthenticated invocations.

UnauthorizedError: You are not authorized at 
RoleAuthorizationService.authorizeRead (/usr/lib/app/main.js:8637:19)

Thanks again,

You can define custom permissions for your database. If you don’t you’ll need extra options. I didn’t show how you can do this in this tutorial but here is a link that’s showing how it works.

You can create a secret key called PERMISSIONS which will be JSON. And in this way you can change permissions of your collections.

2 Likes

Aha! It appears, I need to create multiple SECRETs. :innocent:

Thanks as always!

1 Like

Hello Anyone Reading This:

Firestore is not good enough for large projects when you connect it with Wix’s adapter. Firestore itself is good enough but when connected to Wix with the adapter it’s not working well with large amount of data. (More than 40-50K+)

1 Like

Hi @loeiks, hope all is well.

I’m Max, and I work in the team that develops the adapter. I’d love to know how you came to this conclusion.
Did you use the skip and limit properties during the query?

Hello @maximp I follow the repo you are working on. I don’t remember if I have used limit or skip in the query. What I have done was is create 100K random items and try to query with some filters like .eq

And it was not able to complete the query. With the same adapter there is no problem with MongoDB it’s even very fast.

Thank you for the great guide @loeiks ! :fire:

I was able to connect Wix with Firestore, but I stuck trying to link different fields types other than text (arrays, numbers, dates and booleans).

Could some kind soul shine a light on this GCP newbie, please?

Thank you.

Back in time I have tested almost all field types in Wix and only those are accepted with external collections:

any will not throw an error but techincally not supported

use lowercase when you are typing not like in the image I posted. Like “number” or “datetime” or “date” etc.

1 Like