Sync external DB with Wix Collection

OK.
As promised, I’ve done some reading.
I do have a few questions:

  1. You said that you currently support GET and POST - so should I use POST for insert as well as update?

  2. Can the _id field in my wix collection be set to be identical to a PK field in the MSSQL server, or must the _id be given to the entry by Wix?

  3. From what I gather so far, it seems I can do one of the following:
    a. Call the function once a day, with ALL new and updated items in the request body, and then iterate through them in the function itself.
    b. call one function for all new items, and one for all updated items, iterating thorugh each batch seperately (different HTTP functions).
    c. have a function that just takes one item and handles it. The client calls the function for every item.
    c.1 - the function determines if it’s a new item or an updated one, and handles accrodingly
    c.2 - one function for adding an item, another for updating - the client determines which one to call.
    From a design\efficiency standpoint - which of these solutions should I choose?

Thanks,
Tal

Hi Tal,

  1. yes
  2. you can set the _id to whatever you decide, wix create a uuid if it is set as blank in the insert operation
  3. what are the volumes of the data? we have a limit of the number of inserts/updates you can do per minute to prevent bot attacks or misuse of the system, please keep that in mind. do you have the updates just once a day? shouldn’t it be more updated?
    obviously batches are more optimized over the network (think of few updates to the same record in the same day, you’d only be updated once) and the knowladge if the item is new or not, will also save a read query to the db :slight_smile:

Shlomi

Thanks for the speedy reply.
As for 3:
There aren’t tremendous amounts of data updates - I’d guess it’s about a dozen items a day, max, but I’d definitely say its definitely not more than 100. (I’ll check it with the client, though).
The updates are done nightly, and not in real time. It’s fine.
So you’re saying it’s better if the client (the MSSQL DB) pushes new and updated items separately.
The questions is: when a batch is sent - does it matter if it’s one call per item, or one call per “batch” (so - options (a) or (b) above?

Hi Tal,

in that case, the number of items is really low, you can choose any of the options above :slight_smile:
just do whatever is more comfortable for you to code

Shlomi

Cool. Thanks, Shlomi!
Tal

A year later, is there now a better way to run scheduled sync tasks. Or perhaps even an tutorial about this. Do you now have these backend “tasks”, scheduled tasks or similar?

@vulfox we are just about to release this new capability. Stay tuned :slight_smile:

Hi Tal,

Did you find a way around having to set the collection to allow “anyone” to create/update items?

Cheers,
Scott

Hey Scott.
I’m not 100% sure I understood your question, but if I did, than the answer can be found here: https://www.wix.com/code/reference/wix-data.html#WixDataOptions ( suppressAuth = true ).
If this doesn’t do the trick - please explain what exactly is your problem, and I’ll try my best to answer.

Hi Tal,

I am testing from Postman using http functions in backend code for POST and PUT. The only way I can get it to work is by changing the permissions on the collection to to allow Anyone to create/update records in the collection. This is not ideal as it seems to be a bit of a security risk.

Hope this clears up the question a bit :slight_smile:

Thanks,
Scott

Did you read the info in the link I’d posted? Did you use this in your code?

@tal-orbach
Indeed, I am calling the API from a 3rd party application. I did add it as a test same result.

Cheers,
Scott

@slucas15 Can you post the call from your code to the insert\update operation?

@tal-orbach

Sure this is the PUT from my backed code file. I can get this to work from my application however, I have to change the permissions on the collection in WIX to allow updates/create to Anyone. That doesn’t seem right, I should be able to pass some sort of auth in my request. I couldn’t find a way to even tell the API to use Basic auth in the headers.

export function put_updatePoints(request) {
let options = {
“headers”: {
“Content-Type”: “application/json”

} 

};
// get the request body

return request.body.text()
.then( (body) => {
// update the item in a collection

return wixData.update(“Ranks”, JSON.parse(body));
} )
.then( (results) => {
options.body = {
“inserted”: results
};
return ok(options);
} )
// something went wrong
. catch ( (error) => {
options.body = {
“error”: error
};
return serverError(options);
} );
}

So this is exactly what I wrote you about in my first reply: you’re not using the WixDataOptions in your Update call.
Look again at the syntax for Update: wix-data - Velo API Reference - Wix.com

function update(collectionName: string, item: Object, [options: WixDataOptions]): Promise<Object>

you’re only using the first two parameters (collection, item), you need to also use the third. Go back you the link in my first reply to you, to understand WixDataOptions and how to use it.

type WixDataOptions = {
  suppressAuth: boolean
  suppressHooks: boolean
}

Hi Tal,

I see what did, I fixed the syntax and was able to create/update an item when the permissions where set to admin for everything . However, I am still a little concerned if someone know’s the endpoint, they can send a request to the endpoint and manipulate the data in the collection.
Normally, when integrating systems we have some sort of authentication process, OAuth, Basic etc, I am trying to make sure the endpoints I expose from the backend code are secure and by applying this override, if they know the URL and function the data could be altered.

By the way, thanks a ton for taking time out of your day/night to help :slight_smile:

Learned something new today.

Cheers,
Scott

@slucas15
It’s true that if someone knows the endpoint, they can change your data. Well, the endpoint and the correct format for the request body.
But in much the same way, you can always say that if someone knows your username and password they can change the data.
You can always add more protections (e.g. url parameters such as password/key or parameters within the HTTP request itself), to make it so they need to know far more than just one thing (i.e. the endpoint). Even more advanced (albeit not always relevant) - checking the IP from which the request is sent).
The point is, though, that just as with a username/password scenario - you just need to keep it secret, but with an HTTP request you can make it so far more details are needed.

@shlomi-shemesh Hi, has this been released yet and if yes where could I find the info? Thanks

Hi! I would like to use the WixDataOptions functionality, but the reference file seems to have gone. Is it still supported? I’ve read in the comments above that the documentation should be here:

https://www.wix.com/corvid/reference/wix-data/wixdataoptions

But that does not seem to exist anymore. Where could I find the information I need to obtain permission to run code in the backend that reads the “stores/orders” collection? Thanks in advance!