Backend Code for API calls stops returning results when too many requests made.

Ask the community
,
1

I have a simple Collection thats being updated via API call to a 3rd Party website to get the data to populate it. The API call can only handle one Collection row at a time and i can only make 10 calls per second. If i have more than around 110 records to update the results dont get updated, less than that its fine.
Can anyone help as i need to be updating 500 records if possible.
The process works by querying the collection and then looping though each record with a get request then updating the collection withthe results. I have a pause every 10 calls for 10s via a finction. Appologies for the wall of text.

The main Function is:
export function UpdateNewInfo(){
wixData.query(“NewData”)
.limit(500)
.isNotEmpty(“_id”)
.find()
.then( (results) => {
let items = results.items;
let totalCount = results.items.length;
var mynumber = “”
var j = 0
for ( var i = 0; i <totalCount; i++)
{
var mynumber = results.items[i][“_id”]
j = j +1
if (j > 9) {pausecode(‘10000’); j=0}
getfcainfo(items[i].frn)
.then (fcainfo => {
const collection = (‘NewData’)
let toinsert = {
“_id”: fcainfo.Data[0][“NUM”],
“frn”: fcainfo.Data[0][“NUM”],
“Name”: fcainfo.Data[0][“Name”],
“systemTimestamp”: fcainfo.Data[0][“System Timestamp”],}
wixData.update(collection,toinsert )})
. catch ((err) => {
let errorMsg = err.message;
console.log(errorMsg)});
}
})
. catch ( (error) => {
let errorMsg = error.message;
let code = error.code;} );
}

The Pause Function is:
function pausecode(millis){
var date = new Date();
var curDate = null ;
do {curDate = new Date();}
while ((curDate - date) < millis);}

The API call is
export function getfcainfo(fcanumber) {
const url = ‘MY URL’ + fcanumber;
return fetch(url, {
method: “get”,
headers: {
‘X-Auth-Email’: ‘email’,
‘X-Auth-Key’: ‘key’,
‘Content-Type’: ‘application/json’},})
.then(response => response.json() )
}

2

I would recommend trying bulkUpdate() since it’s specifically intended for such cases.

3

Thx Sam will give it a try, things throwing a Cannot read property ‘_id’ of undefined now :(. Still works but cant spot the error at all.

4

I don’t know whether it’s related to you getting rate limited, but that pause function is very ill conceived. It’s akin to redlining the engine of a car in neutral just because you’re not ready to leave.

export async function get_example(request)
{
const response = {
“headers”: {
“Content-Type”: “application/json”
}
};

const test = new Promise(resolve => setTimeout(() => resolve("Success"), 3000)); 
response.body = await test; 
return ok(response);

}

5

Thx Lee, its to do with the 10 calls per 10 seconds, setTimout() wasnt working and the headers on the API call dont give any info on the rate limit. But to be honest im just copying and pasting code from Google. Really not a very good programmer :slight_smile: