I am aware of the limit of 1000 results in wix queries but how do I make it unlimited without making any serious changes to existing codes?
Existing codes
let squery = wixData . query ( “SEEKERS” )
. gt ( “_createdDate” , S )
. isNotEmpty ( “aiImageSearch” )
. eq ( “_owner” , iitemObj . _owner )
let yquery = wixData . query ( “AIIMAGES” )
. gt ( “_createdDate” , S )
. eq ( “title” , “RECEIVED” )
. eq ( “_owner” , iitemObj . _owner )
Promise . all ([
squery . find (),
yquery . find ()
])
Hi there …
There’s nothing such as unlimited query results, in fact, the maximum limit for a query on Wix is 1K , if you have more than 1K results, you need to fetch these results one page at a time or run a bunch of queries at the same time if you’re into that sort of thing.
Here’s how to get all the results (backend requests are subject to a max 14s before the request timeout).
return query('collection').limit(1000).find().then(async (res) => {
const cache = {
items: [...res.items]
}
while (res.hasNext()) {
const temp = await res.next();
cache.items = cache.items.concat(temp.items);
}
console.log(`We've got ${cache.items.length} items`);
})
The previous methods work in serial, so the more items you have in your DB the longer it’ll take, thankfully, there’s another way to do it with Promise.all( ), which works in parallel.
return Promise.all([
query('collection').limit(1000).find(),
query('collection').limit(1000).skip(1000).find(),
query('collection').limit(1000).skip(2000).find(),
query('collection').limit(1000).skip(3000).find(),
]).then((res) => {
const cache = { items: [] }
for (const result of res) {
cache.items = cache.items.concat(result.items);
}
})
The second method is faster, but you need to put the promises manually, or you can use the map( ) method to create an array of promises if you’re an advanced user.
Hope this helps~!
Ahmad
Thanks for your answer, from the example above, does the skip method skip skip the first 1000 from same collection to bring 2000 from same collection? would you have to keep adding this to get upto a 1 million results returned?
let uquery = wixData . query ( “SEEKERS” )
. isNotEmpty ( “aiImageSearch” )
return Promise . all ([
uquery . limit ( 1000 ). find ()
]. map ( x =>{ for ( let i = 1 ; i <= 100 ; i ++){ x [ i ]= uquery . limit ( 1000 ). skip ( i * 1000 ). find ()}}))
then (( res ) => { const cache = { items :[]}
for ( const result of res ){
cache . items = cache . items . concat ( result . items );
}
})
//Please this does not work, at what point can I get the total results or a field value from the collection?
Hi, the best way is to use the pagination bar. If you load too much data at once it will took lot of time that’s why there is a limit. On the results page, you can add filters/search bar to get the desired results from whole database.
@chidiejike Why would you need to download 1M result?
How would you displayed those items on your page?
If it’s for computation purpose you should use another approach or have a dedicated service out of Wix
Thanks for your answer but filter results can also return more than 1 thousand items that would need to be displayed in a table. But how do I use the pagination to bring back more results as the result of my query would be added to a repeater or pagination
@plomteuxquentin Most of my queries are to get the first item of a field in a collection also to test if the total is greater than small figures eg 1 0r 0, but some of my queries are also to populate a table or repeater or to filter and also to get the total results of a database eg likes, views etc
@chidiejike Please refer to this link & I hope this will help you out - https://www.wix.com/velo/reference/$w/table/pagination
If you’re using repeater, connect the pagination bar to dataset. When you add the setfilter() to filter dataset. As the data changes in the dataset then pagination bar will also get updated.
I’ve not tried this yet but I hope this will help you. Please let me know once you tried this.
@wix-expert-velo-cert Yes I use setfilter to filter a repeater and pagination connected to a dataset and return results but arent those results limited to a maximum amount? eg if you want to .getTotalCount() of the filtered or unfiltered dataset? Does the set filter option and total count bring all the items based on criteria set irrespective of amount?
I think yes, while getting the count it return the total data count. To check the count of filtered items, user dataset.getitems, then use items length to get the count.
@wix-expert-velo-cert In most cases the dataset filteration does not work and for complex filters you would need to do a query that is why I rely more on query results than filtering a dataset that might not respond. $w(“#dataset”).getTotalCount() wont do the job? I have almost 20,000 lines of code, I really need a a basic adjustment to what I have existing. I recently asked a question on how to setfilter to a dataset by comparing new Date() with _createdDate from a collection because that did not filter the dataset for some reason. Even velo forum can do queries to display the amount of views on a post that run into thousands or millions.
Please I need help on implementing a loop to get the items in parallel, this seems like the most viable based on what I have existing
I can understand you point. Another possibility is to use next function.
You can add a button at the end of partial results. Then use next API inside onviewport event.
That would be redoing my entire site
Is there anyway that you know to get all orders from Wix Orders database because limit is 100 and it’s going to timeout. I want to see just X users orders and there is no way to filter it before query (at least I couldn’t find something to do it)
Okay I got it here is the answer (at least 10 times faster and without error):
-note: I just want to search PAID orders so you can remove it!
return wixData.query("Stores/Orders")
.limit(100)
.eq("paymentStatus", "PAID")
.find(options)
.then(async (ordersData) => {
let totalPage = ordersData.totalPages;
let array = [];
let skip = 0;
for (let i = 0; i < totalPage; i++) {
array.push(wixData.query("Stores/Orders").limit(100).eq("paymentStatus", "PAID").skip(skip).find(options))
skip = skip + 100;
}
return Promise.all(array)
.then((res) => {
const cache = { items: [] }
for (const result of res) {
cache.items = cache.items.concat(result.items);
}
return cache;
})
})
I’m creating an array with for loop and totalPages also in this array I’m creating dynamic skip value so I can skip correct amount of data each time. And just resolve all Promises.
10 times faster and better than first method. I couldn’t even get any result with first result I just got timedout every time.
-note: I tested this with 1284 items; with limit(100) if you are doing this on your own database you can limit up to 1000 so it will be faster (probably).