This might be overkill but I am interested in understanding how to best optimize my data model
What fields are indexed in a wix collection? _id is a given but is the default text field that comes with a collection (“title”) also indexed? I am no expert in mongoDB but I thought that maybe that text field is what is called a shard key in mongoDB and therefore is kind of indexed?
On top of this, is it possible to manually set indexes on fields in a collection?
Thanks
Hi.
Please reach out to Wix Customer Support . They will be of great help. This forum is only for code related issues and discussions.
Good luck!
I wouldn’t be here if Wix Support were able to answer this question. As of today, they aren’t. This question isn’t too far from being a code-related question, since collection querying is an integral part of corvid wixData code
@yisrael-wix @alicihan I hope you are doing well and staying safe during Covid-19. I am looking for assistance here, as I found this post regarding an issue that I am facing. I am working on a site that has many collections of which 2 are larger ones that need to be queried.
Basically, it seems as though that only 2 use-worthy fields are indexed: ‘_id’ (aka primarykey) && ‘_createdDate’. Is it possible to set indexing for some fields we need in lieu of waiting for a feature release? - meaning corvid developer-team settings or anything you can advise . I need 1 date field indexed in order to query, otherwise due to this limitation I am encountering query-time-out errors. Which effectively makes these collections useless. I have tested a more complex queries (than my actual one) with ‘_id’ arguments only and then those work. Though any simpler one [e.g.] with a date field fails due to time-out errors. Of course, as a feature release it would be nice to be able to add indexing from Wix Collection’s Settings, for primitive type fields – String, Date, Boolean .
@javierantonf – I don’t know if you have found a work around for this? I have scoured the web and pored over the Wix-Corvid APIs - but no luck, as there doesn’t seem to be the ability to add indexing for a field in a Wix Collection. I had double checked Wix docs before building this one site, to make sure there were no query limitations or table size concerns. Hence it is [really] troubling to endure these hindsight errors. I can create some side tables and a couple services to manage future written records for this limitation, but those would hit a limit too. Hence, I added Yisreal, since he has responded to previous posts.
@reveel I haven’t been able to get an answer for this yet. Can I ask how large these collections are in rows and fields (and what field types they use). My largest collection is 10,000 records and growing (4 ref fields and 1 text) and I haven 't had this issue you mention yet
I am dying for an answer from Wix as this literally will determine whether I need to migrate to a different host once I hit the magic number
These collections many times larger than yours and growing daily. It will depend on the collection properties (fields and field types) and record sizes but my guess is that with an avg of 200kb records it would hit this after 30-50k records. Anyhow the gentlemen I’ve tagged usually does get back.
Being able to index will help and maybe they can do some sharding options in the future (prob at a cost tho).
@yisrael-wix I wanted to add that I had seen previous posts regarding TIME OUT Errors (those being non query types), where many years ago it was deemed as a non-issue. What I am encountering is a real issue meaning it will not proceed since the query becomes a WD_BAD_REQUEST. While playing with tests, I can produce a variety of errors - about 4 types (6 messages). Furthermore, the 14 second timeout period has shown itself as an error, but in no way has 14 seconds passed or would for the query (on per request basis).
Good question. We are actually discussing this in a separate forum. OK, the facts: you cannot create indices, unless on the “VIP tier”. Only system fields are indexed (as you said, _id and, from memory, _owner) in other tiers.
So every search on any other field is sequential (unless VIP-tier). Meaning: search time goes up linear. Problem: if many docs in collection (rows in table, if you like), timeout.
Good news: something´s in the oven to address this. ETA: no idea.
@giri-zano do you know how this is enabled for VIP? Is it an op tion in the editor or do you have to manually request it via email etc?
@javierantonf No, you can just upgrade your license to Business VIP (from Editor/Settings/Upgrade)
EDIT: have never tried it myself, so I do not know if you can upgrade existing Corvid db to that one and if, how. Better ask Support.
@giri-zano I’ve asked Support and the ticket is ongoing but they haven’t provided an answer to this so far. I’ve got a parallel setup in Google Cloud Firestore/AppEngine that does the same thing I’m doing with Wix but with indeces. I may be 24 hours away from migrating away from Wix. Hope Support replies soon because i don’t mind upgrading to VIP if it offers a solution to this. I can’t put my plans on hold much longer because my userbase is growing and can’t rely on a faulty DB. Even having an ETA for indeces would be enough to keep me onboard
I´ll try to bump it up now. EDIT: done.
@giri-zano What is “done”? I received a reply from support saying the answer is in this forum post. But no precise explanation of how the process works once in VIP mode after upgrading or what is “in the oven” for non-VIPs
“Done” means: I tried to bump it up. But no answer yet. This also means I have nothing more to provide: I don´t no either.
I’ve come to the conclusion that rather than being an error originating from the DB that could be fixed with indices the reason of the timeouts must be something else like for example that the server is restarting because of some underlying fault like a garbage collector shutting down due to too much activity etc. It’s just a wild guess. I draw my conclusion from the fact that “timeouts” happen during a continuous 8-minute length of time rather than randomly at different times. They also occur on queries that target very small collections. I guess that people don’t notice this until they start querying a lot more frequently and then they think it’s due to their collection size
I really like Corvid as it is really fast and Wix Data is great. Time for me to move on but either way let us know if this gets fixed at some point in the future