How to prevent or delete duplicate records in CSV file import process to wix database?

Ask the community
,
1

Hi;
I add csv files to my wix database, since the number of records is high, I cannot delete or compare them one by one.

I transfer the data in the csv file that I update everyday. But I do not want to reload the data I transferred the previous day. Each record must be unique.

Do you have a sample code? such as automatic deletion of duplicate records or ignoring file upload

2

Hello,

Check out the links below that should help you start:

https://stackoverflow.com/questions/51934743/how-to-make-the-wix-database-items-unique
https://www.wix.com/corvid/forum/tips-tutorials-examples/example-no-database-duplicates - You can get the idea from this example and adjust it to your needs.
https://support.wix.com/en/article/corvid-scheduling-recurring-jobs - You could also try to set up a recurring job that will check for duplicates on a daily basis.

3

Hi; I’m trying the two codes below but I couldn’t. Where’s my fault?

reference column for comparison : OriginalConnectionDetails
(By comparing url addresses, I do not want to add the existing record back to the database while transferring the csv file if it was added to the database before.)
Database : Germany

Example Code-1

import wixData from ‘wix-data’;

export function searchForDuplicates(value) {
return wixData.query(“Diamond”)
.eq(“giaCode”, value)
.find()
.then((results) => {
return results.items.length;
})
.catch((err) => {
let errorMsg = err;
});
}

export function Diamond_beforeInsert (item) {
return searchForDuplicates(item.giaCode).then((res) => {
if(res > 0) {
return Promise .reject(“Duplicate”);
}
return item;
});
}

-------- Example Code-2

import wixData from ‘wix-data’ ;

export function searchForDuplicates(collection, field, item) {
// info contains the hook context
// use the collectionName property to use function for multiple collections
return wixData.query(collection)
.eq(field, item[field])
.find()
.then((results) => {
return results.items.length;
})
. catch ((err) => {
let errorMsg = err;
});
}

export function Germany_beforeInsert(item, context) {
// Calls routine to check if value already exists in the collection.
// If value not found, then save record.
// Else, if value is found, then reject this insert to prevent duplicate values.
// Note: concurrent inserts may result in duplicates.
// Pass context to searchForDuplicates to use for multiple collections.
return searchForDuplicates(context.collectionName, “OriginalConnectionDetails” , item).then((res) => {
if (res > 0 ) {
return Promise.reject( ‘This item already exists’ );
}
return item;
});
}

4

Can someone please help with this? I need help to edit the code correctly.

5

Hi; I’m trying the two codes below but I couldn’t. Where’s my fault?

reference column for comparison : OriginalConnectionDetails
(By comparing url addresses, I do not want to add the existing record back to the database while transferring the csv file if it was added to the database before.)
Database : Germany

Example Code-1

import wixData from ‘wix-data’; export function searchForDuplicates(value) { return wixData.query(“Diamond”) .eq(“giaCode”, value) .find() .then((results) => { return results.items.length; }) .catch((err) => { let errorMsg = err; }); } export function Diamond_beforeInsert (item) { return searchForDuplicates(item.giaCode).then((res) => { if(res > 0) { return Promise .reject(“Duplicate”); } return item; }); }

-------- Example Code-2

import wixData from ‘wix-data’ ;

export function searchForDuplicates(collection, field, item) {
// info contains the hook context
// use the collectionName property to use function for multiple collections
return wixData.query(collection)
.eq(field, item[field])
.find()
.then((results) => {
return results.items.length;
})
. catch ((err) => {
let errorMsg = err;
});
}

export function Germany_beforeInsert(item, context) {
// Calls routine to check if value already exists in the collection.
// If value not found, then save record.
// Else, if value is found, then reject this insert to prevent duplicate values.
// Note: concurrent inserts may result in duplicates.
// Pass context to searchForDuplicates to use for multiple collections.
return searchForDuplicates(context.collectionName, “OriginalConnectionDetails” , item).then((res) => {
if (res > 0 ) {
return Promise.reject( ‘This item already exists’ );
}
return item;
});
}

6

there is no comment to inform about this?