Relevant tags for searching:
#csv #data_collection #download_file #htmlComponent #utf8 #unicode
A forum member recently posted a question wanting to know how to download the contents of a data collection as a comma separated values (csv) file.
Not being one to shy away from a challenge I decided to share a solution.
Wix doesn’t currently deliver a mechanism to export a data collection to the users desktop from a site page. So this is a particularly challenging question.
WARNING: Before I show you how to accomplish this it is important to recognise that you should always be very careful about providing this type of capability especially with large volumes of data. You should also consider the data that you are downloading and prevailing data protection regulations globally.
As part of this post I am going to try to provide some of the problem solving techniques involved as well as an example solution.
This problem has a number of components.
-
How to extract large quantities of data from a data collection
-
How to enable the downloading of said data.
-
How to trigger the download.
How to extract large quantities of data from a data collection
Point 1 of this list is covered in several Articles in the Wix Code Help Centre . In particular is this one:
- Importing and Exporting Collection Data with Code
This shows a very simple method for copying data from a data collection into a text box as JSON. This can be selected and copied into another web based application ot desktop tool to convert to csv.
So we now know how to access a JSON version of the data collection.
How to enable the downloading of said data.
Now the next problem is how do we download to a file from a web site? Well the answer is that the browser helps us do this using some standard HTML capabilities. But wait we can’t access these from a Wix page!
HTML COMPONENT
Actually we can if we use an html Component. Again there is plenty of information relating how to work with htmlComponents on the Wix Code Help Centre . Again we will pick just one that gives us the key basic capabilities:
- Working with the HTML iframe Element
I urge you to read the whole article, however the link takes you to the section that shows how to communicate between an html component and the page code.
STACK OVERFLOW TO THE RESCUE
OK so we now know how to get our data from the data collection. We also know how to communicate with an HTML Component to ask it to do things. How do we perform the download?
Well this is a case where we can generally rely on our extended developer community at Stack Overflow to help out. In most cases if you ask the right javascript question in the google search bar you are likely to get multiple results from Stack Overflow. The posts I will use are these ones:
-
javascript - How to convert JSON to CSV format and store in a variable - Stack Overflow
-
How to create a dynamic file + link for download in Javascript? - Stack Overflow
As you can see one question is “how to convert json to csv format?” . This is great because we have a JSON file that we created in step 1.
The other question is “How to create a dynamic file link for download in javascript?” . This is great because it gives us the information we need to create the download capability!
Preparing the page
So before we get to the page let’s review the page elements that we are going to use.
We are going to need an html Component. So select the HTML iframe element (see image to the right).
The html element in the example has an ID of html2 (in code this is $w(‘#html2’) ).
Additionally we will use a text input box so that you can enter a Data Collection name.
And a button to force the download!
As you can see our elements are named input1 (in code this is $w(‘#input1’) )
and downloadButton (in code this is $w(‘#downloadButton’) ).
In addition we also have a text element called text1 (in code this is $w('#text1) ) that will display any messages from the htmlComponent. These are the element names you will see in the code below.
The Code
The code is in two sections. We have the HTML code that is added to the html component. We also have page code that we use to control the data collection access and actions sent to the HTML component.
HTML CODE
We will start with the HTML Code…
<html>
<body>
<p id="noticeBoard">Document Loading...</p>
<!-- <button onclick='download_file("my_file.txt", dynamic_text())'>Download</button> -->
<script>
document.onreadystatechange = function () {
if (document.readyState === "complete") {
console.log("Document Loaded!");
updateNoticeBoard("Document Loaded!");
}
}
function updateNoticeBoard(message) {
let messageArea = document.getElementById('noticeBoard');
messageArea.innerHTML = message;
}
window.onmessage = (event) => {
console.log("Message Received: "+(event.data.action ? event.data.action : "Bad Message"));
if (event.data) {
messageReceived(event.data);
}
};
function sendMessageToPage(message) {
window.parent.postMessage(message, "*");
}
function messageReceived(message) {
if (message && message.action === 'data' && message.payload) {
// Convert JSON to csv
download_file("data.csv", csvFromJson(message.payload));
} else if (message && message.payload) {
updateNoticeBoard(message.payload);
}
}
function download_file(name, contents, mime_type) {
mime_type = mime_type || "text/plain";
var blob = new Blob([contents], {type: mime_type});
var dlink = document.createElement('a');
dlink.download = name;
dlink.href = window.URL.createObjectURL(blob);
dlink.onclick = function(e) {
// revokeObjectURL needs a delay to work properly
var that = this;
setTimeout(function() {
window.URL.revokeObjectURL(that.href);
}, 1500);
};
dlink.click();
dlink.remove();
}
function csvFromJson(jsonData) {
const items = JSON.parse(jsonData);
const replacer = (key, value) => value === null ? '' : value // specify how you want to handle null values here
const header = Object.keys(items[0])
let csv = items.map(row => header.map(fieldName => JSON.stringify(row[fieldName], replacer)).join(','))
csv.unshift(header.join(','))
csv = csv.join('\r\n')
console.log(csv)
return csv
}
</script>
</body>
</html>
You add this code to the html component by selecting the element and pasting the code as show in the image to the right.
Page Javascript Code
Now we have the HTML code in place and have established the way to communicate with the component we can now write the code to read from the data collection.
// For full API documentation, including code examples, visit http://wix.to/94BuAAs
import wixData from 'wix-data';
let dataToUpload = null; // We will add our data collection to this buffer
$w.onReady(function () {
$w("#html2").onMessage( (event) => {
$w("#text1").text = event.data; // Write messages from html component to text1.
} );
});
function sendMessageToComponent(message) {
// send message to the HTML element
$w("#html2").postMessage(message);
}
export function downloadButton_click(event) {
//Add your code for this event here:
if ($w('#input1').value && $w('#input1').value.length > 0) {
console.log("downloading "+$w('#input1').value+"...");
sendMessageToComponent({ action:"message", payload:$w('#input1').value});
readDataCollection($w('#input1').value)
}
}
function readDataCollection(collection) {
wixData.query(collection)
.limit(100)
.find()
.then((result) => {
dataToUpload = null;
loadPage(result);
})
.catch((error) => {
console.log(error.stack);
});
}
function loadPage(queryResult) {
console.log("loadingPage "+queryResult.currentPage.toString());
if (!dataToUpload && queryResult.length > 0) {
dataToUpload = queryResult.items;
} else {
dataToUpload = dataToUpload.concat(queryResult.items);
}
if (queryResult.hasNext()) {
queryResult.next()
.then((result) => {
loadPage(result);
})
} else {
// Prep for download
console.log("Create payload for download");
let jsonPayload = JSON.stringify(dataToUpload);
sendMessageToComponent({action:"data", payload:jsonPayload});
}
}
Summary
This post offers a solution to what can be a common need in a variety of different web site use cases.
Make sure the elements described above appear on your page and you should be able to cut and paste the code and voila!
In the solution I have deliberately chosen to discuss how to get to the solution and not provide step by step instructions.
If you want to understand more about how this works then please start by reading the linked posts and try to connect the pieces together. As usual I am happy to answer specific questions that this might raise.
Cheers
Steve
