Query returns collection columns in random order

I created the following collection called “SequenceTest” with a very obvious column sequence

I use the following extremely basic query to get results from the collection:

import wixData from ‘wix-data’ ;
wixData . query (( “SequenceTest” ))
. find ()
. then (( results )=>{
console . log ( results )
})

The query returns the columns in apparently completely random order:

(i.e. the field order returned is “Sunday”,“Tuesday”,“_id”,“_Owner”,“_CreateDate”,“Wednesday”,“_UpdatedDate”,“Monday”,“Friday”,“Thursday”,“Saturday”–in case the image is too small to make out…)

In the real world, there would be multiple rows in the collection. I need to be able to create a user-selectable csv export of all the rows but in the correct columnar order. I have the code to create the csv, but I am not able to correct the column order pushed into it because the query returns results out of order.

Does anyone have a solution that would force the query to return results in the original collection column order?

If you export the collection manually via the editor, the results are in the expected order–but my end users will NOT have access to this function so I need to be able to replicate it through code automation.

Thanks!
Brian Roberts

Show us the code to generate the CSV. Maybe we can help you out there, possibly by correcting how it is handed down. There is little you can do about the order returned from query.

I don’t think that is going to help because the csv is simply created by passing the query results into an HTML element using a json stringify function in whatever order the results are generated.

I’m basically using a modified version of Steve Cropper’s code from here: https://www.wix.com/velo/forum/tips-tutorials-examples/how-to-download-the-contents-of-a-data-collection-in-a-csv-file.

I’m now experimenting with converting specific query results “items” into a new array (with the “items” being pre-defined in the proper order) and then passing that new array into the json stringify function instead of the raw query results. But with around 170 columns to specifically identify, that is going to be one laborious coding process. :frowning:

AFTERTHOUGHT: As I mentioned previously, in Wix editor mode, a manual download of the csv export arrives in the expected order (i.e. the order the columns appear in the Collections editor) and the downloaded .CSV file is even properly named with the Collections title. Would it be possible for the Velo team to expose that function for inclusion in user-generated Javascript coding? Somehow those column indexes are correctly identified in that process but apparently NOT in the wixData.query() process.

You can use the Javascript map() function to “remap” the output to appear as you need. I use this function a lot when I need to “fix things up”.

javascript maps make my head spin. :wink:

Here is the solution at which I arrived:

  1. Create a meta-data Collection which lists each table/collection that should be user-downloadable along with the associated WIX codename associated to each column/field in those collections and the order in which you want those columns/fields to appear in a download.

  2. Use the following javascript and HTML code to query and order your data in the proper columnar order. (I left in all the console.log messages I used to develop this but they are all commented out. You can restore them as necessary to understand what is happening after creating your meta-data table and changing the collection names to reflect your collections).

Your meta-data collection table should be structured something like this:

The most import elements of that meta-data table are the CollectionCode and the FieldCode --which are how WIX Javascript identifies those elements programmatically.

here are the control elements on the website page which access the code:

They look like this in preview mode: (I set the HTML element to “hidden”)

//
here is the javascript code from the web page:
/
/

import wixData from ‘wix-data’ ;

$w . onReady ( function () {
$w ( “#html2” ). onMessage (( event ) => {
console . log ( “event data” , event . data )
$w ( “#text1” ). text = event . data ; // Write messages from html component to text1.
});

});

function sendMessageToComponent ( message ) {
// send message to the HTML element
console . log ( message )
$w ( “#html2” ). postMessage ( message );
}

export async function csvDownload () {
let collectionName = $w ( “#dropdown1” ). value //this dropdown list is data-connected to a meta-data collection which lists all the “end-user download-available” collections on the site
//console.log(collectionName)
let headerOrder = //initialize an array to hold the specific collection column order desired
await wixData . query ( “CollectionsMetaData” ) // this collection contains the collection and field names–and the field names are manually pre-assigned a numberic position value within the collection
. limit ( 1000 ) //I have 7 user-survey data input collections with over cumulative 400 fields so far
. eq ( ‘collectionCode’ , collectionName ) //look for the portion of meta data that applies to the selected collection from the dropdown list
. eq ( ‘includeInReport’ , “Yes” ) //get only the fields marked for inclusion in a download report
. ascending ( ‘position’ ) //MOST IMPORT–order the columnar items as indicated by the meta data position assignment!
. find ()
. then (( headers ) => {
for ( let x = 0 ; x < headers . length ; x ++) { //create an ordered array of “item” identifiers
//console.log(x,headers.items.fieldCode,headers.items.position)
headerOrder . push ([ headers . items [ x ]. fieldCode , headers . items [ x ]. position ]) //we add the fiedl WIX codenames to the array in the order we want them returned in the CSV file
}
//console.log(headerOrder)
})
. then ( async ( nextUp ) => { //now we will query the collection selected in the dorp-down list
await wixData . query ( collectionName )
. limit ( 1000 )
. find ()
. then ( async ( results ) => {
let jsonArr = “” //initialize a json array to be passed to the HTML element embedded in the page which contains the download code
if ( results . items . length > 0 ) { //make sure we have something to process
let tempArr = “” //initialize a temporary array to holed the queried values for one row at a time as they come in
for ( let r = 0 ; r < results . items . length ; r ++) { //determine how many records were detected in the collection
for ( let h = 0 ; h < headerOrder . length ; h ++) { //now determine which field we want to pass to JSON nesxt based on the preveiously queried meta data
let fieldcode = ( headerOrder [ h ][ 0 ]) //this is the WiX collection “codeNaeme” for the desired field
let qCode = “results.items[r].” + fieldcode ; //create a dynamic variable that refences the desired collection column for the results returned from the query collection
//console.log (qCode)
let fieldVal = eval( qCode ) //THIS IS THE SHIZNIT: we dynamically evaluate the value of the field/column we need IN THE PROPER (assigned) COLUMN ORDER
//tempArr.push([ fieldcode, fieldVal ])
let comma = “” ; //now we’re going to emulate what the JSON stringify command does to create a string we can pass to the HTML download code (courtesy of Steve Cropper)
if ( h < headerOrder . length - 1 ) { comma = ‘,’ } else { comma = ‘’ } //if this is our very last column, we don’t need a comma between elements, otherwise we do
tempArr += ‘"’ + fieldcode + ‘": "’ + fieldVal + ‘"’ + comma //format our data lin with the field title, the value, and (if necessaryy) a comma
}
let comma = “” ; //some final JSON stringify formatting (next 3 lines)
if ( r < results . length - 1 ) { comma = ‘,’ } else { comma = ‘’ }
tempArr = “{” + tempArr + “}” + comma //this separates each record detected by the primarily-targeted table collection query
//console.log(tempArr)
jsonArr += tempArr //add this row’s data line to our final jason array we will pass to the HTML code
tempArr = “” //reset the temp array to receive the next row’s data
}
} else { console . log ( “no results found” ) }
//console.log(jsonArr)
//let jsonPayload = JSON.stringify(jsonArr); //we don’t need this anymore–we’ve already created a “stringified” json array
let jsonPayload = “[” + jsonArr . replaceAll ( ‘\n\n’ , ‘\n\n’ ) + “]” //in my case, I had to do some clean-up to eliminate “escaped” new-linecharacters contained in collection text fields
//console.log("headers ",jsonPayload);
sendMessageToComponent ({ name : collectionName + “.csv” , action : “data” , payload : jsonPayload }); //finally, we send the results to Steve Cropper’s HTML code with one additionals value, i.e. “naem: collectionName”) which adds that name to the CSV download
});

    }); 

}

export function button1_click ( event ) {
csvDownload ()
}

//
NOTES REGARDING THE HTML MESSAGE SECTION:
/
/
In order to pass a specific download filename:

this modification to the page javascript: " sendMessageToComponent ({ name : collectionName
requires a modification to Steve Cropper’s HTML Element code as follows:

function messageReceived(message) { 
    if (message && message.action === 'data' && message.payload) { 
        // Convert JSON to csv 
       download_file( **message.name** ,  csvFromJson(message.payload)); 

//
Here is the full HTML code pasted into the HTML page element:
/
/

Document Loading...

document.onreadystatechange = function () { 
    if (document.readyState === "complete") { 
        console.log("Document Loaded!"); 
        updateNoticeBoard("Document Loaded!"); 
    } 
} 

function updateNoticeBoard(message) { 
    let messageArea = document.getElementById('noticeBoard'); 
    messageArea.innerHTML = message; 
} 

window.onmessage = (event) => { 
    console.log("Message Received: "+(event.data.action ? event.data.action : "Bad Message")); 
    if (event.data) { 
        messageReceived(event.data); 
    } 
}; 

function sendMessageToPage(message) { 
    window.parent.postMessage(message, "*"); 
} 

function messageReceived(message) { 
    if (message && message.action === 'data' && message.payload) { 
        // Convert JSON to csv 
        download_file(message.name, csvFromJson(message.payload)); 
    } else if (message && message.payload) { 
        updateNoticeBoard(message.payload); 
    } 
} 

function dynamic_text() { 
    return "create your dynamic text here"; 
} 

// Courtesy https://stackoverflow.com/questions/18249290/generate-csv-for-excel-via-javascript-with-unicode-characters 
function download_file(name, contents, mime_type) { 
    mime_type = mime_type || "text/csv;charset=utf-8"; 

    var blob = new Blob(['\uFEFF', contents], {type: mime_type}); 

    var dlink = document.createElement('a'); 
    dlink.download = name; 
    dlink.href = window.URL.createObjectURL(blob); 
    dlink.onclick = function(e) { 
        // revokeObjectURL needs a delay to work properly 
        var that = this; 
        setTimeout(function() { 
            window.URL.revokeObjectURL(that.href); 
        }, 1500); 
    }; 

    dlink.click(); 
    dlink.remove(); 
} 

function csvFromJson(jsonData) { 
    const items = JSON.parse(jsonData); 
    const replacer = (key, value) => value === null ? '' : value // specify how you want to handle null values here 
    const header = Object.keys(items[0]) 
    console.log(header) 
    let csv = items.map(row => header.map(fieldName => JSON.stringify(row[fieldName], replacer)).join(',')) 
    csv.unshift(header.join(',')) 
    csv = csv.join('\r\n') 

    console.log(csv) 
    return csv 
} 
</script> 

Hope this may be of some use to anybody else facing the same issues.
As “Gramps” says: No cats were harmed in the development of this code"
In fact, they typically walk all over my keyboard while I’m trying to concentrate, therefore, I refuse to be held responsible for any grammatic, programmatic or logical errors.

:wink: