Scrape data from other website and display it as table in Wix

Ask the community
,
1

Hi,

I wanted to scrape some data (specifically some news article titles) from another website and display it on my Wix website as a table. I can figure out the web scraping part, but I can’t find a way to get it onto my website. Should I use a database and save the data there, then connect the database to a table? If so, how do I get the data in the database to change dynamically according to the scraping? Or is there some way to directly scrape the data with JavaScript directly within Wix Code?

Thanks a bunch!

2

Hey
Scraping data from another site is possible using the Wix Fetch library. What you will need to do is to download the data into a string after you have fetched the page. Then you need to create code that will parse out the news you want and make an array of those items. That array can then be connected to the table you want as rows in the table.

That solution will not be that fast because it will take take to download the page and make the array.

I would make a function in a backend module that do the scraping and then add a function in the http-functions.js file that you can add under the Backend folder. That function is reachable from outside by other systems. So go to sites like easycron.com and setup a schedule that will hit that function every hour or something like that and insert the news items into a Data Collection and then just hook up that table to this Data Collection. Will be faster for the user.

1 Like