Google crawler not finding dynamic pages

Ask the community
1

My website relies extensively on dynamic pages, particularly to cast a wide net to get search results. The problem is that the Google Crawler (and Ahrefs) are not detecting any of the dynamic pages by the href links I’m placing on the (All) dynamic page - just via the sitemap.

I’ve done a lot to try to make my (All) pages build referring links to the (Title) pages, but nothing seems to work. Here is the code on my (All) page:

import wixData from 'wix-data';
import wixLocation from 'wix-location';

$w.onReady(function () {
    // Write your JavaScript here
    let now = new Date();
    let today = new Date(now.getFullYear(), now.getMonth(), now.getUTCDate(), 0, 0, 0)
    let oneMonth = new Date();
    oneMonth.setDate(today.getDate() + 31)
    ###1
    wixData.query("Items1")
        .gt("end", today)
        .ne("dummy",true)
        .lt("when", oneMonth)
        .limit(200)
        .ascending('when')
        .find()
        .then((results) => {
            $w("#listRepeater").data = results.items
        })
    $w('#listRepeater').onItemReady(($item, itemData, index) => {
        $item('#image').fitMode = "fit";
        $item('#image').src = itemData.photo;
        $item('#image').link = itemData['link-items-1-title']
        $item('#text9').text = itemData.name
        $item('#text1').text = itemData.title
        ###2
        var link = $item('#text10').html
        link = link.replace("<a>", `<a href="${itemData['link-items-1-title']}">`)
        $item('#text10').html = link
        $item('#container1').onClick(() => {wixLocation.to(itemData['link-items-1-title'])})
        $item('#text8').text = itemData.soldOut == true ? "SOLD OUT" : itemData.cost
    });
});

Several things are going on here, all of which are workarounds Wix has forced me into. The most relevant things are:
###1) I have to do my own query via the Wix Data API so that I can only show future events (I have a separate page for past events). This is done on top of the query the dynamic page is already doing.
###2) I am building my own link because Wix’s dynamic pages don’t seem to let you link to pages dynamically without changing their text.

If you use a tool like the MozBar plugin, all of these links are highlighted as valid, but that seems to be after all JavaScript has executed. Google Search console and Ahrefs are not seeing these dynamically created links, possibly because of execution time, possibly for some other reason.

As a result, my Ahrefs score is garbage, and my performance with Google is below what it should be.

All ideas on how to improve this are welcome :slight_smile:

2

Of course, as soon as I post the question (after weeks and months of trying to figure it out), I figured it out.

The key was found in Velo SEO Best Practices guide. Specifically

For Velo sites, anything that occurs in the onReady() function is rendered on the server, returned as HTML, and seen by search engines.

For most use cases, it's enough to just call the function in onReady(). For asynchronous functions and functions that run following a delay, the onReady() promise may resolve before the function's promise has a chance to resolve. In such cases, if you need the returned content to be indexed, you can block the onReady() from resolving until your function's promise resolves. You can indicate to the server that it should wait by returning the promise in onReady().

So, I modified my code to block while the query is executing and update the page in the .then() blocks.

My Ahrefs score is now 98% (up from mid 40s), and my dynamic (All) pages don’t render once with dummy data and then a second time with real data. They just render once with real data. Yay :slight_smile:

1 Like
3

Hi Pat, i’m facing the same problem here. my site is now listed on a very last page T_T… I’m not a code person. would u mind if i ask how you have modified your code?
I dont know how to use “.then()” code.

please help me out here

Thank you

4

If you are facing issues in your internet or having troubles to change the settings of your ISP, then do check out: https://ispsettings.co/

5

Code Solution

The example shown in the SEO Best Practice uses .then for awaiting the Promise.

If you use Async Await, then try below (which worked for me):

$w.onReady(async function () {
  try {
    await yourFunction1();
    await yourFunction2();
  } catch (error) {
    console.error("Async error:", error);
  }
});

Key Points:

  1. Add async after onReady
  2. Don’t need to add “return”

No Code Solution

In the SEO best practice, it mentioned that:

Note: Database content that is loaded into page elements using a dataset (instead of code) is included in the SSR version of your page and will be seen by search engines.

If the coding way doesn’t work for you, then connect the $w (ui element) using Dataset instead of code.

How to Check if it is Working

  1. Go onto Google Search Counsel and try Live URL test

  2. Right click the page to “View page source”, then Ctrl+F to search for text segments from the page. You should be able to find the content on the page.