IndexNow support?
I have this set up incrementally pushing changed, reasonably-new, main pages. This rotates randomly between IndexNow participants, since they should share all URLs that they receive.
This was provoked by Bing suddenly (unannounced, in the last few days) stopping accepting sitemap pings, rejecting them with a 410 Gone
.
This is pretty much the entire support in the makefile
:
# Recreate/expose the IndexNow key as necessary. # It is not built in to the makefile since it is meant to be 'secret'. # https://www.indexnow.org/documentation IndexNowKeySrc=.work/IndexNow.key.txt IndexNowKey := $(shell cat $(IndexNowKeySrc)) IndexNowKeyFile := $(IndexNowKey).txt all:: $(IndexNowKeyFile) $(IndexNowKeyFile): $(IndexNowKeySrc) @echo "Rebuilding $@" ln $(IndexNowKeySrc) $(IndexNowKeyFile) chmod a+r $(IndexNowKeyFile) # Ping main-page updates to IndexNow and remembers which have been done. # https://www.indexnow.org/ # Errs on the side of under-reporting. # Submits updates incrementally. # Only considers pages up to a few days old. IndexNowMaxDaysOld=7 # Eliminates explicit 'NOINDEX' pages. # Does not attempt to ping any one page more than once between updates. # All pings could be sent to the primary (URL1) or can be shared at random. IndexNowSEURL1=https://yandex.com/indexnow IndexNowSEURL2=https://www.bing.com/indexnow IndexNowFlags=.work/IndexNow.flags .PHONY: IndexNow.ping IndexNow.ping: $(WORKTMP)/IndexNow.ping all:: $(WORKTMP)/IndexNow.ping $(WORKTMP)/IndexNow.ping: makefile $(IndexNowKeyFile) $(SCWPAGES) @echo "Rebuilding $@" @$(LOCKFILENRSLOW) $@.lock @for f in `find $(PAGES) -mtime -${IndexNowMaxDaysOld} | sort -R`; do \ if egrep -q '<!-- *NOINDEX *-->' .$$f; then continue; fi; \ count=0; \ n=$$f; \ flag=${IndexNowFlags}/$$f.log; \ if [ ! -f $$flag -o $$f -nt $$flag ]; then \ echo IndexNow: $$n; \ URL=`( echo ${IndexNowSEURL1} ; echo ${IndexNowSEURL2} ) | sort -R | head -1`; \ wget -O $$flag "$$URL"'?url=$(URLLISTPREFIX)'"$$n"'&key=${IndexNowKey}'; \ count=1; break; \ fi; \ done; \ if [ 0 = "$$count" ]; then echo "All done..."; touch $@; fi @/bin/rm -f $@.lock
2022-01-05:
I note that if I submit a URL to Bing or Yandex, Yandex spiders it immediately. But I don't think I've seen Bing respond at all to a URL submission.
But I can see the Bing-IndexNow submitted URLs in the appropriate section of the Bing Webmaster Tools, with a date and time. They appear immediately, given a (BWT) page refresh.
AggregateRating
).I was sent a canny "Would you like to try again?" marketing email from Sitebulb. So I did, and this time I have ponied up for a 'Lite' licence at least for now. I cannot possibly justify the expenditure as it is basically all my ad revenue, but I have found the tool helpful in its trial version for a number of things. So I think that Sitebulb ought to have at least a little of my money...
A number of observations:
schema.org
structured microdata parsing is currently broken; it does not understand multiple values in a single itemprop
(or itemtype
) attribute.