Programmable Search Engine Blog
The latest news, updates and tips from the Programmable Search Engine team
On-Demand Indexing for a fast-moving web
Thursday, November 13, 2008
Posted by:
Tom Duerig and Nicholas Weininger, Software Engineers
Earlier this year when we
launched Google Site Search
, and AdSense for Search
started using
the Custom Search platform, we created a special Custom Search Engine (CSE) index for enhanced indexing. Webmasters could submit
Sitemaps
in
Webmaster Tools
, and the Custom Search platform indexed URLs from these Sitemaps into a special CSE index for more comprehensive coverage.
This solved an important problem: the need for
enhanced index coverage
for site search. The special CSE index, in addition to the Google index, enabled us to search deeper on selected sites, providing higher search quality.
Today, with our launch of
On-Demand Indexing
, we're addressing the need to provide fast indexing for your important, frequently-changing content. On-Demand indexing allows anyone with a Google Custom Search Engine or Google Site Search to identify and tell us about new pages or recent site changes by submitting a Sitemap to our Webmaster Tools. You can then select this
specific
Sitemap in the Indexing tab of your CSE control panel and hit the new "Index Now" button. We will immediately schedule the relevant pages for crawling and indexing, and these pages will be included in your search results usually within 24 hours - often much faster.
We realize that new or frequently updated content is often crucial to a website. In order to give you more control over how you use the On-Demand Index resources allocated to you, we will honor the
priority
and
lastmod
attributes that you provide in your
Sitemap
- this means you can identify what URLs should be given preference. Please ensure that your Sitemap metadata is up-to-date.
On-Demand Indexing uses a special index that is designed just for Custom Search. The main Google search indexing process is separate and uses different selection criteria, so submitting pages for On-Demand Indexing will not make them appear any faster in the main Google index, or impact ranking on Google.com. For more information on On-Demand Indexing, as well as the allocation per CSE, please refer to our
FAQ
.
We are always listening at our
Custom Search discussion group
, so please continue to tell us what features you'd like to see added to Custom Search.
Archive
2024
Apr
2023
Dec
Oct
Jun
2022
Aug
Jul
Jun
Feb
2021
Apr
Jan
2020
Nov
Oct
Sep
Aug
Jun
Apr
2019
Jul
Jun
Apr
2018
Jul
2017
Jul
Feb
2014
Mar
2013
Dec
Oct
Sep
Mar
Jan
2012
Aug
Jun
May
Mar
Feb
Jan
2011
Dec
Nov
Oct
Sep
Aug
Jul
Jun
Apr
Mar
Feb
Jan
2010
Dec
Nov
Oct
Sep
Aug
May
Apr
Mar
Feb
Jan
2009
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Aug
Jun
May
Apr
Mar
Feb
Jan
2007
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2006
Dec
Nov
Useful links
Programmable Search Engine
Help Center
Support Forum
Developer Documentation
Google AdSense Blog
