Microsoft Internet Security and Acceleration Server 2000

Using FetchUrl to Extend the Cache

The Web proxy service normally fetches objects from the Internet and caches them automatically as proxy clients request them (passive caching). In addition, the Web proxy service automatically keeps certain popular objects "fresh" within its cache by fetching them itself (active caching). However, occasions occur when an application needs a mechanism to override these automatic operations and exercise greater control over the source of objects that are placed in the cache, or the length of time that they are to remain cached. For example, an application may need to:

This function facilitates these operations by allowing an application to cause the Web proxy service to fetch an object from a given location (FetchUrl) and store the object in its cache. The object may also be assigned a new name (CacheUrl), which Web proxy clients will use to retrieve the object from Web proxy cache and/or assigned a particular Time to Live (TTL) within the cache.

FetchURL Usage Examples

Here are three examples of how you might use the FetchURL method.

Example 1

Consider a proxy installation with a very slow Internet connection that requires quick access to the fictional Web site called "fiction.microsoft.com". One solution to this problem would be for the owners of fiction.microsoft.com to produce a CD-ROM that contained all the content found on their Web site and mail it weekly to the proxy customer. An application could use this function to cause the proxy to fetch the data from the CD-ROM and store it in the cache as if it were data from the Internet fiction.microsoft.com site, with a TTL of one week. A complication is that the proxy only supports the HTTP, FTP, and gopher protocols, so the data cannot be fetched directly from the CD-ROM. There are several possible solutions to this problem:

Example 2

In a Web hosting scenario, your organization may regularly update part of the information on its Web site. All or part of the Web site data can be cached on an ISA Server cache. When the Web site information changes, the cache will have to be updated. You can use the FetchUrl method to streamline the update process. In the central database for the Web site information, create an application that generates a file listing all of the files that have changed. Use FetchUrl to retrieve that file. Parse the file to create a list of files that require updating. Then use FetchUrl to retrieve each updated file from the database. This use of FetchUrl allows you to update only the cache files that have changed.

Example 3

In a Web hosting scenario, users may upload updated Web objects to your publishing server at unpredictable frequencies. These items will not be cached by ISA Server if the existing cached object still has a valid TTL. You can use FetchURL to delete the cached object from the cache, so that the updated object will be cached.

Remarks

To retrieve an HTTP object in response to a call to FetchUrl, the Web proxy service uses the normal HTTP GET method to the URL specified by FetchUrl. If there is already a cached copy of the URL specified by CacheUrl (either loaded into the proxy previously by normal operations or by an earlier call to FetchUrl) the proxy includes an "If-Modified-Since" HTTP header in the request, also known as a conditional GET. The response to a conditional GET may be either a new copy of the data that will be stored in the cache or an HTTP status code 304 "Not Modified" response which will cause the proxy to extend the TTL of the object without changing the contents of the data in the cache.

If FetchUrl is used on an array rather than a stand-alone server, the object is automatically fetched by the single array member that is the designated owner of that object according to the CARP algorithm.

Example Code

The following code sample demonstrates VBScript usage of FetchUrl.

' When using VBScript, the enum values should be declared as constants
' or explicit numeric values must be used instead

Const fpcTtlIfNone = 1
Const fpcFetchSynchronous = 4

customTTL = 10	 '

Set root = CreateObject("FPC.Root")
Set myCache = root.Arrays.GetContainingArray.Cache.CacheContents

' we wish to fetch "http://foo.com/bar.htm" and to store it in the cache under the name "http://cache_server/bar.htm"

myCache.FetchURL "http://foo.com/bar.htm", "http://cache_server/bar.htm",  customTTL,  fpcTtlIfNone +  fpcFetchSynchronous