8) What about existing technology, robots.txt and why?
ACAP will work smoothly with the existing robots.txt protocol. More >
Full Responses
8) What about existing technology, robots.txt and why?
ACAP will work smoothly with the existing robots.txt protocol.
We recognise that robots.txt is a well established method for communication between content owners and crawler operators. However, robots.txt is not sophisticated enough for today's content and publishing models. Robots.txt, in its current form as implemented by most search engine operators, provides only a simple choice between allowing and disallowing access. These simple choices are inconsistently interpreted.
Microsoft's Chief Counsel in IP Tom Rubin recently said that using robots.txt in its current form to express permissions in this day and age was like "putting a Fiat engine in a Ferrari." A number of proprietary extensions have been implemented by several of the major search engines, but not all search engines recognise all or even any of these extensions. ACAP provides a standard mechanism for expressing conditional access which is what is now required. ACAP used Robots.txt on the insistence of the search engines. An xml format is being used in other applications.
The initial implementation of ACAP, using the Robots Exclusion Protocol, was appropriate only to the relationship with the search engines (and was undertaken at the insistence of Google) and other aggregators using web crawlers.
The ACAP Technology Working Group, which has done the drafting work on ACAP versions 1.0 and 1.1, is now turning its attention to how ACAP can be applied to the growing range of business models for online delivery of copyright content.
Technical work is ongoing and the next version of ACAP will build upon our experience with versions 1.0 and 1.1, and will add new forms of expression to make ACAP both more expressive and more flexible, capable of being communicated in a variety of ways and interoperable with a wider range of web content delivery applications.