IETF AIPREF: the AI preference vocabulary is still a draft
The IETF AIPREF working group updated the draft vocabulary a site can use to say whether its content may be used for training and answers - what is in it and why it is too early to rely on it
In mid-September 2026 the IETF AIPREF working group published another revision of its draft preference vocabulary. It is an attempt to do what robots.txt cannot: to say not “do not go here” but “do not use this in that way”.
Which problem it solves
robots.txt controls crawling. It does not tell why a bot came: for a page for a search index, for text to train a model, or for an answer to a user’s question.
Today that difference is expressed by bot names: one bot of a company collects training data, another serves search. It works, but requires knowing every bot by name.
What is in the drafts
The group is preparing two documents:
- a vocabulary - the names of kinds of use an owner can allow or forbid: model training, use in AI systems, search;
- a way to attach it - how to state these preferences: in the server response and next to the
robots.txtrules.
What state it is in
Both documents are working group drafts. There is no RFC. The milestone set for August 2026 passed without a published standard.
In parallel there is Content-Signal, a line that Cloudflare documents and adds to its managed robots.txt. It is one company’s initiative, not a standard.
What it does not mean
- That bots already respect these preferences. Bot owners have made no such commitment.
- That
robots.txtis obsolete. It remains the only mechanism bot owners describe directly in their documentation.
What a site owner should do
Nothing new for now. The working tool is the same: robots.txt rules for specific bots in three classes. The bot table checked against owner documentation: AI crawlers and robots.txt.