assistant_only_loss=True , the dataset is not conversational

[cherryshhh]

my dataset is like :
def concat_prompt_completion(example):
prompt = example['prompt']
response = example['completion']
return {
"messages": [
{"role": "user", "content": prompt},
{"role": "assistant", "content": response}
]
}
train_dataset = train_dataset.map(concat_prompt_completion)
test_dataset = test_dataset.map(concat_prompt_completion)
i dont know why it come back ValueError: You set assistant_only_loss=True`, but the dataset is not conversational. This option is only supported for conversational datasets.

[demirciberk]

Confirmed by reading TRL's source (is_conversational in trl/data_utils.py):

python
def is_conversational(example: dict[str, Any]) -> bool:
supported_keys = ["prompt", "chosen", "rejected", "completion", "messages"]
example_keys = {key for key in example.keys() if key in supported_keys}

if example_keys:
    key = example_keys.pop()  # takes an ARBITRARY key from the set
    maybe_messages = example[key]
    if isinstance(maybe_messages, list):
        maybe_message = maybe_messages[0]
        if isinstance(maybe_message, dict) and "role" in maybe_message:
            return True
return False

example_keys is a set, and .pop() on a set returns an arbitrary element, not necessarily "messages". Your .map() call does:

python
def concat_prompt_completion(example):
prompt = example['prompt']
response = example['completion']
return {
"messages": [
{"role": "user", "content": prompt},
{"role": "assistant", "content": response}
]
}
train_dataset = train_dataset.map(concat_prompt_completion)

Dataset.map keeps the original columns by default (prompt, completion) alongside the new messages column you return, unless you pass remove_columns. So each example ends up with keys {"prompt", "completion", "messages"}. When is_conversational builds example_keys, it intersects with ["prompt", "chosen", "rejected", "completion", "messages"] and gets all three back as a set. If .pop() happens to return "prompt" or "completion" (both plain strings, not lists), isinstance(maybe_messages, list) is False, and the function returns False — even though a perfectly valid "messages" list is also present. That's why assistant_only_loss=True (which requires is_conversational(...) to be True) raises:

ValueError: You set assistant_only_loss=True, but the dataset is not conversational. This option is only supported for conversational datasets.

Fix: drop the leftover prompt/completion columns so only messages remains:

python
train_dataset = train_dataset.map(
concat_prompt_completion,
remove_columns=train_dataset.column_names, # or explicitly ["prompt", "completion"]
)
test_dataset = test_dataset.map(
concat_prompt_completion,
remove_columns=test_dataset.column_names,
)

After this, each example only has the messages key, is_conversational deterministically inspects it, sees a list of {"role": ..., "content": ...} dicts, and returns True, so assistant_only_loss=True should work with SFTTrainer/SFTConfig. See the docs on assistant_only_loss and conversational datasets: TRL Dataset formats guide and SFTConfig.assistant_only_loss.

Note: I have not run this fix myself — I traced the logic in the current main branch of trl on GitHub, but I did not execute training code to reproduce/confirm the exact failure end-to-end. If removing the extra columns doesn't resolve it, please share the exact TRL version (pip show trl) and the full dataset schema (train_dataset.column_names / train_dataset.features) before and after the .map() call, since the arbitrary-.pop() behavior can vary with Python's set hashing across keys of different lengths.