I really liked your paper https://arxiv.org/html/2504.04072v1
It'd be interesting to change this slightly to something a bit more biz realistic in how the agents will be deployed. Eg: a team of system admins or coders and managers and various pressures (getting fired, promoted, politics, budgets, adding psycho colleagues, cyber attacks, vulns, bugs, etc). They could be developing software or maintaining an environment, typical and relevant stuff you might use the LLMs for. Marketing and sales and other biz areas are equally relevant.
I think this is one of the best ways of detecting alignment issues I've seen because alignment issues will reveal themselves mostly at pressure points and emergent scenarios that are difficult to think up.
I really liked your paper https://arxiv.org/html/2504.04072v1
It'd be interesting to change this slightly to something a bit more biz realistic in how the agents will be deployed. Eg: a team of system admins or coders and managers and various pressures (getting fired, promoted, politics, budgets, adding psycho colleagues, cyber attacks, vulns, bugs, etc). They could be developing software or maintaining an environment, typical and relevant stuff you might use the LLMs for. Marketing and sales and other biz areas are equally relevant.
I think this is one of the best ways of detecting alignment issues I've seen because alignment issues will reveal themselves mostly at pressure points and emergent scenarios that are difficult to think up.