Evaluations may fail to improve AI risk Understanding or Mitigation and could even cause harm, but evaluations could still be valuable with a few considerations.