the _update_metrics_and_input just calls to_dict on the output. The problem is that its not using json mode with the underlying model, and so frequently the model returns json wrapped in markdown. If ...