The colt was taken out of the field by order of veterinarians at the end of a difficult week for the Derby and the sport.