A New Haven man who was being transported to a correctional facility on an outstanding warrant was caught after briefly escaping from Connecticut State Police custody in New Haven on Wednesday.