Background: The accuracy of ultrasound estimation of fetal weight (EFW) at term may be useful in addressing obstetric complications since birth weight (BW) is a parameter that represents an important prognostic factor for perinatal and maternal morbidity. (2) Methods: In a retrospective cohort study of 2156 women with a singleton pregnancy, it is verified whether or not perinatal and maternal morbidity differs between extreme BWs estimated at term by ultrasound within the seven days prior to birth with Accurate EFW (difference < 10% between EFW and BW) and those with Non-Accurate EFW (difference ≥ 10% between EFW and BW). (3) Results: Significantly worse perinatal outcomes (according to different variables such as higher rate of arterial pH at birth < 7.20, higher rate of 1-min Apgar < 7, higher rate of 5-min Apgar < 7, higher grade of neonatal resuscitation and need for admission to the neonatal care unit) were found for extreme BW estimated by antepartum ultrasounds with Non-Accurate EFW compared with those with Accurate EFW. This was the case when extreme BWs were compared according to percentile distribution by sex and gestational age following the national reference growth charts (small for gestational age and large for gestational age), and when they were compared according to weight range (low birth weight and high birth weight). (4) Conclusions: Clinicians should make a greater effort when performing EFW by ultrasound at term in cases of suspected extreme fetal weights, and need to take an increasingly prudent approach to its management.