If you want to try the extra feedings, it wouldn't hurt anyone and it might give an extra boost to the smaller babies. (If momma will cooperate!)
They don't have to "look" like dwarfs to carry a recessive Dwarfing gene. I've seen some fairly large parents throw peanuts.As others have mentioned, a peanut has physical, internal birth defects that prevent it from digesting food and absorbing nutrients. But your little guy could just be a "runt". (I dislike that word, but lack a better one.)
Knowing how the babies develop in the uterine horns, they don't all start out from fertilized eggs at the same exact moment in time. Considering the rabbit has 2 completely separate uterine horns, that each receive eggs from it's own ovary (andeach ovary may not release eggs at exactly the same time)there is a variable of time that can influence the development of each kit.
An egg that is fertilized a few hours or a day after the others can put that baby at a severe disadvantage at birth. The fertilized egg floats for a few days in the uterusbefore attachment to the uterine wall, and developing a placenta. Again, a late attaching egg may not develop as well as the others.
All the babies are born at the same time (generally speaking). The uterine horns go into contractions and they all have to come out, ready or not! So technically, one or two babies could be at an earlier stage of development than the others. (Sort of like a preemie). And considering they have such a short gestation (28-31 days), a few hours less in the womb can make a huge difference.
Some of these "preemies" may develop normally with a little extra nutrition and care. Some don't. Don't blame yourself or the mother if the tiny one doesn't make it.
You are a wonderful, caring person to take in the momma and her litter. Thank you.