The other 39 passengers, including a woman who had fallen overboard, were rescued after local fishermen raised the alarm.
Ten of the survivors were taken to hospital to be treated for hypothermia.
German navy 'rescued over 10,000 migrants' in 2015
The rubber vessel had set off from Turkey in the early morning in windy weather.
The charity Migrant Offshore Aid Station (MOAS), which helps save migrants and refugees at sea, deployed its fast-rescue Responder boat to help bring the stranded passengers to safety in a joint operation with the Hellenic coastguard.
The toddler's body was pulled out of the water by fishermen, according to the coastguard.
The migrants, including the child's mother, were taken to the port of Pythagorio on Samos, the nearest island, which is 50 kilometres (30 miles) away.
There was no immediate information about their nationalities.
"Nothing can prepare you for the horrific reality of what is going on. Today we came face to face with one of the youngest victims of this ongoing refugee crisis. It is a tragic reminder of the thousands of people who have died trying to reach safety in miserable conditions," said MOAS founder and US entrepreneur Christopher Catrambone in a statement.
Eleven migrants drown after boat sinks off Turkey's western coast
Despite the recent cold and choppy waters, large numbers of migrants and refugees are still setting sail from Turkey to make the hazardous journey across the Aegean in the hope of reaching Greece.
Overall, nearly one million migrants or asylum-seekers landed in Europe in 2015 in search of a better life after setting off in boats from either Turkey or North Africa.
Most were refugees fleeing war and violence in Syria, Afghanistan and Iraq.
More than 3,600 people have died attempting to make the crossing.
Three-year-old Syrian toddler Aylan Kurdi, whose limp body was photographed washed up on a Turkish beach in September, became a heartbreaking symbol of the refugee crisis.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ