Miller, a silver medallist behind Felix at the World Championships in Beijing last year, threw herself across the line to win in 49.44sec.
Felix, the most decorated female track and field athlete in history, took silver in 49.51sec with Shericka Jackson of Jamaica claiming bronze in 49.85.
It was a courageous performance from Miller, the 22-year-old whose great-uncle Leslie Miller competed in the 400m at the 1968 Olympics in Mexico City.
Miller burned through the first 300 metres and had built up a comfortable lead heading into the home stretch.
Bolt completes 100m hat-trick in 9.81sec
But the experienced Felix, the 2012 Olympics 200m champion, who overcame a debilitating ankle injury this season to reach Rio, appeared to have timed her race to perfection.
The 30-year-old gradually reeled in Miller over the closing metres as the finish loomed.
But Miller's desperate late, all-or-nothing dive was to prove the difference and after an agonising delay of a few seconds, she was confirmed as the winner.
It was the second dive finish of the evening.
Earlier, Brazil's Joao Vitor de Oliveira had produced a near identical moment to secure qualification from his 110m hurdles heat, delighting the home crowd.
Farah claims gold despite falling in 10th lap
Felix meanwhile was left to reflect on a disappointing defeat which comes after a year which began with the Los Angeles native eyeing a rare Olympic double.
She had hoped to challenge for both the 200m and 400m titles in Rio, aiming to become the first woman to win both races in the same Olympics since Marie-Jose Perec in 1996.
But a freak training injury in April -- when she twisted an ankle on a medicine ball in the gym -- left her scrambling to recover her peak form.
She failed to make the 200m squad for Rio at the US Olympic trials but won the 400m despite not being fully fit.
Her silver on Monday is the 20th track medal she has won at the Olympics or World Championships since her debut in the Athens Games in 2004. No other female athlete has won more.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ