Meghan Markle's Mom, Doria Ragland, Speaks Out for the First Time in New Netflix Show

Doria Ragland is "ready" to have her "voice heard."

The mother of Meghan Markle spoke out for the first time in her daughter and son-in-law Prince Harry's new Netflix docuseries Harry & Meghan, in which she admits "the last five years have been challenging."

Recalling how Meghan, 41, first told her she was dating the royal, Doria, 66, said, "We were on the phone and she says, 'Mommy, I'm going out with Prince Harry,' and I start whispering, 'Oh my God.' "

"And so it was from the beginning, it was very sort of, 'Oh my God, nobody can know,' " she continued in her interview, which takes place in the series' third episode.

Of Prince Harry, 38, Doria said she noticed he was "handsome" and "really nice" upon first meeting him, with "really great manners."

"And they looked really happy together," she added. "Once it was announced that they were together, it seemed kind of like a novelty."

While she couldn't have been happier that her daughter had found "the one," Doria faced challenges due to the early buzz surrounding Meghan's relationship with Harry — such as feeling "stalked by the paparazzi" herself.

For more on Prince Harry and Meghan Markle's Netflix docuseries, listen below to our daily podcast PEOPLE Every Day.

"I felt unsafe a lot. I can't just go walk my dogs. I can't just go to work. There was always someone there waiting for me," she explained.

Can't get enough of PEOPLE's Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

Doria was the only member of Meghan's family who attended her wedding to Prince Harry back in May 2018 and currently lives near the couple and their two children — son Archie Harrison, 3½, and daughter Lilibet Diana, 18 months — in California.

Speaking about racism Meghan has faced in the public eye, Doria said, "As a parent, in hindsight, absolutely, I would like to go back and have that kind of real conversation about how the world sees you."

RELATED VIDEO: Netflix Releases First Trailer for Meghan Markle and Prince Harry's New Docuseries

Meghan herself explained in the series that while "obviously now people are very aware of my race because they made it such an issue when I moved to the U.K.," it wasn't always that way.

"Before that, most people didn't treat me like a Black woman so that talk didn't have to happen for me," she noted.

But the negative words in the media caused Doria to tell her daughter, " 'This is about race,' and Meg said, 'Mommy, I don't want to hear it.' "

"But this is what is coming down the pike," Doria added.

The first three episodes of Harry & Meghan are now streaming on Netflix.