Tony Rock Talks Growing Up in America's Funniest Family with Brother Chris

Tony Rock credits growing up as one of eight children in Brooklyn with helping him — and his famous brother Chris — cultivate a sense of humor.

“When you have that many siblings, it’s great for your development because you have a bunch of people to bounce ideas off of.” Tony, 43, tells PEOPLE in this week’s issue, on stands now. “We weren’t rich by any means, but we had each other, so we were rich in family. When you don’t have a lot, it just fuels that creativity. So it manifested in us doing characters of people in the neighborhood or doing impersonations of Mom and Dad. The comedy bug, it takes over.”

When it came to making it as a comedian, though, the Think Like a Man actor received tough love from his older brother, 53.

“Chris is pretty much, ‘We gotta do it on our own,’ ” says Tony, who most recently starred on CBS’s Living Biblically. “He’s just as quiet with us as he is with the rest of the world.”

And when his younger brother Jordan wanted to get into the family business, Tony embraced a similar approach. “When he started doing standup, he would beg me to take him on the road with him, but I had to let him pay his dues first,” explains Tony, who’s out now on his Tony Rock Live! comedy tour. “I couldn’t just give him the position just because he’s my brother. Nothing’s given in the Rock family.”

Once Jordan, 27, got settled in the industry, however, Tony brought him along on tour.

“The absolute best part of my career at this point is working with Jordan,” he gushes. “The fact that I’ve been there his entire life from birth to the walking and talking, and now I get to take him on the road with me and stand in the back of the room and watch him absolutely kill a room, it’s just a feeling that I cant explain to you that my baby brother’s so grown up now.”

RELATED VIDEO: Chris Rock and Malaak Compton-Rock Clash Over Child Support

While he shares an industry with his brothers, it’s his sisters that keep him going. “If my sisters like something I do, then I’m very happy with it,” Tony says. “I only listen to my sisters!”

• For more on Tony Rock pick up the latest issue of PEOPLE, on stands Friday

Though all of his siblings have grown up and gotten busy with their own families, they still cherish their time together.

“My older brothers have their own children and they have to go to soccer practice and football practice on the weekends,” Tony says. “They’re doing their thing and on the weekends and I’m on the road. The busier we get, the less time we get to see each other, that’s why when we see each other it’s really, really kind of special.”