Willie Garson and Son Nathen: We Both Wanted This Adoption

When Willie Garson adopted his son Nathen in 2009, it was a joint decision.

“We had a smooth transition because we both wanted this adoption. Fatherhood is what I always wanted it to be,” the White Collar star, 50, tells PEOPLE. “I wanted it so badly and he wanted it so badly. I think often it can be difficult because it’s not that balanced.”

And what Garson wanted was to get right to the fun stuff — which is exactly what he did when Nathen, now 12, arrived. “I don’t know what I would have done with a newborn! I would have dropped him, I would have washed him in the dishwasher,” the single dad jokes.

“Nathen could clean himself, he knew how to eat, he knew how to use the restroom. All of these things I got to skip. I’m on my own, so I don’t think I could have handled a kid that was much younger.”

But while he doubts his skills with a baby, the doting dad does admit his regret at not finding his son earlier. “I do wish I’d met him sooner because I enjoy him so much. Up until six months ago he was always saying, ‘You need to be with me,’ and I’d say, ‘Nathen, in a couple of years you’re not going to say that.'”

Garson continues, “As he becomes a teen, it’s going to be like, ‘Dad, leave me alone!’ It’s gradual. But for now he doesn’t mind me being around.”

The pair, who share a love of food and travel “as long as we stop for ice cream every two hours,” Garson says while recalling a recent European trip with Nathen, have been adjusting perfectly to life as a twosome.

“Nathen and I always talk,” he says. “We talk about everything that a lot of his friends’ parents say, ‘I can’t talk to my kid about that.’ We tend to work it out together because we are partners in this.”

Adds Nathen, “I used to call him Willie, now I call him Dad.”

Father and son have been busy having a blast, which has left little time for Garson’s love life.

“I go out once in a while,” he admits. “But it’s hard for me to justify, when I’m working and I’m away, to then say, ‘I’m going out Tuesday night.’ Where’s the fun in that? I’d just be at the date talking about how my kid is sitting at home and has a math quiz.”

For more from Garson and to read how other celebrity fathers describe their lives as dads, pick up the latest issue of PEOPLE, on newsstands now.

— Michelle Ward and Monica Rizzo