The last few months of 2013 were perhaps the best years of Teen Mom star Amber Portwood's life: Sober at last, she was released from jail after 17 months, and began the long process of rebuilding her relationship with her five-year-old daughter, Leah. But Leah's dad, Gary Shirley, is speaking out to shatter any hope that 2014 will be the year Portwood regains custody. In a new interview with OK! magazine, he insists, "It's going to stay the way it is, to protect Leah."

Shirley won full custody of the tot in December 2011, amid Portwood's struggles with drug addiction and anger management. And though he has forgiven her for how she treated him during that difficult time, he hasn't forgotten.

"Well, everyone knows that she went to jail for drugs and stuff," he explains. "I'm not saying that she'll ever get back into that, but I hear stories of how easy it is to relapse because that pressure is always there."

With that in mind, he says he plans to keep a tight lock on Leah's custody situation. "I don't want to screw myself over by agreeing to joint custody and then have to fight for Leah," Shirley tells OK! "So I think it's going to stay the way it is, just to protect Leah."

In addition, he says, "It's because I've grown, and I have a bond with my daughter that's special, and I don't want that to ever go away. Amber being away gave me a chance to get to know my daughter in a way most guys don't."