I found some more time today to work on the panels. I've tried 4 different types of clips and finally found one that is durable enough not to break. I just found most of the types to be too brittle.
I made a framework that would recess inside the door where I could attach the clips.
3/4" hole saw to make the slots and holes for the clips. You could use a little smaller one but that is what I had.
Here are the clips I found to be the best. I had to enlarge the holes in the door one size bigger for these to fit. They can pretty easily be removed without destroying them.