When my babies started climbing out, I cleaned the nest box out, then turned it on its side and put some hay down to create a little cubby hole for them to snuggle in. I also made sure the mom had something she could hop up onto to get away from babies wanting to constantly nurse from her, as she became totally frantic when all these baby bunnies started wandering around her pen.
I can't say what you should do, just what I did. I didn't have any problems taking them out and playing with them, and the babies are now grown up 7 year old bunnies. The important thing is just making sure the environment they'll be in is safe for them.