Mice are a fact of life here in the country - the field mice find any nice warm spot they can when winter starts. They're dirty and destructive, so I can't allow them to stay in my house. Mouse urine is incredibly corrosive - I've had to throw away a number of items of electronics when the mice got into them.
You can't use DeCon or any other pelleted poison, even if you're careful to put the bait someplace the rabbit can't get to. Mice will spread them around, and they're as dangerous to rabbits as they are to mice.Even though the mice will eventually die, in the meanwhile they're capable of moving pelleted poison throughout the entire house - I found my coffeemaker stuffed full of the stuff one year.
What I do is put solid mouse poison in the basement. You can find the stuff at Agway or other farm supply stores, sold in blocks about four inches long by about an inch around. It's too big for the mice to move, so they consume it on the spot. Upstairs I put old-fashioned snap traps wherever I've seen mouse signs. Bait the snap traps with peanut butter, and use double-sticky tape to fasten them down to something large and heavy so if you catch a mouse it can't drag the trap away someplace hidden where you'll only find the trap and mouse until they're really unpleasant (ironically, old mouse mats from the computer work really well). Put the traps up on counters or shelves where the bunny can't get to them. Most years I'll catch six or eight mice in the snap traps the first week or two, then only one or two the rest of the season as the solid bait in the basement takes effect.