You can buy rodent deterrents that plug into the wall socket - all they do is emit a noise which mice don't like (and we can't hear) so they steer clear of your home. No poison, no traps, no dead mice - all good.
We tried one of the plug in sonic deterrent things, it had absolutely no effect.
In the end we bought a humane trap and caught two of the little critters. Peanut butter works well as a 'bait'.
Once they were caught my husband drove them halfway to work and then let them out in a park.