Baby Mice/ Ethics

The kitchen has mice. Mice live in the kitchen.
For the first time in my life, I live with dogs but no cats, so this is an issue.  I also have two pet rats, who look a lot like big mice.  So. 
Irritated by the nibbles taken from a bag of tortilla chips, I set traps.
I caught two mice, and tried to quash my internal dilemma: if I am a keeper of pet rodents why kill the mice pooping on the counter? (A: because pooping on the counter is gross.)

Then, there was a baby mouse trapped in the sink. Tiny.  So small.  It's whole body would have fit in the circumference of a quarter.  It had freakishly long hind legs though, hopping desperately to escape the sink.
I was too overcome by its cuteness to kill it, so I trapped and released it outside (where it has likely died).
Then I felt bad.  Poor baby mouse!  It had never been outside before.  How was it to survive?  It probably suffered.

Days later: another tiny mouse in the sink.  Tinier, even.  The body small enough to fit in a nickel.  This one I trapped, and released on the kitchen counter, whereupon it scurried into the toaster.

Now: tiny poops in the dishrack every day, and I am anxious about the toaster's cleanliness.  I am skeeved out (wish I'd killed all the mice) and optimistic that I'll find another tiny adorable grasshopper-like baby in the sink (wish I could stroke all the mice).