Believe it or not, kittens don't need milk. And for some cats, cows' milk can actually cause diarrhea. Fresh water is all your kitten needs.
It is very important to make sure your kitten has a bowl of fresh, clean water at all times. If you suspect your kitten is not drinking enough, it may be that she can taste chemicals in the water, so you may want to try filtered or bottled water. Some cats even prefer to drink from flowing water sources, like fountains or dripping taps, so you can buy water fountains designed especially for cats. And don't forget that if she's eating dry, crunchy food, it's essential you give her plenty of water.