In a large bowl, whip together sweet potatoes and butter using a hand held or stand mixer. Once smooth, add eggs one at a time until fully incorporated. Add sugars, flour, salt, baking soda, vanilla, cinnamon and nutmeg. With mixer on low, add milk and mix until blended.