Preheat the oven to 350 degrees. Line two cupcake pans with paper liners.
To make the cupcake batter, combine the flour, cocoa powder, baking soda and salt in a medium bowl. Whisk and set aside. In a liquid measuring cup, combine the sour cream, milk and vanilla extract. In the bowl of an electric mixer fitted with the paddle attachment, cream the butter and sugar together until light and fluffy. Beat in the eggs one at a time, scraping down the bowl as needed. With the mixer on low speed, alternately add the dry ingredients and the sour cream mixture, beginning and ending with the dry ingredients. Mix just until incorporated. Set aside.
To make the filling, combine the confectioners’ sugar, peanut butter, butter and vanilla extract in a bowl. Beat with an electric mixer on medium speed until smooth. Using your hands, roll the mixture into 1-inch balls (at least 20).
Spoon one or two tablespoons of the batter into the bottom of each cupcake liner. Place a ball of the peanut butter filling in each cupcake liner and top with the remaining batter so that all the cups are filled. Bake for 18 to 22 minutes, or until a toothpick inserted in the center comes out clean. Allow to cool in the pan for 5 to 10 minutes, and then transfer to a wire rack to cool completely.
To make the frosting, in the bowl of an electric mixer fitted with the paddle attachment, combine the confectioners’ sugar, cream cheese, butter and peanut butter until smooth. Beat mixture until smooth and well blended. Pipe as desired onto each cupcake.
Store the cupcakes at room temperature for up to 2 hours, or store in the refrigerator until ready to serve.