In a medium-sized mixing bowl, sift the powdered sugar and powdered peanut butter together.
In a stand mixer, cream the butter until all lumps are gone using the paddle attachment.
Add the powdered sugar/powdered peanut butter mixture into the stand mixer bowl and mix on low until combined. Make sure to cover the stand mixer with a towel or add only a cup or so of the mixture at a time. Use a rubber spatula to scrape down the sides of the mixing bowl as needed.
Still mixing on low, add in the vanilla extract and two tablespoons of milk. Mix until creamy. If the consistency is too thick, add in up to two more tablespoons of milk.
Place the peanut butter buttercream into an icing bag to decorate cookies, cakes, or other treats.