In a medium bowl, add one can of sweetened condensed milk and 1 cup of peanut butter.
Whip 1 cup of heavy whipping cream to soft peaks.
Fold ⅓ of the whipped cream into the peanut butter/condensed milk mixture with a rubber spatula. Fold with the spatula in the middle of the peanut butter mix, flipping in the whipped cream. Rotate the bowl and continue to fold in the cream. At first, the peanut butter mix is a little stiff. As you continue to add the whipped cream it will become looser.
Add the remaining whipped cream, ⅓ at a time. Stop as soon as the cream is incorporated.
In another medium-sized bowl, mix the other can of condensed milk with ½ cup of cocoa powder and vanilla extract.
Whip 1½ cups of heavy whipping cream to soft peaks.
Fold the whipped cream into the cocoa/condensed milk mixture with the same process as above.
Place 12 marshmallows on a parchment-lined sheet pan. Put the sheet pan under a broiler for 1-2 minutes until the marshmallows are toasted to your liking.
Add the peanut butter mix and the chocolate mix to the 8x8 pan. Add two dollops of the peanut butter mix in opposite corners. Do the same with the chocolate mix. With a spatula, swirl the two different mixtures into each other. Make sure to reach down to the bottom of the pan so that the swirl is not just on the top.
Randomly place the toasted marshmallows around the top of the ice cream mix. Swirl these into the peanut butter/chocolate mix. Because of the elasticity of the marshmallows, they will be harder to incorporate into the mix.
Place the pan in the freezer. It takes 4-5 hours for the ice cream to harden. Once the ice cream has set, cover with plastic to prevent ice crystals from forming on top.