Access takes it as a personal mission to ensure that data you enter in a form ALWAYS gets saved. Therefore, Access "helps" you and it saved records when it thinks they need to be saved even if you don't explicitly save them. Once you understand the "Access way", you will understand the concept and be able to intuitively know what actions will cause a dirty record to be saved by Access.
The short answer is when you move from one record to another, Access ALWAYS saves the dirty record. Therefore, each record in the subform is saved as you move to a new record or even back to the main form. So, unlike the single form where you haven't left the form or given Access a reason to save it behind your back, you can have an undo process. The exact same functionality works in the subform, except it works ONE record at a time. A subform is exactly the same as a main form, the only difference is that you can see multiple instances of the form. But each instance is a "single" form from the perspective of how it works regarding the saving of data.
Now, we get to - why. Why would the user have created a main record and possibly multiple sub records and then want to abandon them all? You can do this quite easily, just not the way you are doing it. Since all the records are saved at the point the button is pressed, there is no way to undo the process. However, you can delete the main record and if you have cascade delete enabled on the relationship between the main form record and the subform records, Access will automagically also delete the subform records.
Sometimes you have a situation where all the records of the sub form are a "set" and the "set" must balance. For example, you might have to enter a check # with an amount and then for the subform, allocate the money to different accounts. The subform total amt must = the check amount. If it doesn't, the check is out of balance and you don't want to commit the transaction. This is pretty complicated when using bound forms. If this is your situation, you will need a transaction as
@arnelgp suggested.