My suggestion: never use the word “Cancel” in the default action.
To cancel a subscription, you can, for example, say “Remove Subscription” or “Unsubscribe.”
To cancel a download, you can, for example, say “Stop Downloading”.
To cancel a setting, you can, for example, say “Revert Settings”.
Here’s what Facebook does when cancelling a payment subscription (Facebook subscription API).
There’s no reliance on Yes/No. There’s no misleading use of the word cancel. Clear explanation and buttons that clearly define the impending action.
Then they clearly confirm what just happened.
Skype on the other hand shows what not to do. Much confusion!
Edit: Microsoft have since changed Skype’s Cancel button to say Close. It’s a pretty lazy effort at improving the situation, but at least it doesn’t say Cancel any more!
Name the buttons for what they do. If the default is “cancel”, then cancel the cancel should be something simple like “Don’t cancel”.
I know that it’s not ideal to use the word ‘cancel’ in both of them, but it’s the clearest option in this unique situation, and clarity is far more important.
Edit: Some good suggestions from the comments below are to specify what it is that you are cancelling, and to use a term that doesn’t include the word cancel – such as “Keep Subscription”.