"If we believe that you have used or tried to use any ticket or Oyster
card to defraud us we may cancel and not re-issue it. If this happens,
we will not give you a refund of the remaining value of the ticket or
Pre Pay, or refund any deposit paid for the Oyster card."

...which appears (again, IANAL) to give them free rein to withdraw
your Oyster complete with all stored value, if they happen not to like
what you're doing with it.

Not quite. They would still have to prove intent to defraud. And that requires a published
definition of fraudulent use (at the very least by comparing with a leaflet that describes
non-fraudulent use), which seems to be absent.

I seem to remember reading somewhere that Oyster pay as you go
has a 2 hour time limit for each journey in order to prevent misuse.