Well, I got it fixed in the end. After much discussion with T-Mobile on Twitter and then email, they phoned me. And it was fixed in, well, seconds.
It turns out that you can’t download MMS message when you’re using Wi-Fi. Now, this maybe a T-Mobile limited thing, but when you received an MMS, you get a simple text inviting you to download the media that’s attached to it (usually a picture). In my case, I’d click on the download button and it would then state that the download had failed – nothing more.
For, what I assume is, security reasons, this only works when using the phones data connection, not Wi-fi (where I guess it can’t authenticate you properly). If you’re using the latter, the data connection is not in use. So, turn off Wi-fi and the download works.
A simple fix, as it turns out, but frustrating until I knew.
Meantime, T-Mobile have, helpfully, put a useful tip on Twitter…
Having trouble sending MMS in your G2, ensure WiFi is switched off and re-try
Only the day after I found this out. I assume that was aimed at me 😉