It seems to me that you downgrade for a few different reasons:
1. You want to play an ISO of a game on your memory stick.
2. You want to play "homebrew" programs.
3. You want to customize the interface.

Likewise, if these are not a priority for you and you chiefly want to use the PSP to play movies on your memory stick or listen to music on you memory stick, I don't see a real need to downgrade the firmware. My feeling is that most people downgrade so they can play games on their memory stick or for some, because they want to run emulators and their games off of the memory stick. So, if you're priority is to play games on the memory stick, then by all means do what you need to do, but for myself, I'm more interested in portable movies at my beck and call and some audiobooks (though the sound through the speakers is not loud, and it's not much louder on headphones (my iPod sound throws out faaaar more sound through headphones). This is the biggest weakness of the PSP in my opinion... Okay, it's biggest weakness is really all the trouble you have to go to to make it so you can play non-UMD games on it. Still, the sound could use a huge boost and make the player more useful.