Rename file extensions easily with bash

I recently had a misconfiguration in logrotate that left me with a bunch of bzip2 files having a .gz (gzip) extension on their filename instead of .bz2.
Not to worry, it was a simple fix to rename them all with one command:
for filename in *.gz; do mv $filename `basename $filename .gz`.bz2; done