*if* you use binary tarballs they *should not* be extracted in a user writeable location as *no binary* whenever possible should have permissions allowing a ordinary user to change them
This is simply not the way how end users install original Mozilla Firefox binaries.
In addition, if you have write access to ~/, you can also change .bashrc to add paths to executable files and do all sorts of other nasty things.
FWIW I run Mozilla's Firefox nightly builds and they work perfectly fine on Fedora. I've also found the lag behind the official releases annoying, also for some other large end-user packages (blender), for no perceived benefit. But it's not especially painful.