You can turn it off in the options. It is under Tools|Options - Advanced Tab, Multimedia section.
The option is called: Enable Image Toolbar (requires restart)
Although it seems like a silly requirement to turn off an option such as this, a restart was definitely required. I turned it off and the little toolbar continued to show up. So I rebooted and it doesn't pop up anymore.
It might just mean to close down all the browser windows and restart the app, I didn't try that, but rebooting does the trick for sure.