It turns out that MathJax sets cookies to remember your preferences. Specifically, the mjx.menu cookie. I looked around for hints as how to disable this cookie in the configuration, but found none, so here is my solution:
When creating the configuration element, simply replace the MathJax.HTML.Cookie.Set function with a function that does nothing.
var conf = document.createElement ("script");
document.body.appendChild (conf);