How do I remove CSS from a website that loads a script tag, and how do I remove that script tag?

This is a weird specific problem and I can't find any info anywhere.

I have a main React app that dynamically loads other React micro-apps. It does this through the componentDidMount()functionality of a component , using pure JavaScript to add a script tag to the end of the document body whose src address proxies the request to the web server to return a single bundled file that is its sub-application. load. When the component is unmounted, the script tag is removed from the document body and the div root is emptied.

The mini-app needs to be loadable into a bundle (no need to go into it here), I can do this by npm run buildrunning a modified function like the following to build the react app ;

"scripts": {
    "build": "npm run build:sass && npm run build:react && npm run build:bundle",
    "build:react": "react-scripts build",
    "build:bundle": "webpack --config webpack.config.js",
    "build:sass": "node-sass --precision=5 --indent-type=space --output-style=nested --indent-width=2 src/styles/Site.scss src/styles/Site.css"
  }

The bundle portion of the command uses the webpack.config.js file below;

const path = require("path")
const UglifyJsPlugin = require("uglifyjs-webpack-plugin")
const glob = require("glob")

module.exports = {
  entry: {
    "bundle.js": glob.sync("build/static/?(js|css)/*.?(js|css)").map(f => path.resolve(__dirname, f)),
  },
  output: {
    filename: "build/bundle.min.js"
  },
  module: {
    rules: [
      {
        test: /\.css$/,
        use: ["style-loader", "css-loader"],
      },
    ],
  },
  plugins: [new UglifyJsPlugin()],
}

This in turn produces a bundle.min.js file containing all the scripts, node_modules and css in the mini-app, which is loaded by the main app via the following components.

export class MyComponent extends React.PureComponent<Props, object> {

    componentDidMount() {
        const script = document.createElement("script");
        script.src = "myScriptSrc";
        script.async = true;
        script.id = this.AppIdToScriptId();
        document.body.appendChild(script);
    }

    componentWillUnmount() {
        document.getElementById("miniAppRoot").innerHTML = "";
        if (this.props.app) {
            document.getElementById(this.AppIdToScriptId()).remove();
        }
    }

    AppIdToScriptId = () => "d" + this.props.App.Id.replace("-", "");

    public render() {
        return (<div id="miniAppRoot" ></div>);
    }
}

export default MyComponent;

This all works fine, the only issue I'm having is that the css loaded via the react micro-app bundle script persists even after I remove the script tag and empty the html root. Does anyone know why this is happening? I'm not sure if the webpack config is to blame for client side caching or some other possibility, any ideas how to fix this?