Page Size Increase When Activated

I’m noticing about a 500K “weight” increase in my pages when I have the Download Manager plugin activated. Most of the comes from FontAwesome files. The FontAwesome.svg file alone is nearly 300K.

I use the plugin on one private page, so appearance isn’t important. Can I disable the font stuff, or do something else? The plugin is, for some reason, loading these font files on every page on the site. Turning it off on all site pages except where the code is located would be fine, too. Thanks!