Yeah there's going to be a certain amount of DAW overhead even if the plug-in isn't the oscillation/filter or FX source. On the other hand I'm guessing you could also treat it as a regular MIDI hardware device without integration and just route the audio straight into interface and not run VC at all freeing up some CPU, or perhaps use the whole thing as an external FX unit (a pricey FX unit no less!)
