Originally Posted by WebairGerard

It sounds like something is wrong with WP itself or the plugs, that VPS config SHOULD be enough. Likely a misconfig somewhere. Caching will help greatly.

We have some custom WP cache modules that we use too that will help. If you do decide to move to another VPS provider let me know. Hope the above is helpful and all works out soon for you.