Lots of things can be done to speed up the wordpress website, I usually start with Nginx, php5-fpm, mysql , memcache, fastcgi cache as a setup, im sure im forgetting some other stuff. Then it depends on various other elements in your theme, image sizes matter to a point but also offloading some of that to a free cdn like cloudflare.
after you feel like your set run your site though
GTmetrix | Website Speed and Performance Optimization and then look at their suggestions, some of them cant easily be done depending on theme and blocking javascript but you dont have to have it prefect, but if your under 80% or so you have lots of room for improvement it might be a plugin , it might be making sure your stack is optimized for what your doing then maybe offloading some stuff to a cdn.
Their is no real one answer, of course not being an idiot and having 1 meg image files is always a good start (not saying this is you but i find this often when dealing with clients)