Eigen: Open source, header-only matrix and linear algebra library
Boost: A massive collection of open source C++ libraries. In essence an extended STL. Some Virtuoso libraries use portions such as ASIO or container classes. Since Boost is a fairly heavyweight set of libraries, where possible any Boost functionality is optional.
Assimp: Asset importer for 3D models. Uses a uniform interface across various model formats. Provides easy access to geometry, lights, textures, and skeletal animation information. Asset importing and formats can be excruciating to deal with, and this library solves most of your problems for you. The Virtuoso model libraries use Assimp on the backend for importing models created by artists.
STB Image: A set of simple image loaders. Public domain C code by Sean Barrett. Many other useful links on his site.
CMake: Cross platform and compiler agnostic build system that can generate makefiles or project files for the IDE of your choice.
Code::Blocks: A cross platform open source IDE. Easy to adapt to coming from a Visual Studio background. We use this IDE for Windows development.
Doxygen: For those of you who occasionally feel the need to document your code.