There are a number of useful tools out there for writing computer vision software. The three packages that follow are the ones that I have gotten the most use out of over the years. These are very good tools, which I highly recommend to anyone that needs to create computer vision or image processing software.
Matlab Image Processing Toolbox
The Matlab Image Processing Toolbox is a set of functions in Matlab that support image processing and manipulation. The toolbox provides the basic functionality to read in and write out images; in recent years, this has expanded to video files as well. Matlab treats images as matrices of pixel values, making great use of the first-class-citizen support Matlab gives to vectors and matrices. Working with images at a pixel level is just like working with any other matrix in Matlab.
Matlab's great strength is it's ease of use for the programmer. I sometimes joke that Matlab code is "executable pseudocode," and that is precisely the feel it gives while using it. I can code up and try out more ideas, faster, in Matlab than I can in any other language. Creating output plots, figures, and images is very straight forward, and makes checking code behavior a joy. Unfortunately, there are some drawbacks to using Matlab--it's code can only run within the Matlab interpreter; it's dynamic record datatype is nice to work with, but there is no object-oriented support; and code executes very slowly (especially if you use any loops in your code, which is likely with image processing).
I find that my typical development workflow when developing computer vision algorithms will often start with creating prototype code in Matlab. Once I get all of the mathematical and structural kinks worked out, I will sometimes then move on to implementing the code in C++ based on my completed Matlab functions. Reimplementation in C++ is only done if I need real-time performance or faster testing. If performance is no big deal, I find simply using my Matlab code to be acceptable.
OpenCV
The OpenCV C++ library is probably the most widely used computer vision library. There is a lot in here from the low level to the high level. The low level functionality to read in and write out video files is exceptionally useful, and the first step for someone just beginning to write vision software in C++. OpenCV contains a plethora of higher-level vision code, from edge detectors through face detectors through machine learning algorithms.
OpenCV is very usable, and installation is very easy. There is not much that has to be linked against, and the most widely used functions in it work really well. It is an open source project though, and components that do not get much use can exhibit some wonky code. It is pretty big, needing about 4 GB of space. Overall, OpenCV is a great library and a good place to start for someone beginning to write vision code in C++.
VXL
The VXL C++ library is an amazing library, with vast functionality applicable to computer vision projects. I am especially fond of the VNL and VNL Algos sub-libraries. Honestly, they are the closest thing I have found to having Matlab's language-level support for matrices and matrix operations in C++. This isn't to say the VXL library is as intuitive to use as Matlab, but VXL provides a lot of the functionality without having to write all the low-level matrix manipulation functions yourself. What I find especially remarkable about VXL is that all of the code that I have had to go through in it has been very well written, which is not the case with all open source projects.
The major downside of VXL, in my opinion, is the building and installation process. It is a serious rite of passage. If you intend to use VXL for a project of yours, make sure to set aside a day (maybe two) to get to the point where you can link against it. VXL is a library where only the source is provided, and it is up to you to get it compiled and working. There are installation instructions on the VXL site, but they are not very detailed. Here are some detailed VXL installation instructions that I have found to be useful. It is important to also note that VXL is quite large; make sure to have about 10 GB free before you start with it.