After setting up this blog, I discovered that simply adding a picture to your profile would not change your avatar on your WordPress blog.
I did some Googling and found that you will need to sign up for a Gravatar (this stands for Globally Recognized Avatars).
The Gravatar is integrated into your WordPress blog, and so you will need to sign up for an account here then attach that to your WordPress blog. Using Gravatars