The topic of WordPress back-ups is one that gets a lot of attention. I often find that most articles make it out to be far more complicated than it needs to. I follow a few simple steps every couple weeks or every time there is a new release to back-up both my installation directory and my database. To back-up my database I use a tool called Sequel Pro which is only available for the Mac but there are several Windows alternatives, just google "MySQL GUI Windows" and check out some options there.
Connect to your database using Sequel Pro, select the appropriate database from the drop-down (upper left) then simply go to "File->Export->MySQL Dump". Select a location to save the file and boom your database has been backed up. I highly recommend you back-up your back-up using something like Amazon S3, Dropbox or even an off-site FTP.
Next we need to back-up the WordPress installation directory (includes all your media, themes, uploads and other content) because without the database will not provide a full restore. If you are on other than a *nix platform you probably know how to archive/back-up files/folders by now but if you are not using a GUI here is the method I use.
Change directories to one level up from your WordPress root directory and type in the following command [assuming your root directory is named 'wp', if not this will need to reflect the name of your installation folder.]:
tar -zcvf archive_name.tar.gz ./wp
The command above will create a tar.gz archive with everything (recursively) in your installation directory, again back this file up somewhere off-site.
That is it, WordPress is now backed-up and your can upgrade with a clear conscious! If you ever need to restore from these back-ups simply put the .tar.gz file back where you created it and type the following command:
tar -zxvf archive_name.tar.gz ./
The command above will recreate your directory structure and restore all necessary files. Next you will need to recreate the database then "replay" or execute the commands from your SQL file to re-create the tables and repopulate the data.
I hope this helps to dispel some of the myths about how complicated WordPress back-ups need to be and encourages all of you to back-up and back-up often.