If the target is a 3.3V device, it would be best to use a 3.3V compatible programmer, too. The USBtinyISP (since version 2) makes use of 3-state buffers for the signal lines, and has a jumper that can be removed to disconnect the 5V from the USB port from the target.
Parts and Schema
Using the programmer
AVRDUDE is a very popular command-line program for programming AVR chips. Avrdude version 5.5 and higher has built-in support for USBtinyISP!
avrdude knows this programmer as usbtiny