Two answers:
If you wish to be able to use XAMPP on almost any machine (and are willing to live with a few less tools) , say from a thumb drive - install the 'XAMPP Lite'. Or use the 'tip' this page of not using the XAMPP installer to extract and place your files.
http://www.apachefriends.org/en/xampp-windows.htmlIf you want to work with the full product (with all the bells & whistles) consider using a USB drive letter utility on the machines you wish to use (so the drive letter is always mapped consistently) -- then install XAMPP (regular) on the USB drive.
This is based on my own experiences, which also involved a lot of 'futzing around'.
Good Luck