Can you provide the full error message and where you see it?
When you installed XAMPP to your USB stick did you run the setup_xampp.bat file?
This should not be run as it will give a drive letter that makes the USB unportable.
In the XAMPP Control Panel have you any ticks in the Svc check boxes - if so then you must remove the ticks as you have effectively created Windows services which will make the USB unportable and also will create a drive letter for your XAMPP USB installations - this may also allow you to stop and start MySQL.
Also reading your posts it looks like Apache can be started but MySQL can't and that seems to indicate that you may have another instance of MySQL already running - use the currports app below to find out.
You may not have uninstalled your PC XAMPP correctly as it needs that both Apache and MySQL Windows services are uninstalled and the XAMPP Control panel exited before deleting the XAMPP folder.
Are you trying the USB stick in a USB hub - usually a port on the front of your PC?
Try and use a USB port that is directly connected to the mother board - usually on the back of the PC but really should work on the front port, mine does.
Check your firewall is allowing traffic through your USB.
Check your antivirus scanner is also allowing access.
To test the above 2 items try and temporarily disable them and then try localhost again.
Do you have a proxy setting for your lan set in Internet Explorer, if so have you excluded localhost?
Also download this useful little app that does not need installing and is free, to see what ports are being used - port 80 and port 3306 for MySQL should be free.http://www.nirsoft.net/utils/cports.html
Also this may come in handy:http://www.nirsoft.net/utils/usb_devices_view.html
You can have XAMPP on your PC and on a USB stick but you can only run one at a time.