A. - XAMPP is Windows registry agnostic, if it is not installed on a machine using the 'Installer', so a USB stick version won't make entries in the registry.
B. - The other part of your question is very hard o answer. It depends on the level of security tehy have implemented. If they run a tight shop with Windows servers and Group Policies etc. there's proabably nothing you can really hide. IF they let you arbitrarily plug in USB devices, I'm guessing teh security level is not 'paranoid'.
All that said - if they have a policy against it (using unauthorized soaftware), and you could lose your job - plan on losing that job, as 'weird **** happens' - such as how much do you know about how they audit IP traffic???...