You can save the Ultrabeat settings in the project so they'll be there evrytime.
Choose File > Project Settings > Assets
You need to tick the box called 'Include Assets' and then (under Advanced options) 'Copy Ultrabeat samples to project folder'
You can also save EXS files, Space designer files and movie files.
There are other key commands, and ways to access the Assets menu, on page 1005 of the Logic manual.
Rounik also taught us a great way to save files to a separate folder that you can call up in any project:
This was for the ES2 but I think it should work the same for Ultrabeat
Hope that helps!