GarageBand uses the same underlying engine as Logic, and that's a familiar message. Unfortunately it's fairly non-specific. but here are a few things you might look at..
- First, as always, do a hard shutdown/restart
- Try various songs, and see if it's particular to a certain song (or with certain audio files) or if it happens in many songs; test with a brand new Project
- Check your system spec -- ideally there should be an i5 CPU or higher (minimum i3), minimum 8GB RAM (better 16 GB), and there should always be at least 20% free (empty) space on the internal (startup) drive (the drive space is particularly important, when a HD is too full everything can slow down and get glitchy -- if possible you could try playing back the audio files from a different HD)
- In Logic you can try different audio buffer (RAM) sizes, but I guess that's not an option in GB's Preferences
- If you're using a lot of virtual (Software) Instruments, that can put a lot of strain on the system -- make a copy of the song and try removing some to see if the issue goes away or is reduced in severity
Sometimes it's just a mystery -- Apple typically recommends re-installing things, so I guess you could try re-installing GB if all else fails..
Note: if you did any upgrades to your system recently, that could potentially have something to do with it, especially if everything worked fine before, and the problem only started after the upgrades..