To remove the computer from the domain, you'll need access to a local administrator account. This may be as easy as typing
.\Administrator in the
Username: field at login and pushing enter. If the account is disabled or has a password that you do not know, then you'll need to use some kind of boot CD to gain access to it.
As for the performance issue, it's hard to say. It really depends on how that particular domain was configured and what kind of group policies were applied to the machine. For example, if it's still looking for a user profile on a network share that no longer exists, then yeah, you may run in to some odd behavior or even temporary profiles. Basically you're opening yourself to a lot of unknowns.
I would definitely remove the machine from the domain.