When I open a transparent background PNG, I see a grey-white grid.
Is the grid embedded into the file? How can I change it?
That's there to show that it's a transparent background; it's not actually embedded since it's just the way it's displayed in Photoshop. When you drop that onto your web page / document you won't see the grid.
You can change the parameters for the grid in Photoshop → Preferences → Transparency & Gamut.