If you don't have enough memory, find a way to separate the audio and video tracks into separate files, then decrease the quality on the video, creating a new video file (but keeping the original), and then use your editor to open both the low quality video and the audio. Edit the audio. Save the audio. Then comes the final part, combining them together, into your product file.
There's loads of tools for this, and I'm out of the loop for a while, so I can't recommend, but this should work.
You should worry about not changing timestamps (stuff like file lenghts). Also, if you don't need the video to edit the audio, just skip the part where you convert the video to low quality and just edit the audio.