Mencoder can concatenate and transcode videos without temp files ([trivial example here][1]). There's a script floating around somewhere called aconvert that tricks mencoder into doing audio-only (which it normally refuses to do). These two things combined seem like they would satisfy your requirements.

Edit: I found aconvert.sh. Unfortunately, it's only designed to convert one audio file. But here's the code anyway, in case anyone wants to use it to craft a command line that converts multiple audio files.

    #!/bin/sh

    # Author: Jonas Jermann
    # Description: A hack to allow mencoder to encode from an audio only file
    
    if [ "$1" = "" ]; then
        echo "Usage: $0 <\"input file\"> <\"output file\"> <\"options\">"
        exit 0
    fi
    
    options=${3:-"-oac mp3lame"}
    
    mencoder -demuxer rawvideo -rawvideo w=1:h=1 -ovc copy -of rawaudio -endpos `mplayer -identify $1 -frames 0 2>&1 | grep ID_LENGTH | cut -d "=" -f 2` -audiofile $1 -o $2 $options $1

  [1]: http://lglinux.blogspot.com/2007/09/concatenate-avi-files.html