Here's how it work with my setup:
The modem will try to negotiate layer one communication with the DSLAM. I don't know much about this part, so I won't go into that.
I have a router (which in general can be integrated into modem) that initiates PPPoE session and the modem then encapsulates PPPoE into PPPoA. Different set-ups do this part of the story differently. In some cases router may speak PPPoA directly or modem may unpack the PPPoE and repack it as PPPoA. In some cases, Ethernet may be used in provider's core network instead of ATM and then there won't be PPPoA at all.
The router then keeps broadcasting PPP over Ethernet Active Discovery Initiation (PADI) packets and waits for something to respond to it.
The packets go through the core network of my telecom and reach a broadband remote access server. In my case, there are several of them. Each of them sends a reply to the router's WAN side MAC address. The type of reply packet is called PPP over Ethernet Active Discovery Offer (PADO) and contains information such as name of the device that sent the reply, service names, unique identification and so on.
After my router receives first of the PADOs, it sends PPP Active Discovery Request.
It then finishes the PPP over Ethernet Active Discovery phase, initiates a session and starts communication using PPP Link Control Protocol. I think that session number is used to maintain connection with BRAS.
Router sends a PPP LCP Configuration Request packet.
BRAS should first send an LCP Configuration Acknowledge packet and then respond with configuration request type of packet and indicates what types of authentication protocols it supports. In my case that's only Password Authentication Protocol (PAP).
Router receives the configuration request from BRAS and sends out acknowledge.
After that, using PAP in an Authenticate Request router sends out username and password (and since PAP is used, it's in plain-text, open for everyone to see).
BRAS then responds with PAP Authenticate Acknowledge packet indicating successful authentication. The authentication itself uses usually another specialized authentication, authorization and accounting server that's connected to BRAS.
After that, they switch to Internet Protocol Control Protocol. In networks such as this, it's used instead of DHCP.
Router sends IPCP Configuration Request and indicates which parameters it would like to get, in my case IP address and primary and secondary DNS servers.
After that, it sends out IPv6CP Configuration request together with a device identifier.
BRAS then sends out IPCP Configuration Acknowledge packet with IPv4 address.
Next, BRAS sends out IPCP Protocol Reject package and rejects PPP IPv6CP Request.
After that, my router sends out a IPCP Configuration request again and receives IPCP Configuration ACknowledge with IPv4 address and DNS server addresses.
After all that, BRAS sends out PPPoE Active Discovery Session-confirmation packet and from that point on, I have Internet connectivity. Communication then goes onfrom me through my telecom's network into my ISP's network and then onto Internet.
One more thing I should mention is that once every second after LCP starts, my router sends out LCP Echo Request and gets LCP Echo Reply. Using this, it can confirm that it does have Internet connectivity.
Finally in order to terminate session, PPP Active Discovery Termination packet can be sent, or a device can just skip enough Echos to have the connection die to to lack of keepalive messages.
In my case, PADT isn't used. Instead router sends out LCP Termination Request and after it receives LCP Termination Acknowledge, it shuts down the WAN interface.