0

I'm having trouble getting my two interface Ubuntu Server (18.04) to:

  1. have a working pppoe connection to the internet, and;
  2. share that connection with the rest of my LAN.

I mention 2. because in the past I managed to get the WAN connection working on the gateway (no idea how), but then the hard drive failed (it was new!!). After the rebuild I can't even get the gateway to have a working net connection. Grrrr

Currently I can conect to my ISP via pon, but then can't do anything with the connection, not even ping 8.8.8.8. I also tried pinging my ISP nameserver with no sucess.

My netowrk architecture is: LAN <--> Gateway interface enp3s0 <--> Gateway (shorewall/routing) <--> Gateway interface ppp0 (tunnelled via ens33) <--> External Modem <--> Internet

I've included the following file contents to (hopefully) diagnose what I'm missing:

  • Netplan config
  • Result of 'ifconfig'
  • pppoe connection log from syslog (for confirmation the connection has been made)
  • The result of 'route'
  • Contents of resolve.conf
  • Shorewall files: shorewall.comf, zones, policy, interfaces, snat and rules

Any advise/solutions are very much welcome. Thanks and merry christmas!

Netplan:

root@gateway:/etc/shorewall# cat /etc/netplan/01-netcfg.yaml
network:
  version: 2
  renderer: networkd
  ethernets:
    enp3s0:
      dhcp4: no
      addresses:
        - 192.168.54.141/24
      gateway4: 192.168.54.141
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]
    ens33:
      dhcp4: yes

ifconfig:

root@gateway:/etc/shorewall# ifconfig
enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.54.141  netmask 255.255.255.0  broadcast 192.168.54.255
        inet6 fe80::226:2dff:fe24:9143  prefixlen 64  scopeid 0x20<link>
        ether 00:26:2d:24:91:43  txqueuelen 1000  (Ethernet)
        RX packets 6428904  bytes 4050106497 (4.0 GB)
        RX errors 0  dropped 72  overruns 0  frame 0
        TX packets 6303032  bytes 4310814732 (4.3 GB)
        TX errors 4  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 18

ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.10  netmask 255.255.255.0  broadcast 192.168.2.255
        inet6 fe80::213:3bff:fe4a:2934  prefixlen 64  scopeid 0x20<link>
        ether 00:13:3b:4a:29:34  txqueuelen 1000  (Ethernet)
        RX packets 131934  bytes 21509371 (21.5 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 47321  bytes 8213015 (8.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 2219702  bytes 180229868 (180.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2219702  bytes 180229868 (180.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ppp0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1492
        inet 86.146.246.68  netmask 255.255.255.255  destination 172.16.16.5
        ppp  txqueuelen 3  (Point-to-Point Protocol)
        RX packets 62  bytes 2737 (2.7 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3  bytes 54 (54.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

SYSLOG output for pppoe connection:

Dec 25 18:07:16 gateway pppd[8011]: pppd 2.4.7 started by scootyPuff uid 0
Dec 25 18:07:16 gateway pppd[8011]: PPP session is 4206
Dec 25 18:07:16 gateway pppd[8011]: Connected to 24:af:4a:c0:fe:62 via interface ens33
Dec 25 18:07:16 gateway pppd[8011]: Using interface ppp0
Dec 25 18:07:16 gateway pppd[8011]: Connect: ppp0 <--> ens33
Dec 25 18:07:16 gateway systemd-udevd[8012]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Dec 25 18:07:16 gateway networkd-dispatcher[1194]: WARNING:Unknown index 12 seen, reloading interface list
Dec 25 18:07:16 gateway pppd[8011]: CHAP authentication succeeded: CHAP authentication success
Dec 25 18:07:16 gateway pppd[8011]: CHAP authentication succeeded
Dec 25 18:07:16 gateway pppd[8011]: peer from calling number 24:AF:4A:C0:FE:62 authorized
Dec 25 18:07:16 gateway systemd-networkd[29646]: ppp0: Gained carrier
Dec 25 18:07:16 gateway pppd[8011]: not replacing default route to enp3s0 [192.168.54.141]
Dec 25 18:07:16 gateway pppd[8011]: local  IP address 86.146.246.68
Dec 25 18:07:16 gateway pppd[8011]: remote IP address 172.16.16.5
Dec 25 18:07:16 gateway pppd[8011]: primary   DNS address 81.139.57.100
Dec 25 18:07:16 gateway pppd[8011]: secondary DNS address 81.139.56.100

Route:

root@gateway:/etc/shorewall# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.54.141  0.0.0.0         UG    0      0        0 enp3s0
default         192.168.2.1     0.0.0.0         UG    100    0        0 ens33
172.16.16.5     0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 ens33
192.168.2.1     0.0.0.0         255.255.255.255 UH    100    0        0 ens33
192.168.54.0    0.0.0.0         255.255.255.0   U     0      0        0 enp3s0

resolv.conf:

root@gateway:/etc/shorewall# cat /etc/resolv.conf
nameserver 81.139.57.100
nameserver 81.139.56.100
options edns0

SHOREWALL: Version: 5.1.12.2

shorewall.conf:

root@gateway:/etc/shorewall# cat shorewall.conf
###############################################################################
#
#  Shorewall Version 5 -- /etc/shorewall/shorewall.conf
#
#  For information about the settings in this file, type "man shorewall.conf"
#
#  Manpage also online at http://www.shorewall.net/manpages/shorewall.conf.html
###############################################################################
#                      S T A R T U P   E N A B L E D
###############################################################################

STARTUP_ENABLED=Yes

###############################################################################
#                            V E R B O S I T Y
###############################################################################

VERBOSITY=1

###############################################################################
#                               P A G E R
###############################################################################

PAGER=

###############################################################################
#                            F I R E W A L L
###############################################################################

FIREWALL=

###############################################################################
#                              L O G G I N G
###############################################################################

LOG_LEVEL="info"
BLACKLIST_LOG_LEVEL=
INVALID_LOG_LEVEL=
LOG_BACKEND=
LOG_MARTIANS=Yes
LOG_VERBOSITY=2
LOGALLNEW=
LOGFILE=/var/log/messages
LOGFORMAT="%s %s "
LOGTAGONLY=No
LOGLIMIT="s:1/sec:10"
MACLIST_LOG_LEVEL="$LOG_LEVEL"
RELATED_LOG_LEVEL=
RPFILTER_LOG_LEVEL="$LOG_LEVEL"
SFILTER_LOG_LEVEL="$LOG_LEVEL"
SMURF_LOG_LEVEL="$LOG_LEVEL"
STARTUP_LOG=/var/log/shorewall-init.log
TCP_FLAGS_LOG_LEVEL="$LOG_LEVEL"
UNTRACKED_LOG_LEVEL=

###############################################################################
#       L O C A T I O N   O F   F I L E S   A N D   D I R E C T O R I E S
###############################################################################

ARPTABLES=
CONFIG_PATH=":${CONFDIR}/shorewall:${SHAREDIR}/shorewall"
GEOIPDIR=/usr/share/xt_geoip/LE
IPTABLES=
IP=
IPSET=
LOCKFILE=
MODULESDIR=
NFACCT=
PATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin"
PERL=/usr/bin/perl
RESTOREFILE=restore
SHOREWALL_SHELL=/bin/sh
SUBSYSLOCK=""
TC=

###############################################################################
#               D E F A U L T   A C T I O N S / M A C R O S
###############################################################################

ACCEPT_DEFAULT="none"
BLACKLIST_DEFAULT="Broadcast(DROP),Multicast(DROP),dropNotSyn:$LOG_LEVEL,dropInvalid:$LOG_LEVEL,DropDNSrep:$LOG_LEVEL"
DROP_DEFAULT="Broadcast(DROP),Multicast(DROP)"
NFQUEUE_DEFAULT="none"
QUEUE_DEFAULT="none"
REJECT_DEFAULT="Broadcast(DROP),Multicast(DROP)"

###############################################################################
#                        R S H / R C P  C O M M A N D S
###############################################################################

RCP_COMMAND='scp ${files} ${root}@${system}:${destination}'
RSH_COMMAND='ssh ${root}@${system} ${command}'

###############################################################################
#                       F I R E W A L L   O P T I O N S
###############################################################################

ACCOUNTING=Yes
ACCOUNTING_TABLE=filter
ADD_IP_ALIASES=No
ADD_SNAT_ALIASES=No
ADMINISABSENTMINDED=Yes
AUTOCOMMENT=Yes
AUTOHELPERS=Yes
AUTOMAKE=Yes
BALANCE_PROVIDERS=No
BASIC_FILTERS=No
BLACKLIST="NEW,INVALID,UNTRACKED"
CLAMPMSS=Yes
CLEAR_TC=Yes
COMPLETE=No
DEFER_DNS_RESOLUTION=Yes
DELETE_THEN_ADD=Yes
DETECT_DNAT_IPADDRS=No
DISABLE_IPV6=No
DOCKER=No
DONT_LOAD=
DYNAMIC_BLACKLIST=Yes
EXPAND_POLICIES=Yes
EXPORTMODULES=Yes
FASTACCEPT=No
FORWARD_CLEAR_MARK=
HELPERS=
IGNOREUNKNOWNVARIABLES=No
IMPLICIT_CONTINUE=No
INLINE_MATCHES=No
IPSET_WARNINGS=Yes
IP_FORWARDING=On
KEEP_RT_TABLES=No
LOAD_HELPERS_ONLY=Yes
MACLIST_TABLE=filter
MACLIST_TTL=
MANGLE_ENABLED=Yes
MAPOLDACTIONS=No
MARK_IN_FORWARD_CHAIN=No
MINIUPNPD=No
MULTICAST=No
MUTEX_TIMEOUT=60
NULL_ROUTE_RFC1918=No
OPTIMIZE=All
OPTIMIZE_ACCOUNTING=No
PERL_HASH_SEED=0
REJECT_ACTION=
REQUIRE_INTERFACE=No
RESTART=restart
RESTORE_DEFAULT_ROUTE=Yes
RESTORE_ROUTEMARKS=Yes
RETAIN_ALIASES=No
ROUTE_FILTER=Yes
SAVE_ARPTABLES=No
SAVE_IPSETS=No
TC_ENABLED=Internal
TC_EXPERT=No
TC_PRIOMAP="2 3 3 3 2 3 1 1 2 2 2 2 2 2 2 2"
TRACK_PROVIDERS=Yes
TRACK_RULES=No
USE_DEFAULT_RT=Yes
USE_NFLOG_SIZE=No
USE_PHYSICAL_NAMES=No
USE_RT_NAMES=No
VERBOSE_MESSAGES=Yes
WARNOLDCAPVERSION=Yes
WORKAROUNDS=No
ZERO_MARKS=No
ZONE2ZONE=-

###############################################################################
#                       P A C K E T   D I S P O S I T I O N
###############################################################################

BLACKLIST_DISPOSITION=DROP
INVALID_DISPOSITION=CONTINUE
MACLIST_DISPOSITION=REJECT
RELATED_DISPOSITION=ACCEPT
RPFILTER_DISPOSITION=DROP
SMURF_DISPOSITION=DROP
SFILTER_DISPOSITION=DROP
TCP_FLAGS_DISPOSITION=DROP
UNTRACKED_DISPOSITION=CONTINUE

################################################################################
#                       P A C K E T  M A R K  L A Y O U T
################################################################################

TC_BITS=
PROVIDER_BITS=
PROVIDER_OFFSET=
MASK_BITS=
ZONE_BITS=0
root@gatewst:/etc/shorewall#

rules:

root@gateway:/etc/shorewall# cat rules
# Shorewall - Sample Rules File for two-interface configuration.
# For information about entries in this file, type "man shorewall-rules"
######################################################################################################################################################################################################
#ACTION         SOURCE          DEST            PROTO   DEST    SOURCE          ORIGINAL        RATE            USER/   MARK    CONNLIMIT       TIME            HEADERS         SWITCH          HELPER
#                                                       PORT    PORT(S)         DEST            LIMIT           GROUP
?SECTION ALL
?SECTION ESTABLISHED
?SECTION RELATED
?SECTION INVALID
?SECTION UNTRACKED
?SECTION NEW

#       Don't allow connection pickup from the net
#
Invalid(DROP)   net             all             tcp

# Accept DNS connections from the firewall to the network
# Following line commented out as it's covered by the policy allowing all $FW connection to net zone
# DNS(ACCEPT)   $FW             net
 DNS(ACCEPT)    loc             $FW
 DNS(ACCEPT)    loc             net

# Accept NetBOIS requests from LAN
SMB(ACCEPT)     $FW             loc
SMB(ACCEPT)     loc             $FW

# Accept SSH connections from the local network for administration
SSH(ACCEPT)     loc             $FW

# Accept HTTP connections
HTTP(ACCEPT)    loc             net
HTTPS(ACCEPT)   loc             net

# Allow Ping from the local network
Ping(ACCEPT)    loc             $FW
Ping(ACCEPT)    $FW             net
Ping(ACCEPT)    loc             net

# Drop Ping from the "bad" net zone.. and prevent your log from being flooded..
Ping(DROP)      net             $FW
Ping(DROP)      net             loc

ACCEPT          $FW             loc             icmp
ACCEPT          $FW             net             icmp

zones:

root@gateway:/etc/shorewall# cat zones
# For information about entries in this file, type "man shorewall-zones"
###############################################################################
#ZONE   TYPE    OPTIONS                 IN                      OUT
#                                       OPTIONS                 OPTIONS
fw      firewall
net     ipv4
loc     ipv4
#modem  ipv4

interfaces:

root@gateway:/etc/shorewall# cat interfaces
# For information about entries in this file, type "man shorewall-interfaces"
###############################################################################
?FORMAT 2
###############################################################################
#ZONE   INTERFACE       OPTIONS
net     $NET_IF          dhcp,tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,physical=ppp0
loc     $LOC_IF          dhcp,tcpflags,nosmurfs,routefilter,logmartians,physical=enp3s0
#modem   $MODEM_IF        dhcp,tcpflags,nosmurfs,routefilter,logmartians,physical=ens33

policy:

root@gateway:/etc/shorewall# cat policy
# For information about entries in this file, type "man shorewall-policy"
###############################################################################
#SOURCE DEST            POLICY          LOGLEVEL        RATE    CONNLIMIT

loc     all             ACCEPT
net     all             DROP            $LOG_LEVEL

# Allow firewall to talk to internet and LAN
$FW     net             ACCEPT
#loc    $FW             ACCEPT
$FW     loc             ACCEPT

# THE FOLOWING POLICY MUST BE LAST
all     all             REJECT          $LOG_LEVEL

SNAT:

root@gateway:/etc/shorewall# cat snat
# For information about entries in this file, type "man shorewall-snat"
#

# See http://shorewall.net/manpages/shorewall-snat.html for more information
###########################################################################################################################################
#ACTION                 SOURCE                  DEST            PROTO   PORT    IPSEC   MARK    USER    SWITCH  ORIGDEST        PROBABILITY
#
# Rules generated from masq file /home/teastep/shorewall/trunk/Shorewall/Samples/two-interfaces/masq by Shorewall 5.0.13-RC1 - Sat Oct 15 11:41:40 PDT 2016
#
MASQUERADE              10.0.0.0/8,\
                        192.168.54.0/24         $NET_IF
  • Hmm. Do you have packet forwarding enabled? Check /etc/sysctl.conf for net.ipv4.ip_forward=1 . I would also suggest taking a look at the ars technica router guide - my own build is a spin on that though I use firewalld and dnsmasq. Are you using static IPs on clients? – Journeyman Geek Dec 26 '19 at 0:36
  • Packet forwarding is enabled. Double checked /etc/sysctl.conf anyway. I'm meant to be behind a dynamic IP, but my ISP doesn't seem to have changed it in a while. – Ben Dec 26 '19 at 10:44
  • Oh client side. I see no mention of a DHCP server – Journeyman Geek Dec 26 '19 at 10:45
  • I am running my own DHCP server for my LAN. It's bound to the LAN interface port and works just fine, serving IPs from 192.168.54.150 up. IP below are reserved for things I want a static IP for (routers, servers, etc). My netplan file states no DHCP on the LAN interface (enp3s0) as I want an internal static IP for my server. Interface enp3s0 is dynamic and gets its IP from the external modem's DHCP server. The setup tutorials I used said this was the thing to do and allowed my ISP to grant my pppoe IP address. – Ben Dec 26 '19 at 10:52
  • Ah ok. Hmm.how does a traceroute out from your network to 8.8.8.8 look? – Journeyman Geek Dec 26 '19 at 10:55
0

Thank you for the advice everyone. After more 'hacking' and illimination I found the answer.

There were a twoissues:

  1. Despite defining the various options in Shorewall for start on boot, it wasn't. Turns out I needed to define a systemd service script for it. That answer came from here.
  2. I was getting in a real mess with the nameserver IP popping in and out of existance in /etc/resolv.conf AND the default routes. I don't fully understand what was going on, but its probable I was getting my knickers twisted with all my experimenting. I didn't investigate the truth behind that fully, but what I did do was to implement an rc.local script the removed all default routes, then re added the correct route. So now when I do a "ip route show" the correct route is first on the list.

With that all in place I can now reboot the server and have the gateway automatically back up with no intervention/modification. PHEW!

Thanks for you advice & patience!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.