Your LAN will probably work just fine.
Depending on the uplink speed from the modem to the ISP it is possible but rather unlikely that the use of a 100Mbit/s router will slow down your LAN. The explanation is going to get rather technical.
First of all, I will assume what you call gigabit hubs is in fact switches. Back in the days when most networks were running 100Mbit/s hubs became less common and got replaced by switches, which will give you a better network performance than switches. So it is quite unlikely that you are using hubs. I don't know if gigabit hubs are even being manufactured.
The downstream from the internet cannot slow down your LAN, if everything on your LAN except from the router is running at gigabit speeds. You can have at most 100Mbit/s coming from the router onto the LAN, and any other link those packets would travel through on the LAN will have plenty of capacity to handle the packets from the internet.
The switch receiving packets from the router will have to buffer the entire packet from the router before it can start sending it to the next device. This won't slow down your LAN. It may increase the roundtrip time to the internet, and with this happening in both directions it may cost you as much as 0.3ms on the roundtrip time to the internet. If you try to measure this slowdown, it will probably get lost in the variations in roundtrip times introduced by lots of other factors.
The point where things get interesting and potentially problematic is on the upstream. We'll consider the scenario where a computer on your LAN is sending data to a server on the internet as fast as your setup can keep up with.
Less than 100Mbit/s upstream
If the upstream speed from your modem to your ISP is less than 100Mbit/s. The roundtrip time on your internet connectivity will increase.
As TCP on the sending computer starts, it will slowly increase the sending speed until it reaches the limit. At some point it will be sending data faster than the upstream from the modem to the ISP permits. At this point the modem will start buffering packets. For instance packets may enter this buffer at 100Mbit/s from the modem but only leave towards the ISP at 50Mbit/s. As the buffer usage grows, the internet roundtrip time increases. And since these buffers in modems tend to be too large, the roundtrip time can increase significantly. But even a too large buffer will eventually fill up, and the modem will have to drop packets.
TCP on the sending computer will notice the increased roundtrip time or lost packets and adjust its sending speed accordingly. Thus it will stabilize at a sending speed lower than the 100Mbit/s which can be send from the switch to the router. Thus your LAN will not be affected, but internet connectivity will feel very slow for any interactive usage.
More than 100Mbit/s upstream
If the speed of the upstream from the modem to the ISP is increased to more than 100Mbit/s, the bottleneck is going to move. And the problem will look quite differently.
The sending computer will not see any increase in latency or packet drops until the speed goes above 100Mbit/s. Once TCP crosses that speed threshold, the switch on the left of your diagram will be receiving packets faster than 100Mbit/s. But the switch knows that it cannot send data faster than 100Mbit/s to the router, and rather than dropping packets it will signal the sender to slow down.
This means if a computer connected to the internet and to another computer on the LAN, things will behave weird. Before when the upstream to the ISP was only 50Mbit/s, it would send 50Mbit/s to the server on the internet and the other 950Mbit/s could be used to send to another computer on the LAN.
But as the upstream to the ISP is increased to more than 100Mbit/s, this computer would send 100Mbit/s to a computer on the internet and then the switch would tell the network interface to slow down. This would likely cause the computer to only send 100Mbit/s to a server on the internet and another 100Mbit/s to the other computer on the LAN. So in this scenario, increasing the internet connection speed caused the effective LAN speed to drop from 1Gbit/s to 200Mbit/s.
If the computer sending to the internet was connected to the rightmost switch, this problem would not only affect the sending computer, but also the connectivity between the two switches.
Which of the two problems you could experience depends on whether the upstream capacity from modem to ISP is more or less than 100Mbit/s. But there is a single solution, which can solve both problems.
If your router can do bandwidth management, you can configure it to limit the maximum traffic it will send to the modem. Configure this such that it will a few percent be less than the capacity of the bottleneck (which is the smaller number of the upstream from modem to ISP and the 100Mbit/s through the router).
When the router reaches this configured capacity, it must either use ECN or drop packets to signal that capacity has been reached. This will cause TCP on the sending computer to adjust to your upstream capacity before introducing any of the problems explained in this answer.
If the server receiving the traffic being send from your LAN is behaving maliciously, it can get even worse. By sending crafted acknowledgements, the receiving server could trick the sending computer on your LAN into believing there is more upstream capacity than is really the case.
Under those circumstances it is possible for a computer on your LAN to keep increasing its sending speed past the point of any bottleneck. This obviously doesn't remove the bottleneck but may create multiple bottlenecks where there should be only one.
If your internet upstream capacity was 50Mbit/s, the malicious receiver could still cause a sender on your LAN to send with the full 100Mbit/s. This will quickly fill up the buffers on the modem, and cause switches on your LAN to push back on sending speeds.
Using bandwidth management on the router would only protect the the modem buffers from filling up. It would not protect your LAN. This is because though the data send from the router to the malicious server will clearly indicate a bottleneck, the malicious server can still send acknowledgements back which give the impression that there is no bottleneck.
In this case naively implemented bandwidth management on the router could backfire. Because it eliminates the latency increase while still allowing the malicious server to hide any packet loss from the sender. That way your LAN would suffer even faster.
The good news is that whatever upload you were performing to such a malicious server will finish much quicker than you expected. And you will probably just stop using such malicious servers. And the only harm done was a temporary slowdown of your network.