Secure Boot is not about disallowing a physical attacker access to the system. It’s designed to prevent software from manipulating the boot process by checking the cryptographic integrity of the bootloader. Think Blue Pill.
Boot steps are responsible for verifying the next step code. Secure Boot itself is only concerned with the very first step.
A software attacker does not need to access UEFI setup or change anything about the boot configuration. They would just replace the UEFI executable that’s set to run. A password lock on UEFI setup does nothing about that.
Secure Boot on the other hand verifies the UEFI executable using cryptographic methods. It would detect this change and refuse to boot.
To ensure trusted boot and operating, all kernel-mode code must be signed. This is the case on 64-bit versions of Windows which also enforce signatures on device drivers.
Operating systems or the software they run are not secure. In most cases, it is impossible to achieve formally provable security. As such, attackers will always be able to achieve enough access to perform privileged operations such as replacing the bootloader.