The problem with Outlook is that attachments in previous emails are only accessible through an individual email and not seen in a thread. I managed to solve the problem by changing the Conversation Settings from the View tab to group messages from a thread in my feed and be able to view each of them individually (thus getting access to attachments).
The problem is that the emails are only grouped if they were sent or copied to me. But if I want to access attachments from an email that was sent before I was added to the conversation I'm out of luck because such emails are not grouped and I can't view them individually, thus the problem with accessing attachments from the previous emails stays relevant in such case.
How do I solve it?