I use conditional formatting for this purpose. This answer can not possibly give you the step by step for your scenario, but it will get you started.
You will need to modify it according to how your data is laid out, but it should work. There is a tutorial at the source below to help you learn how to do this.
In essence, you are going to use conditional formatting with a formula.
Create a rule and choose Use a formula to determine which cells to format
Below is what it looks like for one that is within 7 days of being overdue.
The formula is cut off in the image. It is =IF(ISBLANK($J2), AND(MEDIAN(TODAY()+1,$E2,TODAY()+7)=$E2),"")
Below is for those that are overdue.
I end up with two rules applied to the same range.
Again, you will have to make this work for your data, but it will work with three conditions. Follow the examples in the source linked below to learn how to do this. It's a great tool I use daily.
Highlight due dates in Excel – Show items due, overdue and completed in different colors