What is the right course of action here?

Your answer is correct because you and the yellow car both have the right of way to proceed ahead of the red car.

Here's why:

The yellow diamond traffic sign indicates you are on a priority road. The small sign below it shows that vehicles coming from the right must yield. This means both you (going straight or turning left) and the yellow car (coming from the priority road on the right) have priority over the red car, which must yield because it is coming from a secondary road.

Therefore, you may proceed ahead of the red car, and the yellow car may also proceed ahead of the red car. The red car must wait until both of you have passed.

Understanding priority roads and yield signs will help you know when you can proceed safely and who must wait at intersections.