What is the right course of action?

You chose correctly that you may drive through first.

The key rule here is the priority road sign (the yellow diamond) which grants you the right of way. This means you do not have to yield to other vehicles turning onto the priority road from side streets. The shape sign below it indicates a junction where the priority continues straight or to the left, so vehicles coming from the right (like the blue car) must yield to you. The green car on the left is also not on the priority road and must yield.

In short, when you see a priority road sign, you have right of way over cars coming from side streets until the priority road ends. This allows you to proceed without stopping or yielding in this scenario.