What is the right course of action?

You are correct in choosing to move first. In this situation, the yellow car is positioned at a minor road with a yield sign, indicating it must give way to traffic on the main road. You are on the main road, so you have the right of way. The yellow car must wait for you to pass before it can enter or cross the main road.

German traffic rules clearly state that vehicles on the main road have priority over vehicles coming from secondary or yield roads. Therefore, you do not need to allow the yellow car to pass first. Your chosen action to move first aligns perfectly with these right-of-way rules.