In general, the first letter of a word beginning with בג"ד כפ"ת will usually have a דגש. However, if the word is connected to the word preceding it and that word ends with a vowel sound, the דגש is removed. The letters that are capable of producing a vowel sound are א, ה, ו, י. I've always found AHOY to be the easiest mnemonic for this rule. There are two exceptions to this rule in the תורה and both are found in this week's פרשה. In fact, they are both in אז ישיר.
The first is טו:יא מִי-כָמכָה בָּאֵלִם ה', מִי כָּמכָה נֶאְדָּר בַּקּדֶשׁ. The first מִי-כָמכָה follows the rule. The second does not. מנחת שי cites two reasons for this. First, he says that since the name מיכה is the name of an עבודה זרה, it is not proper for the name to "appear" to be mentioned next to השם's name which it would if you were not to pause sufficiently between מי and כמכה. The second reason is am allegorical explanation in the name of ר' יוסף קרא. [Perhaps there is a difference between the two as to whether or not one should bother to go back if they slip up. According to the first approach, once you've said it, it's pretty much too late. But perhaps according to the second approach you do accomplish something by going back. Nevertheless, this should NOT be corrected in shul.]
The second example is טו:טז בִּגְדל זְרוֹעֲךָ יִדְּמוּ כָּאָבֶן. It should be כָאָבֶן. Once again, according to מנחת שי, we are afraid of an improper stringing of words. Without the דגש, it might sound like יִדְּמוּךָ אָבֶן, which would essentially be stating that a rock is comparable to חס ושלום ,השם.
The Mesorah Gedolah in Daniel perek 5 lists all the exceptions to the "AHOY" rule. In "Az Yashir" you have the two you mention plus 3 more: "Ga'oh Ga'ah" which appears twice (once in the Shira and once by Miriam) and also "Am zu Ga'alta" where the Gimmel has a dagesh even though it shouldn't. The Minchas Shai explains this last one as needing a dagesh because without a dagesh one might confuse the shoresh as being from the word "M'Go'al" which is derogatory.