Penny lasted only a few warmup pitches in the top of the fourth inning before leaving.
Penny and Joel Pineiro, the pitcher he replaced in the Cardinals' rotation, were both gone after three innings.
Pineiro (3-5) entered with 17 1-3 consecutive scoreless innings but was clubbed for nine runs on nine hits against the team that allowed him to leave as a free agent after a 15-win season. Penny was shaky for the second straight start while giving up four runs on five hits.
Jason Motte (2-1), the third St. Louis pitcher, struck out two in two perfect innings.
Felipe Lopez followed Penny's third career homer, and first since 2003, with a solo shot that barely cleared the wall in right and was the first confirmed by video replay at Busch Stadium.
Lopez also had an RBI triple and Colby Rasmus had a two-run triple for the Cardinals, who are 4-1 since Albert Pujols and Matt Holliday switched spots in the order, although the pair was a combined 0 for 4 with an intentional walk the first three innings while St. Louis was building a 9-4 lead.
Tied at 4 after two, the teams combined for 10 hits, a wild pitch, passed ball and hit batter before the first strikeout: Juan Rivera to end the top of the third.
Mike Napoli had a two-run homer and Torii Hunter had two hits for the Angels, whose starters have a 1.70 ERA in 20 wins and 7.65 ERA in 24 losses.
Skip Schumaker drew an intentional walk to load the bases with two outs in the third before Penny, who had been 2 for 18 with a pair of singles, hit the first pitch from Pineiro well over the left-field wall for an 8-4 lead.