Boylan was promoted from his assistant's role Thursday, replacing Scott Skiles, who was fired on Christmas Eve after the Bulls' horrendous start. Chicago is now 10-17.
As his first major move, Boylan changed the starting lineup, replacing Gordon with Chris Duhon, hoping that Gordon
- former sixth man of the year - would provide instant scoring off the bench.
Gordon responded with a big final quarter when he went 10-of-11 from the line. Luol Deng added 28 for Chicago. Michael Redd paced the Bucks with 34.
Bucks coach Larry Krystkowiak was ejected with 1:18 left after picking up his second technical foul when he angrily came on the floor toward official Derrick Collins during a timeout.
The blowup came seconds after Deng's follow shot had put the Bulls up 95-91. Moments earlier, Chicago had been awarded possession after a loose ball scramble. A replay appeared to show the ball went off Chicago's Kirk Hinrich and Krystkowiak was livid.
Gordon made two free throws after the technicals on Krystkowiak to put Chicago up 97-91.
Redd made two of three free throws after he was fouled on a long attempt with 24 seconds left to make it 98-95. After two more free throws by Gordon, Redd again was fouled on a 3-point attempt and made two to cut it to 100-97 with 14 seconds to to play.
Milwaukee was only 7-for-15 from the line for the game and missed seven free throws in the final quarter.
Mo Williams had 15 assists and 11 points for the Bucks, while Andrew Bogut added 15 points and 11 rebounds. Deng shot 12-for-18 from the field and had 10 rebounds.