NEW YORK — The Orioles played with fire too many times Sunday and finally got burned in a double whammy of an eighth inning at Yankee Stadium that robbed them of a series win. Instead, the club will travel back to Baltimore wondering how this has happened so many times this season — early runs, no additions and a late collapse.

The top of the eighth inning ended with Cedric Mullins slamming his bat into the dirt, his frustration spilling outwardly after the Orioles struck out three times with a runner in scoring position. The bottom of the eighth inning ended with New York holding a lead in a game the Yankees had narrowly trailed since the beginning.

The 4-2 loss to end the road trip supplies a sour note to Baltimore’s upturn this month. What makes it more sour: This last loss was avoidable, had the Orioles capitalized in other situations.

Instead, right-hander Bryan Baker pitched for the fourth time in six days. He allowed a two-run double to Jazz Chisholm Jr., and Chisholm later scored on a fielder’s choice when catcher Gary Sánchez couldn’t complete the catch and tag.

“You get a couple guys on base right there, you’re hoping to kind of expand the lead,” interim manager Tony Mansolino said. “With how used the bullpen has been here over the last really two to three weeks, at some point there was gonna be a break, and unfortunately it was today.”

It appeared as though the Orioles might skate out of danger time and again. Right-hander Dean Kremer and left-hander Keegan Akin combined to escape the sixth inning unscathed. Right-hander Seranthony Domínguez walked a batter to bring Aaron Judge to the plate with two runners on in the seventh, but Domínguez buckled down for strike three.

But, after Baltimore wasted a chance in the eighth when left-hander Tim Hill walked Gunnar Henderson and Ryan O’Hearn with no outs, the dam broke. Baker allowed two singles, Chisholm doubled for a second time and the fourth run helped right-hander Devin Williams earn a save.

The three strikeouts in the eighth were notable, but they weren’t the only missed chances. Baltimore finished 2-for-11 with runners in scoring position and stranded nine.

“I felt like we had some opportunities to get there. Just didn’t come through,” Henderson said. “But ultimately I was happy with the way we played on the road trip.”

Baltimore has won 12 of its 20 games this month, and while that is an upgrade from earlier this season, the tides aren’t rising quickly enough for a club that seems stranded on a sandbar. The Orioles are 11 games below .500 (33-44) and are in last place in the American League East. Although a wild-card spot is possible, six teams are between them and the final playoff place.

“We played a lot better than 3-4,” Baker said. “I think we probably should have been 5-2, kind of let a few get away from us. We’ve got to clean a few things up and execute and put away games. 3-4, we’d like to be better than that, especially in-division, against a couple good teams.”

Kremer broke a streak of 10 straight appearances against the Yankees (45-32) in which he allowed a home run, and it came at the right time. Baltimore’s bullpen covered a heavy load through the first two games of the series due to short starts from Tomoyuki Sugano and Zach Eflin. But Kremer navigated what can be a challenging test for him (in his previous 12 starts against New York, Kremer held a 4.40 ERA).

This, then, became Kremer’s 13th start against the Yankees — the most against a single team in his career. He found particular success Sunday with his splitter, which forced six of his 11 whiffs, and he tightroped out of danger in the sixth inning with the help of Dylan Carlson and Akin.

Kremer walked Judge to begin the frame and then left a splitter over the heart of the plate that Ben Rice throttled for a 110.1 mph double to right.

The Orioles’ one-run lead could have eroded there. But Carlson chased down a flare in right field off the bat of Cody Bellinger for the first out, and he got the ball into the infield quickly enough to prevent it turning into a sacrifice fly. Then Kremer fired a four-seam fastball past Giancarlo Stanton for strike three.

Before Chisholm came to the plate, Mansolino turned the game over to Akin from the bullpen. He forced Chisholm into an inning-ending flyout.

The 5 2/3 innings from Kremer with one run against him marked his best start in June, continuing from his performance against the Tampa Bay Rays at the beginning of the road trip. Kremer’s June began in rocky fashion, with a combined nine runs against him in his first two outings. But he has produced consecutive one-run starts, and his ERA is down to 4.60 — its lowest of the season.

And Kremer’s start was just the second time in Baltimore’s last 11 games that a starter worked into the sixth inning.

“Command was good. Shapes were good,” Kremer said. “The guys have eaten quite a lot of innings over this road trip. Weather has been grueling. So, just trying to do my job and give us the best chance to win and take as much stress as I can off the bullpen.”

The lone run scored against Kremer in the second, when Chisholm doubled and DJ LeMahieu singled. It proved painful, because Chisholm collided with catcher Maverick Handley along the third-base line, which forced Handley to leave the game — just a day after catcher Adley Rutschman’s oblique strain resulted in an injured list placement.

Mansolino said Handley’s full body is being evaluated. He called the scene “scary” and something not frequently seen since rules were put in place to protect catchers from collisions in 2014.

Still, Kremer worked well with a lead he was immediately handed. The Orioles plated two runs against right-hander Will Warren in the first inning, courtesy of RBI hits from O’Hearn and Colton Cowser. They missed out on an opportunity to add on in the eighth, though, when the first two batters reached to begin the inning.

Right-hander Fernando Cruz entered from the bullpen to inherit those runners, and he struck out Ramón Laureano, Cowser and Mullins to end the threat. That left the Orioles on unsteady footing in the bottom of the frame, and Baker couldn’t continue a run of scoreless innings.

“I thought we did a good job hanging with them,” Henderson said. “I wish we could’ve got the one today, but yeah, kind of stalled out towards the end of the game and they did a good job of pushing some runs across.”

This article has been updated.