Milne has accepted a one-match suspension for unnecessary contact to Essendon defender Courtenay Dempsey's face during the fourth quarter of Saturday night's 37-point loss at Etihad Stadium.
As the rest of the Saints travelled to New Zealand on Tuesday morning ahead of the Anzac Day clash at Westpac Stadium, Milne remained behind with a week of training ahead of him.
Watters said it was "disappointing" as the Saints needed their leaders on the field with potentially at least one debutant to face to the Swans.
"If you want my blunt assessment, it was just stupid. I don't think there was any major malice in it," Watters said at Melbourne Airport.
"It was certainly on a low level but it was just something that didn't need to happen.
"It was not the sort of act that I expect from our leaders, Milney knows that, he's disappointed that he can't be there where he needs to be, which is actually playing alongside a number of our younger players."
Watters said he hoped Milne would "reflect" this week and understand the opportunity he was missing, with the Saints and Swans to play the first AFLgame for premiership points outside of Australia.
He said there was no guarantee the veteran would be called up to face Collingwood in round six on May 3.
"He'll train and prepare himself to be up for selection the following week, and there's no guarantees with that," he said.
"Once you come out of the side, you have to fight your way back in and he's no different."
The Saints travelled with a squad of 26, with injured defender James Gwilt (knee) among them.
While he is not expected to play, Sean Dempster (hand) remains a chance to return while Justin Koschitzke and Tom Lee made the trip as potential replacements for Beau Maister (wrist).
Watters said there would likely be at least one fresh face from the group of Brodie Murdoch, Josh Saunders and Sam Dunell.
"I think sometimes when you do debut young players, the type of game they come into can define their career for a long way down the track," he said.
"It wouldn't surprise if the final selection you do see a new Saints player play on Anzac Day in New Zealand.
"It might be a little bit of old and a little bit new. We'll wait and see."
The team will be finalised after the Saints touch down in Wellington on Tuesday afternoon.
Meanwhile, Milne has also accepted a $3000 third offence melee sanction.
The other Saints involved in the half time scuffle that broke out in Saturday evening's game – Nathan Wright, Tom Simpkin, Ahmed Saad and Jarryn Geary – have accepted $1200 fines for first offences.
Stephen Milne is a forward in NAB AFL Fantasy. He averaged 68.45 points in 2012. Register your team at our AFL Fantasy Hub.
Jennifer Phelan is a reporter for AFL Media. Follow her on Twitter @AFL_JenPhelan.
