Milne is on an indefinite club suspension - which could last three to four weeks - after Victoria Police charged him with four counts of rape last Tuesday.
The AFL backed the Saints' announcement the following day that they were standing down Milne, but the AFL Players' Association strongly opposed the decision.
Milne's teammates were also unhappy, prompting a meeting the day after the announcement between club board members and senior players.
Grant Thomas, who was St Kilda coach when Milne's alleged incident happened in 2004, told The Age that the AFL needed a clear policy on how the game handles players facing police charges.
But AFL deputy CEO Gillon McLachlan said that would not work.
"I have to say the Stephen Milne one was very difficult," he told Channel Seven's Game Day.
"There's such a huge amount of things to consider in that and I do think you have to consider every case on (its) merits.
"I know people would like black-and-white policies on this sort of stuff, I just don't think that's possible.
"Every case is different and this one has a number of difficulties with it."
McLachlan said while the circumstances at the club surrounding the suspension might have been awkward, the Saints had made the right call.
"The outcome they came to was really sensible, considered," he said.
"I know there might be some processes, some communication issues around that, but I do think the decision the board was made was well-considered and a good one."
McLachlan would not comment on how long Milne's suspension should last.
"That will be a decision for the footy club," he said.
The suspension meant Milne was a spectator at the MCG on Saturday as teammates Nick Riewoldt and Nick Dal Santo marked their 250-game milestones with a win over Melbourne.
