Only 12,619 people turned up for the Hawks' clash with Fremantle at Aurora Stadium last Saturday, which was Hawthorn's lowest crowd at the venue since it started playing there in 2001.
"The people from Hobart are not coming up," Van Zetten, who has been Launceston's mayor since 2007, told AFL.com.au.
"Whether that's 10 per cent (of the previous crowds at Aurora Stadium) or whatever, it's still a number of people and it adds up.
"Local football is another big factor. That's why I'd prefer Sunday games, so there's no local footy on.
"Or I'd love to see the local footy played on the Sunday if we do have an AFL game on a Saturday.
"Add those Hobart people and the people at local football back in, and the crowd is probably back to the average of around 16,000, which would be much better for us."
Many northern Tasmanians harbour resentment that the state government turned its back on its original pledge to make Launceston the exclusive home of AFL matches in Tassie, with Hobart compensated by having exclusive rights to international cricket.
"There is some disappointment there, because having six games (of AFL football) played in the state rather than four was always going to make a difference," Van Zetten said.
"When money's tight, people are going to choose what they spend their money on. AFL football is one of those things, and you can only go to so many games - especially people in southern Tasmania, who are now going to the games close by."
The poor attendance at Aurora Stadium last weekend highlighted a worrying trend.
Hawthorn's average home and away crowds in Launceston have declined from more than 17,500 in 2008 to 15,687 in 2012 (and last year's average was bolstered by a bumper attendance of 19,217 when Hawthorn played the Sydney Swans).
The Hawks' second-lowest crowd at the venue came last season when just 12,959 people turned up to see them play Fremantle.
"We're getting the same teams coming quite a lot, so the novelty factor has certainly worn off," Van Zetten said.
The clash between the Kangaroos and the Sydney Swans drew a sell-out crowd of 13,221 to Hobart's Blundstone Arena in round three.
Although the crowd that attended the Hawthorn-Fremantle game a week later in Launceston a week later was not much lower, it was well down on Aurora Stadium's capacity of more than 20,000.
"Fremantle just hasn't got the pulling power to bring people in," Van Zetten said.
"The neutral supporters don't want to go and watch Freo again because we've seen them last year and the year before.
"We know why we get them; they don't pull a crowd in Melbourne either."
Crowds are unlikely to improve at Aurora Stadium during the rest of the season, as the Hawks' remaining three games are against Greater Western Sydney, the Western Bulldogs and the Brisbane Lions.
"If we get an Essendon or Collingwood or Carlton we'll be right," Van Zetten said.
"That won't happen, but it would be lovely."
Follow the reporter on Twitter at @AFL_AdamMcNicol