When asked by someone on Twitter if he smoked cannabis, councillor Ben Duncan, the party's Home Affairs spokesman, replied:
"I only smoke weed when I'm murdering, raping and looting! Is the fact that you have a wife admission of your humanity?"
The local Labour Party have called for Duncan to resign, branding his comments "best irresponsible, and at worst highly offensive to anyone who has suffered from rape".
Councillor Geoffrey Theobald, Leader of the Conservative group on Brighton and Hove City Council said:
"The latest comments by Cllr. Duncan (who let’s not forget is the Council’s sole representative on the Sussex Police Authority and spokesman on community safety) are grossly insensitive, but sadly don’t come as a great surprise"
Conservative councillor Dawn Barnett told The Huffington Post UK that Duncan's remarks were "highly insensitive to people who have been attacked or raped".
"I thoroughly condemn his comments on Twitter", she said adding: "He is a complete lunatic".
Mike Weatherley, Tory MP for Hove and Portslade, said Duncan was "already known for his crass jokes".
"There are some things that you just don’t laugh about and rape is obviously one of them. Comments like this must put his Green Party colleagues in a very tricky position on what action to now take.
"As for the subject of cannabis cafes, I despair as to what madcap experiments the Greens wish to test on the people of Brighton And Hove next. You couldn’t make some of this stuff up.”
Duncan's comments came after he suggested that Brighton could licence cannabis cafes in order to capitalise on tourists from Amsterdam.
A Dutch judge has supported a proposed ban on foreigners smoking cannabis in Amsterdam. The ban will take effect in Holland by the end of the year, with many warning that the number of tourists will fall.
“Think of all the millions our shops and hotels would make if all those tourists being turned away from Amsterdam by the Dutch Tories came here to spend their holiday cash instead!"
He urged people to consider the idea “for the sake of our tourist industry and the health and wellbeing of those living in and visiting our city”.
"Of course, in Brighton and Hove, we know only too well the damage that the current drug policy of complete criminalisation of drug use is causing rather than seeing it as a health issue."
“Recently Caroline Lucas MP and the city’s top cop, Graham Bartlett called for decriminalisation of drug use and for good reasons.
"Cannabis use can be harmful, but all analysis shows that it's much less likely to harm you than, say, driving a car, or crossing a road.
Councillor Duncan asked whether Brighton could be "the liberal, tolerant, tourist capital of Europe?”
Councillor Duncan's call for legalising cannabis cafes comes as the Home Affairs Committee pursues its inquiry into drug use. The actor Russell Brand recently appeared before MPs and said drug addicts needed "love and compassion".
London Lib Dem mayoral candidate Brain Paddick has said he would decriminalise cannabis in the capital. He argued that prosecuting people for cannabis prosecution was a waste of police time and money, when they should be focusing on "more serious crimes".
HuffPost UK contacted the councillor's office for his response to calls for his resignation and he was not available for comment.