Like some have said SeeBass, it's only natural that more people will turn up when the team is doing well and the entertainment level is higher. There are a lot of things out there for people to spend their hard earned cash on and if the "product" is crap then a lot of people are just not going to bother. In the same way as if a musician releases a rubbish album, people don't buy it and don't go to the concert. I see nothing fake in that, it is personal choice.
It's great that you continue to support the club through thick and thin though and good on you for that.