"But I have my reasons. First of all, I used to spend time at weekends with my daughters, so I didn't go as often.
"It's quite a lot of money at Stamford Bridge, so it was a waste of my money.
"Secondly, I lost a bit of connection with the club. I've got to be honest with you because we spent a lot of money and we were sacking a lot of fans and it didn't fit my values as much.
"I was getting angry with them and I was thinking 'why am I going every week'.
"But I still like Chelsea and I still like watching them on TV."
Lovejoy also admitted that going to football was making him sad as he opened up on his struggles with depression.
Angry Chelsea fan slams Todd Boehly's transfer policy and claims no Blues supporters are happy with his running of the 'unrecognisable' club
He explained that a match at Anfield in which he was abused by Liverpool fans left him feeling "lonely".
He recalled: "The other thing is I had a bout of depression and it wasn't making me happy being there.
"It made me feel kind of sad for some reason. I remember going to Anfield with Simon Rimmer... He's a Liverpool fan and he took me up to Anfield.
"Loads of Liverpool fans spot me, they're all standing up and they're going 'Lovejoy is a rent boy '
"I just felt so lonely, depression hit me and going to football didn't do it for me anymore. However, watching it on TV really did.
"My idea of heaven is Super Sunday or Monday Night Football."
Lovejoy was the star of Soccer AM from 1996 until 2007 but left after a pay dispute.
In 2020, he told the Athletic that he believed that people who appeared on screen should be paid more.
He said: "I'd done 11 years, and I absolutely adored the show. But we were being treated like a production team of any other part of Sky,
"For a few years I'd been saying: if someone appears on screen can they get a few quid more?
"It started getting embarrassing in the end. Some of them were getting paid so little money, they were saying to me 'Tim, I've got to leave soon, because I've been doing eight years of this.'
"I went to see the bosses and said: 'Look, is there any way that we can pay the guys proper money?'
"They just said: 'No, they're researchers and assistant producers'. I really pleaded with them to pay the team what they deserved. But I understand why the bosses couldn't do it."