I was watching Girl in the Fireplace with my mother this evening and I had a thought about the time agency.
The ship SS Madame de Pompadour is a massively powerful 51st century vessel with the ability to open multiple holes in time and space. Of course with Jack Harkness being a 51st century time agent, it's not a great leap to say that the ship could be related to the same time agency. I’m not the first person to make that argument and I’m sure I’m not going to be the last.
What comes next is based on the assumption that they’re one and the same. It's a bit science fiction, but if this isn't a series prone for wild leaps of the imagination, that what is?
In the Torchwood season two premiere, Captain John Hart tells Jack that the time agency closed, but he doesn't state a reason. What if the slaughter of the crew of the SS Madame de Pompadour by the maintenance droids - more than fifty people, according to the Doctor - and the subsequent loss of the ship itself took enough of a bite out the agency that it was unable to cope and collapsed? John said there were only seven agents left (although he didn't indicate if that number included himself and Jack or not).
Of course, this is all speculation. We'll probably never find out what really happened, but it’s interesting to take the tiny shreds of information we do get and imagine how they might fit together.
EDIT:
cieria rightly pointed out that John is, of course, a compulsive liar. We only have his word that the agency is even closed - he had a million reasons to manipulate Jack.