When you start an RP session with your PCs trapped in a bunker, you have created a
bottle episode.
EDIT:
Some people mistakenly use the term “bottle episode” to refer specifically to an episode set in a single location <= Yes, this.
Maybe watch a few as prep for running a session like this? I was quite impressed by the Family Guy episode with Stewie and Brian stuck in a bank vault.
The lesson is this: Bottle episodes are about dramatic dialogue and interpersonal conflicts, not procedural action and overcoming external obstacles... which mainstream RPGs are not well equipped to handle.
While the threat of frenzy is a lovely in-game way to stage the impending negative outcome (indeed, I wonder if you could have started the session by telling the players that someone is going to frenzy by the end of the session, the only question being who and when), the way to get there was not simply a series of courage rolls. As pointed out, this produced the wrong kind of boredom. What you wanted was argument, irritation, stifling desire, etc.
Ideally, this would have been the moment for the PCs to get on each other's cases about stuff...
"So, I haven't said anything about this until now because I really respect you, but I need to talk to you about the black lines in your aura." <boom BOOM>
"Explain to me again why you had to kill my ghoul?" <tat tat tat... BOOM!>
"Why do you Brujah have to be so hot headed about everything? It's your fault we're stuck in this mess!" <eeeeEEEEE BOOM!>
But if your players need some help, maybe you could have used the NPCs they were trapped with (it sounds like you sort of did with the Sabbat pack getting antsy).
"You Camarilla think you're so cool. Let me tell you what your elders are REALLY up to." <boomboomBOOM>
"What do you mean I can't have some fun with the soldiers stuck in here with us? No one's going to miss them." <tat tat tat... tat tat...>
"Why do you keep hiding from the beast honey? You should let it out a little more... It feels so good!" <BOOOOOM!!!>
Just like in any good TV show, the context of being trapped in a bunker with the shells getting closer and closer becomes a metaphor for the emotional pressure building up between the characters. "Why won't she just drop it? I've got to get out of here! When is this going to END? Shut UP! JUST SHUT UP!!!"
But as I said above, mainstream game systems (White Wolf included) don't really help you with this, so you are either on your own doing some dramatic improv (which can be fun) or you need to do some tweaking. You were on track with this, but perhaps from the wrong direction: Instead of trying to model shell-shock, you want to mechanically motivate your players to engage in the kind of wheedling and needling of each other described above.
One place to look would be Robin D. Laws' Drama System, the only RPG I know at all intimately that aims to produce dramatic instead of procedural stories. While you could just port it over for the evening, here are some lessons:
1) Set up relationships of dramatic tension beforehand: Get your players to come to the session with (or perhaps even better spend some time at the beginning of the session crafting) dramatic relationships that involve the other PCs. Each player should pick another PC that they want something emotional from that that PC is motivated not to give them. Maybe they want an apology, or forgiveness. Maybe they want a gesture of respect or even love. Maybe they want that person to experience guilt. Maybe they want to make them happy, or maybe they just really want to get in that person's pants... er... neck.
2) Reward patterns of play that may not be spontaneous: Robin's worry is that the petitioner (the person who wants something) will be too slick and strategic and the granter (the person who the thing is wanted from) will simply always say no. Therefore he sets up a points system that rewards the petitioner for being denied (motivating him or her to be unreasonable and irritating in his/her demands) and that rewards the granter for acquiescing (motivating him or her to cooperate in the long run, even if he/she is going to regret it later).You might do the same, but you might bend it towards the aforementioned threat of frenzy. What if...
Every time you voluntarily give up a temporary point of self control (or willpower or whatever) in response to something your interlocutor has said to you (your character is getting really annoyed by the other PC's demands OR is more and more frustrated that the other PC won't give them what they want just this once), you get rewarded?
If you ever run out of self control this way, you frenzy immediately, but at the end of any scene where the granter did NOT acquiesce to the petitioner's demands, both sides have to roll self control to see if they frenzy.
This would hopefully motivate the players to spend some time "farming" the reward by getting on each other's nerves and running down each other's self control/WP, while creating an interesting end game in which the granter has to decide if he/she has the resources left to risk saying no.
The gesture of ticking off boxes as you are talking is also fairly unobtrusive and would let the dialogue flow pretty naturally.
What's the reward? The obvious answer is XP. There is some math to do here... Figure out how much XP PCs should earn for the session, divide it by how many self control you expect them to lose. The only thing to keep in mind would be whether they are going to be involved in uneven amounts of conversation and whether you're okay with that (uneven spotlight and uneven reward).
A more meta way to proceed would simply be to say that the bombing doesn't stop until a certain amount of self control has been lost and let the players figure out who it's going to come from.
Also, is it just me or is this THE session you need background sound effects and mood lighting for?
Cheers,
Ogreboy88