...shoutouts to
mook for How to be a GURPS GM. It's been infinitely helpful.
So starting out, what resources are there for Horror in GURPS? Are there any rules outside the big two books for things like fear and spooky scary monsters? Is there a GURPS Cthulhu book anywhere?
Ooh, "infinitely helpful"! I like it -- thank you mucho, I'm really glad it helped present GURPS as something you might dig.
Okay, link avalanche in three... two... one...
The GURPS Books page has a super-handy section
specific to Horror which lists all their Horror-related products (and labeled as being 3rd edition or 4th).
For specific books, I'll mention a few. (Not that all book pages have a link to a free preview PDF, which always includes at least the Table of Contents and Index so you can see
exactly what is covered and what is not for any book you're curious about).
Horror is an absolute delight of a book, one of my all-time faves-- I recommend it without hesitation. (Though I'd still like to mention, you don't ever actually
need anything outside the Basic Set -- everything else is just a "nice to have and saves me a lot of work"). It's kind of a gamer's
Danse Macabre... an overview of the entire genre and all of its most common manifestations, with copious suggestions for how to accomplish all that through GURPS as the medium.
As you mentioned fear, you'll notice on the preview PDF's Table of Contents that a large chunk of the book explores humanity's most primal fears (madness, the unnatural, mutilation, etc.) as expressed by monsters from legend and more modern tales. This sums up the approach nicely (Horror, p. 56):
"...the remainder of this chapter is organized into sections that address some basic human fears. Each fear has a "signature" monster, some variations, and game notes attached to it. The monster given for a particular fear is a starting point; this organization is descriptive, not prescriptive. If you decide that vampires are fundamentally not about the fear of taint and disease, but rather the fear of the unloved dead, then revamp (ahem) your bloodsucker accordingly. Some creatures – and fears – have pointers and advice for such adjustments. Add new monsters for specific fears, and recombine monster templates with Frankensteinian abandon. The goal here is to help the GM think about monsters in terms of their role as thematic elements first and foremost, as portents of the uncanny, rather than as opponents."
Another interesting one is
The Madness Dossier, by Kenneth Hite, conspiracy-recounter extraordinaire. I don't actually own this one, but Hite is always an entertaining read, and it sounds like it deals with a lot of issues of shifting reality, perception, etc. that sound like they'd fit in the kind of game you're running.
The only specifically Cthulhu GURPS property I can recall is the 3rd edition
Cthulhupunk, which is basically the Mythos in a Cyberpunk world. The stats-type stuff would be off a bit (though easy enough to adapt), but all of the fluffy narrative bits would be just fine as-is. (Unfortunately, as a 3rd edition book, no preview PDF, but the description at the linked page gives some clues).
Finally, honorable mention to
Zombies and
Zombies: Day One... doesn't sound like you're game is much about survival horror, but they are a fine companion to Horror dealing exhaustively with just all the various flavors of walking (leaping, running) dead.
Oh, and
finally finally, it may be a bit more militaristic than what you're looking for, but the
Monster Hunters series is also great, not only for suggestions on battling monsters but for ways to streamline the rules for a faster pace.
Sounds like a fun game, and hits right in the GURPS sweet spot. I hope it continues going well, and you find GURPS a solid fit for what you're trying to do!