Horror is a disreputable yet strangely durable genre. It is attacked, disparaged, and often simply ignored; and yet it is defiantly alive, or at least stubbornly undead. What better time than Hallowe’en to examine its curious appeal, its many tricks and treats?

First of all, what exactly is horror? According to the wondrous fount of knowledge that is Wikipedia , horror is “a genre of literature which is intended to, or has the capacity to, frighten … scare or startle readers by inducing feelings of horror and terror.” Immediately, one might feel justified in questioning the definition. American Psycho and Revolutionary Road are examples of books that scared and startled me personally, and yet I doubt that either would be tagged as “horror” in the conventional sense of the word. Still, it’s as valid a definition as any I’ve heard. Horror is, above all, about fear.

Fear. It’s a primal response, every bit as primal as hunger or sexual desire, and in most traditional systems of morality these base animal instincts are seen as things be overcome, to be subordinated to reason or godliness or the good of society, or whichever higher truth you happen to believe in. Horror, however, far from encouraging us to conquer our fears, actively seeks to provoke fear.

Horror also evokes the terrifying spectre of the loss of agency, the loss of control. In horror, bad stuff happens, and generally the hapless characters are unable to prevent it. Occasionally, a protagonist may be rewarded for his or her courage, intelligence or integrity, but usually the good and bad alike fall victim to the monster or murderer. In these aspirational days, then, horror is refreshingly unaspirational: this perhaps accounts for its core demographic of teens, pessimists, and loafers like me. Normal, reasonably well-functioning adults, you might think, should have little need for horror; life is precarious enough without having to worry about zombies or swamp monsters. Adolescents and no-hopers, on the other hand, are accustomed to feeling powerless, so the giddy, out-of-control sensation that horror inspires is probably less likely to alarm them.

Another reason for horror’s disreputability may be horror writers’ occasional tendency to pander to the lowest common denominator, to churn out the pulpy rubbish that the literary elite loathe. It has ever been so, since the days when the “penny dreadfuls” were hawked about for the decidedly unedifying entertainment of the masses. But then again, does the suspicion really stand up to scrutiny? Very few people would deny the greatness of Poe, an unashamed horror writer. Besides, horror can take many forms, from the relative subtlety of the ghost story, with its themes of loss, grief and temporal dislocation, to the no-holds barred gore-fest so beloved of teenagers. The ghost story is, indeed, relatively respectable, not least because it has attracted practitioners as skilled as Charles Dickens, Edith Wharton, Elizabeth Gaskell and Henry James.

At its best, the ghost story presents us with a satisfying psychological dance between life and death, light and darkness. Can we make peace with the past? Can we live with it, and not be consumed by it? The haunting is often the external manifestation of an internal, psychological state that the haunted is not able to consciously confront; the ghost, for good or ill, forces that confrontation. The haunted house, with its dark corners and echoing corridors, is a physical representation of the tortured mind. Examination of profound and troubling questions is, indeed, a feature of much quality horror. Frankenstein is an enquiry into the nature of existence itself; The Turn of the Screw is a queasy insight into a mind that is on the verge of cracking; The Fall of the House of Usher is a strange ode to dissolution and decay.

Horror is, simply, about being human: about being at the mercy of immensely powerful forces, both internal and external, that we cannot defeat or control; about being, frankly, afraid. A ghost probably won’t creep up on you tonight, but you’ll still be haunted by the need to make your peace with an imperfect past. You’re unlikely to get it in the neck from a vampire, but you’ll nevertheless continue to be threatened by amoral, parasitic predators in their all-too-human form. The world is full of malign and often irrational powers. Horror reminds us of this basic fact, and affords us a thrill not least because it questions our ability to deal with them.

Smashing pumpkins: some creepy Hallowe’en reads

The Hound of the Baskervilles, Arthur Conan Doyle

Curious, if True, Strange Tales, Elizabeth Gaskell

The Legend of Sleepy Hollow, Washington Irving

The Haunting of Hill House, Shirley Jackson

The Turn of the Screw, Henry James

The Shining, Stephen King

Carmilla, Sheridan Le Fanu

Complete Tales and Poems, Edgar Allen Poe

Frankenstein: Or, the Modern Prometheus, Mary Shelley

The Strange Case of Dr Jekyll and Mr Hyde, Robert Louis Stevenson

Dracula, Bram Stoker

The Invisible Man, H.G. Wells

The Picture of Dorian Gray, Oscar Wilde