Despite being marketed as Edgar Allan Poe’s The Haunted Palace, the story in this is really H.P. Lovecraft’s The Case of Charles Dexter Ward. It gets its title from a poem by Poe that would become part of The Fall of the House of Usher. 

Everyone in Arkham believes that Joseph Curwen (Vincent Price) is some kind of evil magician. Well, they’re right, as he and his lover Hester Tillinghast (Cathie Merchant) keep something unspeakable in the basement that they use to terrify young women in rituals. Exra Weeden (Leo Gordon) and a mob catch them and decide to burn Curwen alive. Before he goes up in an inferno of small-minded townspeople, he curses everyone and says he will come back for revenge.

A century later, Curwen’s great-great-grandson Charles Dexter Ward (Price) and his wife Anne (Debra Paget) come to town. The people hate them already; the people are strange and deformed. Charles becomes overly interested in a picture of his ancestor to the point that it starts to change who he is. This pleases caretaker Simon (Lon Chaney Jr.).

Dr. Marinus Willet (Frank Maxwell) explains to Charles and Anne that his predecessor had a black magic tome, the Necronomicon, and had summoned both Cthulhu and Yog-Sothoth. He mated them with teenage girls, which led to the town being full of freaks. Well, good news: Charles is slowly and surely becoming Joseph, and everyone in the city is going to pay. 

This begins with the Poe poem — “And travelers, now, within that valley, through the red-litten windows see vast forms, that move fantastically to a discordant melody.” — and ends with it: “While, like a ghastly rapid river, through the pale door, a hideous throng rush out forever and laugh, but smile no more.”