Your setting reminds me of the old fairy/faerie stories.
For the human side, how you want the two races to interact will help you decide what kind of technology and society they have. A medieval kingdom will respond to the forest in a different way than a modern state or simple hunter-gatherer group would.
Don't worry too much about lack of characters or plot. Until you go to write a story in the world you don't necessarily need specific characters (unless they're really, really important) and history functions as a plot.
Theram, when you said that there was nothing weird about it, did you mean that this has been done? If so, then please tell me the book/film/whatever that contains it.
I'm currently thinking about introducing that old chestnut of elves being vulnerable to iron. The reason for it is that the elves have copper in their blood (humans and most Earth creatures have iron, which is why our blood is red; squids and such have copper, which is why it's blue/green), and they have some sort of allergic reaction to iron. Would that be a good idea? Another thing is that the elves are seen as magical and powerful because they know how to induce hallucinations and poison the humans; they might not be all they seem. I also have a creature that needs a home: it looks like a small, cute rodent (large eyes, the kind of cuteness that a gerbil has), and is seen riding atop cats. This creature is called a Cat-rider, and features in folk tales. However, what the people don't know is that the cat-rider is a parasite and controls the cat's brain; it sends in small "tentacles" into the cat's head, much like the roots of a plant, and then it uses the cat for something. I haven't yet figured out why it needs to ride a cat. Is that a good idea, should I put it in my fantasy world?
Elves and the Fair Folk have been linked to illusions since time immemorial, Puella Magi Madoka Magica is currently thoroughly exploring the idea of Mind Screw places that serve its inhabitnts (as did Perry Rhodan before that... see also the Genus Loci page), and the "their magic is actually drugs" joke has been around since super mario.
Not that any of this makes it bad or worthless in any way, shape or form, heh. *winks and gives OP a thumbs up* This combination might actually yield interesting results!
I'd give more suggestions, but that'd be unfair towards Zolnier who's been waiting wayyy too long for my contributions already.
Just ask, and I'll answer you at a later point, okay?
Warning about the copper-based blood: make sure you do the research on that. I could see it being very easy to fall into You Fail Biology Forever if you don't get that exactly right.
But too much iron is bad for humans, too. Perhaps, because they don't use it for their blood, what level is 'too much' becomes much lower. They should also be less prone to copper poisoning than humans, too.
If I'm asking for advice on a story idea, don't tell me it can't be done.

I'm thinking about writing a fantasy story. In this world, there are humans and elves. The elves live deep in the forests, and the forests are very dangerous, full of poisonous creatures and mushrooms spewing forth toxic spores. Some of these poisons are hallucinatoric for humans. The elves are a sentient species that have adapted to living in the forest and they can also use different poisons for their own ends. I'm not sure of what technology level the humans should have, and I imagine the elves to have some sort of tribal society. The humans tell stories about the elven forests, of people who got lost there and never returned, or who returned completely insane. When inside the forest, it's very easy to get lost because of all the hallucinations, and you can't really be sure if those footsteps behind you were real or just imagined, can you?
I have no plot and no characters. Is this idea a good one or should I scrap it? Be as nitpicky as you can. I haven't yet thought about the shape of the planet (I imagine all fantasy worlds as separate planets), the climate (is the elven forest a jungle? a pine forest?), or any such things. However, I welcome all suggestions and questions.