I would say no, Frodo and Bilbo do not die. I think Tolkien intentionally leaves this a bit vague. This is a simple question but there is a lot of information to take into account. I am typing this without referring to the books so I could be mistaken.
At one point, I think it's when Pippin is hanging out with Gandalf in Minas Tirith just before the Siege of Gondor begins, Pippin and Gandalf are talking about what comes after life. Gandalf explains how the mortals have no clue what comes after death, that is the nature of the mortality of men. He explains, as does the Silmarillion, that when the elves die they go to the halls of the gods, which reside in Valinor.
In the Silmarillion it is implied that mens souls are destroyed, when they die. Their death is the balancing factor in their heightened capability for emotion and physical strength.
Now, the reasons I think that Frodo and Bilbo become immortal when they reach Valinor is because of the life extending, preserving nature of elven magic. (I think Aragorn mentions something about how time's effects are changed in Rivendell, and there is another bit about time in Lórien). Valinor, is the home of the Ainur, the gods and their magic are obviously more powerful than the elves. Also, In the Silmarillion, several of the greatest heroes, (Túrin if I am not mistaken) are brought to the halls of the Ainur to rest forevermore.
As I mentioned earlier, I could be mistaken on any number of points. If I am I would love to be corrected. =-)