1. The fact that he died suddenly surprised me.

2. Do you like the idea that we have to go to New York to be wealthy?

3. The fear that the World Wall 3 will break out is widespread.

4. I would like to have the possibility that he will come back soon.
(Are the sentences all grammatical? They contain apposition clauses. Can we omit 'that' in each sentence?)

#1 is ok, but simply removing the "that" is awkward.

Note that it humorously ambiguous:
"died suddenly," or "suddenly suprised me"?

#2 is syntactically correct, but an unusual usage.
I'd not remove the "that"

#3 I'd not use "the" since World War III is a name. Leave "that"

#4 is syntactically ok, but I'd never say it.

1. The fact that he died suddenly surprised me.

2. Do you like the idea that we have to go to New York to be wealthy?

3. The fear that World War III will break out is widespread.

4. I would like to have the possibility that he will come back soon.

Steve's comment re sentence 1 involves the placement of the word "suddenly" -- and if you write "suddenly died" it'll be clear what was sudden (the dying or the surprising).

2 and 3 are fine.

4 is very odd and wordy. We'd be far more likely to say or write, "I hope he comes back soon."

1. Yes, all of the sentences are grammatically correct. They all contain apposition clauses, which provide additional information about the nouns (subjects) in the main clauses.

To answer the second part of your question, in these sentences, 'that' is a subordinating conjunction introducing the apposition clauses. In standard English, 'that' is often omitted in apposition clauses. Therefore, you can rewrite the sentences without using 'that':

1. The fact he died suddenly surprised me.
2. Do you like the idea we have to go to New York to be wealthy?
3. The fear the World War 3 will break out is widespread.
4. I would like to have the possibility he will come back soon.

Note that although 'that' can be omitted, its inclusion does not make the sentences incorrect or unnatural. The choice to include or omit 'that' is a matter of personal style or context.