In a speech in London, the Oscar-winning Hollywood actress said there was a “duty that falls on all of us” to help those fleeing their homes, warning the alternative was “chaos”.
The American star also had criticism for US presumptive Republican presidential nominee Donald Trump, who has proposed building a wall to stop Mexican immigrants and called for a temporary ban on Muslims entering the United States.
“It is hard to hear that this is coming from somebody who is pressing to be an American president,” she said in a question and answer session after her speech.
“America is built on people coming together for freedom, especially freedom of religion.”
Conflicts, including the five-year war in Syria, have fuelled a global refugee crisis, with 60 million refugees and displaced people across the world.
Jolie Pitt, a United Nations special envoy for refugees, said she recognised that some people felt “angry” about the numbers of people on the move, and no longer had faith that institutions could deal with the issue.
“It has given space to a false air of legitimacy to those who promote the politics of fear and separation,” she said.
“It has created the risk of a race to the bottom, with countries competing to be the toughest in the hope of protecting themselves whatever the cost or challenge to their neighbours, and despite their international responsibilities.”
However, she warned: “If your neighbour’s house is on fire you are not safe if you lock your doors. Strength lies in being unafraid.”
Jolie Pitt called on the international community to be more generous towards refugees, who were each “a person with an equal right to stand in dignity on this planet”.
“This is a duty that falls on all of us, to the next UN secretary-general, to all governments, to civil society, to everyone of us,” she said.
“Whether we succeed will help define this century. The alternative is chaos.”
The UN set out a plan last week that aims to resettle at least 10 percent of the global refugee population every year, as it tries to tackle the crisis.
UN Secretary-General Ban Ki-moon hopes the new “global compact on responsibility-sharing” will lift some of the burden on developing countries.