The Nikkei 225 index at the Tokyo Stock Exchange fell 47.23 points to 17,510.81 by the break, while the Topix index of all first-section shares was down 0.22 percent, or 3.05 points, at 1,405.70.
The market swung in and out of positive territory as the higher oil prices boosted energy firms but hit tyre makers and airlines.
Crude enjoyed some much-needed buying after data showed the recent slump in prices has led to a decline in the number of rigs drilling.
US benchmark West Texas Intermediate for March delivery rose 36 cents to $49.93 while Brent crude for March rose 18 cents to $54.93.
On Wall Street, the Dow closed up 1.14 percent Monday, the S&P 500 jumped 1.30 percent and the Nasdaq gained 0.89 percent.
The oil price rise will come as welcome relief for markets after oil slumped more than 50 percent from its June 2014 peak owing to a global glut and a strong dollar.
“A sense of relief is beginning to spread among investors,” Toshihiko Matsuno, chief strategist at SMBC Friend Securities, told Bloomberg News.
“We’re also seeing some easing of worries over a slowdown in economic growth and less risk of defaults or credit downgrades of countries and companies.”
However, the rising yen took the wind out of the market.
In forex trading, the greenback bought 117.29 yen, compared with 117.64 yen in New York.
In share trading, Bridgestone fell 2.87 percent to 4,499.5 yen, Toyo Tire & Rubber was 2.27 percent lower at 2,586.0 yen and Japan Airlines dropped 3.31 percent to 3,795.0 yen.
Energy giant Inpex jumped 6.11 percent to 1,371.5 yen, while Sony rose 1.91 percent to 2,750.0 yen after Credit Suisse raised its price target on the stock and it announced the sale of its US-based online games unit.
Panasonic rose 2.3 percent to 1,378.5 yen while Sharp jumped 4.74 percent to 243.0 yen. Both firms publish financial results later in the day. (AFP)