I created a new page (keeping the Page URL as default; which is blank). Then I went to the settings of the page and deleted the "Page URL", and clicked on "Update" button just beside the setting.
Now the newly created page goes to the home page, which is trivial. However I couldn't find a way to revert what I did. Normally in such cases we go to Admin->Page Management and change the "Page URL" setting of the newly created page. But The "Page URL" setting is not accessible from there.