I have been told by the sales person at the store that this is a common issue with the Note. The dialer and contacts take anywhere from 2-5 seconds to load up the first time you press the respective buttons. Very frustrating if you ask me because the main functionality of this device is that of a phone. Once you bring up the dialer (or contacts), the next few times it comes up in a snap while its in memory but if you leave it closed for some time, the next time you click on it, it takes time to load again. Samsung should really make the dialer and contacts apps stay in memory all the time. I don’t see a reason why they should not be able to do it. I have used the TouchPal dialer and contacts manager and they come up much faster than the stock applications.
BTW I have also tried disabling animations, power saving etc but the problem persists. And I have less than 900 contacts that includes my phone contacts and synced contacts with gmail and skype. Of these only the contacts with numbers are programmed to be displayed. Thats less than 250 contacts. With a 1.4gig processor I would expect better loading speed for such a small number of contacts.