9 times out of 10 if Outlook starts the moment the PC is turned on then its because its in the start-up folder (not registry)

Check by going:

[Start] [Programs] [Startup]

If outlook is in there then simply delete it (this WILL NOT delete outlook, so dont worry about deleting from this folder)

Then restart and Outlook will not start.

Alternatively by holding down shift when starting windows you will prevent ANY startup programms running.