Got an alienware x51 r2. and im going to get a 1060 gpu. What power supply (400w) would be compatable with the computer?. I'm struggling to find one that would work because of the laptop like port for the power supply.
I'm not sure if there is anything available off the shelf. There would be lots of forum posts around if people had managed to work it out as it's been a big issue for years. I had a quick search and found the same as you with the older style connection. Maybe there's an adapter somewhere.
I think you could be ok with the 330w PSU. The 1060 draws 120w and if you have the 3770 that draws about 150w at max load. The rest of your parts shouldn't draw much more. There's a vid out there showing a guy running a 1060 and his whole system is only drawing 180w. Nvidia suggest a 400w psu but they would over estimate just to cover themselves. I think I'll get myself a 1060 now
The GPU? From what I've read it's short enough to fit in easily. It's some of the 1070's and 80's that have trouble. Just make sure you get a blower style GPU and not one with the fans as you need the hot air to blow out of the machine. It's a lot harder to find the blower style these days. You need something like the Nvidia Founders Edition. Some people say that cards with a single fan seem to be ok but it's still keeping the hot air inside the case with all your other parts.
The 1060 is as plug and play as it gets. Many people here (myself included) have been using a 1060 in R1, R2, and R3 with the stock 330w external PSU without any problems or case mods. You just need 1) the 330W - not the 240, 2) the latest flashed BIOS, 3) A blower style card - recommended.