Motherboard suggestions

Ferguson said:

In case anyone is interested, an update.

I decided to get rid of spinning disks entirely, so went with the Intel 750 as a system disk (better deep-queue performance, 1.2TB), not in a raid configuration.

For photos I put in two 2TB 850 EVO SSD's in a mirror storage space. That way I get redundancy, and I did some tests and mirrored performance is right up there with single drive performance even on writes (a real surprised). I planned for catalog and photos on that.

Then I put a 2TB 850 EVO SSD alone for scratch, previews, temp.

And.... it didn't work. The U.2 connection for the 750 takes up ports 2 and 3 of the 6 SATA ports, but ports 0 and 1 are not working fully. I can't get a drive to run in either one, leaving only ports 5 and 6, one short for this configuration (and two short for what I eventually suspect I'll need, another drive in the mirror).

Lots of experimenting, checking conflicts, a new bios, making sure it was not drives or cables -- it's the ports.

So a new board is on the way. My concern is whether it's a design failure or an actual hardware failure. The board is new, not many people using it, and most seem to be gamers who don't use a lot of drives, or people putting in an M.2 and U.2 in a raid-0 set for real high speed disk. So who knows if they really tested it as I am using.

Anyway... now the floor of my office is a wreck and I'm sitting half-assembled waiting for a new board.

While waiting, I am spending a lot of time debating ordering another 850 EVO and just building a 2x2 mirror storage space. It will run a lot faster in writes (the stripe aspect) and seeing that the stand-alone 850 didn't really run faster for scratch/temp, this is a more flexible configuration (if more expensive).

Having spare time is always costly.

The bad news is I haven't gotten far enough along to do any lightroom testing. I'm building with the old system working, so I will try to get some comparative runs for repeatable things like preview builds or exports when both are running.

Click to expand...

Thank you for the update. Sorry about the delay. Have you considered getting a RAID controller on the PCI Express bus instead?
This would eliminate the port issues/conflicts and give you the speed you need. In addition, some of the PCI Express cards now are 16 channels and reasonably priced.

PhilBurton said:

Ferguson,

Which version of Win 10? Or are you going to transfer the Win 10 "digital entitlement" from your old system?

Can I ask you how much you have spent on this dream rig? Was this more than you planned?

Phil

Click to expand...

I am going to try to transfer my W10 digital entitlement. I installed the ANniversary edition, which allows you to do activation based on a microsoft windows ID; whether it will work I do not know. Microsoft has written completely contradictory indications on that. Initially they said retail versions would transfer, not they say any upgraded version will not. So who knows.

It was MUCH more than I planned. I haven't added it all up, but I ended up really blowing any idea of a budget when I decided to go all SSD including photos. I've got now a 1.2TB Intel plus 4 x 2TB SSD's, so that's over $3000 in storage alone (ouch... I bought those in pieces, now you made me add it up). But I tend to keep systems for 4-5 years, and I just did not want to start with spinning disks. No good rationale reason, just trying to rush the future along I think.

The actual processor and MB were not bad, something like $500 to get the fastest Z170 chipset processor and MB, I don't really the memory but 64GB was not a lot. It was the SSD's that really blew the bank.

The real killer was getting the MB working. All that time let me think and think and I just kept hitting the "buy" button.

Don't tell my wife.

tspear said:

Thank you for the update. Sorry about the delay. Have you considered getting a RAID controller on the PCI Express bus instead?
This would eliminate the port issues/conflicts and give you the speed you need. In addition, some of the PCI Express cards now are 16 channels and reasonably priced.

Click to expand...

Actually it doesn't eliminate it exactly. I've considered it, but there are a few issues. Low end raid controllers seem to get mixed reviews. HIgh end are REALLY expensive. So a bit nervous about getting a good one. But in addition to that, I can't add a x4 much less an x8 raid controller without running the GPU at x8 instead of x16. I have no idea if that matters -- the whole "will the GPU help LR speed" is so nebulous, that all I can do is put in a decent one, cross my fingers and hope. Cutting its transfer rate in half made me nervous.

What I considered a bit more strongly was shifting from the Z170 chipset to X99, which adds a lot more unencumbered PCIe lanes, but the CPU's for it get more speed by more cores, not faster cores. The fastest processor for it is slower, per core, than in the Z170 chipset. I still think much (most?) of LR's develop performance is spent in very threads, and not sure more cores (than 4) is any help. Certainly tests on long running renders in Photoshop show a diminishing return as you get to 3, 4, 5 etc cores, though it is not clear if that transfers to LR.

Really what I have SHOULD work. The U.2 adapter should take 2 SATA ports off, leaving me 4 to use. New board comes tomorrow so I'll know more then.

Been doing some timing tests if anyone is interested. These are timings of repeatable things. I did this:

1) Took a folder of 76 images (D4/D5 size mixed, already processed).
2) Select all, change exposure
3) Immediately do a "Build 1:1 previews" and time it
4) Export them all (quality 92, full size, sharpen for screen) and time it

I did this with the old computer with and without GPU, and as expected no difference (it only affects develop, but wanted to confirm, so did not test on new).

I then did it with the new system with faster everything, in order:

a) With hyperthreading enabled (default)
b) With hyperthreading disabled
c) With memory overclocked to 3000 (vs 2133 as standard)



As you can see, the new system is faster, but not by much. The (several years old) I7-3770K is 3.2Ghz, then brand new I7-6700K is 4Ghz. Neither over-clocked.

The difference in speed is similar to the processor speed difference; a bit better, but not hugely so despite about 4 years difference and generations in processor.

Memory speed makes very little difference, a bit, but not nearly proportional.

Hyperthreading seems to hurt a bit, though apparently only in preview builds not export. I am not sure quite why, though I speculate that Previews are more highly parallel than exports, but not so parallel that 8 threads would help. In other words, the virtual thread switching had more harm than the additional virtual cores. Many people just keep hyper-threading on all the time, but there are a lot of workloads where it varies from moot to harmful (I always turned it off on big database servers, for example).

But... this is just about as fast of a desktop as you can build today, without over-clocking. More cores will not help (e.g. X99 chipset). And lightroom is still slow.

By the way, this system has 64G of memory and extremely fast disks (Intel 750 SSD for system on x4 PCIe, and 4 x Samsung 850 EVO drives in a 2x2 storage system -- both are much faster than regular SSD's, and the catalog on the former (+ cache and temp) and the photos on the other. So this was NOT disk limited.

I did experiment a bit with develop, and the combination of faster base system, faster GPU, and (probably) faster PCIe transfers seemed to make it much less laggy. But that's a vague subjective impression quite possibly due to wishful thinking. I still think we need a way to benchmark Develop stuff.

Anyway... FWIW.

Ferguson said:

Been doing some timing tests if anyone is interested. These are timings of repeatable things. I did this:

1) Took a folder of 76 images (D4/D5 size mixed, already processed).
2) Select all, change exposure
3) Immediately do a "Build 1:1 previews" and time it
4) Export them all (quality 92, full size, sharpen for screen) and time it

I did this with the old computer with and without GPU, and as expected no difference (it only affects develop, but wanted to confirm, so did not test on new).

I then did it with the new system with faster everything, in order:

a) With hyperthreading enabled (default)
b) With hyperthreading disabled
c) With memory overclocked to 3000 (vs 2133 as standard)

View attachment 8142

As you can see, the new system is faster, but not by much. The (several years old) I7-3770K is 3.2Ghz, then brand new I7-6700K is 4Ghz. Neither over-clocked.

The difference in speed is similar to the processor speed difference; a bit better, but not hugely so despite about 4 years difference and generations in processor.

Memory speed makes very little difference, a bit, but not nearly proportional.

Hyperthreading seems to hurt a bit, though apparently only in preview builds not export. I am not sure quite why, though I speculate that Previews are more highly parallel than exports, but not so parallel that 8 threads would help. In other words, the virtual thread switching had more harm than the additional virtual cores. Many people just keep hyper-threading on all the time, but there are a lot of workloads where it varies from moot to harmful (I always turned it off on big database servers, for example).

But... this is just about as fast of a desktop as you can build today, without over-clocking. More cores will not help (e.g. X99 chipset). And lightroom is still slow.

By the way, this system has 64G of memory and extremely fast disks (Intel 750 SSD for system on x4 PCIe, and 4 x Samsung 850 EVO drives in a 2x2 storage system -- both are much faster than regular SSD's, and the catalog on the former (+ cache and temp) and the photos on the other. So this was NOT disk limited.

I did experiment a bit with develop, and the combination of faster base system, faster GPU, and (probably) faster PCIe transfers seemed to make it much less laggy. But that's a vague subjective impression quite possibly due to wishful thinking. I still think we need a way to benchmark Develop stuff.

Anyway... FWIW.

Click to expand...

And if you can develop scripts, I'll be happy to run benchmarks on my 4 year old X79-based system with 32 GB, an old video card and an SSD only for scratch. Regular SATA II drive @ 7200 rpm for bulk image file storage. I might even use these benchmarks as an incentive to finally overclock both memory and CPU.

Linwood, I was very surprised at those timings, specifically the huge difference between Preview build and Export. Given that building 1:1 previews and exporting are pretty similar, it would make sense that the timings would be similar as well. That's certainly been my experience in all the performance timing tests that I've been running since LR3.....in LR3, 4 and 5 there was hardly any difference, though with LR6's parallel exports it was then noticeable that multiple exports were always quicker than building 1:1 previews for the same files. Hence my surprise at your timings.....either the preview building is remarkably quick, or the exports are significantly slow (or a bit of both!). Worth further investigation, I would think.

If I read these numbers correctly you are getting between 40% and 48 % improvement.

What were you expecting.


I suggest you try a simple timed o/s copy of the files and compare the old to new. This leaves Lr out of the equation.

The improvement could be down to processing speed rather than disc i/o as the image processing has to take place before the copy to disk operation ..... which begs the question ..... what improvement would you have got my just improving the cpu.

Fergeson,
Nice system. When you look at Lr, it is CPU processing serially and has limited benefit from multiple cores; I have seen this documented with multiple blogs doing performance tests on Lr. Depending on the task, Lr seems to peak around 4 cores; with additional cores getting very limited usage.
The iCore 7 generational changes deal with mostly cache, max clock speed, memory speed, bus speed, graphics, heat, energy usage and available number of cores. Basically everything around the CPU itself.
The result, going from a iCore 7 gen 4 to an iCore 7 gen 6 for Lr in most cases will only be faster based on clock speed. So for Lr, counter to what many people look when shopping for a CPU, you want pure clock speed (which you seemed to have picked) and most systems today seem to place an emphasis on number of cores.

So I am stuck waiting for the another generation or two (this why I have not bothered to upgrade from iCore Gen 4).

Tim

Jim Wilde said:

Linwood, I was very surprised at those timings, specifically the huge difference between Preview build and Export. Given that building 1:1 previews and exporting are pretty similar, it would make sense that the timings would be similar as well. That's certainly been my experience in all the performance timing tests that I've been running since LR3.....in LR3, 4 and 5 there was hardly any difference, though with LR6's parallel exports it was then noticeable that multiple exports were always quicker than building 1:1 previews for the same files. Hence my surprise at your timings.....either the preview building is remarkably quick, or the exports are significantly slow (or a bit of both!). Worth further investigation, I would think.

Click to expand...

I just report the news, I don't explain it.

In watching it run, I did notice that the progress bar for previews jumped; my guess is that it was doing several in parallel. The progress bar, and watching the target directory fill up, seemed to indicate exports were one at a time.

Try it.

Didn't they withdraw the parallel export aspect?

Gnits said:

If I read these numbers correctly you are getting between 40% and 48 % improvement.

What were you expecting.

Click to expand...

Fair question. I think there's "expecting" and "hoping".

I was hoping for magic - that the collection of additive improvements - cache, memory, bus, disk -- would, you know... add.

I am an engineer, so I was expecting more or less what you see.



I think what led to this, at least in part, is the continual discussions here and elsewhere that go more or less like this:

"Lightroom is awfully slow".

"It's fast for me, must be your computer"

"My computer is X, yours is Y, should be similar".

"Well, LR runs just fine for me".

I'm always on the slow end. Clearly expectations play a role here, but there's always the chance that somewhere hidden within my aging computer was a hidden bottleneck that was strangling performance, even if I could not see it.

So I set out to build the fastest non-overclocked computer I could (with respect to lightroom), and sure enough Lightroom got faster, but not much.

So fundamentally, Lightroom was the issue all along in that it did not meet my expectations, there was no hidden bottleneck, and my hopes for a "throw hardware at it" solution are dashed. It will be better than it was, but any real solution lies with Adobe.

tspear said:

Nice system. When you look at Lr, it is CPU processing serially and has limited benefit from multiple cores; I have seen this documented with multiple blogs doing performance tests on Lr. Depending on the task, Lr seems to peak around 4 cores; with additional cores getting very limited usage.
The iCore 7 generational changes deal with mostly cache, max clock speed, memory speed, bus speed, graphics, heat, energy usage and available number of cores. Basically everything around the CPU itself.
The result, going from a iCore 7 gen 4 to an iCore 7 gen 6 for Lr in most cases will only be faster based on clock speed. So for Lr, counter to what many people look when shopping for a CPU, you want pure clock speed (which you seemed to have picked) and most systems today seem to place an emphasis on number of cores.

Click to expand...

Right. I've pretty consistently said that (and frankly that is where I think one fault lies with Adobe -- they have not done well at creating parallelism opportunities. You would think in image processing, with those huge bitmaps, that massively parallel operations would be possible (indeed, that's where the GPU comes in -- it's not some super computer, it just has hundreds or thousands of processors, of sorts).

While I took a small jump in CPU speed, the GPU was more like an order of magnitude (GT640 to GTX970) of capability, and I could see negligible benefit when I swapped to just that. If I was running, say, a video rendering program like Resolve, I bet I would see a huge jump. Adobe does not seem to have figured out how to get a lot of bang for the buck (literally) out of GPU's for LR.

tspear said:

So I am stuck waiting for the another generation or two (this why I have not bothered to upgrade from iCore Gen 4).

Click to expand...

Honestly I probably should not have. This all started with a memory failure, made me start thinking it was aging and maybe time to replace so I started going for just a MB and CPU, of course memory followed, that made me realize NVMe was out there, so new OS drive; oops, no raid, can't keep a catalog on that, so I need raid SSD's, don't want to use the old ones... it doesn't take very long pulling this sort of thing to run up a huge bill.

But at least I can put to be "it must be something in your computer". Maybe it will save you and others from thinking there's a hardware fix to what is fundamentally a software problem.

Linwood

Ferguson said:

I just report the news, I don't explain it.

In watching it run, I did notice that the progress bar for previews jumped; my guess is that it was doing several in parallel. The progress bar, and watching the target directory fill up, seemed to indicate exports were one at a time.

Try it.

Didn't they withdraw the parallel export aspect?

Click to expand...

Try it? I already have, very many times, and so far as LR6 is concerned exports are always faster than previews. I can't explain what you think you are seeing, previews seem to be built sequentially, and exports are (from LR6.6.1, IIRC) now once again run in parallel (3 at a time, it looks like from my progress bar).

Jim,

Could it be a difference in export settings and plugin?

Ferguson said:

Fair question. I think there's "expecting" and "hoping".

Click to expand...

Great reply.


My compliments for the job that you have done and sharing this with the community.

I hope someone in Adobe also reads this and be encouraged to do something. My view is that Adobe develop / optimise to the minimum they have to (eg Books Module, Library usability, etc), preferring to work on widgets and new apps and techie leading edge stuff, etc.. I am not limited this view to Lr, but could include InDesign and PS.

I recall a serious o/s techie mainframe performance expert telling me (at 4 am in the morning), as the client manager with a client performance issue, that performance can only be gained via lots and lots of little steps. We had a 100k h/w upgrade in the car which we were doing our best to avoid using. Those words have stayed in my mind since.

At least you know that you have done all that you can and ..... hopefully .. hopefully ... be in a good position to take advantage of any performance improvements which come down the line. Also, you have given yourself a platform for the next 5 years or so and an update on your own knowledge set for the current level of technology available.

Amazingly ... I did what you have just done approx 5 years ago..... and felt it must be time now to upgrade again. Having looked at your results .... I may wait a little longer.

I would still think there is value to compare the time it takes to copy a bunch of files on both systems, just to see some indicator of performance gain of the upgrade... independently of Lr.

Thanks for sharing.

tspear said:

Jim,

Could it be a difference in export settings and plugin?

Click to expand...

It could be anything, Tim. I'm simply exporting at 100% quality, no resizing, using the standard LR export function.

So I really have no idea what could be causing that disparity, it'll be up to Linwood to decide if he wants to go digging around to try to find a reason for it.

Jim Wilde said:

I can't explain what you think you are seeing...

Click to expand...

Now that's a comment that is sure to encourage me. :speechless:

The nice thing about science is that it doesn't depend on what you think I think you think... By stating what I did, others can try the same thing and either see something similar or different and from observation, not opinion, a consensus can be formed.

Maybe it's Mac vs Windows, maybe a version difference (2015.6), maybe it's being closer to Stonehenge that time is a bit distorted there.

If someone is so inclined, maybe they can experiment.

I "think" I'll move on to other things.

Gnits said:

I would still think there is value to compare the time it takes to copy a bunch of files on both systems, just to see some indicator of performance gain of the upgrade... independently of Lr.

Click to expand...

I did some benchmarks of disk, though I was convinced on the old computer I had disk I/O excluded by having catalogs on SSD, cache/temp on another SSD, and images on a Raid 0+1.

If you're curious, here are what the raw disks look like:



The top/left is the NVMe drive (intel 750), the others are identical 850 EVO's. One is broken and is going to be replaced, the bottom right will only run at Sata II speeds regardless of port or cable. But it runs.

For the actual run, I combined the four drives into a 2x2 mirror storage space, and it's performance is:


This is the fastest drive on the old system (I use it for cache/temp), the system and image disks are slower. So depending on type of IO, all the drives range from a lot faster to a little faster. I did decide to forgo raid for the system drive, mostly so I didn't need to buy two 750's, and because I expect the Intel drive to be pretty reliable.

While I'm putting all this stuff together, in case curious -- do not buy a new computer to save electricity.

I did some measurements of power usage (I'm waiting for a replacement SDD before I actually load everything and move, so bored). These are for the computer only, power into the power supply, and do not take into account the monitors or external peripherals like a EHD (which wouldn't be plugged in long).

I was interested to see the old computer drew 14 watts while turned off (not asleep -- off, the PSU is providing power for various things). The new 7 watts.

Both computers, with my normal workloads did not vary much whether at idle (but up) and running something (I used a disk benchmark, and was surprised the old HDD did not really see a difference). The old was about 147 watts, the new about 89. I would expect most of that is in SDD vs HDD, maybe a bit in a better PSU (though that's probably more at idle than running).

Turn that into annual 24x7 cost and it's about $157/year for the old, $95 for the new at $0.13/KHW.

Power Saving payoff time not including inflation is about 80-100 years.

One small help, the new computer so far seems to sleep and wake up fine, the old would never sleep properly (more precisely it would not wake up properly)- never found if software, hardware, bios, etc. So I may save another $30 or so if I get it to sleep right, pulling down the payoff period to only a couple of (human) generations.

Thanks for info. Useful numbers to reference down stream.

Ferguson said:

One small help, the new computer so far seems to sleep and wake up fine, the old would never sleep properly (more precisely it would not wake up properly)- never found if software, hardware, bios, etc.

Click to expand...

I ran into sleep / wake issues (mostly would not wake) when trying to get my Macrium Reflect software to wake up and complete unattended backups. I traced the problem down to issues between the motherboard and the then version of Windows. I could not upgrade the motherboard drivers because there were no updated drivers for my ver of o/s. I was up a creek without a paddle. This was a factor pushing me towards an upgrade.

However, Windows 10 seems to have solved the issue for me as it wakes up and sleeps just fine now.

Just to keep this current, I finally got a new Motherboard that worked (ASUS Z170-WS), the Gigabyte had some kind of compatibility issue with the Samsung 850 EVO 2TB's.

With four disks finally working correctly, the speed of the 2x2 storage spaces is now fast, not as fast as the 750, but darn close and redundant.



So now I'm starting to exercise everything and finally going to load it up.

I swear I've had data centers install faster than this PC build. Darn Gigabyte.

Thank you for sharing your findings. I just wanted to add that they are fully in line with my personal experience:

I have recently upgraded from a X79 based system with spinning hard drives and 16 GB RAM to a new X99 based system with an NVMe PCI-E drive plus SSD RAID 0 and 64 GB RAM and did upgrade from Windows 7 to Windows 10 at the same time. While the speed improvement in Lightroom was noticeable, it was nowhere near of what I had expected. I even tried RAM disks for the previews and Camera RAW cache folders, but it made no noticeable difference in browsing or in the develop module. Not measured scientifically, just personal impression.

Robert Reiser said:

I have recently upgraded from a X79 based system

Click to expand...

Thanks for sharing.... what motherboard did you opt for ( and any key reasons for choice would be great...)

Gnits said:

Thanks for sharing.... what motherboard did you opt for ( and any key reasons for choice would be great...)

Click to expand...

Frustrated with Gigabyte (though recognizing that none of these vendors have good customer service), I went back to ASUS and hunted for U.2 support natively, without an add-on card. The add-on cards are probably OK, but it shows the board was not designed with U.2 in mind, so much as retrofitted.

I stumbled on the Z170-WS. It's labeled a "Workstation" board. It is a strange thing for ASUS, as it is hidden away. If you come into their site with their selectivity tools, e.g. picking Intel, then Z170 chip, etc. it does not appear. It's labeled "Commercial". So if you start from the commercial side and search you end up only with other chipsets, you can't find that one.

It's an orphan, lost between commercial space (usually X99 and Xeon) and the gamers. But in reading about it, it appeared aimed at excessive durability and peripheral use (it's the only one I found that could do 4 full size graphics cards, due to a built-in PCIe bridge). I don't need four, but that also brought with it reportedly more duable components, better power regulation (certainly more beefy). It also can have two M.2 cards, one U.2 (takes the place of one M.2) out of the box.

It also had six different fan headers, all PWM/Voltage controllable and a dedicated water pump header (not using it myself, but still shows they are serious about cooling control).

It's definitely not gamer oriented, no flashy paint job, no LED's (other than functional ones). But it is built for overclocking in a serious way as well.

I've always had the hypothesis, with no real data to support it (well, other than 5 or so builds of my own) that buying the Overclocking capable boards and unlocked Intel processors intended for high workloads at high frequencies would yield a more stable and reliable board at standard usage. So in that sense this looked doubly suitable - aimed at even beefier components.

Unfortunately it was quite a bit more expensive ($320) than originally planned, but I'm already deeply into this build, and won't hurt me to skip a few meals.

Gnits said:

Thanks for sharing.... what motherboard did you opt for ( and any key reasons for choice would be great...)

Click to expand...

Sure... I did choose the ASRock X99 Extreme 6, based on these requirements:

• Support for ECC RAM (just to be future proof, in case I want to upgrade to 128 GB at some point. 8 unbuffered DIMMs are usually not supported, but registered ECC is less of a burden for the controller). Currently, I have 4 x 16 GB ECC modules installed.
• Support for 8 DIMM sockets with a maximum of 128 GB (see above)
• Support for XEON CPUs (they are required for ECC RAM, and anyway I don't like the idea of overclocking my equipment. Keeping everything stable is hard enough even without overclocking). I went for the 6-core E5-1650 v4 (Broadwell-EP). For me, that is the price/performance sweet spot.
• Support for an M.2 slot with 4 x PCI-E lanes for the NVMe boot disk
• ATX format (I don't like these huge computer cases)
• Socket 2011-3 (because of the total # of PCI-E lanes and quad-channel RAM)

There are not a lot of desktop boards on the market supporting all my requirements. You can buy a workstation board, but they have their price and their own challenges. In hindsight, the ASRock X99 Extreme 4 would have worked as well for me. Let me know if you need additional information.

My apologies, I now realize Gnits was asking Robert. Sorry to have jumped in.