Color is just a fun bonus to me, conformation comes first and foremost, always. But, there are times color affects conformation and that is when it can matter. Particularly on pintos, markings can make a horse look longer or steeper than they really are. On solids crooked blazes can make a head look off and asymetrical front stockings can make a driving horse look out of rhythm.
There are a few other things I want to stay away from when it comes to color. I would rather not have a horse with a white tail because of the cleaning aspect but it's not a major issue. The only colors I do actually avoid for color's sake are cremello, perlino, etc as I don't want to risk sunburn. They are not found very often in the lines I like, thankfully. I also stay away from LWO+ horses.
On a whole, I've been blessed to find quality horses in amazing colors and patterns and I consider that an incredible bonus!