Everybody understands frequency response. Phase is a little harder to understand, but it is also important. Flat phase with zero degrees on frequency plot means that all frequencies comes out with the same phase. That means they come at the same time. So if you want to have group delay constant, you must have zero phase. Also Pic.1 and Pic.2  show how much different waveform is when 200Hz and 400Hz signal are in phase and with 400Hz delayed by 90 degrees. There are debates is phase audible for real life music listening. But tests shows that simple acoustic signals generated anechoically display clear phase audibility on headphones. On the Lipshitz paper  they say "normal music or speech signals phase distortion appears not to be generally audible, although it was heard with 99% confidence on some recorded vocal material. It is clear that more work needs to be done to ascertain acceptable limits for the phase linearity of audio components-limits which might become more stringent as improved recording/reproduction systems become available".

   Linear phase also makes possible to implement theoretical perfect impulse response 4-way system. Otherwise phase shift will make problems at crossover regions.

  This link on linkwitz site talks about linear phase is easy to read and understand and it has measurements. Recommend to take a look. 

   So building high resolution system phase has to be taken into consideration !