We believe that the narrative of only listening to experts or trusting the wisdom of the crowd blindly is flawed. Instead we have developed a system that weighs experts and lay-people differently and dynamically and show that a good balance is required. We show that our methodology leads to a 15 percent improvement in mean performance, 15 percent decrease in variance, and almost 30 percent increase in Sharpe-type ratio in a real online market.