"They need a heavyweight to back the 'Out' campaign and they don't have one," said Charles Grant, director of the Centre for European Reform in London.