I don't think that our investment in UK-based brokers will be under any threat even if they do leave the EU. That said, I am wondering how conditions will change. After the referendum the FCA came out and said that for the time being there won't be any changes and everything will continue as it was (for now), but sooner or later the UK and the EU will be renegotiating trading agreements and the like. What will happen when a EU-based trader wants to withdraw money from a UK-based broker? Will the banking fees increase? What about investing in them? That's what bothering me more.
I don't think there will be big changes, but that's better to not holding the whole capital in UK brokers and banks..