Mixture of Experts

Different parts of the network focusing on parts of the questions

Routing, distribution

activating