MLOPS talk
SML
Two opposites
Intent based conversations, decision trees, easy to run, hard to build features., controlled
lllms , flexible ,natural,
Nextwork of SLM, offline processing, task search , general qa. Need to be strung together
Complex graph knowledge for how to converse wiith models
Neural decision parser , the brain
Slm expresses code for the intent of the system.
Given user utterance what will be done.
T5 Model:
Phased policy
Contextual question vs general purlose qus
Distiliation : vs sme experts.
Teacher system: student slm
Let models do the parts that are hard to scale.
Let SME s guide the models.
Humans in the loop
Have humans define the task, ie the input
SLM for inference, LLM for ..training
Focus of LLMs has move from makijg it bigger , to does it solve an actual problem.
Distilation networks of SLMs
Knowledge discovery, how to get sme knowledve out. What is sourced and what is fluff.
Small focused system.
Its difficult to get an embeeding space for a full business, better be specific.
Models for search, page ranking,
What does distilation look like in a RAG setup , teacher does hard bits,
Specific models , does what they should do at a the appropraite time.
Smaller can be better, scalable high quality data is essential.
Llm are good for general.
Distialition, is good when things scale, and costs are constricted.
How to evaluate a model/process, cant inprove what you cant measure.
How to add variabilty within a data set, so that the model is general to the context. Data collection process, . You need a good sampling of the problem space.
How to do data analysis , with interaction with llms. Tool use, python engine, wolfram.
Responsiveness of the system is important.
Swarm intelliengce: decision making with lots of SLM, to get a good conclusion. Use SLM to deliberate. To enhance performance. To a varitey of inputs ie hr, tech, ceo ect.