14  Static & Dynamic Models

14.1 1. Key Message

👍 Thought conversion 1:

“Soft” communication shares many qualities of “hard” machine learning. “Hard” deep learning is ground-breaking because of, not despite of its “soft” qualities.

14.2 2. Imagine the parameters to a model

• These are the most intuitive dimensions of a model to consider.
• Parameters draw a path from the input to the output and how they interact with each other.
  • e.g. X in the following model:

\[y_i = \beta_{0} + \beta_{1} x_{i1} + \cdots + \beta_{p} x_{ip} + \varepsilon_i\]

• Of the known parameters, we aim to define its coefficient, i.e. how much it influences the outcome.
  • known & actually measured influencing variables
• Of the know parameters, sometimes this is not possible
  • Known influencing variables.
• Some parameters will not even be considered
  • Unknown influencing variables

14.3 3. Two extremes

	Static/Finite Disciplines	Dynamic/Infinite Disciplines
Classical models (goal/object of discipline)	Seek universal truths, e.g. In the form of natural laws & robust theories	Seeking unified understanding (universal truths are rarer and more difficult to develop)
Techniques	Closed, controlled environments, where the entirety of a system may be described	Even with closed environments, the infinite complexity of the real world is difficult to capture. At most we can define broad trends and generalizations.
Certainty	Concrete thinking	Probabilistic thinking
Approach	Analytical Thinking	Systems thinking
Logic	Deductive reasoning	Inductive reasoning
Disciplines	Classical physics & chemistry and to a point molecular genetics	Systems Biology (ecology, earth science) & psychology

14.4 4. A Scenario

Now imagine the simple actions of two people talking to each other. We can imagine two models are at play for each person depending on what they are doing:

	Talking	Listening
Model	Describes an encoder transmiting information.	Describes a decoder receiving information.
Input	Thought	Speech
Output	Speech	Thought

TalkingDescribes a model where an encoder transmitsInput in thoughtoutput is speech ListeningDescribes a model where a decoder receivesinput is speechoutput is thought Input, model and output are Interdependent, constrained contextual. Dynamic/Infinite models are also present in project management, decision making & other areas outside of actual technical work.

14.5 5. In Classical Definitions

Static/Finite models in classically soft disciplines

Dynamic/Infinite models in classically hard disciplines

cf. cases where e.g. biology or psychology try to trace a direct path between input and output. This is possible, (but it appears quite rare). Classical Mendialian genetics, which is actually hite rare. Diagnostic markers or a disearst (biomarkers, genetic variants) which are also rare.

Deep learning is a great example of the development of dynamic/infinite models combined with classical hard disciplines, like maths. Early AI attempted to predict all possible outcomes of a system, which turned out to be an imposible task. ML algorithims defines the model and the hyperparameters then tried to use the right model on the data Then DL let the model redefine according to the data Upcoming: Foundational models Data-centric models, instead of modifying models, get more data and clean up the data to get more information. (see Andrew Ng.)